Dexterous hand motion planning method and system based on deep learning

Abstract

The invention relates to the technical field of dexterous hand motion planning, in particular to a dexterous hand motion planning method and system based on deep learning, and the method comprises the steps: obtaining the state information of a dexterous hand, carrying out the multi-modal feature fusion to generate environment perception information, carrying out the dynamic obstacle detection and cooperative signal analysis, generating an obstacle avoidance path, and optimizing the motion path planning. Real-time data are collected through the sensor module, multi-modal features are fused, and environment sensing information is generated to support motion planning of the dexterous hand; collaborative decision-making among multiple dexterous hands is realized by utilizing an internal communication module, and the adaptive ability and task execution efficiency in a complex dynamic environment are remarkably improved. In addition, dynamic obstacle information is shared by prompting a cooperation signal, conflicts are avoided, and the response speed is increased. The method is suitable for efficient motion planning of the dexterous hand in a complex dynamic scene.

Description

Technical Field

[0001] This invention belongs to the field of dexterous hand motion planning technology, specifically a dexterous hand motion planning method and system based on deep learning. Background Technology

[0002] With the rapid development of robotics technology, dexterous hands are increasingly used in complex tasks, especially in scenarios such as precision grasping and sorting goods, demonstrating great potential. However, existing dexterous hand motion planning methods still have significant shortcomings in terms of intelligent decision-making, adaptability, and efficiency, making it difficult to meet the needs of diverse task scenarios. For example, patent CN114800490B proposes a dexterous hand adaptive admittance control system and method for precision grasping. Through the design of the task planning layer and the underlying control layer, it achieves precise control of contact position and contact force and solves the adaptability problem for different objects. However, this technical solution relies on parameter identification of the dynamic model of the unknown environment and requires real-time calculation of impedance relationships to achieve a unified framework for force control and position control. Although this method improves the accuracy of grasping, its computational complexity is high, and its ability to respond quickly to dynamic environments is limited, making it difficult to adapt to complex and ever-changing task requirements. Furthermore, patent CN119260722B proposes a motion planning and control method and system for robot sorting goods. It accurately calculates camera depth maps using feature correction methods and volume rendering technology, and combines an improved whale optimization algorithm to plan the motion trajectory and grasping posture of the robotic arm and dexterous hand. This method can significantly improve grasping speed and accuracy, but its core algorithm is optimized for specific application scenarios, resulting in poor versatility. At the same time, this technical solution does not fully utilize the advantages of deep learning; its planning efficiency and intelligence level still need improvement when facing high-dimensional, unstructured environments.

[0003] The aforementioned problems with existing technologies indicate that current dexterity hand motion planning methods still have certain shortcomings in terms of intelligent decision-making, adaptability to dynamic environments, and versatility, particularly exhibiting significant limitations when handling complex, dynamic, and unstructured tasks. Therefore, there is an urgent need for an innovative technical solution that introduces deep learning models to improve the intelligence level of motion planning, enhance adaptability to dynamic environments, and simplify the planning process for complex tasks, thereby meeting the efficient operational needs of dexterity hands in diverse scenarios. Based on this background, this invention proposes a deep learning-based dexterity hand motion planning method and system, aiming to solve the technical problems existing in the prior art and promote the further development of dexterity hands in practical applications. Summary of the Invention

[0004] This invention provides a deep learning-based method for dexterous hand motion planning, the main purpose of which is to improve the dexterous hand's adaptability and task execution efficiency in complex dynamic environments.

[0005] To achieve the above objectives, this invention provides a deep learning-based method for dexterous hand motion planning, comprising: The initial state information of the dexterous hand is obtained, and the joint angles, finger postures, and spatial position of the target object are extracted from the initial state information. Real-time environmental data is collected by the sensor module of the dexterous hand, and multimodal feature fusion processing is performed on the real-time environmental data to obtain environmental perception information; Determine whether the environmental perception information contains dynamic obstacle information, and use the internal communication module of the dexterous hand to determine whether there is a cooperative signal from a neighboring dexterous hand; When the environmental perception information does not contain dynamic obstacle information and no cooperative signal is received, return to the steps described above for collecting real-time environmental data based on the sensor module of the dexterous hand; When the environmental perception information contains dynamic obstacle information and no coordination signal is received, the power control module of the dexterous hand is used to generate an obstacle avoidance action path based on the environmental perception information and generate a prompt coordination signal. When the environmental perception information contains dynamic obstacle information and a coordination signal is received, the coordination signal is parsed to obtain the action intention information of the nearby dexterous hand. The environmental perception information and the action intention information are combined to make a comprehensive decision, generate an optimized action path, and generate a second prompt coordination signal.

[0006] Optionally, the step of performing multimodal feature fusion processing on the real-time environmental data to obtain environmental perception information includes: The real-time environmental data is subjected to outlier removal and missing value imputation to obtain standardized data, and the standardized data is then spatially aligned to obtain consistent data. Multi-dimensional feature extraction is performed on the consistent data to obtain a feature set, and semantic annotation based on scene segmentation is performed on the feature set to obtain a scene object; The scene objects are classified and identified to obtain object categories, and key areas of the scene objects are marked to obtain marker points; Within a preset time window, the dynamic change trajectory of the marker points is acquired to obtain a dynamic feature sequence; By summarizing the scene objects, object categories, and dynamic feature sequences, environmental perception information is obtained.

[0007] Optionally, the power control module of the dexterous hand generates an obstacle avoidance path based on the environmental perception information and generates a prompting and coordination signal, including: Based on the environmental perception information, the dexterous hand is used to plan the movement path to obtain an obstacle avoidance path; The power control module is used to control the movement of the dexterous hand according to the obstacle avoidance path; During the dexterous hand's motion control process, the influence range of the force of the power control module is obtained, and the environmental perception information, obstacle avoidance path, and influence range of the force are used to generate a prompt coordination signal.

[0008] Optionally, the step of planning the movement path of the dexterous hand based on the environmental perception information to obtain an obstacle avoidance path includes: Based on the dynamic feature sequence in the environmental perception information, the motion direction and relative speed of the dynamic obstacle are obtained; Based on the scene objects in the environmental perception information, obtain the boundary range of the obstacles; The target motion path of the dexterous hand is obtained, and the nearest intersection point between the motion direction of the dynamic obstacle and the target motion path is identified to obtain the nearest point of the obstacle and the nearest point of the dexterous hand. The vector information from the nearest point of the obstacle to the nearest point of the dexterous hand is obtained to obtain the obstacle avoidance direction and the nearest distance. Obtain the operating radius of the dexterous hand, calculate the sum of the obstacle boundary range and the operating radius to obtain the collision range, and calculate the difference between the collision range and the nearest distance to obtain the minimum obstacle avoidance distance; Based on the obstacle avoidance direction and minimum obstacle avoidance distance, obtain the nearest safe operating point; The real-time posture of the dexterous hand is obtained, and based on the relative speed, the motion path between the nearest safe operation point and the real-time posture is constructed to obtain the obstacle avoidance path.

[0009] Optionally, after generating the cue coordination signal, the method further includes: The movement speed of the dexterous hand is obtained, and the ratio of the relative speed to the movement speed is calculated to obtain the speed weight. Based on the speed weight, the preset cooperation range is weighted and adjusted to obtain the signal coverage range; The internal communication module is used to propagate the prompting and coordination signal according to the signal coverage area.

[0010] Optionally, after obtaining the velocity weights, the method further includes: Determine whether the speed weight exceeds a preset high-risk threshold; When the speed weight exceeds the high-risk threshold, the behavior pattern of the dynamic obstacle is obtained, wherein the behavior pattern includes an inertial sliding mode and a sudden turning mode, the interference area of ​​the inertial sliding mode is a linearly expanding region, and the interference area of ​​the sudden turning mode is a fan-shaped spreading region. Based on the interference area, the boundary range of the dynamic obstacle is updated to obtain an updated boundary range, and an updated obstacle avoidance path is obtained using the updated boundary range.

[0011] Optionally, the step of comprehensively deciding on the environmental perception information and action intention information to generate an optimized action path includes: Acquire the force distribution of neighboring dexterous hands, the movement paths of neighboring dexterous hands, and historical environmental perception information from the action intent information; Obtain the correlation between the historical environmental perception information and the environmental perception information; Based on the aforementioned correlation, time correction is performed on the force distribution and movement path of the neighboring dexterous hand to obtain the corrected force distribution and corrected movement path. By summarizing the environmental perception information, the distribution of corrective forces, and the corrective action path, comprehensive decision information is obtained, and an optimized action path is generated based on the comprehensive decision information.

[0012] Optionally, generating the optimized action path based on the comprehensive decision information includes: Based on the target action route, environmental perception information in the comprehensive decision information, and the corrected action path, the direction of the dexterous hand action and the direction of obstacle avoidance are obtained. The intersection direction of the dexterous hand movement direction and the obstacle avoidance direction is obtained to obtain the second movement direction; Obtain the nearest safe operation point in the second action direction to obtain the second safe operation point; The motion path of the real-time attitude and the second safe operation point is obtained to obtain the optimized motion path; Obtain the force vector sequence of the optimized action path in the corrected force distribution; The dexterous hand is controlled by using the force vector sequence and optimized motion path.

[0013] Optionally, after performing motion control on the dexterous hand, the method further includes: Obtain the execution parameters of the action control, and generate an action log based on the execution parameters; The action log is sent to a pre-built task management system to obtain feedback information from the task management system, and the actions of the dexterous hand are corrected based on the feedback information.

[0014] To achieve the above objectives, the present invention also provides a dexterous hand motion planning system based on deep learning, comprising: The state acquisition module is used to acquire the initial state information of the dexterous hand and extract the joint angles, finger postures and spatial position of the target object from the initial state information; An environmental perception module is used to collect real-time environmental data based on the sensor module of the dexterous hand, and to perform multimodal feature fusion processing on the real-time environmental data to obtain environmental perception information. The collaborative detection module is used to determine whether the environmental perception information contains dynamic obstacle information, and to use the internal communication module of the dexterous hand to determine whether there is a collaborative signal from a neighboring dexterous hand; The motion planning module is used to return to the steps of collecting real-time environmental data based on the sensor module of the dexterous hand when the environmental perception information does not contain dynamic obstacle information and no coordination signal is received; and when the environmental perception information contains dynamic obstacle information and no coordination signal is received, to generate an obstacle avoidance motion path and a prompt coordination signal using the power control module of the dexterous hand based on the environmental perception information; and when the environmental perception information contains dynamic obstacle information and a coordination signal is received, to parse the coordination signal to obtain the motion intention information of the adjacent dexterous hand, to make a comprehensive decision by combining the environmental perception information and the motion intention information, to generate an optimized motion path, and to generate a second prompt coordination signal.

[0015] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: Memory, storing at least one instruction; The processor executes the instructions stored in the memory to implement the deep learning-based dexterity hand motion planning method described above.

[0016] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the deep learning-based dexterity hand motion planning method described above.

[0017] To address the problems described in the background art, this invention first collects real-time environmental data through a sensor module and performs multimodal feature fusion to generate environmental perception information. Then, based on this information, preliminary movement planning for the dexterous hand is performed. In this invention, when a dexterous hand generates an obstacle avoidance path, a cueing coordination signal is generated. This signal can be transmitted to neighboring dexterous hands, allowing them to adjust their movements in advance to avoid conflicts. Furthermore, due to the transmission of the cueing coordination signal, the dynamic obstacle information acquired by the current dexterous hand can be shared with neighboring dexterous hands that lack complete information, achieving collaborative decision-making among multiple dexterous hands. This not only improves the dexterous hand's adaptability to dynamic environments but also enhances execution efficiency in complex task scenarios. Moreover, communication between dexterous hands in this invention is directly accomplished through an internal communication module, significantly improving response speed. Therefore, this invention can significantly improve the movement planning ability and task execution efficiency of dexterous hands in complex dynamic environments. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a deep learning-based dexterity hand motion planning method provided in an embodiment of the present invention; Figure 2 A functional block diagram of a dexterous hand motion planning system based on deep learning provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the deep learning-based dexterous hand motion planning method according to an embodiment of the present invention. Detailed Implementation

[0019] This invention provides a deep learning-based method and system for dexterous hand motion planning. Its core lies in significantly improving the dexterous hand's adaptability and task execution efficiency in complex dynamic environments through multimodal data fusion and collaborative decision-making mechanisms. Combined with... Figures 1 to 3 As shown, the specific implementation of the present invention will be described in detail below, including the functional implementation of each module, algorithm design and actual operation process.

[0020] First, see Figure 1 The core process of this invention begins with the state acquisition module, which is responsible for collecting the initial state information of the dexterous hand and extracting joint angles, finger postures, and the spatial position of the target object. This information forms the basis for subsequent motion planning, and its accuracy directly affects the performance of the entire system. In practical applications, dexterous hands are typically equipped with various sensors, such as inertial measurement units (IMUs), torque sensors, and vision sensors, to monitor the state of the dexterous hand in real time. By processing the data from these sensors, the initial state information of the dexterous hand can be accurately obtained. For example, in a specific embodiment, the initial state information of a dexterous hand shows that its joint angles are... The fingers are in a bent position, and the spatial position of the target object is... This data is then fed into subsequent modules for further processing.

[0021] Next, the environmental perception module performs multimodal feature fusion processing on the real-time environmental data collected by the dexterous hand's sensor module to generate environmental perception information. This process includes several steps: First, outlier removal and missing value imputation are performed on the real-time environmental data to obtain standardized data; then, spatial alignment is performed on the standardized data to ensure consistency between data from different sensors; subsequently, multi-dimensional feature extraction is performed on the consistent data to obtain a feature set, and semantic annotation based on scene segmentation is performed on the feature set to obtain scene objects; next, scene objects are classified and identified to obtain object categories, and key areas of scene objects are marked to obtain marker points; finally, within a preset time window, the dynamic change trajectory of the marker points is acquired to obtain a dynamic feature sequence, and the scene objects, object categories, and dynamic feature sequences are summarized to form complete environmental perception information. For example, in one embodiment, the environmental perception module detects a dynamic obstacle moving to the right with a relative speed of v=2m / s and an obstacle boundary range of r=0.3m, while marking key areas of the obstacle such as edges and center points. This information provides important basis for subsequent motion path planning.

[0022] The collaborative detection module is responsible for determining whether the environmental perception information contains dynamic obstacle information and using the dexter's internal communication module to determine whether there is a collaborative signal from a neighboring dexter. When the environmental perception information does not contain dynamic obstacle information and no collaborative signal is received, the system returns to the real-time environmental data acquisition step to continue monitoring environmental changes. However, if the environmental perception information contains dynamic obstacle information but no collaborative signal is received, the motion planning module uses the power control module to generate an obstacle avoidance path based on the environmental perception information and generates a prompting collaborative signal. The purpose of the prompting collaborative signal is to notify neighboring dexterities of the current dynamic obstacle information so that they can adjust their actions in advance to avoid conflicts. For example, in one embodiment, when dexter A detects a dynamic obstacle and generates an obstacle avoidance path, it sends a prompting collaborative signal to neighboring dexter B through its internal communication module. The signal includes information such as the obstacle's direction of movement, relative speed, boundary range, and obstacle avoidance path. This collaborative mechanism not only improves the obstacle avoidance capability of a single dexter but also enhances the cooperation efficiency among multiple dexterities.

[0023] When the environmental perception information includes dynamic obstacle information and a coordination signal is received, the motion planning module parses the coordination signal to obtain the motion intention information of the neighboring dexterous hand. It then combines the environmental perception information and the motion intention information to make a comprehensive decision, generating an optimized motion path and a second cue coordination signal. The comprehensive decision-making process includes several steps: First, it acquires the force distribution of the neighboring dexterous hand, the motion path of the neighboring dexterous hand, and historical environmental perception information from the motion intention information. Next, it acquires the correlation between historical and current environmental perception information. Then, it performs time correction on the force distribution and motion path of the neighboring dexterous hand based on the correlation, obtaining a corrected force distribution and a corrected motion path. Finally, it summarizes the environmental perception information, the corrected force distribution, and the corrected motion path to obtain comprehensive decision information, and generates an optimized motion path based on this comprehensive decision information. For example, in one embodiment, after dexterous hand A receives a coordination signal from dexterous hand B, it parses that B's motion path is a straight line forward, with the force distribution concentrated in the forward area. Simultaneously, it combines the current environmental perception information to detect a dynamic obstacle ahead. In this scenario, dexterous hand A generates an optimized motion path to bypass the obstacle and sends a second cueing coordination signal to other dexterous hands via the internal communication module to update the global collaborative decision.

[0024] In the process of generating obstacle avoidance motion paths, the motion planning module includes the following steps: First, based on the dynamic feature sequence in the environmental perception information, the motion direction and relative velocity of the dynamic obstacle are obtained; then, based on the scene objects in the environmental perception information, the obstacle boundary range is obtained; next, the target motion path of the dexterous hand is obtained, and the nearest intersection point between the motion direction of the dynamic obstacle and the target motion path is identified to obtain the nearest point of the obstacle and the nearest point of the dexterous hand, and the vector information from the nearest point of the obstacle to the nearest point of the dexterous hand is obtained to obtain the obstacle avoidance direction and the nearest distance; subsequently, the operating radius of the dexterous hand is obtained, the sum of the obstacle boundary range and the operating radius is calculated to obtain the collision range, and the difference between the collision range and the nearest distance is calculated to obtain the minimum obstacle avoidance distance; finally, based on the obstacle avoidance direction and the minimum obstacle avoidance distance, the nearest safe operating point is obtained, and based on the real-time posture and relative velocity of the dexterous hand, the motion path between the nearest safe operating point and the real-time posture is constructed to obtain the obstacle avoidance path. For example, in one embodiment, the target movement path of the dexterous hand is a straight line forward, the dynamic obstacle moves to the right with a relative speed of v = 2 m / s, the obstacle boundary range is r = 0.3 m, and the dexterous hand's operating radius is R = 0.2 m. Calculations show that the collision range is r + R = 0.5 m, the closest distance is d = 0.7 m, and the minimum obstacle avoidance distance is d - (r + R) = 0.2 m. Based on these parameters, the system ultimately generates a safe movement path to bypass the obstacle.

[0025] Furthermore, to improve the propagation efficiency of the coordination signal, this invention introduces the concept of speed weighting. Specifically, after generating the cue coordination signal, the system acquires the movement speed of the dexterous hand, calculates the ratio of relative speed to movement speed to obtain a speed weight, and then adjusts a preset coordination range according to the speed weight to obtain the signal coverage range. Subsequently, the cue coordination signal is propagated using an internal communication module based on the signal coverage range. For example, in one embodiment, the movement speed of the dexterous hand is... The relative speed is Then the speed weight is Assuming the preset scope of collaboration is... The weighted signal coverage area is then... In this way, the system can dynamically adjust the propagation range of the collaborative signal according to the actual situation, thereby improving the overall response speed and collaborative efficiency.

[0026] In certain high-risk scenarios, the system also needs to analyze the behavior patterns of dynamic obstacles. For example, when the speed weight exceeds a preset high-risk threshold, the system acquires the behavior patterns of the dynamic obstacles, including inertial sliding patterns and sudden turning patterns. The interference area of ​​the inertial sliding pattern is a linearly expanding region, while the interference area of ​​the sudden turning pattern is a fan-shaped spreading region. Based on the interference area, the system updates the boundary range of the dynamic obstacle to obtain an updated boundary range, and uses the updated boundary range to obtain an updated obstacle avoidance path. For example, in one embodiment, when the speed weight exceeds the high-risk threshold... At that time, the system detected that the dynamic obstacle's behavior pattern was a sudden turning pattern, and its interference area was a fan-shaped diffusion area with an angle of [missing information]. The radius is r = 0.5m. Based on these parameters, the system updates the boundary range of the dynamic obstacle and generates a safer obstacle avoidance path.

[0027] Finally, after completing motion control, the system generates a motion log and sends it to a pre-built task management system to obtain feedback and correct the dexterous hand's movements. The motion log includes execution parameters for motion control, such as the motion path, force distribution, and execution time. The task management system generates feedback information based on these parameters, such as motion deviation and execution efficiency, and sends this feedback back to the dexterous hand system for further motion correction. For example, in one embodiment, if the dexterous hand's motion path deviates from the target path by 0.1m and the execution time is t=5s, the system adjusts the motion path based on the feedback and re-executes the task, ultimately achieving higher task execution accuracy.

[0028] In summary, this invention significantly improves the adaptability and task execution efficiency of dexterous hands in complex dynamic environments through multimodal data fusion, collaborative decision-making mechanisms, and dynamic path planning algorithms. Combined with... Figures 1 to 3 The embodiments shown demonstrate that the specific implementation of the present invention covers the complete process from data acquisition to action execution, ensuring the feasibility and practicality of the technical solution.

Claims

1. A deep learning-based dexterous hand motion planning method, characterized in that, The method comprises: acquiring initial state information of the dexterous hand, and extracting joint angles, finger postures, and spatial positions of target objects from the initial state information; collecting real-time environmental data according to a sensor module of the dexterous hand, and performing multi-modal feature fusion processing on the real-time environmental data to obtain environmental perception information; judging whether the environmental perception information contains dynamic obstacle information, and judging whether there is a coordination signal of a neighboring dexterous hand by using an internal communication module of the dexterous hand; when the environmental perception information does not contain dynamic obstacle information and no coordination signal is received, returning to the step of collecting real-time environmental data according to the sensor module; when the environmental perception information contains dynamic obstacle information and no coordination signal is received, generating an obstacle avoidance action path according to the environmental perception information by using a power control module of the dexterous hand, and generating a prompt coordination signal; when the environmental perception information contains dynamic obstacle information and a coordination signal is received, analyzing the coordination signal to obtain action intention information of a neighboring dexterous hand, comprehensively deciding by combining the environmental perception information and the action intention information, generating an optimized action path, and generating a second prompt coordination signal.

2. The deep learning-based dexterous hand motion planning method of claim 1, wherein, The multi-modal feature fusion processing on the real-time environmental data to obtain environmental perception information comprises: performing outlier rejection and missing value filling operations on the real-time environmental data to obtain standardized data, and performing spatial alignment processing on the standardized data to obtain consistent data; performing multi-dimensional feature extraction operations on the consistent data to obtain a feature set, and performing semantic labeling operations on the feature set based on scene segmentation to obtain scene objects; performing classification identification on the scene objects to obtain object categories, and performing key region marking operations on the scene objects to obtain marker points; acquiring dynamic change trajectories of the marker points within a preset time window to obtain dynamic feature sequences; summarizing the scene objects, object categories, and dynamic feature sequences to obtain environmental perception information.

3. The deep learning-based dexterous hand motion planning method of claim 1, wherein, The generation of an obstacle avoidance action path according to the environmental perception information by using a power control module of the dexterous hand and the generation of a prompt coordination signal comprise: performing action path planning on the dexterous hand according to the environmental perception information to obtain an obstacle avoidance path; performing action control on the dexterous hand according to the obstacle avoidance path by using the power control module; acquiring an influence range of the power control module during the execution of the action control of the dexterous hand, and generating a prompt coordination signal by using the environmental perception information, the obstacle avoidance path, and the influence range.

4. The deep learning-based dexterous hand motion planning method of claim 3, wherein, The action path planning on the dexterous hand according to the environmental perception information to obtain an obstacle avoidance path comprises: acquiring a motion direction and a relative speed of a dynamic obstacle according to a dynamic feature sequence in the environmental perception information; acquiring an obstacle boundary range according to a scene object in the environmental perception information; Obtain a target action route of the dexterous hand, identify a nearest intersection point between a motion direction of the dynamic obstacle and the target action route, obtain an obstacle nearest point and a dexterous hand nearest point, and obtain vector information from the obstacle nearest point to the dexterous hand nearest point to obtain an obstacle avoidance direction and a nearest distance; Obtain an operation radius of the dexterous hand, calculate a sum of a boundary range of the obstacle and the operation radius to obtain a collision range, and calculate a difference between the collision range and the nearest distance to obtain a minimum obstacle avoidance distance; Obtain a nearest safe operation point according to the obstacle avoidance direction and the minimum obstacle avoidance distance; Obtain a real-time posture of the dexterous hand, construct an action path between the nearest safe operation point and the real-time posture according to the relative speed to obtain an obstacle avoidance path.

5. The deep learning-based dexterous hand motion planning method of claim 3, wherein, After the prompt cooperation signal is generated, the method further comprises: Obtain an action speed of the dexterous hand, calculate a proportion of the relative speed and the action speed to obtain a speed weight; Weight-adjust a preset cooperation range according to the speed weight to obtain a signal coverage range; Use the internal communication module to propagate the prompt cooperation signal according to the signal coverage range.

6. The deep learning-based dexterous hand motion planning method of claim 5, wherein, After the speed weight is obtained, the method further comprises: Determine whether the speed weight exceeds a preset high-risk threshold; When the speed weight exceeds the high-risk threshold, obtain a behavior mode of the dynamic obstacle, wherein the behavior mode comprises an inertial sliding mode and a sudden turning mode, an interference area of the inertial sliding mode is a linear expansion area, and an interference area of the sudden turning mode is a fan-shaped diffusion area; Update a boundary range of the dynamic obstacle according to the interference area to obtain an updated boundary range, and use the updated boundary range to obtain an updated obstacle avoidance path.

7. The deep learning-based dexterous hand motion planning method of claim 1, wherein, The comprehensive decision-making on the environment perception information and the action intention information to generate an optimized action path comprises: Obtain a neighboring dexterous hand force distribution, a neighboring dexterous hand action path and historical environment perception information in the action intention information; Obtain an association relationship between the historical environment perception information and the environment perception information; Time-correct the neighboring dexterous hand force distribution and the neighboring dexterous hand action path according to the association relationship to obtain a corrected force distribution and a corrected action path; Summarize the environment perception information, the corrected force distribution and the corrected action path to obtain comprehensive decision-making information, and generate an optimized action path according to the comprehensive decision-making information.

8. The deep learning-based dexterous hand motion planning method of claim 7, wherein, The generation of the optimized action path according to the comprehensive decision-making information comprises: Obtain a dexterous hand action direction and an obstacle avoidance direction according to the target action route, the environment perception information in the comprehensive decision-making information and the corrected action path; Obtain an intersection direction of the dexterous hand action direction and the obstacle avoidance direction to obtain a second action direction; Obtain a nearest safe operation point in the second action direction to obtain a second safe operation point; Obtain an action path of the real-time posture and the second safe operation point to obtain an optimized action path; Obtain a force vector sequence of the optimized action path in the corrected force distribution; The force vector sequence and the optimized motion path are used to control the motion of the dexterous hand.

9. The deep learning-based dexterous hand motion planning method of claim 8, wherein, After the motion control of the dexterous hand, the method further comprises: obtaining execution parameters of the motion control, and generating a motion log according to the execution parameters; sending the motion log to a pre-constructed task management system, obtaining feedback information of the task management system, and correcting the motion of the dexterous hand according to the feedback information. 10.A dexterous hand motion planning system based on deep learning, characterized in that, The system comprises: a state acquisition module configured to acquire initial state information of the dexterous hand, and extract joint angles, finger postures and spatial positions of target objects from the initial state information; an environment perception module configured to collect real-time environment data according to a sensor module of the dexterous hand, and perform multi-modal feature fusion processing on the real-time environment data to obtain environment perception information; a cooperative detection module configured to determine whether the environment perception information contains dynamic obstacle information, and determine whether there is a cooperative signal of a neighboring dexterous hand by using an internal communication module of the dexterous hand; a motion planning module configured to return to the step of collecting real-time environment data according to the sensor module when the environment perception information does not contain dynamic obstacle information and no cooperative signal is received, generate an obstacle avoidance motion path and a prompt cooperative signal according to the environment perception information by using a power control module of the dexterous hand when the environment perception information contains dynamic obstacle information and no cooperative signal is received, and analyze the cooperative signal to obtain motion intention information of the neighboring dexterous hand when the environment perception information contains dynamic obstacle information and the cooperative signal is received, make a comprehensive decision by combining the environment perception information and the motion intention information, generate an optimized motion path and a second prompt cooperative signal.

Images