A dexterous hand motion planning method and system based on deep learning

Abstract

The application relates to the technical field of dexterous hand motion planning, in particular to a dexterous hand motion planning method and system based on deep learning, which comprises acquiring dexterous hand state information, generating environment perception information through multi-modal feature fusion, detecting dynamic obstacles and analyzing collaborative signals, generating an obstacle avoidance path and optimizing an action path plan. Real-time data is collected by a sensor module and multi-modal features are fused to generate environment perception information to support the action planning of the dexterous hand; internal communication modules are used to realize collaborative decision-making among multiple dexterous hands, thereby significantly improving the self-adaptability and task execution efficiency in a complex dynamic environment. In addition, dynamic obstacle information is shared through prompt collaborative signals to avoid conflicts and enhance response speed. The application is suitable for efficient motion planning of dexterous hands in complex dynamic scenarios.

Description

Technical Field

[0001] This invention belongs to the field of dexterous hand motion planning technology, specifically a dexterous hand motion planning method and system based on deep learning. Background Technology

[0002] With the rapid development of robotics technology, dexterous hands are increasingly used in complex tasks, especially in scenarios such as precision grasping and sorting goods, demonstrating great potential. However, existing dexterous hand motion planning methods still have significant shortcomings in terms of intelligent decision-making, adaptability, and efficiency, making it difficult to meet the needs of diverse task scenarios. For example, patent CN114800490B proposes a dexterous hand adaptive admittance control system and method for precision grasping. Through the design of the task planning layer and the underlying control layer, it achieves precise control of contact position and contact force and solves the adaptability problem for different objects. However, this technical solution relies on parameter identification of the dynamic model of the unknown environment and requires real-time calculation of impedance relationships to achieve a unified framework for force control and position control. Although this method improves the accuracy of grasping, its computational complexity is high, and its ability to respond quickly to dynamic environments is limited, making it difficult to adapt to complex and ever-changing task requirements. Furthermore, patent CN119260722B proposes a motion planning and control method and system for robot sorting goods. It accurately calculates camera depth maps using feature correction methods and volume rendering technology, and combines an improved whale optimization algorithm to plan the motion trajectory and grasping posture of the robotic arm and dexterous hand. This method can significantly improve grasping speed and accuracy, but its core algorithm is optimized for specific application scenarios, resulting in poor versatility. At the same time, this technical solution does not fully utilize the advantages of deep learning; its planning efficiency and intelligence level still need improvement when facing high-dimensional, unstructured environments.

[0003] The aforementioned problems with existing technologies indicate that current dexterity hand motion planning methods still have certain shortcomings in terms of intelligent decision-making, adaptability to dynamic environments, and versatility, particularly exhibiting significant limitations when handling complex, dynamic, and unstructured tasks. Therefore, there is an urgent need for an innovative technical solution that introduces deep learning models to improve the intelligence level of motion planning, enhance adaptability to dynamic environments, and simplify the planning process for complex tasks, thereby meeting the efficient operational needs of dexterity hands in diverse scenarios. Based on this background, this invention proposes a deep learning-based dexterity hand motion planning method and system, aiming to solve the technical problems existing in the prior art and promote the further development of dexterity hands in practical applications. Summary of the Invention

[0004] This invention provides a deep learning-based method for dexterous hand motion planning, the main purpose of which is to improve the dexterous hand's adaptability and task execution efficiency in complex dynamic environments.

[0005] To achieve the above objectives, this invention provides a deep learning-based method for dexterous hand motion planning, comprising: The initial state information of the dexterous hand is obtained, and the joint angles, finger postures, and spatial position of the target object are extracted from the initial state information. Real-time environmental data is collected by the sensor module of the dexterous hand, and multimodal feature fusion processing is performed on the real-time environmental data to obtain environmental perception information; Determine whether the environmental perception information contains dynamic obstacle information, and use the internal communication module of the dexterous hand to determine whether there is a cooperative signal from a neighboring dexterous hand; When the environmental perception information does not contain dynamic obstacle information and no cooperative signal is received, return to the steps described above for collecting real-time environmental data based on the sensor module of the dexterous hand; When the environmental perception information contains dynamic obstacle information and no coordination signal is received, the power control module of the dexterous hand is used to generate an obstacle avoidance action path based on the environmental perception information and generate a prompt coordination signal. When the environmental perception information contains dynamic obstacle information and a coordination signal is received, the coordination signal is parsed to obtain the action intention information of the nearby dexterous hand. The environmental perception information and the action intention information are combined to make a comprehensive decision, generate an optimized action path, and generate a second prompt coordination signal.

[0006] Optionally, the step of performing multimodal feature fusion processing on the real-time environmental data to obtain environmental perception information includes: The real-time environmental data is subjected to outlier removal and missing value imputation to obtain standardized data, and the standardized data is then spatially aligned to obtain consistent data. Multi-dimensional feature extraction is performed on the consistent data to obtain a feature set, and semantic annotation based on scene segmentation is performed on the feature set to obtain a scene object; The scene objects are classified and identified to obtain object categories, and key areas of the scene objects are marked to obtain marker points; Within a preset time window, the dynamic change trajectory of the marker points is acquired to obtain a dynamic feature sequence; By summarizing the scene objects, object categories, and dynamic feature sequences, environmental perception information is obtained.

[0007] Optionally, the power control module of the dexterous hand generates an obstacle avoidance path based on the environmental perception information and generates a prompting and coordination signal, including: Based on the environmental perception information, the dexterous hand is used to plan the movement path to obtain an obstacle avoidance path; The power control module is used to control the movement of the dexterous hand according to the obstacle avoidance path; During the dexterous hand's motion control process, the influence range of the force of the power control module is obtained, and the environmental perception information, obstacle avoidance path, and influence range of the force are used to generate a prompt coordination signal.

[0008] Optionally, the step of planning the movement path of the dexterous hand based on the environmental perception information to obtain an obstacle avoidance path includes: Based on the dynamic feature sequence in the environmental perception information, the motion direction and relative speed of the dynamic obstacle are obtained; Based on the scene objects in the environmental perception information, obtain the boundary range of the obstacles; The target motion path of the dexterous hand is obtained, and the nearest intersection point between the motion direction of the dynamic obstacle and the target motion path is identified to obtain the nearest point of the obstacle and the nearest point of the dexterous hand. The vector information from the nearest point of the obstacle to the nearest point of the dexterous hand is obtained to obtain the obstacle avoidance direction and the nearest distance. Obtain the operating radius of the dexterous hand, calculate the sum of the obstacle boundary range and the operating radius to obtain the collision range, and calculate the difference between the collision range and the nearest distance to obtain the minimum obstacle avoidance distance; Based on the obstacle avoidance direction and minimum obstacle avoidance distance, obtain the nearest safe operating point; The real-time posture of the dexterous hand is obtained, and based on the relative speed, the motion path between the nearest safe operation point and the real-time posture is constructed to obtain the obstacle avoidance path.

[0009] Optionally, after generating the cue coordination signal, the method further includes: The movement speed of the dexterous hand is obtained, and the ratio of the relative speed to the movement speed is calculated to obtain the speed weight; Based on the speed weight, the preset cooperation range is weighted and adjusted to obtain the signal coverage range; The internal communication module is used to propagate the prompt coordination signal according to the signal coverage area.

[0010] Optionally, after obtaining the velocity weights, the method further includes: Determine whether the speed weight exceeds a preset high-risk threshold; When the speed weight exceeds the high-risk threshold, the behavior pattern of the dynamic obstacle is obtained, wherein the behavior pattern includes an inertial sliding mode and a sudden turning mode, the interference area of ​​the inertial sliding mode is a linearly expanding region, and the interference area of ​​the sudden turning mode is a fan-shaped spreading region. Based on the interference area, the boundary range of the dynamic obstacle is updated to obtain an updated boundary range, and an updated obstacle avoidance path is obtained using the updated boundary range.

[0011] Optionally, the step of comprehensively deciding on the environmental perception information and action intention information to generate an optimized action path includes: Acquire the force distribution of neighboring dexterous hands, the movement paths of neighboring dexterous hands, and historical environmental perception information from the action intent information; Obtain the correlation between the historical environmental perception information and the environmental perception information; Based on the aforementioned correlation, time correction is performed on the force distribution and movement path of the neighboring dexterous hand to obtain the corrected force distribution and corrected movement path. By summarizing the environmental perception information, the distribution of corrective forces, and the corrective action path, comprehensive decision information is obtained, and an optimized action path is generated based on the comprehensive decision information.

[0012] Optionally, generating the optimized action path based on the comprehensive decision information includes: Based on the target action route, environmental perception information in the comprehensive decision information, and the corrected action path, the direction of the dexterous hand action and the direction of obstacle avoidance are obtained. The intersection direction of the dexterous hand movement direction and the obstacle avoidance direction is obtained to obtain the second movement direction; Obtain the nearest safe operation point in the second action direction to obtain the second safe operation point; The motion path of the real-time attitude and the second safe operation point is obtained to obtain the optimized motion path; Obtain the force vector sequence of the optimized action path in the corrected force distribution; The dexterous hand is controlled by utilizing the force vector sequence and optimized motion path.

[0013] Optionally, after performing motion control on the dexterous hand, the method further includes: Obtain the execution parameters of the action control, and generate an action log based on the execution parameters; The action log is sent to a pre-built task management system to obtain feedback information from the task management system, and the actions of the dexterous hand are corrected based on the feedback information.

[0014] To achieve the above objectives, the present invention also provides a dexterous hand motion planning system based on deep learning, comprising: The state acquisition module is used to acquire the initial state information of the dexterous hand and extract the joint angles, finger postures and spatial position of the target object from the initial state information; An environmental perception module is used to collect real-time environmental data based on the sensor module of the dexterous hand, and to perform multimodal feature fusion processing on the real-time environmental data to obtain environmental perception information. The collaborative detection module is used to determine whether the environmental perception information contains dynamic obstacle information, and to use the internal communication module of the dexterous hand to determine whether there is a collaborative signal from a neighboring dexterous hand; The motion planning module is used to return to the steps of collecting real-time environmental data based on the sensor module of the dexterous hand when the environmental perception information does not contain dynamic obstacle information and no coordination signal is received; and when the environmental perception information contains dynamic obstacle information and no coordination signal is received, to generate an obstacle avoidance motion path and a prompt coordination signal using the power control module of the dexterous hand based on the environmental perception information; and when the environmental perception information contains dynamic obstacle information and a coordination signal is received, to parse the coordination signal to obtain the motion intention information of the adjacent dexterous hand, to make a comprehensive decision by combining the environmental perception information and the motion intention information, to generate an optimized motion path, and to generate a second prompt coordination signal.

[0015] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: Memory, storing at least one instruction; The processor executes the instructions stored in the memory to implement the deep learning-based dexterity hand motion planning method described above.

[0016] To address the aforementioned problems, the present invention also provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the deep learning-based dexterity hand motion planning method described above.

[0017] To address the problems described in the background art, this invention first collects real-time environmental data through a sensor module and performs multimodal feature fusion to generate environmental perception information. Then, based on this information, preliminary movement planning for the dexterous hand is performed. In this invention, when a dexterous hand generates an obstacle avoidance path, a cueing coordination signal is generated. This signal can be transmitted to neighboring dexterous hands, allowing them to adjust their movements in advance to avoid conflicts. Furthermore, due to the transmission of the cueing coordination signal, the dynamic obstacle information acquired by the current dexterous hand can be shared with neighboring dexterous hands that lack complete information, achieving collaborative decision-making among multiple dexterous hands. This not only improves the dexterous hand's adaptability to dynamic environments but also enhances execution efficiency in complex task scenarios. Moreover, communication between dexterous hands in this invention is directly accomplished through an internal communication module, significantly improving response speed. Therefore, this invention can significantly improve the movement planning ability and task execution efficiency of dexterous hands in complex dynamic environments. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a deep learning-based dexterity hand motion planning method provided in an embodiment of the present invention; Figure 2 A functional block diagram of a dexterous hand motion planning system based on deep learning provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of an electronic device that implements the deep learning-based dexterous hand motion planning method according to an embodiment of the present invention. Detailed Implementation

[0019] This invention provides a deep learning-based method and system for dexterous hand motion planning. Its core lies in significantly improving the adaptability and task execution efficiency of the dexterous hand in complex dynamic environments through multimodal data fusion and collaborative decision-making mechanisms. Combined with... Figures 1 to 3 As shown, the specific implementation of the present invention will be described in detail below, including the functional implementation of each module, algorithm design and actual operation process.

[0020] First, see Figure 1 The core process of this invention begins with the state acquisition module, which is responsible for collecting the initial state information of the dexterous hand and extracting joint angles, finger postures, and the spatial position of the target object. This information forms the basis for subsequent motion planning, and its accuracy directly affects the performance of the entire system. In practical applications, dexterous hands are typically equipped with various sensors, such as inertial measurement units (IMUs), torque sensors, and vision sensors, to monitor the state of the dexterous hand in real time. By processing the data from these sensors, the initial state information of the dexterous hand can be accurately obtained. For example, in a specific embodiment, the initial state information of a dexterous hand shows that its joint angles are... The fingers are in a bent position, and the spatial position of the target object is... These data are then fed into subsequent modules for further processing.

[0021] Next, the environmental perception module performs multimodal feature fusion processing on the real-time environmental data collected by the dexterous hand's sensor module to generate environmental perception information. This process includes several steps: First, outlier removal and missing value imputation are performed on the real-time environmental data to obtain standardized data; then, spatial alignment is performed on the standardized data to ensure consistency between data from different sensors; subsequently, multi-dimensional feature extraction is performed on the consistent data to obtain a feature set, and semantic annotation based on scene segmentation is performed on the feature set to obtain scene objects; next, scene objects are classified and identified to obtain object categories, and key areas of scene objects are marked to obtain marker points; finally, within a preset time window, the dynamic change trajectory of the marker points is acquired to obtain a dynamic feature sequence, and the scene objects, object categories, and dynamic feature sequences are summarized to form complete environmental perception information. For example, in one embodiment, the environmental perception module detects a dynamic obstacle moving to the right with a relative speed of v=2m / s and an obstacle boundary range of r=0.3m, while marking key areas of the obstacle such as edges and center points. This information provides important basis for subsequent motion path planning.

[0022] The collaborative detection module is responsible for determining whether the environmental perception information contains dynamic obstacle information and using the dexter's internal communication module to determine whether there is a collaborative signal from a neighboring dexter. When the environmental perception information does not contain dynamic obstacle information and no collaborative signal is received, the system returns to the real-time environmental data acquisition step to continue monitoring environmental changes. However, if the environmental perception information contains dynamic obstacle information but no collaborative signal is received, the motion planning module uses the power control module to generate an obstacle avoidance path based on the environmental perception information and generates a prompting collaborative signal. The purpose of the prompting collaborative signal is to notify neighboring dexterities of the current dynamic obstacle information so that they can adjust their actions in advance to avoid conflicts. For example, in one embodiment, when dexter A detects a dynamic obstacle and generates an obstacle avoidance path, it sends a prompting collaborative signal to neighboring dexter B through its internal communication module. The signal includes information such as the obstacle's direction of movement, relative speed, boundary range, and obstacle avoidance path. This collaborative mechanism not only improves the obstacle avoidance capability of a single dexter but also enhances the cooperation efficiency among multiple dexterities.

[0023] When the environmental perception information includes dynamic obstacle information and a coordination signal is received, the motion planning module parses the coordination signal to obtain the motion intention information of the neighboring dexterous hand. It then combines the environmental perception information and the motion intention information to make a comprehensive decision, generating an optimized motion path and a second cue coordination signal. The comprehensive decision-making process includes several steps: First, it acquires the force distribution of the neighboring dexterous hand, the motion path of the neighboring dexterous hand, and historical environmental perception information from the motion intention information. Next, it acquires the correlation between historical and current environmental perception information. Then, it performs time correction on the force distribution and motion path of the neighboring dexterous hand based on the correlation, obtaining a corrected force distribution and a corrected motion path. Finally, it summarizes the environmental perception information, the corrected force distribution, and the corrected motion path to obtain comprehensive decision information, and generates an optimized motion path based on this comprehensive decision information. For example, in one embodiment, after dexterous hand A receives a coordination signal from dexterous hand B, it parses that B's motion path is a straight line forward, with the force distribution concentrated in the forward area. Simultaneously, it combines the current environmental perception information to detect a dynamic obstacle ahead. In this scenario, dexterous hand A generates an optimized motion path to bypass the obstacle and sends a second cueing coordination signal to other dexterous hands via the internal communication module to update the global collaborative decision.

[0024] In the process of generating obstacle avoidance motion paths, the motion planning module includes the following steps: First, based on the dynamic feature sequence in the environmental perception information, the motion direction and relative velocity of the dynamic obstacle are obtained; then, based on the scene objects in the environmental perception information, the obstacle boundary range is obtained; next, the target motion path of the dexterous hand is obtained, and the nearest intersection point between the motion direction of the dynamic obstacle and the target motion path is identified to obtain the nearest point of the obstacle and the nearest point of the dexterous hand, and the vector information from the nearest point of the obstacle to the nearest point of the dexterous hand is obtained to obtain the obstacle avoidance direction and the nearest distance; subsequently, the operating radius of the dexterous hand is obtained, the sum of the obstacle boundary range and the operating radius is calculated to obtain the collision range, and the difference between the collision range and the nearest distance is calculated to obtain the minimum obstacle avoidance distance; finally, based on the obstacle avoidance direction and the minimum obstacle avoidance distance, the nearest safe operating point is obtained, and based on the real-time posture and relative velocity of the dexterous hand, the motion path between the nearest safe operating point and the real-time posture is constructed to obtain the obstacle avoidance path. For example, in one embodiment, the target movement path of the dexterous hand is a straight line forward, the dynamic obstacle moves to the right with a relative speed of v = 2 m / s, the obstacle boundary range is r = 0.3 m, and the dexterous hand's operating radius is R = 0.2 m. Calculations show that the collision range is r + R = 0.5 m, the closest distance is d = 0.7 m, and the minimum obstacle avoidance distance is d - (r + R) = 0.2 m. Based on these parameters, the system ultimately generates a safe movement path to bypass the obstacle.

[0025] Furthermore, to improve the propagation efficiency of the coordination signal, this invention introduces the concept of speed weighting. Specifically, after generating the cue coordination signal, the system acquires the movement speed of the dexterous hand, calculates the ratio of relative speed to movement speed to obtain a speed weight, and then adjusts a preset coordination range according to the speed weight to obtain the signal coverage range. Subsequently, the cue coordination signal is propagated using an internal communication module based on the signal coverage range. For example, in one embodiment, the movement speed of the dexterous hand is... The relative speed is Then the speed weight is Assuming the preset scope of collaboration is... The weighted signal coverage area is then... In this way, the system can dynamically adjust the propagation range of the collaborative signal according to the actual situation, thereby improving the overall response speed and collaborative efficiency.

[0026] In certain high-risk scenarios, the system also needs to analyze the behavior patterns of dynamic obstacles. For example, when the speed weight exceeds a preset high-risk threshold, the system acquires the behavior patterns of the dynamic obstacles, including inertial sliding patterns and sudden turning patterns. The interference area of ​​the inertial sliding pattern is a linearly expanding region, while the interference area of ​​the sudden turning pattern is a fan-shaped spreading region. Based on the interference area, the system updates the boundary range of the dynamic obstacle to obtain an updated boundary range, and uses the updated boundary range to obtain an updated obstacle avoidance path. For example, in one embodiment, when the speed weight exceeds the high-risk threshold... At that time, the system detected that the dynamic obstacle's behavior pattern was a sudden turning pattern, and its interference area was a fan-shaped diffusion area with an angle of [missing information]. The radius is r = 0.5m. Based on these parameters, the system updates the boundary range of the dynamic obstacle and generates a safer obstacle avoidance path.

[0027] Finally, after completing motion control, the system generates a motion log and sends it to a pre-built task management system to obtain feedback and correct the dexterous hand's movements. The motion log includes execution parameters for motion control, such as the motion path, force distribution, and execution time. The task management system generates feedback information based on these parameters, such as motion deviation and execution efficiency, and sends this feedback back to the dexterous hand system for further motion correction. For example, in one embodiment, if the dexterous hand's motion path deviates from the target path by 0.1m and the execution time is t=5s, the system adjusts the motion path based on the feedback and re-executes the task, ultimately achieving higher task execution accuracy.

[0028] In summary, this invention significantly improves the adaptability and task execution efficiency of dexterous hands in complex dynamic environments through multimodal data fusion, collaborative decision-making mechanisms, and dynamic path planning algorithms. Combined with... Figures 1 to 3 The embodiments shown demonstrate that the specific implementation of the present invention covers the complete process from data acquisition to action execution, ensuring the feasibility and practicality of the technical solution.

Claims

1. A method for dexterous hand motion planning based on deep learning, characterized in that, The method includes: The initial state information of the dexterous hand is obtained, and the joint angles, finger postures, and spatial position of the target object are extracted from the initial state information. Real-time environmental data is collected by the sensor module of the dexterous hand, and multimodal feature fusion processing is performed on the real-time environmental data to obtain environmental perception information; Determine whether the environmental perception information contains dynamic obstacle information, and use the internal communication module of the dexterous hand to determine whether there is a cooperative signal from a neighboring dexterous hand; When the environmental perception information does not contain dynamic obstacle information and no cooperative signal is received, return to the steps described above for collecting real-time environmental data based on the sensor module. When the environmental perception information contains dynamic obstacle information and no coordination signal is received, the power control module of the dexterous hand generates an obstacle avoidance action path based on the environmental perception information and generates a prompt coordination signal. When the environmental perception information contains dynamic obstacle information and a coordination signal is received, the coordination signal is parsed to obtain the action intention information of the nearby dexterous hand. The environmental perception information and the action intention information are combined to make a comprehensive decision, generate an optimized action path, and generate a second prompt coordination signal. The environmental perception information and action intention information are integrated to make a decision and generate an optimized action path, including: Acquire the force distribution of neighboring dexterous hands, the movement paths of neighboring dexterous hands, and historical environmental perception information from the action intent information; Obtain the correlation between the historical environmental perception information and the environmental perception information; Based on the aforementioned correlation, the force distribution and movement path of the neighboring dexterous hand are time-corrected to obtain the corrected force distribution and corrected movement path. By summarizing the environmental perception information, the distribution of corrective forces, and the corrective action path, comprehensive decision information is obtained, and an optimized action path is generated based on the comprehensive decision information.

2. The deep learning-based dexterous hand motion planning method as described in claim 1, characterized in that, The process of performing multimodal feature fusion processing on the real-time environmental data to obtain environmental perception information includes: The real-time environmental data is subjected to outlier removal and missing value imputation to obtain standardized data, and the standardized data is then spatially aligned to obtain consistent data. Multi-dimensional feature extraction is performed on the consistent data to obtain a feature set, and semantic annotation based on scene segmentation is performed on the feature set to obtain a scene object; The scene objects are classified and identified to obtain object categories, and key areas of the scene objects are marked to obtain marker points; Within a preset time window, the dynamic change trajectory of the marker points is acquired to obtain a dynamic feature sequence; By summarizing the scene objects, object categories, and dynamic feature sequences, environmental perception information is obtained.

3. The deep learning-based dexterous hand motion planning method as described in claim 1, characterized in that, The power control module utilizing the dexterous hand generates an obstacle avoidance path and a prompting coordination signal based on the environmental perception information, including: Based on the environmental perception information, the dexterous hand is used to plan the movement path to obtain an obstacle avoidance path; The power control module is used to control the movement of the dexterous hand according to the obstacle avoidance path; During the execution of the dexterous hand's motion control, the influence range of the force of the power control module is obtained, and the environmental perception information, obstacle avoidance path, and influence range of the force are used to generate a prompt coordination signal.

4. The deep learning-based dexterity hand motion planning method as described in claim 3, characterized in that, The step of planning the movement path of the dexterous hand based on the environmental perception information to obtain an obstacle avoidance path includes: Based on the dynamic feature sequence in the environmental perception information, the motion direction and relative speed of the dynamic obstacle are obtained; Based on the scene objects in the environmental perception information, obtain the boundary range of the obstacles; The target motion path of the dexterous hand is obtained, and the nearest intersection point between the motion direction of the dynamic obstacle and the target motion path is identified to obtain the nearest point of the obstacle and the nearest point of the dexterous hand. The vector information from the nearest point of the obstacle to the nearest point of the dexterous hand is obtained to obtain the obstacle avoidance direction and the nearest distance. Obtain the operating radius of the dexterous hand, calculate the sum of the obstacle boundary range and the operating radius to obtain the collision range, and calculate the difference between the collision range and the nearest distance to obtain the minimum obstacle avoidance distance; Based on the obstacle avoidance direction and minimum obstacle avoidance distance, obtain the nearest safe operating point; The real-time posture of the dexterous hand is obtained, and the motion path between the nearest safe operation point and the real-time posture is constructed based on the relative speed to obtain the obstacle avoidance path.

5. The deep learning-based dexterous hand motion planning method as described in claim 4, characterized in that, After generating the cue coordination signal, the method further includes: The movement speed of the dexterous hand is obtained, and the ratio of the relative speed to the movement speed is calculated to obtain the speed weight; The signal coverage range is obtained by weighting the preset cooperation range according to the speed weight. The internal communication module is used to propagate the prompt coordination signal according to the signal coverage area.

6. The deep learning-based dexterous hand motion planning method as described in claim 4, characterized in that, The step of generating an optimized action path based on the comprehensive decision information includes: Based on the target action route, environmental perception information in the comprehensive decision information, and the corrected action path, the direction of the dexterous hand action and the direction of obstacle avoidance are obtained. The intersection direction of the dexterous hand movement direction and the obstacle avoidance direction is obtained to obtain the second movement direction; Obtain the nearest safe operation point in the second action direction to obtain the second safe operation point; The motion path of the real-time attitude and the second safe operation point is obtained to obtain the optimized motion path; Obtain the force vector sequence of the optimized action path in the corrected force distribution; The dexterous hand is controlled by the force vector sequence and the optimized motion path.

7. The deep learning-based dexterous hand motion planning method as described in claim 5, characterized in that, After obtaining the velocity weights, the method further includes: Determine whether the speed weight exceeds a preset high-risk threshold; When the speed weight exceeds the high-risk threshold, the behavior pattern of the dynamic obstacle is obtained, wherein the behavior pattern includes an inertial sliding mode and a sudden turning mode, the interference area of ​​the inertial sliding mode is a linearly expanding region, and the interference area of ​​the sudden turning mode is a fan-shaped spreading region. The boundary range of the dynamic obstacle is updated based on the interference area to obtain the updated boundary range, and the updated obstacle avoidance path is obtained using the updated boundary range.

8. The deep learning-based dexterous hand motion planning method as described in claim 6, characterized in that, After controlling the movement of the dexterous hand, the method further includes: Obtain the execution parameters of the action control, and generate an action log based on the execution parameters; The action log is sent to a pre-built task management system to obtain feedback information from the task management system, and the actions of the dexterous hand are corrected based on the feedback information.

9. A dexterous hand motion planning system based on deep learning, characterized in that, The system includes: The state acquisition module is used to acquire the initial state information of the dexterous hand and extract the joint angles, finger postures and spatial position of the target object from the initial state information; An environmental perception module is used to collect real-time environmental data based on the sensor module of the dexterous hand, and to perform multimodal feature fusion processing on the real-time environmental data to obtain environmental perception information. The collaborative detection module is used to determine whether the environmental perception information contains dynamic obstacle information, and to use the internal communication module of the dexterous hand to determine whether there is a collaborative signal from a neighboring dexterous hand; The motion planning module is used to return to the steps of collecting real-time environmental data based on the sensor module when the environmental perception information does not contain dynamic obstacle information and no coordination signal is received; and to generate an obstacle avoidance motion path and a prompt coordination signal using the power control module of the dexterous hand based on the environmental perception information when the environmental perception information contains dynamic obstacle information and no coordination signal is received; and to parse the coordination signal to obtain the motion intention information of the adjacent dexterous hand when the environmental perception information contains dynamic obstacle information and a coordination signal is received, and to make a comprehensive decision by combining the environmental perception information and the motion intention information to generate an optimized motion path and a second prompt coordination signal. The environmental perception information and action intention information are integrated to make a decision and generate an optimized action path, including: Acquire the force distribution of neighboring dexterous hands, the movement paths of neighboring dexterous hands, and historical environmental perception information from the action intent information; Obtain the correlation between the historical environmental perception information and the environmental perception information; Based on the aforementioned correlation, the force distribution and movement path of the neighboring dexterous hand are time-corrected to obtain the corrected force distribution and corrected movement path. By summarizing the environmental perception information, the distribution of corrective forces, and the corrective action path, comprehensive decision information is obtained, and an optimized action path is generated based on the comprehensive decision information.

Images