Robot movement and manipulation control method based on deep learning

Abstract

The invention discloses a robot movement and manipulation control method based on deep learning, and belongs to the technical field of artificial intelligence and intelligent robot systems. The problems that an existing method is difficult to process complex natural language instructions, and in a dynamic environment, the cooperation efficiency of a robot moving task and a mechanical arm control task is low are solved. According to the method, the input complex natural language instruction is processed by adopting the large language model to obtain the subtask sequence, so that the understanding of the robot on the natural language instruction can be realized; through a hierarchical reinforcement learning framework, environment semantic information is deeply fused, an interpretable risk assessment mechanism is introduced, and the system is endowed with continuous learning and generalization ability; the cooperative efficiency of autonomous navigation and manipulation tasks of the mobile robot in complex, dynamic and unknown environments can be remarkably improved, and the safety and the intelligent level of the robot are guaranteed. The method can be applied to the field of robot movement and manipulation control.

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and intelligent robot systems, and specifically relates to a robot movement and manipulation control method based on deep learning. Background Art

[0002] With the rapid development of artificial intelligence technology and the widespread application of robotics, intelligent robot systems are playing an increasingly important role in industrial automation, intelligent warehousing, service robots, and exploration tasks. Especially in scenarios that require complex interactions with the environment and the execution of multi-step tasks, higher requirements are placed on the autonomy, intelligence, and flexibility of robots.

[0003] Existing mobile robot systems usually have certain navigation and obstacle avoidance capabilities, such as using LiDAR or visual SLAM technology to build a geometric map of the environment, and combining path planning algorithms such as A* and D* to move in the map. At the same time, the robotic arms on fixed or mobile robots have also been used to perform various manipulation tasks, such as grasping, placing, and assembling, which usually rely on fixed sequential decisions.

[0004] However, existing technologies still face many challenges when dealing with multi-step tasks in complex and unstructured environments: 1. Currently, robot tasks are usually assigned through programming, graphical interfaces or simple instruction sets, which are difficult to handle complex task requirements expressed in natural language. Automatically decomposing high-level, ambiguous human instructions into robot-executable, time-ordered logic subtask sequences is the key bottleneck for achieving truly intelligent interaction. Traditional rule-based or simple script-based methods are difficult to cover a variety of task instructions and lack robustness and generalization capabilities.

[0005] 2. A mobile robot system capable of performing multi-task operations requires a high degree of coordination between the mobile platform (such as an omnidirectional mobile car), the perception system (such as a camera, a lidar), the decision system (task planning), and the execution system (such as a robotic arm). How to integrate information from different modules (such as target detection results, map information, and own status) in real time and efficiently, and coordinate movement, perception, planning, and operation to ensure the smooth completion of the task is a complex system engineering problem. Especially in a dynamic environment, path planning, target recognition and tracking, and precise operation of the robotic arm require close coordination.

[0006] In summary, existing methods are still difficult to handle complex natural language instructions, and in a dynamic environment, the collaborative efficiency of robot movement and manipulator task execution is still low. Therefore, it is an urgent need to meet the needs of people's daily lives to propose a control method that can understand natural language instructions, intelligently plan and decompose complex tasks, and at the same time can efficiently coordinate robot movement and manipulator manipulation tasks. Summary of the Invention

[0007] The purpose of the present invention is to solve the problems that existing methods are difficult to handle complex natural language instructions and in a dynamic environment, the collaborative efficiency of robot movement tasks and manipulator manipulation tasks is low. A robot movement and manipulation control method based on deep learning is proposed. Users only need to input natural language instructions to generate a task sequence of the robot and complete a series of tasks through automatic planning.

[0008] The technical solution adopted by the present invention to solve the above technical problems is: A robot movement and manipulation control method based on deep learning, the method specifically includes the following steps: Step 1: The user inputs a natural language instruction into a large language model, and the large language model outputs a subtask sequence S according to the input natural language instruction; Step 2: Initialize the subtask s = 1; Step 3: Select the next operation according to the type of the s-th subtask in the subtask sequence S, specifically: If the s-th subtask in the subtask sequence S is a path planning task, then execute Step 4; If the s-th subtask in the subtask sequence S is a manipulator manipulation task, then execute Step 5; Step 4: Use the first-layer reinforcement learning network and the second-layer reinforcement learning network for path planning, and the robot moves to the target position according to the path planning result; then execute Step 6; Step 5: Use the target detection model deployed on the robot to identify the position of the target object in the environment, and calculate the target angles of each joint of the robot's manipulator according to the position of the target object; Execute a grasping action on the target object according to the target angles of each joint of the manipulator; then execute Step 6; Step 6: Whether all the subtasks in the subtask sequence S have been executed; If there are still subtasks not executed, then let s = s + 1, and return to execute Step 3; If all subtasks have been executed, then end.

[0009] The beneficial effects of the present invention are:

[0010] The present invention proposes an intelligent path planning method based on hierarchical reinforcement learning and interpretable semantic risk perception. By using a large language model to process the input complex natural language instructions, a subtask sequence is obtained, which enables the robot to understand natural language instructions. Then, through a hierarchical reinforcement learning framework, deeply integrating environmental semantic information, introducing an interpretable risk assessment mechanism, and endowing the system with the ability of continuous learning and generalization, the collaborative efficiency of the mobile robot in complex, dynamic and unknown environments for autonomous navigation and manipulation tasks can be significantly improved, and the safety and intelligence level of the robot can be ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a flowchart of a robot movement and manipulation control method based on deep learning according to the present invention; Figure 2 is an architecture diagram of a multi-task manipulation robot system. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] DETAILED DESCRIPTION OF THE EMBODIMENT 1: With reference to Figure 1 this embodiment will be described. A robot movement and manipulation control method based on deep learning according to this embodiment specifically includes the following steps: Step 1: The user inputs natural language instructions into the large language model, and the large language model outputs a subtask sequence S according to the input natural language instructions; Step 2: Initialize the subtask s = 1; Step 3: Select the next operation according to the type of the s-th subtask in the subtask sequence S (the subtask type only includes path planning tasks and manipulation tasks), specifically: If the s-th subtask in the subtask sequence S is a path planning task, then execute Step 4; If the s-th subtask in the subtask sequence S is a robotic arm manipulation task, then execute Step 5; Step 4: Use the first-layer reinforcement learning network and the second-layer reinforcement learning network for path planning, and the robot moves to the target position according to the path planning result; then execute Step 6; Step 5: Use the target detection model deployed on the robot to identify the position of the target object in the environment, and calculate the target angles of each joint of the robotic arm of the robot according to the position of the target object; Execute a grasping action on the target object according to the target angles of each joint of the robotic arm; then execute Step 6; Step 6: Whether all the subtasks in the subtask sequence S have been executed; If there are still subtasks not executed, then set s = s + 1, and return to execute Step 3; If all subtasks have been executed, then end.

[0013] According to the characteristics of the omnidirectional mobile platform and multi-degree-of-freedom robotic arm, a mobile manipulation robot system equipped with a deep learning model is constructed. The multi-task manipulation robot system includes a car platform, a robotic arm platform and two computing devices, which communicate through the ROS framework and collaborate to complete tasks. The overall system hardware structure and communication process are as follows Figure 2 shown.

[0014] The core of the robot system lies in the high coordination of the mobile platform, perception system, task planning system and robotic arm grasping. The robot can perform environmental perception and task decision-making, and realize complex multi-task manipulation functions based on natural language instructions. When the task sequence is generated, the ROS framework is relied on to realize message transmission and service calls between modules: (1) Movement and navigation execution: The system implements precise positioning and navigation based on the navigation subtasks in the task sequence, relying on the pre-built environment semantic map and combining multi-sensor fusion technology (such as lidar, IMU, wheel encoder and other data). It can quickly generate the initial path from the current position to the target area. The first-layer reinforcement learning network will perform global planning based on the initial path, and the second-layer reinforcement learning network will fine-tune the path based on the global path planning results.

[0015] (2) Visual guidance and target positioning: When the robot reaches the predetermined target area, it uses the camera and depth sensor to collect environmental image data, and applies the lightweight YOLOv5 target detection model based on deep learning to quickly identify the category and position of the target object in the image. Then, combined with the camera's intrinsic and extrinsic parameters, the three-dimensional position and posture of the target object in the robot arm base coordinate system are accurately calculated, providing accurate spatial information support for the robot arm's grasping.

[0016] (3) Robotic arm grasping / placing: Based on the target posture information provided by the vision system, the robot arm control system uses the inverse solution of its kinematic model to calculate the target angle of each joint and drive the robot arm to accurately perform grasping and other operations.

[0017] (4) System status feedback and integration: During the entire task execution process, each functional module can display status information (such as navigation completion status, target detection results, grasping execution start and end information, etc.) through the ROS message mechanism. The system can comprehensively judge the execution effect of each subtask (success, failure, or need to retry, etc.) based on the status information, dynamically adjust the task status, and trigger subsequent subtasks in sequence according to the preset task sequence logic, ensuring the coherence and efficiency of the entire multi-task operation process, ensuring the stable operation of the overall system and the safety of personnel and equipment.

[0018] Specific implementation method 2: The difference between this implementation method and specific implementation method 1 is that the large language model is a GLM, and the input of the large language model also includes an environmental semantic map of the target area.

[0019] The other steps and parameters are the same as those in the first specific implementation manner.

[0020] To enable the robot to understand natural language instructions from humans and execute complex multi-step tasks, the present invention introduces the domestic open-source large model GLM for task understanding and planning. The user issues a high-level instruction u to the system using natural language. Based on the understanding of the instruction and the cognitive semantic map of the current environment, the model GLM automatically decomposes the complex high-level instruction u into a series of ordered basic subtask sequences S that the robot can execute. Taking the input "Please pick up the red book on the table and then put it on the bookshelf" as an example, the large language model is responsible for parsing the instruction u to identify key information such as the intention, actions (pick up, place), target objects (book, bookshelf), and their attributes (red, on the table, on the bookshelf) in the instruction, and outputs the task sequence S as [move, classroom][operate, red book].

[0021] Specific implementation manner three: The difference between this implementation manner and the first or second specific implementation manner is that the specific process of step four is as follows: Step four one: Adopt The algorithm plans a path from the current position of the robot to the target location of the s-th subtask. The first-layer reinforcement learning network performs global path planning according to the algorithm planning result and the current environmental information; Step four two: The second-layer reinforcement learning network fine-tunes the global path planning result according to the global path planning result and the perceived local environmental information, and the robot moves towards the target position along the fine-tuned path.

[0022] The other steps and parameters are the same as those in the first or second specific implementation manner.

[0023] The present invention proposes an intelligent path planning method based on hierarchical reinforcement learning and interpretable semantic risk perception. It specifically includes a high-level policy network (High-Level Policy, HLP, that is, the first-layer reinforcement learning network) and a low-level policy network (Low-Level Policy, LLP, that is, the second-layer reinforcement learning network). The HLP is responsible for making long-term abstract decisions based on global tasks and macro environmental information, and selecting a navigation strategy mode or setting phased sub-goals. The LLP is responsible for performing short-term real-time motion control according to the instructions issued by the HLP and local fine environmental perception.

[0024] After the robot is actually deployed, it continuously collects experience tuples and stores them in the experience replay pool. Using these real data, the parameters of the pre-trained first-layer reinforcement learning model and the second-layer reinforcement learning model can be updated regularly or under specific triggering conditions (such as task failure) Perform fine-tuning. The update process of the policy network (Actor) can be expressed as: The update process of the value network (Critic) can be expressed as: Among them, is the experience replay pool.

[0025] Exploration strategy: Combine -greedy and UCB (Upper Confidence Bound) to encourage the robot to explore unknown states and semantic configurations in the environment on the premise of safety, so as to collect more valuable training data and accelerate the adaptation to a specific working environment.

[0026] Input of the policy network: Use these semantic embeddings as part of the state input of the policy network (especially the first-layer reinforcement learning network). The first-layer reinforcement learning network learns not isolated semantic labels, but the semantic concepts represented by these embedding vectors.

[0027] Generalization mechanism: When encountering new objects or regions that have not explicitly appeared in the training, if their semantic embeddings are close to those of known high-risk / low-risk objects or regions in the vector space, the model can generalize the learned navigation strategies based on this similarity. For example, if the robot learns to be cautious about "fragile item A", when encountering a new "fragile item B" (whose embedding is close to A), it can also adopt a similar strategy.

[0028] The method of the present invention can overcome the limitations of traditional path planning methods in complex dynamic environments, and improve the autonomous navigation ability, safety and environmental adaptability of the robot in unknown and uncertain situations. It has the following advantages: (1) Hierarchical decision-making framework: Simulate the human "policy first, execution later" navigation mode, decompose complex navigation tasks into high-level policy selection and low-level action execution, and improve the planning efficiency and the ability to handle complex scenarios.

[0029] (2) Deep semantic fusion: Not only use semantic information to identify objects, but also deeply bind it with navigation risks and behavior constraints, so that the robot can "understand" the potential dangers and specific requirements of the environment.

[0030] (3) Explainable risk perception: By explicitly modeling semantic risks in the reinforcement learning reward function, the path selection behavior of the robot becomes more explainable, and it can actively adopt avoidance or cautious strategies according to risk assessment.

[0031] (4)Active adaptation and continuous learning: Enable the robot to continuously optimize the navigation strategy through interaction with the environment and generalize it to new environments and new semantics.

[0032] Embodiment 4: The difference between this embodiment and one of Embodiments 1 to 3 is that the agent state space of the first-layer reinforcement learning network is defined as ; Among them, is a vector composed of the semantic category of the target location of the s-th sub-task (the type of the target location needs to be mapped to a digital number. For example, when the target location is an office area, the corresponding digital number of the target location is 1, and when the target location is a step, the corresponding digital number of the target location is 2. Use the digital number to represent the semantic category of the location) and the coordinates of the target location (that is, the entrance coordinates of the target location); represents the sequence of semantic categories of all areas that need to be passed from the current position of the robot to the target location in the planning result of the algorithm; It should be noted that in the present invention, the centroid coordinates of the robot are used as the current position of the robot; is a vector composed of the current position coordinates of the robot and the current attitude angle of the robot; In the present invention, the robot attitude angle is defined as the angle between the forward direction of the robot and the X-axis of the global coordinate system; represents the historical average passing time required for the robot to pass through the current area; The action space of the agent of the first-layer reinforcement learning network is defined as ; Among them, represents the first, second,..., the th areas that the robot needs to pass through in sequence in the environmental semantic map in the current step action (that is, all areas that the robot needs to pass through from the current position to the target location).

[0033] Other steps and parameters are the same as those in one of Embodiments 1 to 3.

[0034] Embodiment 5: The difference between this embodiment and one of Embodiments 1 to 4 is that the reward function adopted in the training process of the first-layer reinforcement learning network is: Among them: represents the indicator function. If the s-th sub-task is successfully completed, the value of the indicator function is 1, otherwise the value of the indicator function is 0; It represents the navigation efficiency of the planned path from the starting point to the target position obtained according to the actions output by the agent; Obtain the time required for the robot to move from the starting point to the target position along the planned path, and the navigation efficiency can be obtained by dividing the straight-line distance from the starting point to the target position by the time; It represents the semantic risk assessment value on the path from the starting point to the target position output by the agent; It represents the time required for the robot to move from the starting point to the target position along the planned path of the agent; and and and are the weight coefficients for each item respectively.

[0035] Other steps and parameters are the same as those in any one of the specific implementation manners one to four.

[0036] The reward function designed in this implementation manner is used to guide the first-layer reinforcement learning network to make optimal macro global decisions.

[0037] Specific implementation manner six: The difference between this implementation manner and any one of the specific implementation manners one to five is that the state space of the second-layer reinforcement learning network is defined as ; Among them, is a vector composed of the channel entrance position coordinates of a sub-goal and the semantic category of the sub-goal in the global path planning result; In the present invention, each sub-goal in the actions output by the first-layer reinforcement learning network needs to be sent to the second-layer reinforcement learning network in sequence to be added to the state of each step of the second-layer reinforcement learning network agent; is a vector composed of the position coordinates of the dynamic obstacle and the speed of the dynamic obstacle obtained according to the local environment perception information (through lidar or depth camera); is a vector composed of the current position coordinates of the robot (taking the centroid coordinates of the robot as the position coordinates of the robot), linear velocity, angular velocity, and the wheel speeds of each Mecanum wheel of the robot; Among them, the linear velocity includes the forward / backward direction velocity and the lateral translation velocity . For indoor small and medium-sized robots, the maximum speed is usually between and . Therefore, a reasonable range can be . Due to the characteristics of the Mecanum wheels, the lateral movement ability may be slightly inferior to the forward movement, or its maximum value may be limited for stability considerations. For example . Robots usually have good in-place rotation ability, with an angular velocity range of rad / s; is a vector composed of the labels of the objects around the robot (the labels are obtained by mapping the objects to specific values according to their categories) and the confidence of the objects around the robot; Use the depth camera on the robot to obtain images around the robot, and then use a lightweight object detection network to detect the surrounding objects and output the confidence of the objects at the same time; The actions output by the agent of the second-layer reinforcement learning network are the desired linear velocity and desired angular velocity of the robot.

[0038] Other steps and parameters are the same as those in any one of the first to fifth specific embodiments.

[0039] The action space of the second-layer reinforcement learning network is , , which directly corresponds to the three main degrees of freedom of the robot in the body coordinate system. The unique structure of the Mecanum wheels enables the robot to independently control these three velocity components, realizing forward movement, backward movement, lateral movement, diagonal movement, and in-place rotation. The second-layer reinforcement learning network can learn how to combine these motion primitives to complete the navigation task most efficiently and safely under different local environments and the first-layer instructions. For example, in a narrow passage, the second-layer reinforcement learning network may learn to use more lateral movement ( ) rather than large-angle turning ( ) to adjust the position. When it is necessary to quickly avoid lateral dynamic obstacles, the second-layer reinforcement learning network can quickly output a large instruction.

[0040] Specific embodiment seven: The difference between this embodiment and any one of the first to sixth specific embodiments is that the reward function adopted in the training process of the second-layer reinforcement learning network is: Where: represents the navigation efficiency; Calculate the distance between the position of the robot before executing the current-step action output by the agent and the target position , and then calculate the distance between the position of the robot after executing the current-step action output by the agent and the target position , subtract the calculated distances , divide the subtraction result by the initial distance value between the starting point and the target point of the current subtask , and the calculation result is the navigation efficiency . That is: Normalize the distance between the robot's position after executing the current step action and the target position to a ratio relative to the initial distance, which is convenient for unifying the rewards of different scale tasks. When the robot moves towards the target, , is positive; when moving away from the target, is negative. It is calculated once at the beginning of each subtask and remains unchanged until the subtask is completed or switched. If the distance between the subtask target point and the starting point is very close, too small may cause large fluctuations in the navigation efficiency . When is less than the set threshold, it is necessary to adjust the value of to the threshold to avoid large fluctuations in navigation efficiency.

[0041] represents the indicator function. If a collision occurs after the agent outputs the action in the current step, the value of the indicator function is 1; otherwise, the value of the indicator function is 0; represents the local semantic risk assessment value of the robot executing the action output by the agent in the current step; represents the shortest vertical distance from the robot's current position to the path planned by the first-layer reinforcement learning network; are the weight coefficients of each item.

[0042] Other steps and parameters are the same as those in any one of the specific implementation manners one to six.

[0043] The reward function designed in this implementation manner is used to guide the second-layer reinforcement learning network to generate smooth, safe, and efficient local trajectories.

[0044] Specific implementation manner eight: The difference between this implementation manner and any one of the specific implementation manners one to seven is that the calculation methods of the semantic risk assessment values and are as follows: Where: represents the global path output by the agent of the first-layer reinforcement learning network; represents the path on the th sub-goal area; represents corresponding basic risk value; when there is a glass door in a certain sub-goal area to be passed, the basic risk values for the open and closed states of the glass door can be set respectively, and for other types of sub-goal areas, a fixed basic risk value can be set respectively; Indicates confidence (i.e., when using the lightweight object detection network to recognize the environmental semantic map, the corresponding confidence can be obtained); Where: Indicates the path that needs to be passed for the current step action output by the executing agent; Indicates the current position of the robot; Indicates centered at the position with as the radius of the circular area; The value of is related to the speed of the robot and the reaction time of the robot, Indicates the path and the sub-region within the circular area; Indicates the sub-region corresponding basic risk value; Indicates the sub-region confidence (using the lightweight object detection network to recognize the environmental semantic map, the corresponding confidence can be obtained).

[0045] Other steps and parameters are the same as those in any one of the specific embodiments one to seven.

[0046] By taking and as negative reward terms (penalty terms) and introducing them into the reward function, in the process of maximizing the cumulative reward by the reinforcement learning agent, it will naturally learn to avoid high semantic risk areas or take more cautious behaviors in high-risk areas (for example, the low-level policy outputs a lower speed).

[0047] Since the semantic risk is explicitly modeled, when the robot selects a non-optimal path in the traditional sense (such as not the shortest path), it is possible to judge whether the choice is made due to avoiding a certain high semantic risk area by analyzing the reward composition in the decision-making process of the first-layer reinforcement learning network, which enhances the interpretability of the path planning result.

[0048] Specific Embodiment Nine: The difference between this embodiment and any one of the specific embodiments one to eight is that the object detection model in step five is a lightweight YOLOv5 object detection model, and the lightweight YOLOv5 object detection model is the YOLOv5 network after replacement obtained by using the MobileNetV3 model to replace the backbone network of YOLOv5.

[0049] The other steps and parameters are the same as those in any one of the first to eighth specific embodiments.

[0050] In this embodiment, a target detection technology based on deep learning is adopted to provide accurate target pose information for the robotic arm. To further optimize the model performance, the original backbone network of the YOLOv5 target detection model is replaced with the MobileNetV3 network structure to obtain a lightweight YOLOv5 target detection model. The lightweight YOLOv5 target detection model can reduce the model volume and calculation amount while maintaining a high detection accuracy, enabling the model to have a fast inference speed and low computational resource requirements, and being suitable for deployment on a mobile robot embedded platform. The robot uses the depth camera mounted on it to collect environmental images in real time. The images are input into the lightweight YOLOv5 model for processing, and the object categories in the environment and their positions in the images are identified through the lightweight YOLOv5 model. Combining the real-time positioning information of the robot (which can be provided by technologies such as SLAM) and the internal and external parameters of the camera provides support for task planning and robotic arm grasping.

[0051] Specific Embodiment Ten: The difference between this embodiment and any one of the first to ninth specific embodiments is that the specific process of step five is as follows: Step Five One: Use the depth camera deployed on the robot to collect the color image img and depth information depth in real time. The color image img is detected by the lightweight YOLOv5 target detection model to obtain the two-dimensional bounding box where the target position is located ; According to the two-dimensional bounding box where the target position is located and the depth information depth, and combining with the internal parameters of the depth camera, the coordinates of the target position in the three-dimensional space are obtained ; Step Five Two: Take the coordinates of the target position in the three-dimensional space as the expected grasping point , and according to the inverse kinematics algorithm and the expected grasping point calculate the target angle vector of each joint of the robotic arm , respectively representing the target angles of the 1st, 2nd,..., Mth joints of the robotic arm, and M represents the total number of joints of the robotic arm; Step Five Three: Execute the grasping action according to the target angles of each joint of the robotic arm.

[0052] The other steps and parameters are the same as those in any one of the first to ninth specific embodiments.

[0053] This embodiment effectively integrates visual recognition and spatial positioning, significantly improving the accuracy and robustness of the system in grasping targets in complex scenarios, and also laying a foundation for multi-task manipulation robots to effectively execute task sequences.

[0054] The above numerical examples of the present invention are only for explaining in detail the calculation model and calculation process of the present invention, rather than limiting the embodiments of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made based on the above description. It is impossible to list all the embodiments here. Any obvious changes or modifications derived from the technical solutions of the present invention still fall within the protection scope of the present invention.

Claims

1. A method for controlling the movement and manipulation of a robot based on deep learning, characterized in that, The method specifically includes the following steps: Step 1: The user inputs a natural language instruction into the large language model, and the large language model outputs a subtask sequence S according to the input natural language instruction; Step 2: Initialize subtask s = 1; Step 3: Select the next operation according to the type of the s-th subtask in the subtask sequence S. Specifically: If the s-th subtask in the subtask sequence S is a path planning task, then execute Step 4; If the s-th subtask in the subtask sequence S is a robotic arm manipulation task, then execute Step 5; Step 4: Use the first-layer reinforcement learning network and the second-layer reinforcement learning network to perform path planning, and the robot moves to the target position according to the path planning result; then execute Step 6; Step 5: Use the target detection model deployed on the robot to identify the position of the target object in the environment, and calculate the target angles of the joints of the robotic arm of the robot according to the position of the target object; Execute a grasping action on the target object according to the target angles of the joints of the robotic arm; then execute Step 6; Step 6: Whether all the subtasks in the subtask sequence S have been executed; If there are still subtasks not executed, then set s = s + 1, and return to execute Step 3; If all subtasks have been executed, then end.

2. The method for controlling the movement and manipulation of a robot based on deep learning according to claim 1, wherein The large language model is GLM, and the input of the large language model also includes the environmental semantic map of the target area.

3. The method for controlling the movement and manipulation of a robot based on deep learning according to claim 2, wherein The specific process of Step 4 is: Step 4.

1. Use the algorithm to plan a path from the current position of the robot to the target location of the s-th subtask. The first-layer reinforcement learning network performs global path planning based on the algorithm planning result and the current environmental information; Step 4-2: The second-layer reinforcement learning network fine-tunes the global path planning result according to the global path planning result and the perceived local environmental information, and the robot moves towards the target position according to the fine-tuned path.

4. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 3, characterized in that The agent state space of the first-layer reinforcement learning network is defined as ; Among them, is a vector composed of the semantic category of the target location of the s-th sub-task and the coordinates of the target location ; Indicates in In the planning result of the algorithm, the semantic category sequence composed of all areas that need to be passed from the current position of the robot to the target location ; is a vector composed of the current position coordinates of the robot and the current attitude angle of the robot; Indicates the historical average passage time required for the robot to pass through the current area; The action space of the agent of the first-layer reinforcement learning network is defined as ; Among them, represents the 1st, 2nd, …, th areas that the robot needs to pass through in sequence in the environmental semantic map during the current step action.

5. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 4, characterized in that, The reward function used in the training process of the first-layer reinforcement learning network is as follows: Wherein: represents an indicator function. If the sth sub-task is successfully completed, the value of the indicator function is 1; otherwise, the value of the indicator function is 0. Indicates the navigation efficiency of the planned path from the starting point to the target position obtained according to the actions output by the agent; Represents the semantic risk assessment value on the path from the starting point to the target position output by the agent; Indicates the time required for the robot to move from the starting point to the target position along the planned path of the agent; , , and are the weight coefficients for each item, respectively.

6. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 5, characterized in that, The state space of the second-layer reinforcement learning network is defined as ; Among them, is a vector composed of the channel entrance position coordinates of a sub-goal in the global path planning result and the semantic category of the sub-goal; is a vector composed of the dynamic obstacle position coordinates and the dynamic obstacle speed obtained according to the local environment perception information; is a vector composed of the current position coordinates, linear velocity, angular velocity of the robot, and the wheel speeds of the respective Mecanum wheels of the robot; is a vector composed of the labels of the objects around the robot and the confidence levels of the objects around the robot; The actions output by the agent of the second-layer reinforcement learning network are the desired linear velocity and desired angular velocity of the robot.

7. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 6, characterized in that, The reward function used during the training process of the second-layer reinforcement learning network is as follows: Wherein: represents the navigation efficiency; represents an indicator function. If a collision occurs after the agent outputs an action in the current step, the value of the indicator function is 1; otherwise, the value of the indicator function is 0. Indicates the local semantic risk assessment value of the action output by the agent for the current step executed by the robot; Indicates the shortest vertical distance from the current position of the robot to the path planned by the first-layer reinforcement learning network; is the weight coefficient of each item.

8. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 7, characterized in that, The semantic risk assessment value and are calculated as follows: Wherein: represents the global path output by the agent of the first-layer reinforcement learning network; Indicates the path The semantic category of the Indicate corresponding basic risk value; Indicates confidence; Wherein: represents the path that needs to be passed through for the current step action output by the execution agent; Indicates the current position of the robot; Indicates a circular area centered at the position with as the radius; Indicates a path Sub-regions on and within the circular region; Indicates a sub-region The corresponding basic risk value; Indicates the confidence level of the sub-region ​ 9. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 8, characterized in that, The target detection model in Step 5 is a lightweight YOLOv5 target detection model, and the lightweight YOLOv5 target detection model is the YOLOv5 network after replacement obtained by using the MobileNetV3 model to replace the backbone network of YOLOv5.

10. A method for controlling the movement and manipulation of a robot based on deep learning according to claim 9, characterized in that, The specific process of Step 5 is: Step 5-1: Use the depth camera deployed on the robot to collect the color image img and depth information depth in real time. The color image img is detected by a lightweight YOLOv5 object detection model to obtain the two-dimensional bounding box where the target is located ; According to the two-dimensional bounding box where the target position is located and the depth information depth, and combined with the internal parameters of the depth camera, the coordinates of the target position in the three-dimensional space are obtained ; Step Five Two: Use the coordinates of the target position in the three-dimensional space as the desired grasping point , and calculate the target angle vector of each joint of the robotic arm according to the inverse kinematics algorithm and the desired grasping point where , respectively represent the target angles of the 1st, 2nd, …, Mth joints of the robotic arm, and M represents the total number of joints of the robotic arm; Step 5-3: Execute a grasping action according to the target angles of the joints of the robotic arm.

Images