Body-equipped robot control system and method based on multi-modal perception and deep learning

Abstract

The invention discloses a body-equipped robot control system and method based on multi-modal perception and deep learning, and the system comprises a multi-modal perception module which is used for obtaining and fusing visual information, tactile information and proprioceptive information, and generating a unified state representation; the deep reinforcement learning control module adopts a reinforcement learning algorithm based on an actor-commentator framework and is used for representing an output control strategy according to the unified state, and the reinforcement learning algorithm adopts an experience playback mechanism and a target network technology to realize stable learning; the hierarchical motion planning module is connected with the deep reinforcement learning control module and is used for decomposing a complex task corresponding to the control strategy into high-level strategy planning and bottom-level motion execution; the self-adaptive impedance control module is connected with the hierarchical motion planning module and is used for dynamically adjusting impedance parameters of the robot according to interaction information of the robot and the environment; and the safety guarantee module is used for applying constraint optimization to the control strategy or the motion execution so as to ensure the operation safety of the robot.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of robot control, and particularly relates to a somatic robot control system and method based on multi-modal perception and deep learning. BACKGROUND

[0002] With the rapid development of artificial intelligence and robot technology, robots are increasingly applied in the fields of industrial manufacturing, medical services, home assistance and special operations. However, robots still face great challenges in achieving human-like autonomous, dexterous and safe operation in complex, dynamic and unstructured real-world environments. The core of this challenge lies in how to enable robots to have human-like multi-modal environmental perception ability, autonomous decision-making learning ability, flexible interaction control ability and internal safety guarantee mechanism.

[0003] Currently, the mainstream robot control methods can be mainly divided into two categories: traditional control methods based on models and intelligent control methods based on data-driven. Traditional control methods, such as pre-programmed control and PID (Proportion-Integral-Derivative) control, rely on accurate environmental models and pre-set rules. This kind of method is stable in structured and deterministic environments, but its flexibility is seriously insufficient. When the environment changes unpredictably or the task needs to be adjusted, manual reprogramming or parameter adjustment is often required, making it difficult to adapt to dynamic environments. More importantly, traditional methods usually rely only on a single sensor modality (such as position encoder), and the environmental perception ability is weak, which cannot effectively understand and integrate multi-modal information such as vision and touch to build a comprehensive environmental state cognition. This is an inherent defect in scenarios that require fine operation or human-robot collaboration.

[0004] To overcome the limitations of traditional methods, researchers have introduced data-driven approaches such as deep reinforcement learning. Deep reinforcement learning, especially algorithms based on the actor-critic framework (such as DDPG), enables robots to autonomously learn control strategies through interaction with their environment, demonstrating strong self-learning and adaptive potential. However, directly applying deep reinforcement learning to embodied robot control still presents significant challenges: First, insufficient utilization of multimodal information. Existing methods often focus on single-state inputs (such as joint angles), lacking mechanisms for effectively fusing multi-source heterogeneous information such as vision, touch, and proprioception. This results in a one-sided understanding of the robot's environment, limiting its performance in complex tasks. Second, low training efficiency and stability. Deep reinforcement learning suffers from low sample efficiency, high training costs and risks on physical robots, and the training process is prone to instability and slow convergence. Third, lack of safety mechanisms. Learning strategies purely driven by task rewards may produce dangerous or destructive actions, and there is a lack of systematic methods that integrate safety as a hard constraint into the learning and control process. Fourth, insufficient interaction compliance. Most control strategies output direct position or torque commands, failing to adjust their own impedance characteristics (stiffness and damping) in real time according to the dynamic changes in the contact state. This results in a harsh interaction process, which can easily cause damage to themselves or the environment.

[0005] In summary, existing technologies have failed to systematically address the core challenges of multimodal perception fusion, efficient and stable learning, and compliant and safe interaction—the three aspects of embodied robots. Therefore, there is an urgent need for an innovative control system and method that can integrate multimodal perception, deep reinforcement learning, adaptive compliant control, and active safety constraints to comprehensively enhance the autonomy, adaptability, safety, and operational performance of embodied robots in complex real-world environments. Summary of the Invention

[0006] This invention addresses the aforementioned problems and overcomes the shortcomings of existing technologies by providing an embodied robot control system and method based on multimodal perception and deep learning, which can achieve multimodal information fusion, possess self-learning capabilities, and has high security and training efficiency.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: An embodied robot control system based on multimodal perception and deep learning includes a multimodal perception module, a deep reinforcement learning control module, a hierarchical motion planning module, an adaptive impedance control module, and a safety assurance module. The multimodal perception module is used to acquire and fuse visual information, tactile information, and proprioceptive information to generate a unified state representation; the state fusion formula of the multimodal perception module is: ,in, s f In a state of fusion,f v As a visual feature, f t Tactile characteristics, f p Features of proprioception W f To fuse the weight matrix, b f For the fusion bias vector; The deep reinforcement learning control module employs a reinforcement learning algorithm based on the actor-critic framework to output a control policy according to the unified state representation. The reinforcement learning algorithm uses an experience replay mechanism and target network technology to achieve stable learning. The value function update formula of the deep reinforcement learning control module is: ,in, α For learning rate, gamma As a discount factor, r For instant rewards, s This is the current state. a For the current action, s′ For the next state, a′ For the next action; The hierarchical motion planning module, connected to the deep reinforcement learning control module, is used to decompose the complex task corresponding to the control strategy into high-level policy planning and low-level motion execution; the high-level policy planning aims to maximize cumulative reward, and its objective function is: ,in, J g For the high-level objective function, gamma As a discount factor, r g As a reward for senior management, s t For time step t state, g t For time step t Sub-targets; The adaptive impedance control module, connected to the hierarchical motion planning module, is used to dynamically adjust the robot's impedance parameters based on the interaction information between the robot and the environment; the adaptive impedance control formula is: ,in, tau For output torque, K p For adaptive stiffness matrix, K d For adaptive damping matrix, q d For the desired position, q For actual location, dFor the desired speed, For actual speed, tau External torque; The safety assurance module is used to apply constraints to the control strategy or motion execution to ensure the safe operation of the robot; the mathematical expression of the safety constraints is: The constraints are ,in, text nominal Nominal torque, h i For security constraint functions; The multimodal perception module, deep reinforcement learning control module, hierarchical motion planning module, adaptive impedance control module, and safety assurance module work together to optimize the robot's overall control performance.

[0008] Furthermore, the multimodal perception module includes a visual feature extraction unit, a tactile feature extraction unit, and a proprioceptive feature extraction unit; The visual feature extraction unit uses a convolutional neural network to extract visual features, using the following formula: in, I For raw visual input, theta v These are the parameters of the convolutional neural network; The tactile feature extraction unit uses a multilayer perceptron to extract tactile features, using the following formula: in, T For raw tactile input, theta t These are the parameters of the multilayer perceptron; The proprioceptive feature extraction unit uses a multilayer perceptron to extract proprioceptive features, using the following formula: in, q 1 This refers to the joint position. For joint velocity, theta p These are the parameters of the multilayer perceptron.

[0009] Furthermore, the deep reinforcement learning control module includes an actor network, a critic network, an experience replay mechanism, and a target network. The policy update of the actor network is based on the policy gradient, and the value update of the critic network is based on the temporal difference error. The actor network is used to generate control policies, and its policy update formula is as follows: ;in, theta For actor network parameters, J ( theta Let ) be the objective function. π θ (a | s ) is the policy function. Q π ( s, a ) is the action value function; The network of critics is used to assess value, and its value update formula is: , ;in, delta For timing difference error, w For the critic's network parameters, J ( w Let Q be the objective function. w ( s, a () is the value function; The experience replay mechanism samples data from the experience pool according to the sampling probability, and the sampling probability formula is: ;in, Priority α 1 is the priority index. It is a small constant; The target network technology is updated via a soft update method, and the soft update formula is as follows: ;in, theta ′ For the target network parameters, β To update the coefficients, β≤ 1.

[0010] Furthermore, the hierarchical motion planning module includes a high-level strategy planning unit, a low-level motion execution unit, and a value function decomposition unit; The high-level strategy planning unit is used to generate sub-objectives, and its sub-objective update formula is as follows: ,in, π h For high-level policy functions, s This is the current state. g t1 For the current sub-goal; The underlying motion execution unit is used to output control torque, and its control law is as follows: ,in, π l This refers to the underlying strategy function; the underlying motion execution unit outputs the current sub-target based on the higher-level strategy. g t1 It generates underlying control commands through control laws; The value function decomposition formula of the value function decomposition unit is: ,in, Q h ( s, g () represents the high-level value function. Q l( s , a | g ) is the underlying conditional value function.

[0011] Furthermore, the adaptive impedance control module dynamically adjusts the stiffness and damping parameters through a neural network, and the learning formulas include the stiffness matrix learning formula and the damping matrix learning formula. The formula for learning the stiffness matrix is ​​as follows: ,in, f k For a parameterized neural network, s represents the current state. F ext For external force; the damping matrix learning formula: ,in, g ϕ For parameterized neural networks; It also includes an impedance parameter learning objective function, the expression of which is: ,in, For positional error, For speed error, tau measured To measure torque, R For regularization terms, lambda This is the regularization coefficient.

[0012] Furthermore, the safety constraint functions of the safety assurance module include joint constraint functions, collision constraint functions, and torque constraint functions; The joint constraint function expression is as follows: ,in, q i,max and q i,min These represent the maximum and minimum limits of the joint position, respectively. This represents the joint velocity limit; The expression for the collision constraint function is: ,in, d Let it be a distance function. d safe To maintain a safe distance; The torque constraint function expression is as follows: ,in, tau max Maximum permissible torque; It also includes real-time solutions to constrained optimization problems, the formula for which the real-time solution to constrained optimization problems is: The constraints are ,in, tau desired For the desired torque, delta t For time step, MThis is the quality matrix.

[0013] Furthermore, the overall optimization objective function of the system is: ,in, The objective function is the task objective function. gamma As a discount factor, r For instant rewards; The objective function is security. Let the efficiency objective function be... lambda 1 and lambda 2 represents the weighting coefficient.

[0014] A control method for embodied robots based on multimodal perception and deep learning, implemented using the aforementioned embodied robot control system based on multimodal perception and deep learning, includes the following steps: S1. Multimodal Perception Fusion Steps: The multimodal perception module acquires the robot's visual, tactile, and proprioceptive information, and performs feature fusion to generate a unified state representation; the state fusion formula is then used. ,in, s f In a state of fusion, f v As a visual feature, f t Tactile characteristics, f p Features of proprioception W f To fuse the weight matrix, b f For the fusion bias vector; S2. Deep Reinforcement Learning Control Steps: Using a deep reinforcement learning control module and a deep reinforcement learning algorithm based on the actor-critic framework, a control policy is generated through the actor network according to the unified state representation, and value is evaluated through the critic network. The training process is stabilized using an experience replay mechanism and target network technology. The value function update formula is then used. ,in, α For learning rate, gamma As a discount factor, r For instant rewards, s This is the current state. a For the current action, s′ For the next state, a′ For the next action; S3. Hierarchical Motion Planning Steps: The hierarchical motion planning module decomposes the complex task corresponding to the control strategy into high-level strategy planning and low-level motion execution; the high-level strategy planning aims to maximize cumulative reward, and its objective function is: ,in,J g For the high-level objective function, gamma As a discount factor, r g As a reward for senior management, s t For time step t state, g t For time step t Sub-targets; S4. Adaptive Impedance Control Steps: The adaptive impedance control module dynamically adjusts the robot's impedance parameters based on real-time interaction information between the robot and its environment, using an adaptive impedance control algorithm to execute the underlying motion. The adaptive impedance control formula is as follows: ,in, tau For output torque, K p For adaptive stiffness matrix, K d For adaptive damping matrix, q d For the desired position, q For actual location, d For the desired speed, For actual speed, tau External torque; S5. Security Assurance Steps: Through the security assurance module, constraint optimization is applied during the generation, planning, and execution of control strategies in steps S2 to S4 to ensure that security constraints are met; the mathematical expression of the security constraints is: The constraints are ,in, text nominal Nominal torque, h i This is a safety constraint function.

[0015] Furthermore, the deep reinforcement learning control steps include: Actor Network Strategy Update Formula: ,in, theta For actor network parameters, J ( theta Let ) be the objective function. π θ ( a | s ) is the policy function; For the dominant function, V ( s ) is the state value function; The formula for updating the value of a critic's network: ,in This refers to timing difference error; The exploration strategy employs the Ornstein-Uhlenbeck process: Where ε is the mean regression rate, mu This is the long-term mean. sigma For volatility, W t This is the Wiener process.

[0016] Furthermore, the method also includes a training optimization step: a course learning strategy, the difficulty of which increases with the number of training steps, as shown in the formula: The initialization steps for imitation learning are as follows: ,in, a expert For expert actions; meta-learning rapid adaptation steps, the formula is: , where θ 1 ′ To adapt to subsequent parameters, theta For actor network parameters, α L is the learning rate. Ti This is the task loss function.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention significantly improves the control performance of embodied robots by constructing an integrated system that coordinates five modules: multimodal perception fusion, deep reinforcement learning decision-making, hierarchical task planning, adaptive impedance control, and active safety assurance. Specifically, this invention has the following comprehensive beneficial effects: 1. Comprehensive perception and intelligent decision-making capabilities: This invention integrates visual, tactile, and proprioceptive information to construct a unified environmental cognition, overcoming the problems of single perception modality and one-sided environmental understanding in traditional methods; combined with deep reinforcement learning based on the actor-critic framework, it enables the robot to learn autonomously from interactions and dynamically optimize strategies, significantly enhancing its adaptability and autonomy in unknown or changing environments.

[0018] 2. Efficient and stable learning and precise compliant control: This invention introduces mechanisms such as experience playback, target network and course learning, which greatly improves training efficiency and stability and shortens convergence time. It adopts hierarchical motion planning to decompose complex tasks and combines them with adaptive impedance control to realize the dynamic adjustment of rigid-flexible characteristics with the interactive environment. This significantly improves the compliance and safety of the robot's interaction with the environment or people while ensuring trajectory tracking accuracy.

[0019] 3. Systematic Safety and Overall Performance Optimization: This invention embeds safety as a hard constraint into the control loop, effectively avoiding dangerous actions and reducing collision and violation risks through real-time monitoring and constraint optimization. Through a multi-objective collaborative overall optimization function, the system balances tracking accuracy, control energy consumption, and safety boundaries, resulting in significant improvements in various robot performance indicators. Experiments show that this system outperforms traditional DDPG and PID methods in control accuracy, learning speed, environmental adaptability, and operational safety, demonstrating outstanding comprehensive performance and practical value. Attached Figure Description

[0020] Fig. 1 This is a schematic diagram of the overall architecture of an embodied robot control system based on multimodal perception and deep learning according to the present invention (Note: a directed graph is used for representation).

[0021] Fig. 2 This is a schematic diagram of the multimodal sensing fusion of the present invention (Note: a directed graph is used for representation).

[0022] Fig. 3 This is a schematic diagram of the deep reinforcement learning training process of the present invention (Note: a directed graph is used for representation).

[0023] Fig. 4 This is a schematic diagram of the adaptive impedance control principle of the present invention (Note: a directed graph is used for representation).

[0024] Fig. 5 This is a schematic diagram of the performance comparison analysis of the present invention (Note: a network diagram is used for representation). Detailed Implementation

[0025] To make the technical problems solved, the technical solutions, and the beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0026] like Figs. 1-5 As shown in the figure, an embodiment of the present invention provides an embodied robot control system based on multimodal perception and deep reinforcement learning, including a multimodal perception module, a deep reinforcement learning control module, a hierarchical motion planning module, an adaptive impedance control module, and a safety assurance module; the multimodal perception module is used to acquire and fuse visual information, tactile information, and proprioceptive information to generate a unified state representation; the state fusion formula of the multimodal perception module is: ,in, s f In a state of fusion, f v As a visual feature, f t Tactile characteristics,f p Features of proprioception W f To fuse the weight matrix, b f Let be the fusion bias vector; where the fusion weight matrix is... W f The adaptive adjustment formula: The deep reinforcement learning control module employs a reinforcement learning algorithm based on the actor-critic framework to output a control policy according to the unified state representation. The reinforcement learning algorithm uses an experience replay mechanism and target network technology to achieve stable learning. The value function update formula of the deep reinforcement learning control module is: ,in, α For learning rate, gamma As a discount factor, r For instant rewards, s This is the current state. a For the current action, s′ For the next state, a′ For the next action; The hierarchical motion planning module, connected to the deep reinforcement learning control module, is used to decompose the complex task corresponding to the control strategy into high-level policy planning and low-level motion execution; the high-level policy planning aims to maximize cumulative reward, and its objective function is: ,in, J g For the high-level objective function, gamma As a discount factor, r g As a reward for senior management, s t For time step t state, g t For time step t Sub-targets; The adaptive impedance control module, connected to the hierarchical motion planning module, is used to dynamically adjust the robot's impedance parameters based on the interaction information between the robot and the environment; the adaptive impedance control formula is: ,in, tau For output torque, K p For adaptive stiffness matrix, K d For adaptive damping matrix, q d For the desired position, q For actual location, d For the desired speed, For actual speed, tau External torque; The safety assurance module is used to apply constraints to the control strategy or motion execution to ensure the safe operation of the robot; the mathematical expression of the safety constraints is: The constraints are ,in, text nominal Nominal torque, h i For security constraint functions; The multimodal perception module, deep reinforcement learning control module, hierarchical motion planning module, adaptive impedance control module, and safety assurance module work together to optimize the robot's overall control performance.

[0027] Specifically, the multimodal perception module includes a visual feature extraction unit, a tactile feature extraction unit, and a proprioceptive feature extraction unit; the visual feature extraction unit uses a convolutional neural network to extract visual features, with the following formula: in, I For raw visual input, theta v The parameters are those of the convolutional neural network; the tactile feature extraction unit uses a multilayer perceptron to extract tactile features, and the formula is: in, T For raw tactile input, theta t The parameters are those of the multilayer perceptron; the proprioceptive feature extraction unit uses the multilayer perceptron to extract proprioceptive features, and the formula is: in, q 1 This refers to the joint position. For joint velocity, theta p These are the parameters of the multilayer perceptron.

[0028] Specifically, the deep reinforcement learning control module includes an actor network, a critic network, an experience replay mechanism, and a target network. The policy update of the actor network is based on the policy gradient, and the value update of the critic network is based on the temporal difference error. The actor network is used to generate the control policy, and its policy update formula is as follows: ;in, theta For actor network parameters, J ( theta Let ) be the objective function. π θ ( a | s ) is the policy function. Q π ( s, a) represents the action value function; the commentator network is used to evaluate value, and its value update formula is: , ;in, delta For timing difference error, w For the critic's network parameters, J ( w Let Q be the objective function. w ( s, a The value function is defined as follows: The experience replay mechanism samples data from the experience pool according to the sampling probability, and the sampling probability formula is as follows: ;in, Priority α 1 is the priority index. It is a small constant; the target network technology is updated through a soft update method, and the soft update formula is: ;in, theta ′ For the target network parameters, β To update the coefficients, β≤ 1.

[0029] Specifically, the hierarchical motion planning module includes a high-level strategy planning unit, a low-level motion execution unit, and a value function decomposition unit; the high-level strategy planning unit is used to generate sub-objectives, and its sub-objective update formula is: ,in, π h For high-level policy functions, s This is the current state. g t1 The current sub-target; the underlying motion execution unit is used to output control torque, and its control law is: ,in, π l This refers to the underlying strategy function; the underlying motion execution unit outputs the current sub-target based on the higher-level strategy. g t1 The underlying control commands are generated through control laws; the value function decomposition formula of the value function decomposition unit is: ,in, Q h ( s, g () represents the high-level value function. Q l ( s , a | g ) is the underlying conditional value function.

[0030] Specifically, the adaptive impedance control module dynamically adjusts the stiffness and damping parameters through a neural network, and the learning formulas include a stiffness matrix learning formula and a damping matrix learning formula; the stiffness matrix learning formula is as follows: ,in, f k For a parameterized neural network, s represents the current state. F ext For external force; the damping matrix learning formula: ,in, g ϕ This is a parameterized neural network; it also includes an impedance parameter learning objective function, the expression of which is: ,in, For positional error, For speed error, tau measured To measure torque, R For regularization terms, lambda This is the regularization coefficient.

[0031] Specifically, the safety constraint functions of the safety assurance module include joint constraint functions, collision constraint functions, and torque constraint functions; the expression for the joint constraint function is: ,in, q i,max and q i,min These represent the maximum and minimum limits of the joint position, respectively. The joint velocity limit; the collision constraint function expression: ,in, d Let it be a distance function. d safe For a safe distance; the torque constraint function expression is: ,in, tau max This includes the maximum permissible torque; it also includes the real-time solution of the constraint optimization problem, the formula for the real-time solution of the constraint optimization problem is: The constraints are ,in, tau desired For the desired torque, delta t For time step, M This is the quality matrix.

[0032] Specifically, the overall optimization objective function of the system is: ,in, The objective function is the task objective function. gamma As a discount factor, r For instant rewards; The objective function is security. Let the efficiency objective function be... lambda 1 and lambda 2 represents the weighting coefficient.

[0033] In addition, the robot control method based on multimodal perception and deep reinforcement learning provided in this embodiment of the invention includes the following steps: S1. Multimodal Perception Fusion Steps: The multimodal perception module acquires the robot's visual, tactile, and proprioceptive information, and performs feature fusion to generate a unified state representation; the state fusion formula is then used. ,in, s f In a state of fusion, f v As a visual feature, f t Tactile characteristics, f p Features of proprioception W f To fuse the weight matrix, b f For the fusion bias vector; S2. Deep Reinforcement Learning Control Steps: Using a deep reinforcement learning control module and a deep reinforcement learning algorithm based on the actor-critic framework, a control policy is generated through the actor network according to the unified state representation, and value is evaluated through the critic network. The training process is stabilized using an experience replay mechanism and target network technology. The value function update formula is then used. ,in, α For learning rate, gamma As a discount factor, r For instant rewards, s This is the current state. a For the current action, s′ For the next state, a′ For the next action; S3. Hierarchical Motion Planning Steps: The hierarchical motion planning module decomposes the complex task corresponding to the control strategy into high-level strategy planning and low-level motion execution; the high-level strategy planning aims to maximize cumulative reward, and its objective function is: ,in, J g For the high-level objective function, gamma As a discount factor, r g As a reward for senior management, s t For time step t state, g t For time step t Sub-targets; S4. Adaptive Impedance Control Steps: The adaptive impedance control module dynamically adjusts the robot's impedance parameters based on real-time interaction information between the robot and its environment, using an adaptive impedance control algorithm to execute the underlying motion. The adaptive impedance control formula is as follows: ,in, tau For output torque, K p For adaptive stiffness matrix, K d For adaptive damping matrix, q d For the desired position, q For actual location, d For the desired speed, For actual speed, tau External torque; S5. Security Assurance Steps: Through the security assurance module, constraint optimization is applied during the generation, planning, and execution of control strategies in steps S2 to S4 to ensure that security constraints are met; the mathematical expression of the security constraints is: The constraints are ,in, text nominal Nominal torque, h i This is a safety constraint function.

[0034] Specifically, the deep reinforcement learning control steps include: Actor Network Strategy Update Formula: ,in, theta For actor network parameters, J ( theta Let ) be the objective function. π θ ( a | s ) is the policy function; For the dominant function, V ( s ) is the state value function; The formula for updating the value of a critic's network: ,in This refers to timing difference error; The exploration strategy employs the Ornstein-Uhlenbeck process: Where ε is the mean regression rate, mu This is the long-term mean. sigma For volatility, W t This is the Wiener process.

[0035] Specifically, the method further includes a training optimization step: a course learning strategy, the difficulty of which increases with the number of training steps, as shown in the formula: The initialization steps for imitation learning are as follows: ,in, a expert For expert actions; meta-learning rapid adaptation steps, the formula is: , where θ 1 ′ To adapt to subsequent parameters, theta For actor network parameters, α L is the learning rate. Ti This is the task loss function.

[0036] The performance of this invention is compared and analyzed with that of traditional DDPG and traditional PID in the prior art. The main performance indicators are compared experimentally in terms of learning convergence speed, control accuracy, safety, environmental adaptability and stability. The comparison results are shown in Table 1 below.

[0037] Table 1 Comparison of Performance Indicators Performance index The present invention Traditional DDPG Traditional PID Learning convergence speed 85% 65% 45% Control accuracy 88% 70% 80% Safety score 92% 75% 65% Environmental adaptability 85% 60% 40% Stability 90% 68% 55% Through experimental verification of the technical effects, the present invention has the following significant advantages over the prior art: Improved training efficiency: convergence speed increased by 40%, training time reduced by 35%; Improved control accuracy: position tracking error reduced by 28%, force control accuracy improved by 32%; Enhanced safety: violation operation reduced by 65%, collision probability reduced by 72%; Improved adaptability: adaptation time to new tasks shortened by 55%, robustness to environmental changes improved by 48%.

[0038] The above technical effects are quantified and verified using the following indicator: Average cumulative reward: Task success rate: Safety indicators: Energy efficiency: This invention achieves a comprehensive improvement in the control performance of embodied robots through technological innovations such as multimodal perception fusion, deep reinforcement learning control, adaptive impedance control, and hierarchical planning, and has significant theoretical value and practical application prospects.

[0039] It is understood that, although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A embodied robot control system based on multimodal perception and deep learning, characterized in that, It includes a multimodal perception module, a deep reinforcement learning control module, a hierarchical motion planning module, an adaptive impedance control module, and a safety assurance module; The multimodal perception module is used to acquire and fuse visual information, tactile information, and proprioceptive information to generate a unified state representation; the state fusion formula of the multimodal perception module is: ,in, s f In a state of fusion, f v As a visual feature, f t Tactile characteristics, f p Features of proprioception W f To fuse the weight matrix, b f For the fusion bias vector; The deep reinforcement learning control module employs a reinforcement learning algorithm based on the actor-critic framework to output a control policy according to the unified state representation. The reinforcement learning algorithm uses an experience replay mechanism and target network technology to achieve stable learning. The value function update formula of the deep reinforcement learning control module is: ,in, α For learning rate, γ As a discount factor, r For instant rewards, s This is the current state. a For the current action, s′ For the next state, a′ For the next action; The hierarchical motion planning module, connected to the deep reinforcement learning control module, is used to decompose the complex task corresponding to the control strategy into high-level policy planning and low-level motion execution; the high-level policy planning aims to maximize cumulative reward, and its objective function is: ,in, J g For the high-level objective function, γ As a discount factor, r g As a reward for senior management, s t For time step t state, g t For time step t Sub-targets; The adaptive impedance control module, connected to the hierarchical motion planning module, is used to dynamically adjust the robot's impedance parameters based on the interaction information between the robot and the environment; the adaptive impedance control formula is: ,in, τ For output torque, K p For adaptive stiffness matrix, K d For adaptive damping matrix, q d For the desired position, q For actual location, d For the desired speed, For actual speed, τext External torque; The safety assurance module is used to apply constraints to the control strategy or motion execution to ensure the safe operation of the robot; the mathematical expression of the safety constraints is: The constraints are ,in, τ nominal Nominal torque, h i For security constraint functions; The multimodal perception module, deep reinforcement learning control module, hierarchical motion planning module, adaptive impedance control module, and safety assurance module work together to optimize the robot's overall control performance.

2. The embodied robot control system based on multimodal perception and deep learning according to claim 1, characterized in that, The multimodal perception module includes a visual feature extraction unit, a tactile feature extraction unit, and a proprioceptive feature extraction unit; The visual feature extraction unit uses a convolutional neural network to extract visual features, using the following formula: in, I For raw visual input, θ v These are the parameters of the convolutional neural network; The tactile feature extraction unit uses a multilayer perceptron to extract tactile features, using the following formula: in, T For raw tactile input, θ t These are the parameters of the multilayer perceptron; The proprioceptive feature extraction unit uses a multilayer perceptron to extract proprioceptive features, using the following formula: in, q 1 This refers to the joint position. For joint velocity, θ p These are the parameters of the multilayer perceptron.

3. The embodied robot control system based on multimodal perception and deep learning according to claim 1, characterized in that, The deep reinforcement learning control module includes an actor network, a critic network, an experience replay mechanism, and a target network. The policy update of the actor network is based on the policy gradient, and the value update of the critic network is based on the temporal difference error. The actor network is used to generate control policies, and its policy update formula is as follows: ; in, θ For actor network parameters, J ( θ Let ) be the objective function. π θ ( a | s ) is the policy function. Q π ( s,a ) is the action value function; The network of critics is used to assess value, and its value update formula is: , ;in, δ For timing difference error, w For the critic's network parameters, J ( w Let Q be the objective function. w ( s,a () is the value function; The experience replay mechanism samples data from the experience pool according to the sampling probability, and the sampling probability formula is: ;in, Priority α 1 is the priority index. It is a small constant; The target network technology is updated via a soft update method, and the soft update formula is as follows: ;in, θ ′ For the target network parameters, β To update the coefficients, β≤ 1.

4. The embodied robot control system based on multimodal perception and deep learning according to claim 1, characterized in that, The hierarchical motion planning module includes a high-level strategy planning unit, a low-level motion execution unit, and a value function decomposition unit. The high-level strategy planning unit is used to generate sub-objectives, and its sub-objective update formula is as follows: ,in, π h For high-level policy functions, s This is the current state. g t1 For the current sub-goal; The underlying motion execution unit is used to output control torque, and its control law is as follows: ,in, π l This refers to the underlying strategy function; the underlying motion execution unit outputs the current sub-target based on the higher-level strategy. g t1 It generates underlying control commands through control laws; The value function decomposition formula of the value function decomposition unit is: ,in, Q h ( s,g () represents the high-level value function. Q l ( s , a | g ) is the underlying conditional value function.

5. The embodied robot control system based on multimodal perception and deep learning according to claim 1, characterized in that, The adaptive impedance control module dynamically adjusts the stiffness and damping parameters through a neural network, and the learning formulas include stiffness matrix learning formulas and damping matrix learning formulas. The formula for learning the stiffness matrix is ​​as follows: ,in, f k For a parameterized neural network, s represents the current state. F ext For external force; the damping matrix learning formula: ,in, g ϕ For parameterized neural networks; It also includes an impedance parameter learning objective function, the expression of which is: ,in, For positional error, For speed error, τ measured To measure torque, R For regularization terms, λ This is the regularization coefficient.

6. The embodied robot control system based on multimodal perception and deep learning according to claim 1, characterized in that, The safety constraint functions of the safety assurance module include joint constraint functions, collision constraint functions, and torque constraint functions; The joint constraint function expression is as follows: ,in, q i,max and q i,min These represent the maximum and minimum limits of the joint position, respectively. This represents the joint velocity limit; The expression for the collision constraint function is: ,in, d Let it be a distance function. d safe To maintain a safe distance; The torque constraint function expression is as follows: ,in, τ max Maximum permissible torque; It also includes real-time solutions to constrained optimization problems, the formula for which the real-time solution to constrained optimization problems is: The constraints are ,in, τ desired For the desired torque, Δt For time step, M This is the quality matrix.

7. The embodied robot control system based on multimodal perception and deep learning according to claim 1, characterized in that, The overall optimization objective function of the system is: ,in, The objective function is the task objective function. γ As a discount factor, r For instant rewards; The objective function is security. Let the efficiency objective function be... λ 1 and λ 2 represents the weighting coefficient.

8. A control method for embodied robots based on multimodal perception and deep learning, characterized in that, The embodied robot control system based on multimodal perception and deep learning as described in claim 1 is implemented by including the following steps: S1. Multimodal Perception Fusion Steps: The multimodal perception module acquires the robot's visual, tactile, and proprioceptive information, and performs feature fusion to generate a unified state representation; the state fusion formula is then used. ,in, s f In a state of fusion, f v As a visual feature, f t Tactile characteristics, f p Features of proprioception W f To fuse the weight matrix, b f For the fusion bias vector; S2. Deep Reinforcement Learning Control Steps: Using a deep reinforcement learning control module and a deep reinforcement learning algorithm based on the actor-critic framework, a control policy is generated through the actor network according to the unified state representation, and value is evaluated through the critic network. The training process is stabilized using an experience replay mechanism and target network technology. The value function update formula is then used. ,in, α For learning rate, γ As a discount factor, r For instant rewards, s This is the current state. a For the current action, s′ For the next state, a′ For the next action; S3. Hierarchical Motion Planning Steps: The hierarchical motion planning module decomposes the complex task corresponding to the control strategy into high-level strategy planning and low-level motion execution; the high-level strategy planning aims to maximize cumulative reward, and its objective function is: ,in, J g For the high-level objective function, γ As a discount factor, r g As a reward for senior management, s t For time step t state, g t For time step t Sub-targets; S4. Adaptive Impedance Control Steps: The adaptive impedance control module dynamically adjusts the robot's impedance parameters based on real-time interaction information between the robot and its environment, using an adaptive impedance control algorithm to execute the underlying motion. The adaptive impedance control formula is as follows: ,in, τ For output torque, K p For adaptive stiffness matrix, K d For adaptive damping matrix, q d For the desired position, q For actual location, d For the desired speed, For actual speed, τext External torque; S5. Security Assurance Steps: Through the security assurance module, constraint optimization is applied during the generation, planning, and execution of control strategies in steps S2 to S4 to ensure that security constraints are met; the mathematical expression of the security constraints is: The constraints are ,in, τ nominal Nominal torque, h i This is a safety constraint function.

9. The embodied robot control method based on multimodal perception and deep learning according to claim 8, characterized in that, The deep reinforcement learning control steps include: Actor Network Strategy Update Formula: ,in, θ For actor network parameters, J ( θ Let ) be the objective function. π θ ( a | s ) is the policy function; For the dominant function, V ( s ) is the state value function; The formula for updating the value of a critic's network: ,in This refers to timing difference error; The exploration strategy employs the Ornstein-Uhlenbeck process: Where ε is the mean regression rate, μ This is the long-term mean. σ For volatility, W t This is the Wiener process.

10. The embodied robot control method based on multimodal perception and deep learning according to claim 8, characterized in that, The method also includes a training optimization step: a course learning strategy, the difficulty of which increases with the number of training steps, using the following formula: ; The learning initialization steps are imitated, and the formula is: ,in, a expert For expert actions; meta-learning rapid adaptation steps, the formula is: , where θ 1 ′ To adapt to subsequent parameters, θ For actor network parameters, α L is the learning rate. Ti This is the task loss function.

Images