139 results about "Strategic learning" patented technology

An agent policy learning method with privacy protection in mobile edge computing, which belongs to the technical field of mobile communications. The method comprises: establishing an edge collaborative computing unloading model which comprises a service caching model, a task unloading model and a system cost model; on the basis of the edge collaborative computing unloading model, using the minimization of the processing cost of a task as a target to establish an optimization problem of joint task unloading, service caching, computing power resource allocation and transmission power control; abstracting an optimal problem into a partially observable Markov decision process; and on the basis of the Markov decision process, using a multi-agent deep reinforcement learning algorithm based on a federated learning method to autonomously learn a task unloading, service caching, computing power resource allocation and transmission power control policy. In the present invention, an edge collaborative computing unloading model is established, such that an agent network protects privacy security of data and sensitive information of users by means of training based on federated learning.

The invention provides an intelligent storage robot group cooperative scheduling method based on deep reinforcement learning, and relates to the technical field of intelligent storage, and the method comprises the steps: constructing a group perception module through a hierarchical attention mechanism, and generating a dynamic cooperative perception matrix; establishing a deep reinforcement learning model for strategy learning; and designing a multi-level reward function and optimizing a training process through an adaptive weight adjustment mechanism. According to the invention, the cooperative efficiency of warehouse robot group scheduling is improved, the task conflict rate is reduced, and the adaptability of the system to a complex dynamic environment is enhanced.

The invention provides an unsupervised continuous emotion electroencephalogram analysis method and device based on deep reinforcement learning, and the method comprises the steps: in a prototype learning stage, obtaining sample features of time sequence signals corresponding to sample objects, carrying out the clustering processing of the sample features based on a K-Means algorithm, obtaining the global distribution information of emotion information in the time sequence signals, and carrying out the clustering processing of the global distribution information; in the reinforcement learning stage, a distribution-prototype reward function is determined according to the global distribution information, an Actor-Critic framework is adopted, a near-end strategy optimization algorithm PPO is used, an Actor executes a current action for a given time sequence signal based on a current strategy, Critic determines a current score of the current action according to a distribution-prototype reward fed back by the environment, and the current score of the current action is determined according to the distribution-prototype reward fed back by the environment. And the Actor performs strategy learning based on the current score until a preset iteration termination condition is reached, and a model used for determining key signals in the target time sequence signals according to the target time sequence signals is obtained. Therefore, the accuracy and reliability of the finally obtained model are improved.

The invention is applicable to the technical field of robot motion control, and provides a robot skill learning method, which comprises the following steps of: generating an auxiliary reward component function by utilizing a large language model according to learning prompt information, constructing a multi-branch value network according to a reward component function set formed by a plurality of reward component functions and auxiliary reward component functions in the learning prompt information, and performing iterative strategy training on the strategy network by using the multi-branch value network based on the weight coefficient of each value branch until the target function of the strategy network is converged, so as to obtain the target function of the strategy network. A strategy network is used for guiding robot skill learning, so that the intelligent decision-making capability of a large language model and fine-grained feedback of a multi-branch value network are combined, the strategy learning efficiency and the overall performance in reinforcement learning are improved, it is ensured that different skill obtaining requirements can be met in the strategy optimization process, and the robot skill learning efficiency is improved. And different environments and complex robot tasks can be handled, and the accuracy, precision and efficiency of task execution are improved.

The invention belongs to the technical field of artificial intelligence, and particularly relates to a voice question and answer task type dialogue strategy learning method based on a large model, which comprises the following specific steps: data preprocessing and intention labeling; pre-training a large model according to the intention label; dialogue strategy learning; and the model is iteratively updated through supervised learning and reinforcement learning. According to the method, accurate prediction and adaptation to the user intention can be realized, and the continuity and consistency of the dialogue are improved, so that more accurate and efficient customer service is realized in the task type dialogue, the user experience is enhanced, and the performance of an intelligent question and answer assistant is improved.

The invention provides a multi-agent communication strategy learning method based on human feedback. The method comprises the steps that S1, a multi-agent reinforcement learning model is constructed, modeling is conducted on a multi-agent system, a dynamic communication topological structure in the multi-agent system is constructed, and the overall calculation process of the model is determined; s2, an offline pre-training process of the multi-agent system is carried out, training is carried out only through historical trajectory data in the process, interaction is not participated, and a suboptimal behavior strategy is rapidly learned; s3, collecting human feedback reward data and historical training data of the multi-agent system based on the specific task scene model; and S4, dynamically adjusting a communication strategy, carrying out auxiliary online training according to reward data fed back by human beings, helping the multi-agent system to better carry out communication decision making, and improving learning efficiency. Human feedback data is applied to multi-agent system training, and the limitation of a traditional multi-agent reinforcement learning method is effectively improved.

The invention relates to the field of robot operation and three-dimensional perception, in particular to a monocular image-based three-dimensional perception robot operation knowledge distillation method, which comprises the following steps of: establishing a strategy learning framework comprising a student model and a teacher model; constructing a strategy prediction model in the strategy learning framework; training the strategy prediction model, wherein a three-level knowledge distillation mechanism is adopted in the training process to complete knowledge migration between the teacher model and the student model; combining the three distillation losses with strategy optimization losses to form a total loss function, and performing end-to-end training on the student model until convergence to obtain a deployable monocular strategy model; the deployable monocular strategy model only retains a student model, and generates a robot operation instruction. The method has the beneficial effects that the robot under monocular RGB input has three-dimensional perception and high-precision operation capabilities while the reasoning efficiency is kept, the task success rate and generalization performance are remarkably improved, and the effectiveness and robustness of the method are verified.

The invention provides a bridge group multi-target maintenance decision-making method fusing an evolutionary algorithm and artificial intelligence, and relates to the technical field of civil engineering and artificial intelligence crossing. The method comprises the steps of defining bridge group maintenance cost and structure failure risks, representing preferences of decision makers for different decision targets by weight combinations, and constructing a multi-target maintenance decision optimization model; encoding the weight combination into an individual of a multi-objective evolutionary algorithm, and randomly generating an initial population; aiming at each generation of weight combination, constructing a bridge group Markov decision-making environment; learning an optimal maintenance strategy by adopting an A2C training reinforcement learning agent; the optimal maintenance strategy is evaluated, and an evaluation result is used as individual fitness to be fed back to the multi-objective evolutionary algorithm; using a multi-objective evolutionary algorithm to perform evolutionary search on the multi-objective weight combination; and through a closed-loop feedback mechanism, outputting a Pareto optimal solution set containing an optimal maintenance strategy under various weight combinations, thereby realizing collaborative optimization of weight optimization and strategy learning.

The invention relates to an unmanned aerial vehicle cluster formation decision making and hunting method based on reinforcement learning, and belongs to the technical field of multi-agent formation control and decision making. The expected position corresponding to each agent and the control instruction of the dynamic model of the multi-agent are obtained, the central node does not need to quickly realize the expected formation configuration given by the decision-making system in a distributed mode, and the formation control problem of the multi-agent is solved; secondly, the formation of the multiple agents can be visually manipulated to adapt to dynamically changing tasks, and the formation manipulation problem of the multiple agents is solved; according to an actual confrontation task, learning and evolutionary online autonomous decision-making of the multi-agent system is realized, and the formation autonomous decision-making problem of the multi-agent system is solved; and finally, the interpretability and safety of learning decision are enhanced by utilizing a control model mechanism, and the interpretability and safety problems in the formation strategy learning process are considered.

The invention discloses a multi-agent reinforcement learning method. The method comprises the following steps: step 1, system initialization and edge device modeling; 2, generating a data source and constructing a task package; step 3, edge equipment trajectory planning and movement acquisition; 4, local task execution and unloading strategy decision making; 5, communication link modeling and bandwidth resource allocation are carried out; step 6, scheduling optimization driven by an information age and depreciation mechanism; step 7, multi-agent strategy optimization based on Actor-Critic is carried out; and step 8, reward function design and reinforcement learning process. The technical problems that an existing centralized scheduling or heuristic algorithm cannot perform efficient learning and scheduling in a resource heterogeneous and data dynamic environment, and cannot still have good learning ability and generalization ability under the condition of lack of global information are solved; the method is suitable for system modeling and optimization work of efficient strategy learning and collaborative decision making of multiple agents in a complete cooperation task.

The invention discloses a complex network disintegration method based on evolution deep reinforcement learning. According to the method, an encoder-decoder model fusing a graph convolutional neural network and a deep Q network is constructed, and is used for efficiently extracting importance features of nodes in a complex network and realizing dynamic decision-making of a node disassembling sequence according to the importance features. In order to optimize model parameters and improve search capability, an evolutionary algorithm is introduced to perform global exploration on the model parameters, and the problem that a directional optimization strategy is easy to fall into local optimum is avoided. Meanwhile, deep mining is carried out on an evolution result in combination with a reinforcement learning strategy, the overall optimization process is accelerated, and advantage complementation of parameter evolution and strategy learning is achieved. Experimental results show that the method significantly improves the efficiency and precision of network disassembly while maintaining the robustness of the model, and has good practical value and wide application prospects.

The invention provides a multi-modal fusion virtual simulation experiment teaching system, and relates to the technical field of virtual simulation, and the system comprises an information collection module which is used for obtaining the interaction information of a learner; the information fusion module is used for constructing a multi-modal input data set; the instruction analysis module is used for analyzing and generating an operation instruction sequence; the teaching intervention strategy generation module is used for generating a self-adaptive teaching intervention strategy; the teaching intervention strategy execution module is used for executing a self-adaptive teaching intervention strategy; and the learning effect evaluation module is used for quantifying the learning effect evaluation score. The technical problems that in traditional virtual simulation experiment teaching, multi-modal operation behaviors of learners are difficult to accurately capture in real time, personalized teaching intervention is lacked and learning effect evaluation is incomplete can be solved, and virtual experiment simulation and intelligent teaching intervention driven by multi-modal information fusion are achieved. And the experimental interaction experience and the learning effect evaluation precision are improved.

The invention discloses a digital twinning multi-agent reinforcement learning intelligent decision-making system and method with a secure memory playback mechanism, and the system comprises a digital twinning module which is used for constructing a virtual model and synchronizing the virtual model with a physical entity in real time; the multi-agent reinforcement learning module is used for carrying out strategy learning based on a constrained Markov decision process and balancing performance and safety through a Lagrange multiplier; the safe memory playback module is used for weighting and playing back the experience samples according to the risk and the timeliness so as to improve the learning safety; the reversible grey influence network module is used for causal modeling and reasoning and enhancing decision interpretability; the double-loop self-constraint control module ensures that a control action is always in a physical safety boundary through a barrier function and safety projection; and the convergence and stability criterion module is used for verifying strategy security convergence and system asymptotic stability. According to the method, the problems of strategy border crossing, virtual-real mismatching and the like in the high-risk manufacturing process are solved, and multi-target optimal control under the safety constraint is realized.

The invention provides an incremental reinforcement learning path planning method based on dynamic reward remodeling, which is used for solving the problems of training efficiency and composite optimization in path planning. According to the method, a rapid exploration random tree and an artificial potential field method are combined with reinforcement learning, a dynamic reward function is designed, real-time feedback is provided for an intelligent agent, and strategy learning is accelerated. The system specifically comprises a reward remodeling module which provides process rewards through a fast exploration random tree and an artificial potential field method; the sub-target course learning module is used for gradually increasing task complexity and optimizing sample utilization; the progressive reward adjustment module dynamically adjusts the reward weight and solves the reward coupling problem; and the strategy fine tuning module dynamically adjusts the learning rate and supports strategy optimization. Experimental results show that in static and dynamic path planning tasks, compared with a traditional method, the method has the advantages that the training efficiency and the composite optimization performance are remarkably improved, the reward coupling problem is effectively solved, and the intelligent agent can efficiently learn and converge to a global optimal strategy.

The invention relates to the technical field of multi-objective optimization, in particular to a multi-mechanism deep integrated multi-objective optimization framework and an optimization method and application thereof, comprising a reinforcement learning module, a genetic algorithm module, a bidirectional evolutionary learning coupling module, a Pareto experience pool module, a preference management module and a graph neural network module, the bidirectional evolutionary learning coupling module is used for realizing bidirectional information interaction between the reinforcement learning module and the genetic algorithm module, and the reinforcement learning module generates candidate solutions and feeds the candidate solutions back to the genetic algorithm module; the genetic algorithm module screens the elite solution with the crowding distance larger than a set threshold value in the first Pareto layer and updates the strategy of the feedback reinforcement learning module. According to the method, the strategy learning ability of reinforcement learning and the global population search ability of the genetic algorithm are fused, and the overall optimization efficiency and the approximation ability to the complex Pareto frontier are remarkably improved.

The invention provides a cluster confrontation method and system based on expert knowledge-assisted deep reinforcement learning, and the method and system improve the initial strategy learning speed and overall combat effectiveness of the system by introducing an expert knowledge base and an imitation learning technology and combining deep reinforcement learning to optimize the collaborative decision-making efficiency of an intelligent agent. The method aims at providing an effective initial strategy acquisition mechanism, utilizing an expert knowledge base to accelerate the early strategy learning of the agents, reducing the training time, optimizing the strategies of the agents in a complex dynamic environment through a multi-agent deep reinforcement learning algorithm, and improving the cooperative combat ability. According to the scheme, the time required for initial strategy learning can be greatly shortened, a more optimized strategy is obtained in combination with deep reinforcement learning, the high efficiency of strategy tuning is guaranteed, and then the real-time guarantee in large-scale cluster confrontation is guaranteed.

The invention relates to the technical field of data processing, and discloses a manufacturing industry data intelligent analysis method and system based on deep reinforcement learning. The method comprises the following steps: carrying out time sequence processing on manufacturing industry equipment state, order characteristics, inventory level and quality index data to obtain a four-dimensional production data matrix, carrying out strategy learning through an LSTM-Actor-Critic algorithm to obtain a manufacturing decision strategy network, carrying out classification processing according to a production cycle to obtain a hierarchical data set, constructing an intelligent experience playback buffer area, and carrying out intelligent experience playback. And carrying out collaborative optimization on order scheduling, inventory replenishment and equipment task allocation decisions to obtain a manufacturing industry data intelligent analysis result. The technical problem that an existing manufacturing industry data analysis method lacks adaptive learning ability and cannot process multi-domain collaborative decision optimization is solved.

The invention discloses an adaptive task unloading mechanism driven by meta-reinforcement learning in an edge computing environment. According to the mechanism, a system architecture composed of a user equipment layer and an edge server layer is constructed, and task analysis, state perception and strategy optimization processes are combined to realize unloading scheduling optimization of a multi-task dependent structure. Task unloading is modeled as a Markov decision process, a double-layer training mechanism is adopted, local strategy training is realized by utilizing a near-end strategy optimization algorithm, and the generalization ability of the system is improved in combination with cross-task meta-strategy learning. According to the method, the sequence is fused into the sequence neural network structure and the multi-head attention mechanism, and the accuracy and efficiency of unloading strategy generation are improved while the task dependency relationship is modeled. The mechanism has good task adaptability and delay optimization performance in a dynamic edge computing environment, and is suitable for various mobile computing scenes.

The invention is suitable for the technical field of deep reinforcement learning, and provides a deep reinforcement learning method and system for building energy control, and the method comprises the steps: constructing an intelligent agent based on a target building environment; collecting common experience data when the intelligent agent interacts with the target building environment; constructing a prompt text; inputting the prompt text into the large language model to obtain an action range of the control action; performing correction processing on the common empirical data according to the action range to obtain empirical data based on LLM correction; and according to the common experience pool and the LLM experience pool, training the intelligent agent by minimizing a loss function. According to the method, the action range of the control action in different environment states is analyzed by utilizing the large language model, so that low-quality empirical data in random exploration of the intelligent agent is guided and corrected, the intelligent agent can obtain high-quality experience, and the learning efficiency and the overall performance of the building energy control strategy are improved.

The embodiment of the invention provides an insurance pricing method and device based on strategy learning, electronic equipment and a medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring behavior data of a user, and acquiring target behavior data of the behavior data in a preset time period; determining an insurance premium initial pricing according to the data category of the target behavior data; performing feature extraction on the target behavior data through a multi-view learning model to obtain a feature vector; performing pricing prediction on the premium initial pricing and the feature vector through a strategy network; according to the pricing ratio, the premium variance and the premium initial pricing obtained by the pricing prediction, determining an estimated current pricing; inputting the estimated current pricing, the initial premium pricing and the historical feature vector of the previous time step of the current time step into a preset value network for pricing evaluation, and outputting an evaluated value; and adjusting and estimating the current price according to the evaluation value. According to the embodiment of the invention, the pricing can be dynamically adjusted according to the behavior data of the user.

The invention discloses a multi-AUV safe path planning method based on deep reinforcement learning. According to the method, a deep reinforcement learning algorithm model based on an MATD3 method is adopted, a policy network and a value network are updated by adopting security constraints, the security of policy learning is improved, the expected reward revenue is maximized under the condition that the expected security cost constraints are met, a frequent minimum and maximum optimization process is avoided by adopting first-order penalty optimization, and the security of policy learning is improved. Meanwhile, safety correction based on a safety layer is added in the training process to guarantee safety in the early stage of training, and exploratory and safety balance is brought to strategy optimization of reinforcement learning through safety constraint and safety correction. According to the multi-AUV path planning method, the time cooperation constraint and the space cooperation constraint of multi-AUV path planning are comprehensively considered, the centralized training and decentralized decision-making architecture is applied to multi-AUV path planning, the path planning method capable of ensuring cooperation safety is provided for a multi-AUV system, and the safety and the reliability of path planning are improved.

The invention belongs to the technical field of artificial intelligence, and relates to a strategy learning method and system for Android agent beyond behavior cloning, and the method comprises the steps: 1) carrying out the behavior cloning of an Android agent base model according to expert demonstration data; and 2) constructing a reward function on the basis of the Android agent base model after behavior cloning, and performing off-line reinforcement learning RWR and AWR training on the reward function according to expert demonstration data to obtain a final Android agent. According to the method, a strategy learning method exceeding traditional behavior cloning is designed by combining the advantages of behavior cloning and offline reinforcement learning, compared with a traditional behavior cloning method, the method can learn a better strategy, and compared with implicit behavior cloning, generative adversarial imitation learning and other methods, the method is more suitable for the field of Android agents; moreover, the method occupies less resources, is simple in training mode, does not need to train a discriminator network compared with an offline reinforcement learning method, and is also effective under the condition of sparse rewards.

The invention belongs to the technical field of multi-micro-energy-network cooperative control, and particularly relates to a communication lightweight multi-micro-energy-network cooperative control method for reward recovery assisted by a large model, and the method comprises the following steps: S1, constructing a mathematical model of distributed energy in a micro-energy network, and enabling the mathematical model to be used for describing the dynamic behaviors of energy conversion, storage and constraint; s2, modeling a micro energy network cooperative control problem as a partially observable Markov decision process POMDP; s3, solving the POMDP of the step S2 by using an optimized MAPPO method to obtain a cooperative control scheme; and issuing to each micro-energy network terminal for execution. The method can break through the limitation of the communication bottleneck on the premise of ensuring the learning quality of the control strategy.

The invention belongs to the technical field of intelligent task-based dialogue, and particularly relates to a task-based dialogue strategy learning method and system based on reinforcement learning, which adopts a soft actor commentator algorithm to be combined with behavior cloning and maximum entropy inverse reinforcement learning to relieve a cold start problem. According to the method, maximum entropy inverse reinforcement learning is utilized, a reward value is calculated, and a user target in a dialogue is accurately deduced according to maximum log likelihood estimation of a human dialogue and a simulated dialogue; a potential reward function is extracted from a successful track through inverse reinforcement learning, manual reward design is replaced, accumulated reward maximization is pursued during strategy optimization, diversity and exploratory performance of the strategy are encouraged, and the dialogue strategy is prevented from being converged to a single mode too early.

The invention discloses a federated backdoor defense method based on decoupling contrast learning, and the method comprises the steps: training a backdoor model based on a backdoor sample, and immediately stopping training after the backdoor model converges on the backdoor sample; respectively extracting a penultimate layer vector of the backdoor model and the local model from a sample pair held by the malicious client as a backdoor feature and a clean feature; comparing and learning the separated back door features and the clean features, and learning the clean features for the local model by using a sample weighting strategy to train the local model to obtain a trained local model; and sending local model parameters of the trained local model to a global server, and generating model parameters of a new global model based on the local model parameters through an aggregation function. The method aims at reducing information dependence between backdoor features and clean features through comparative learning, so that local model learning is free of backdoor representation, and the robustness of a global model is improved.

The invention discloses an efficient progressive strategy enhancement (EPPE) framework based on deep reinforcement learning (DRL) and used for a path planning scene, and the framework assists a DRL algorithm and is used for solving dual challenges of training efficiency and combinatorial optimization faced by DRL in path planning. The framework combines the advantages of sparse rewards and process rewards, the sparse rewards help the intelligent agent to realize a global optimal strategy, and the process rewards provide real-time feedback, so that the intelligent agent can adjust the strategy in time. The strategy learning efficiency is remarkably improved, and the reward coupling problem caused by process rewards is effectively solved, so that the global optimization of the strategy is realized. The method can be applied to the research field of robot path planning, and plays an important role in the application fields of automatic driving, logistics distribution and the like.

The invention discloses a multi-agent collaborative decision-making method based on experience screening and individual reward generation. The method comprises the following steps: S1, generating an experience buffer area; the experience buffer area comprises experience blocks in a plurality of rounds; the experience block of each turn comprises actions of a plurality of corresponding time steps; s2, round sampling is carried out in the empirical buffer area, and the local optimal state under multiple time steps in each round is determined according to the individual Q value and the total Q value; s3, internal rewards of the corresponding agents in the current state are confirmed according to the local optimal state, and global rewards are confirmed according to the internal rewards of all the agents; s4, the individual rewards are calculated according to the contribution degree of the single agent to the global rewards, and the individual rewards are used for updating the individual neural network of the agent; according to the method, accurate credit distribution of multiple agents can be realized in a sparse reward environment, and strategy learning is guided.

The invention specifically discloses an end-to-end control system and method fusing multi-modal sensing and a strategy collaborative optimization mechanism. The system comprises a multi-modal sensing module; the key frame selection module and the time sequence modeling module are used for dynamically screening key frames from the historical sensing frames and carrying out time sequence modeling and feature coding by adopting a time sequence Transform model to obtain time sequence state features with semantic continuity; the strategy optimization module is used for constructing an Actor-Critic strategy network based on a maximum entropy reinforcement learning algorithm; the system is composed of a multi-mode sensing module, a time sequence modeling module, a strategy learning module and an end-to-end control module, a sensing-decision-control integrated closed-loop structure is formed, the system takes multi-source sensor data in a real traffic scene as input, and the multi-source sensor data in the real traffic scene are subjected to uniform feature coding and time sequence modeling, so that the real traffic scene is obtained. The method has the advantages of being short in information processing link, high in response speed, high in adaptive capacity and the like.

The invention relates to the crossing field of water management and artificial intelligence. The method disclosed by the invention solves the problems of difficulty in coordination of multi-subject benefit conflicts, insufficient utilization of a basin spatial topological structure, separation of physical constraints and decision-making processes, limitation of adaptive ability and the like in the prior art. The method is characterized by comprising the steps of abstracting a drainage basin into a directed graph structure; generating a node embedding vector by using a physically guided graph attention network, and explicitly introducing a physical constraint factor in an attention mechanism; configuring an intelligent agent for each sub-basin to perform distributed strategy learning; cooperative training is carried out by adopting a multi-agent depth deterministic strategy gradient algorithm, fair distribution is guided by a global reward function in a Nash product form, and physical constraints such as water balance and the like are ensured in combination with a local reward function containing a nonlinear physical penalty term. According to the method, distributed, physically consistent and self-adaptive optimal configuration of drainage basin water resources is realized, and the method is mainly used for improving fairness, efficiency and feasibility of water resource distribution.

The invention discloses a robust self-adaptive multi-skill learning method for a quadruped robot based on key frame guidance. The robust self-adaptive multi-skill learning method comprises the following steps: determining a target key frame set of multi-skill learning; in a flat terrain environment, probability sampling is carried out on the key frames according to skill task values to generate skill instructions, the quadruped robot is controlled to execute actions, and training tracks are collected and stored in a strategy learning experience playback pool and a self-simulation learning experience playback pool respectively. Training the strategy network and the value function network through strategy learning circulation; a self-mimicking learning discriminator network is trained through a self-mimicking learning loop to extract a high quality trajectory. And after the flat terrain training is completed, entering a complex terrain environment, and carrying out adaptive training under the guidance of the trained strategy network and the self-simulation learning discriminator network so as to realize the migration of the skill primitives to the complex environment. According to the method, multi-skill learning and terrain adaptive capacity can be effectively considered, and the robustness, flexibility and multi-task execution performance of the quadruped robot are improved.