612 results about "Imitation learning" patented technology

The invention discloses an energy efficiency data detection processing method and system for a data center, and the method comprises the steps: deploying a sensor and a distributed collection network, and achieving the collection of energy efficiency data through the calculation of an energy efficiency change rate and the adjustment of a self-adaptive sampling frequency; dynamic energy efficiency prediction is realized by adopting adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion; an energy efficiency perception load evaluation index is introduced to realize energy efficiency maximization; an intelligent group decision model is constructed, and global allocation of computing resources is realized through dynamic balance scheduling optimization, task allocation stability analysis and adaptive convergence adjustment; based on an energy efficiency autonomous learning framework, initial strategy training, reinforcement learning optimization scheduling strategy, adaptive tuning and dynamic adjustment are carried out by utilizing imitation learning, and adaptive optimization is realized. According to the invention, under the condition that the business of the data center changes rapidly, the allocation of the computing resources can be adjusted accurately, rapidly and dynamically, and the optimal energy efficiency is achieved.

The invention relates to the technical field of information retrieval, and discloses a multi-modal fusion and reinforcement learning collaborative retrieval enhancement generation method and system. The method comprises the following steps: receiving an original query input by a user, and generating a sub-query based on a large language model in combination with a multi-modal context of a current iteration step; forming a current state in combination with the sub-query and the multi-modal context, modeling a retrieval enhancement generation task as a Markov decision process, and adaptively selecting an optimal action from a predefined action set in the current state by utilizing a large language model according to a decision strategy; executing a corresponding multi-modal retrieval operation according to the optimal action, fusing the obtained multi-modal information, generating an intermediate answer or a final answer of the sub-query, and updating a multi-modal context by using the intermediate answer; off-line training optimization is carried out on the large language model through imitation learning and a calibration chain, and decision strategies and sub-queries are inferred online through the model after fine adjustment. According to the invention, more efficient and accurate complex query processing is realized.

The invention provides an operation control model training method and a robot operation control method and system, and the method comprises the steps: carrying out the reinforcement learning training of an initial strategy network according to the first sample state information of a simulation robot and a reward function corresponding to each action task, and obtaining a target strategy network corresponding to each action task; according to second sample state information of the simulation robot, a task label corresponding to a preset training task and expected action information corresponding to the second sample state information output by a target strategy network corresponding to each action task, carrying out joint training of transfer learning, reinforcement learning training and imitation learning on the initial operation control model; and constructing an operation control model according to the trained initial operation control model. Through joint training of multiple training tasks, smooth switching among different action tasks of the robot can be ensured while multi-task compatible processing of the robot can be realized, and the operation control performance of the robot is effectively improved.

The invention discloses a visual servo double-arm robot migration simulation learning method from simulation to reality, which comprises the following steps: S110, constructing a virtual simulation scene according to a real task scene, and establishing a relation between the virtual simulation scene and the real task scene; s120, the collected teaching mechanical arm and executing mechanical arm operation data are replayed and optimized in the virtual simulation scene, so that a data set used for final training is constructed; s130, a deep imitation learning network is designed and achieved, input of the deep imitation learning network comprises visual data obtained in the real task scene and state data of all joint motors of the teaching mechanical arm, and output of the deep imitation learning network is predicted states of all joint motors of the execution mechanical arm at the next moment; and S140, migrating the fully trained and converged deep imitation learning network from a virtual simulation scene to a double-arm robot in a real task scene. According to the invention, the exploration efficiency and generalization ability of the two-arm robot learning network are improved.

The invention discloses a virtual power plant scheduling method based on a large language model and deep reinforcement learning, and belongs to the technical field of virtual power plant scheduling. Comprising the following steps: constructing a virtual power plant multi-agent cloud edge collaborative scheduling framework based on large language model driving; predicting wind power, photovoltaic power and load power based on a large language model; constructing a mathematical model of virtual power plant optimization scheduling; converting the virtual power plant optimization scheduling model into a Markov game process in combination with a large language model; performing initialization training on the strategy network of the edge layer intelligent agent by adopting imitation learning to obtain a pre-trained edge layer intelligent agent strategy network; and based on the pre-training strategy network of the boundary layer intelligent agent, combining with a large language model and adopting an improved multi-agent near-end strategy optimization algorithm to solve a scheduling strategy.

The invention provides a multi-mode intelligent robot control method and device, and the method comprises the steps: obtaining a current mode instruction which at least comprises one of a video instruction, a text instruction, a picture instruction and an audio instruction; on the basis of a pre-trained multi-modal large model, according to the current modal instruction, obtaining post-collapse characterization; on the basis of a pre-trained robot imitation learning network, according to the representation after collapse and current environment observation, predicting an output target action; the multi-mode intelligent robot with the body is controlled to operate according to the target action; wherein the robot imitation learning network is obtained by performing training optimization according to a training sample set formed by single-modal data samples, corresponding environment observation data and real actions of the robot. According to the method, the post-collapse representation is obtained by pre-training the multi-modal large model, and the post-collapse representation is utilized to predict the target action, so that the requirement on multi-modal manual annotation data when a training robot simulates a learning network is avoided in the process, the effect of multi-modal reasoning can be achieved only according to single-modal data training, and the efficiency of multi-modal reasoning is improved. Not only is data collection and labeling cost reduced, but also the reasoning ability of the model is improved.

The invention belongs to the technical field of robot learning, and particularly discloses a robot continuous imitation learning method and system based on a double diffusion model. Comprising the following steps: generating old task data in a submerged space by utilizing a pre-trained latent diffusion model, generating a model through guidance of a task identifier in combination with conditional diffusion, generating an image and state data related to an old task, modeling to construct a diffusion model, realizing multi-modal mapping from a state to an action, and generating a diversified action sequence; high-quality images and state data are generated through a pre-trained variational auto-encoder and a conditional diffusion model, and the consistency of the generated data in time and the accuracy of global semantics are ensured through time sequence continuity constraint and semantic consistency constraint; and performing feedback optimization training on the latent diffusion model. According to the method, the loss weight is dynamically adjusted, and the model is ensured to show good generalization ability and stability in a complex task in combination with collaborative training of an imitation learning strategy and a generator.

The invention provides a mechanical arm grabbing method and system based on multi-modal information fusion, and belongs to the technical field of robot intelligent control. Comprising the steps that the conversion relation between a camera coordinate system and a mechanical arm base coordinate system is established through camera calibration, a deep learning neural network is used for conducting grabbing pose estimation on an obtained RGB-D image, and multiple candidate grabbing poses are determined; analyzing a natural language instruction input by a user based on a multi-modal large model, and recognizing a target object region from the RGB-D image by combining a target detection and image segmentation technology; based on the obtained candidate grabbing poses and the target object area, an optimal grabbing pose is screened through a scoring mechanism and mapped to a mechanical arm base coordinate system; and then a dynamic grabbing path is generated by adopting an imitation learning algorithm, and the mechanical arm is controlled to execute grabbing operation. Through multi-modal semantic understanding, accurate grabbing of the mechanical arm in a complex environment can be achieved.

The embodiment of the invention provides an automatic driving method and device based on a multi-dimensional reward function, and the method and device achieve the control of a driving motion through the combination of an imitation learning framework and a reinforcement learning framework, collection of environment information through a plurality of cameras, and construction of a strategy generation network and a discriminator network. A multi-dimensional reward function model is designed, behaviors such as red light running, line pressing, lane departure and collision are detected and evaluated in real time, and a driving behavior reward and punishment matrix is constructed. Based on an actor evaluation network architecture, environment information and a navigation instruction are input into an actor network to generate an optimal driving action, and the action value is evaluated through the evaluation network to realize dynamic parameter optimization. According to the method, the defects of the traditional technology in the aspects of driving behavior evaluation, action value judgment and the like are effectively overcome, and the safety and reliability of the automatic driving system are remarkably improved.

The invention belongs to the technical field of mechanical arm control, and discloses a mobile mechanical arm control method based on depth element imitation learning, which introduces a depth element imitation learning (DMIL) algorithm, and the algorithm combines depth element learning and imitation learning (IL) technologies to improve the adaptability and efficiency of a mobile mechanical arm in such tasks. The expert trajectory is elaborately made and carefully checked in a AppeliaSim simulation environment, and the interaction of the real world is evaluated at the same time. The position and the direction of the button lock are determined by adopting a deep learning 6D attitude estimation method, and key operation reference points are visually identified. In the whole IL stage, through integration of resistance inverse reinforcement learning (AIRL) and variable impedance control (VIC) technologies, a strategy for controlling operation of the mechanical arm is enhanced. The expert-guided trajectory and tangible force feedback data from real-world settings enhance this.

The invention belongs to the related technical field of artificial intelligence, and discloses a robot time sequence imitation learning method and system based on a Mama coding complete history, and the method comprises the steps: receiving a multi-modal observation sequence in a task execution process of a robot, the multi-modal observation sequence comprising observation data of at least one sensor; processing the multi-modal observation sequence by using a sequence processing module based on a state space model, and updating a time sequence output of complete historical information of one code up to the current time step at each time step; and predicting the next step or a series of future actions of the robot based on the time sequence output of the current time step so as to control the robot to simulate. The time sequence processing module based on the state space model is utilized to process and encode the complete observation history in the task execution process of the robot, so that a non-Markov decision-making imitation learning method is realized, and the learning efficiency and the execution success rate of the robot in a complex and state-dependent long time sequence operation task are improved.

The invention belongs to the field of power distribution networks, and discloses a power distribution network and micro-grid collaborative optimization scheduling method and a related device, and the method comprises the steps: obtaining the state observation data of a power distribution network intelligent body and each micro-grid intelligent body; adopting a mixed multi-agent soft actor commentator algorithm, based on a pre-trained strategy network, according to the state observation data of the power distribution network agent and each micro-grid agent, solving a preset power distribution and micro-grid collaborative optimization model, and obtaining a control strategy of the power distribution network and each micro-grid; wherein the pre-trained strategy network is obtained through the following steps: designing the distribution microgrid collaborative optimization model into a reinforcement learning model in a multi-agent environment, and pre-training a strategy network of the reinforcement learning model based on a generative adversarial imitation learning mode to obtain the pre-trained strategy network. The method solves the problem that an existing method for solving the problem of cooperative optimization scheduling of the power distribution and micro-grids oriented to high-proportion distributed resource access is not economical, efficient and reliable, and improves the efficiency and effect of cooperative optimization scheduling of the power distribution and micro-grids.

The invention discloses a force feedback exoskeleton and a humanoid robot imitation learning method. The force feedback exoskeleton comprises a bearing part, an arm exoskeleton part and a data glove part. The number of the arm exoskeleton parts is two, the two arm exoskeleton parts are symmetrically connected to the backpack part, and the number of the data glove parts is two so as to collect position and posture information of the left hand and the right hand respectively. The humanoid robot imitation learning method is based on same double-arm characteristics and operator visual angle image input, is combined with nodal topology correlation organization of human joints, integrates multi-modal data information, and outputs control quantity which is almost the same as expert operation. The system is applied to the technical field of exoskeletons, high-quality data acquisition can be realized while the exoskeleton is lightweight and integrated, the number of degrees of freedom of data acquisition of traditional exoskeletons is greatly increased, and the system can be used for guiding collaborative movement of double mechanical arms and further effectively applied to flexible operation tasks of humanoid robots.

The invention discloses a self-adaptive memory enhancement imitation learning method for a robot control task, which comprises the following steps of: performing standardization processing and time sequence feature extraction on original multi-modal data, and constructing hierarchical feature representation; establishing a layered structure including short-term memory, working memory and long-term memory, and evaluating memory importance and dynamically updating memory content based on use frequency, task correlation and novelty; retrieving related memories according to the current state and carrying out multi-modal feature fusion; a shaping reward is calculated based on neighbor sample search; and optimizing strategy network parameters through a multi-loss function combination. Through adaptive memory management and a multi-modal feature fusion mechanism, the learning efficiency and execution performance of the robot control system are improved.

The invention discloses a power grid safety enhanced intelligent scheduling method and device based on twinborn adversarial imitation learning and a storage medium, and belongs to the technical field of power scheduling. The method comprises the following steps: acquiring current real-time state information of a power grid system; inputting the real-time state information into a pre-trained scheduling strategy agent to generate a corresponding real-time scheduling action; and judging whether the real-time scheduling action in the current state is an expert-level action or not based on a state action data pair pool formed by the scheduling strategy agent in a training stage, and outputting a final scheduling strategy to a real environment according to a judgment result. Through real-time data analysis, intelligent decision and expert verification, the method can automatically adapt to the complexity and dynamic nature of the power grid, not only realizes efficient, safe and stable operation of the power grid, but also provides powerful support for intelligent management of the power grid.

In various examples, systems and methods are disclosed relating to training machine learning models using human demonstration of segments of a task, where other segments of the task are performed by a planning method, such as a Task and Motion Planning (TAMP) system. A method may include segmenting a task to be performed by a robot into segments, determining a first set of instructions of a plurality of sets of instructions for operating the robot to perform a first objective of a first segment, determining that the plurality of sets of instructions is inadequate to perform a second objective of a second segment, receiving from a user device a second set of instructions for operating the robot for the second segment following an end of the first segment, and updating a machine learning model for controlling the robot using the second set of instructions for the second segment.

The invention discloses a reinforcement learning method and system, and relates to the field of reinforcement learning, and the method comprises the steps: obtaining suboptimal demonstration data in a target application scene; performing imitation learning through the intelligent agent according to the suboptimal demonstration data to obtain an initial strategy; determining a plurality of behavior sequences based on the initial strategy; performing iterative training on the reward model according to the plurality of behavior sequences to obtain a trained reward model; based on the trained reward model, introducing a regular term into a contrast reinforcement learning algorithm of KL divergence to obtain a contrast reinforcement learning loss function with the regular term; and according to the plurality of behavior sequences, based on the trained reward model and the contrast reinforcement learning loss function with the regular term, performing reinforcement learning to obtain a final reinforcement learning strategy in the target application scene. According to the method, the reward model and the loss function of contrast reinforcement learning are improved, and after the agent simulates and learns the suboptimal demonstration data, a better strategy can be quickly learned.

The invention discloses a visual language navigation method based on memory driving, and the method comprises the following steps: S1, constructing an MDQT model which comprises a panoramic encoder, a text embedding layer, a Q-Former, an action prediction module and a memory updating module; s2, training an MDQT model by using three pre-training tasks of image-concerned mask language modeling, instruction track matching and instruction track contrast learning; and S3, performing fine tuning on the MDQT model by using imitation learning and reinforcement learning. According to the invention, a learnable memory vector with a fixed length is used to encode historical information. In each navigation step, the memory vector interacts with the extracted panoramic information and instruction information, and visual information most relevant to the current instruction information is extracted according to the current memory state of the robot for decision making. The memory of the robot to the historical navigation steps is effectively maintained under limited resources, and the navigation success rate and efficiency of the robot are improved.

The invention relates to a humanoid robot imitation learning method and device, computer equipment and a storage medium. The method comprises the steps of obtaining training data; according to the training data, training an initial model to obtain an imitation learning model; enabling a target humanoid robot to execute an action according to the imitation learning model; wherein the training data is data acquired by taking the simulation robot as a center when the simulation robot simulates an original object, and the training data comprises an actual joint position and an actual tail end force. According to the invention, the simulation learning model can be closer to the action habits of human beings, and the control of the humanoid robot can be more accurate and meticulous.

The disclosed concepts relate to training a machine learning model to provide help sessions during a video game. For instance, prior video game data from help sessions provided by human users can be filtered to obtain training data. Then, a machine learning model can be trained using approaches such as imitation learning, reinforcement learning, and / or tuning of a generative model to perform help sessions. Then, the trained machine learning model can be employed at inference time to provide help sessions to video game players.

The invention relates to a robot skill continuous learning method and device based on imitation learning, and the method comprises the steps: sequentially aiming at each task in a plurality of tasks; before a preset termination condition is met, circularly executing the following operations: inputting the observation image, the state information, a pre-acquired text instruction of the current task and a skill code book of the current task into a simulation learning strategy network, and predicting to obtain action information of the robot at the current moment, the skill code book of the current task is a plurality of pre-initialized learnable sub-skill vectors, and the skill code book comprises skill information of the skill code book of the previous task under the condition that the previous task exists; calculating loss based on the predicted action information of the robot and the real action information of the robot in the expert demonstration data; and updating the imitation learning strategy network and the skill code book of the current task based on the loss. The problems that skill selection is not flexible, and it is difficult to abstract the sub-skill hierarchical structure through manual skill design are solved.

A robot control method based on tactile prediction pre-training comprises the following steps: acquiring and generating a human playing data set consisting of three-channel image tensors in an offline stage, and training a constructed conditional diffusion model comprising a tactile encoder, a tactile decoder and an action and visual encoder; in the online stage, the trained conditional diffusion model is integrated into a standard imitation learning strategy network, and an action instruction of the robot is generated according to the state of the robot, the current visual features and the tactile feature vectors extracted by the imitation learning strategy network. According to the method, a specific agent task is completed by training a deep neural network model, that is, a future tactile signal sequence is predicted according to historical information and future action intentions; the model is enabled to characterize generic haptic features contacting physical dynamic laws for further migration into downstream robot control tasks.

According to the humanoid robot multi-motion switching control method and system based on imitation learning, imitation learning is performed on the basis of a generative adversarial network, and the sampling probability of each motion skill is dynamically adjusted according to the performance condition of the humanoid robot for executing each motion skill; according to the method, the humanoid robot can uniformly master different motion skills, the humanoid robot can integrate different motion skill combinations, the severity of the mode collapse problem is effectively relieved, and good flexibility and expandability are achieved.

The invention provides a driving training teaching method and system based on imitation learning, and relates to the technical field of intelligent driving, and the method comprises the steps: obtaining the driving behavior demonstration data of a plurality of human drivers in a plurality of driving scenes, and extracting the driving behavior characteristics in the driving behavior demonstration data; calculating a first feature expectation of an actual driving behavior trajectory of the human driver based on the driving behavior demonstration data; a reinforcement learning algorithm is adopted to optimize a driving strategy of the intelligent agent, the intelligent agent executes the driving strategy in a simulation environment to obtain a simulation driving behavior track of the intelligent agent, and a second feature expectation is calculated; alternately optimizing and updating the parameters of the implicit reward function by minimizing the difference between the first feature expectation and the second feature expectation and combining a generative adversarial network; and applying the intelligent agent driving strategy to the auxiliary driving decision of the intelligent agent.

The invention provides a new energy power grid prospective scheduling method and device, and relates to the technical field of electric power system intraday economic scheduling. The method comprises the following steps: firstly, constructing an opportunity constraint optimal power flow model of prospective scheduling, analyzing the influence of new energy uncertainty on opportunity constraint, establishing a constrained Markov decision process of prospective scheduling, and then utilizing a risk evaluator network fitting risk function probability distribution and an actuator network considering extreme scene performance to determine the opportunity constraint optimal power flow model of prospective scheduling. The processing capability of the intelligent agent on a prospective scheduling scene containing a new energy extreme climbing event is enhanced; and finally, the training of the intelligent agent is accelerated by utilizing an imitation learning technology in a power grid prospective scheduling off-line simulation environment. According to the method, the solving speed and the strategy robustness and safety of the double-layer robust optimization model of the look-ahead scheduling can be considered.

The invention discloses a zero-sample cross-domain imitation learning method based on three-dimensional semantic point cloud and related equipment, and the method comprises the steps: constructing a training scene in a physical simulation engine, randomly generating an operation object pose, and enabling a robot to act to complete an operation task; an original image is observed, semantic segmentation privilege information is obtained by utilizing a simulation environment, an operation object is segmented, and semantic point cloud of the operation object is obtained by combining a depth map generated by the simulation environment; synchronously recording robot joint angles, and constructing a training data set; inputting the semantic point cloud of the operation object and the training data set into an imitation learning model, performing action blocking, extracting features, performing feature splicing, generating a predicted action sequence, and completing model training; deploying the trained model to a real environment, obtaining the semantic point cloud of an operation object, reading the current joint angle of the robot, sending the angle to the model for reasoning, outputting a robot action sequence, and completing a specified task. According to the method, the generalization ability of the strategy can be improved while the operation precision is ensured.

The invention discloses an unmanned aerial vehicle autonomous navigation method based on imitation learning and reinforcement learning fusion, and belongs to the technical field of unmanned aerial vehicle processing, and the method comprises the following steps: S1, collecting a data set of an unmanned aerial vehicle, and carrying out normalization processing on the data set to obtain a standard data set; s2, constructing an Actor network by utilizing expert operation data of the standard data set; s3, inputting the depth image and the state information of the standard data set into an Actor network, and generating a pre-judgment action; and S4, inputting the pre-judgment action into a discriminator to complete autonomous navigation. Through introduction of the generative adversarial imitation learning module, the unmanned aerial vehicle can continuously optimize a navigation strategy, and high robustness and adaptability can be maintained even in a complex environment.

The invention discloses an unmanned aerial vehicle navigation obstacle avoidance method based on selective imitation enhanced deep reinforcement learning, imitation learning and deep reinforcement learning are combined to be used for fixed-wing unmanned aerial vehicle navigation and obstacle avoidance, and an intelligent agent directly maps non-ideal sensor data into a control instruction by learning an end-to-end strategy. For a sparse reward scene, an artificial potential field is constructed to generate a non-expert guidance action by accessing privilege state information which cannot be acquired by an intelligent agent, and a navigation strategy is continuously optimized according to an observation state and a sparse reward signal through real-time interaction with an environment and fusion of imitation learning and deep reinforcement learning. A Q-value-driven dynamic decision-making mechanism is adopted, and the motion of the fixed-wing unmanned aerial vehicle is determined by comparing the motion suggested by the instructor with the expected income of the motion generated by the learner, so that exploration-utilization balance is realized, and invalid exploration and local optimal decision are effectively avoided.

The invention relates to the technical field of edge computing, and discloses a self-adaptive pervasive edge computing task unloading method based on imitation learning, and the method comprises the steps: firstly constructing a task unloading strategy imitation model and an environment adaptability evaluation model, the former is used for learning and simulating a historical task unloading strategy, and the latter is used for evaluating the influence of environment change on the strategy; a deep reinforcement learning algorithm is introduced to train a simulation model, and real-time feedback of an environment evaluation model is combined to continuously optimize and adjust an unloading strategy so as to generate a highly adaptive task unloading decision. According to the method, the task unloading instruction can be intelligently generated and executed according to the real-time edge computing environment state and the task characteristics, the dynamic change of the environment is effectively coped with, and efficient utilization of resources and rapid execution of tasks are achieved. Compared with a traditional method depending on a static rule or a predefined strategy, the task unloading flexibility is remarkably improved, and powerful support is provided for intelligent application in the edge computing environment.

Implementations described herein relate to training and refining robotic control policies using imitation learning techniques. A robotic control policy can be initially trained based on human demonstrations of various robotic tasks. Further, the robotic control policy can be refined based on human interventions while a robot is performing a robotic task. In some implementations, the robotic control policy may determine whether the robot will fail in performance of the robotic task, and prompt a human to intervene in performance of the robotic task. In additional or alternative implementations, a representation of the sequence of actions can be visually rendered for presentation to the human can proactively intervene in performance of the robotic task.