516 results about "Imitation learning" patented technology

The invention discloses an energy efficiency data detection processing method and system for a data center, and the method comprises the steps: deploying a sensor and a distributed collection network, and achieving the collection of energy efficiency data through the calculation of an energy efficiency change rate and the adjustment of a self-adaptive sampling frequency; dynamic energy efficiency prediction is realized by adopting adaptive time window adjustment, short-term and long-term error feedback correction, energy efficiency trend modeling and multi-level prediction fusion; an energy efficiency perception load evaluation index is introduced to realize energy efficiency maximization; an intelligent group decision model is constructed, and global allocation of computing resources is realized through dynamic balance scheduling optimization, task allocation stability analysis and adaptive convergence adjustment; based on an energy efficiency autonomous learning framework, initial strategy training, reinforcement learning optimization scheduling strategy, adaptive tuning and dynamic adjustment are carried out by utilizing imitation learning, and adaptive optimization is realized. According to the invention, under the condition that the business of the data center changes rapidly, the allocation of the computing resources can be adjusted accurately, rapidly and dynamically, and the optimal energy efficiency is achieved.

The invention relates to the technical field of information retrieval, and discloses a multi-modal fusion and reinforcement learning collaborative retrieval enhancement generation method and system. The method comprises the following steps: receiving an original query input by a user, and generating a sub-query based on a large language model in combination with a multi-modal context of a current iteration step; forming a current state in combination with the sub-query and the multi-modal context, modeling a retrieval enhancement generation task as a Markov decision process, and adaptively selecting an optimal action from a predefined action set in the current state by utilizing a large language model according to a decision strategy; executing a corresponding multi-modal retrieval operation according to the optimal action, fusing the obtained multi-modal information, generating an intermediate answer or a final answer of the sub-query, and updating a multi-modal context by using the intermediate answer; off-line training optimization is carried out on the large language model through imitation learning and a calibration chain, and decision strategies and sub-queries are inferred online through the model after fine adjustment. According to the invention, more efficient and accurate complex query processing is realized.

The invention discloses a visual servo double-arm robot migration simulation learning method from simulation to reality, which comprises the following steps: S110, constructing a virtual simulation scene according to a real task scene, and establishing a relation between the virtual simulation scene and the real task scene; s120, the collected teaching mechanical arm and executing mechanical arm operation data are replayed and optimized in the virtual simulation scene, so that a data set used for final training is constructed; s130, a deep imitation learning network is designed and achieved, input of the deep imitation learning network comprises visual data obtained in the real task scene and state data of all joint motors of the teaching mechanical arm, and output of the deep imitation learning network is predicted states of all joint motors of the execution mechanical arm at the next moment; and S140, migrating the fully trained and converged deep imitation learning network from a virtual simulation scene to a double-arm robot in a real task scene. According to the invention, the exploration efficiency and generalization ability of the two-arm robot learning network are improved.

The invention discloses a virtual power plant scheduling method based on a large language model and deep reinforcement learning, and belongs to the technical field of virtual power plant scheduling. Comprising the following steps: constructing a virtual power plant multi-agent cloud edge collaborative scheduling framework based on large language model driving; predicting wind power, photovoltaic power and load power based on a large language model; constructing a mathematical model of virtual power plant optimization scheduling; converting the virtual power plant optimization scheduling model into a Markov game process in combination with a large language model; performing initialization training on the strategy network of the edge layer intelligent agent by adopting imitation learning to obtain a pre-trained edge layer intelligent agent strategy network; and based on the pre-training strategy network of the boundary layer intelligent agent, combining with a large language model and adopting an improved multi-agent near-end strategy optimization algorithm to solve a scheduling strategy.

The invention belongs to the technical field of robot learning, and particularly discloses a robot continuous imitation learning method and system based on a double diffusion model. Comprising the following steps: generating old task data in a submerged space by utilizing a pre-trained latent diffusion model, generating a model through guidance of a task identifier in combination with conditional diffusion, generating an image and state data related to an old task, modeling to construct a diffusion model, realizing multi-modal mapping from a state to an action, and generating a diversified action sequence; high-quality images and state data are generated through a pre-trained variational auto-encoder and a conditional diffusion model, and the consistency of the generated data in time and the accuracy of global semantics are ensured through time sequence continuity constraint and semantic consistency constraint; and performing feedback optimization training on the latent diffusion model. According to the method, the loss weight is dynamically adjusted, and the model is ensured to show good generalization ability and stability in a complex task in combination with collaborative training of an imitation learning strategy and a generator.

The invention provides a mechanical arm grabbing method and system based on multi-modal information fusion, and belongs to the technical field of robot intelligent control. Comprising the steps that the conversion relation between a camera coordinate system and a mechanical arm base coordinate system is established through camera calibration, a deep learning neural network is used for conducting grabbing pose estimation on an obtained RGB-D image, and multiple candidate grabbing poses are determined; analyzing a natural language instruction input by a user based on a multi-modal large model, and recognizing a target object region from the RGB-D image by combining a target detection and image segmentation technology; based on the obtained candidate grabbing poses and the target object area, an optimal grabbing pose is screened through a scoring mechanism and mapped to a mechanical arm base coordinate system; and then a dynamic grabbing path is generated by adopting an imitation learning algorithm, and the mechanical arm is controlled to execute grabbing operation. Through multi-modal semantic understanding, accurate grabbing of the mechanical arm in a complex environment can be achieved.

The embodiment of the invention provides an automatic driving method and device based on a multi-dimensional reward function, and the method and device achieve the control of a driving motion through the combination of an imitation learning framework and a reinforcement learning framework, collection of environment information through a plurality of cameras, and construction of a strategy generation network and a discriminator network. A multi-dimensional reward function model is designed, behaviors such as red light running, line pressing, lane departure and collision are detected and evaluated in real time, and a driving behavior reward and punishment matrix is constructed. Based on an actor evaluation network architecture, environment information and a navigation instruction are input into an actor network to generate an optimal driving action, and the action value is evaluated through the evaluation network to realize dynamic parameter optimization. According to the method, the defects of the traditional technology in the aspects of driving behavior evaluation, action value judgment and the like are effectively overcome, and the safety and reliability of the automatic driving system are remarkably improved.

The invention belongs to the technical field of mechanical arm control, and discloses a mobile mechanical arm control method based on depth element imitation learning, which introduces a depth element imitation learning (DMIL) algorithm, and the algorithm combines depth element learning and imitation learning (IL) technologies to improve the adaptability and efficiency of a mobile mechanical arm in such tasks. The expert trajectory is elaborately made and carefully checked in a AppeliaSim simulation environment, and the interaction of the real world is evaluated at the same time. The position and the direction of the button lock are determined by adopting a deep learning 6D attitude estimation method, and key operation reference points are visually identified. In the whole IL stage, through integration of resistance inverse reinforcement learning (AIRL) and variable impedance control (VIC) technologies, a strategy for controlling operation of the mechanical arm is enhanced. The expert-guided trajectory and tangible force feedback data from real-world settings enhance this.

The invention belongs to the related technical field of artificial intelligence, and discloses a robot time sequence imitation learning method and system based on a Mama coding complete history, and the method comprises the steps: receiving a multi-modal observation sequence in a task execution process of a robot, the multi-modal observation sequence comprising observation data of at least one sensor; processing the multi-modal observation sequence by using a sequence processing module based on a state space model, and updating a time sequence output of complete historical information of one code up to the current time step at each time step; and predicting the next step or a series of future actions of the robot based on the time sequence output of the current time step so as to control the robot to simulate. The time sequence processing module based on the state space model is utilized to process and encode the complete observation history in the task execution process of the robot, so that a non-Markov decision-making imitation learning method is realized, and the learning efficiency and the execution success rate of the robot in a complex and state-dependent long time sequence operation task are improved.

The invention belongs to the field of power distribution networks, and discloses a power distribution network and micro-grid collaborative optimization scheduling method and a related device, and the method comprises the steps: obtaining the state observation data of a power distribution network intelligent body and each micro-grid intelligent body; adopting a mixed multi-agent soft actor commentator algorithm, based on a pre-trained strategy network, according to the state observation data of the power distribution network agent and each micro-grid agent, solving a preset power distribution and micro-grid collaborative optimization model, and obtaining a control strategy of the power distribution network and each micro-grid; wherein the pre-trained strategy network is obtained through the following steps: designing the distribution microgrid collaborative optimization model into a reinforcement learning model in a multi-agent environment, and pre-training a strategy network of the reinforcement learning model based on a generative adversarial imitation learning mode to obtain the pre-trained strategy network. The method solves the problem that an existing method for solving the problem of cooperative optimization scheduling of the power distribution and micro-grids oriented to high-proportion distributed resource access is not economical, efficient and reliable, and improves the efficiency and effect of cooperative optimization scheduling of the power distribution and micro-grids.

The invention discloses a visual language navigation method based on memory driving, and the method comprises the following steps: S1, constructing an MDQT model which comprises a panoramic encoder, a text embedding layer, a Q-Former, an action prediction module and a memory updating module; s2, training an MDQT model by using three pre-training tasks of image-concerned mask language modeling, instruction track matching and instruction track contrast learning; and S3, performing fine tuning on the MDQT model by using imitation learning and reinforcement learning. According to the invention, a learnable memory vector with a fixed length is used to encode historical information. In each navigation step, the memory vector interacts with the extracted panoramic information and instruction information, and visual information most relevant to the current instruction information is extracted according to the current memory state of the robot for decision making. The memory of the robot to the historical navigation steps is effectively maintained under limited resources, and the navigation success rate and efficiency of the robot are improved.

The invention relates to a humanoid robot imitation learning method and device, computer equipment and a storage medium. The method comprises the steps of obtaining training data; according to the training data, training an initial model to obtain an imitation learning model; enabling a target humanoid robot to execute an action according to the imitation learning model; wherein the training data is data acquired by taking the simulation robot as a center when the simulation robot simulates an original object, and the training data comprises an actual joint position and an actual tail end force. According to the invention, the simulation learning model can be closer to the action habits of human beings, and the control of the humanoid robot can be more accurate and meticulous.

The disclosed concepts relate to training a machine learning model to provide help sessions during a video game. For instance, prior video game data from help sessions provided by human users can be filtered to obtain training data. Then, a machine learning model can be trained using approaches such as imitation learning, reinforcement learning, and / or tuning of a generative model to perform help sessions. Then, the trained machine learning model can be employed at inference time to provide help sessions to video game players.

A robot control method based on tactile prediction pre-training comprises the following steps: acquiring and generating a human playing data set consisting of three-channel image tensors in an offline stage, and training a constructed conditional diffusion model comprising a tactile encoder, a tactile decoder and an action and visual encoder; in the online stage, the trained conditional diffusion model is integrated into a standard imitation learning strategy network, and an action instruction of the robot is generated according to the state of the robot, the current visual features and the tactile feature vectors extracted by the imitation learning strategy network. According to the method, a specific agent task is completed by training a deep neural network model, that is, a future tactile signal sequence is predicted according to historical information and future action intentions; the model is enabled to characterize generic haptic features contacting physical dynamic laws for further migration into downstream robot control tasks.

The invention provides a new energy power grid prospective scheduling method and device, and relates to the technical field of electric power system intraday economic scheduling. The method comprises the following steps: firstly, constructing an opportunity constraint optimal power flow model of prospective scheduling, analyzing the influence of new energy uncertainty on opportunity constraint, establishing a constrained Markov decision process of prospective scheduling, and then utilizing a risk evaluator network fitting risk function probability distribution and an actuator network considering extreme scene performance to determine the opportunity constraint optimal power flow model of prospective scheduling. The processing capability of the intelligent agent on a prospective scheduling scene containing a new energy extreme climbing event is enhanced; and finally, the training of the intelligent agent is accelerated by utilizing an imitation learning technology in a power grid prospective scheduling off-line simulation environment. According to the method, the solving speed and the strategy robustness and safety of the double-layer robust optimization model of the look-ahead scheduling can be considered.

The invention discloses a zero-sample cross-domain imitation learning method based on three-dimensional semantic point cloud and related equipment, and the method comprises the steps: constructing a training scene in a physical simulation engine, randomly generating an operation object pose, and enabling a robot to act to complete an operation task; an original image is observed, semantic segmentation privilege information is obtained by utilizing a simulation environment, an operation object is segmented, and semantic point cloud of the operation object is obtained by combining a depth map generated by the simulation environment; synchronously recording robot joint angles, and constructing a training data set; inputting the semantic point cloud of the operation object and the training data set into an imitation learning model, performing action blocking, extracting features, performing feature splicing, generating a predicted action sequence, and completing model training; deploying the trained model to a real environment, obtaining the semantic point cloud of an operation object, reading the current joint angle of the robot, sending the angle to the model for reasoning, outputting a robot action sequence, and completing a specified task. According to the method, the generalization ability of the strategy can be improved while the operation precision is ensured.

The invention discloses an unmanned aerial vehicle navigation obstacle avoidance method based on selective imitation enhanced deep reinforcement learning, imitation learning and deep reinforcement learning are combined to be used for fixed-wing unmanned aerial vehicle navigation and obstacle avoidance, and an intelligent agent directly maps non-ideal sensor data into a control instruction by learning an end-to-end strategy. For a sparse reward scene, an artificial potential field is constructed to generate a non-expert guidance action by accessing privilege state information which cannot be acquired by an intelligent agent, and a navigation strategy is continuously optimized according to an observation state and a sparse reward signal through real-time interaction with an environment and fusion of imitation learning and deep reinforcement learning. A Q-value-driven dynamic decision-making mechanism is adopted, and the motion of the fixed-wing unmanned aerial vehicle is determined by comparing the motion suggested by the instructor with the expected income of the motion generated by the learner, so that exploration-utilization balance is realized, and invalid exploration and local optimal decision are effectively avoided.

The invention relates to the technical field of edge computing, and discloses a self-adaptive pervasive edge computing task unloading method based on imitation learning, and the method comprises the steps: firstly constructing a task unloading strategy imitation model and an environment adaptability evaluation model, the former is used for learning and simulating a historical task unloading strategy, and the latter is used for evaluating the influence of environment change on the strategy; a deep reinforcement learning algorithm is introduced to train a simulation model, and real-time feedback of an environment evaluation model is combined to continuously optimize and adjust an unloading strategy so as to generate a highly adaptive task unloading decision. According to the method, the task unloading instruction can be intelligently generated and executed according to the real-time edge computing environment state and the task characteristics, the dynamic change of the environment is effectively coped with, and efficient utilization of resources and rapid execution of tasks are achieved. Compared with a traditional method depending on a static rule or a predefined strategy, the task unloading flexibility is remarkably improved, and powerful support is provided for intelligent application in the edge computing environment.

Implementations described herein relate to training and refining robotic control policies using imitation learning techniques. A robotic control policy can be initially trained based on human demonstrations of various robotic tasks. Further, the robotic control policy can be refined based on human interventions while a robot is performing a robotic task. In some implementations, the robotic control policy may determine whether the robot will fail in performance of the robotic task, and prompt a human to intervene in performance of the robotic task. In additional or alternative implementations, a representation of the sequence of actions can be visually rendered for presentation to the human can proactively intervene in performance of the robotic task.

The invention provides a strategy model training method and device for a robot and a storage medium, and the method comprises the steps: in an offline learning stage, firstly collecting expert demonstration data, enabling the robot to carry out the imitation learning based on the expert demonstration data, and obtaining an initial strategy model and an offline data pool; and entering an online learning stage, enabling the robot to perform reinforcement learning on the basis of expert strategies, determining training data of a current time step according to manual intervention information, adding the training data into an online data pool, performing model parameter updating according to the offline data pool and the online data pool, and after all tasks are finished, performing online learning. And taking the obtained initial strategy model as a target strategy model. In the online reinforcement learning process of the robot, autonomous exploration and manual intervention of the model are combined, the robustness of the robot strategy network is further improved, and efficient transformation of the robot strategy from demonstration to autonomy is realized.

Imitation learning, or artificial intelligence-based learning from demonstration, aims to acquire an agent policy by observing and mimicking the behavior demonstrated in expert demonstrations. Imitation learning can be used to generate reliable and robust learned policies in a variety of tasks involving sequential decision-making, such as autonomous driving and robotics tasks. However, current imitation learning solutions are limited in their ability to generalize states or goals unseen from the expert's demonstrations. The present disclosure integrates a diffusion model into generative adversarial imitation learning, which, in terms of prior solutions, can provide superior performance in generalizing to states or goals unseen from the expert's demonstrations, provide data efficiency for varying the amounts of available expert data, and capture more robust and smoother rewards.

The invention provides an anthropomorphic virtual test scene multi-background vehicle collaborative decision-making method, which comprises the following steps of: constructing a dynamic traffic map of multiple vehicles based on a graph structure, and constructing a parameter-shared generative adversarial imitation learning framework, inputting the node features optimized by the graph attention network to a reinforcement learning network to generate a serialized control instruction, and comparing the generated control instruction with a state-action pair of expert data by using a discriminator sharing scoring parameters to iteratively optimize a control instruction generator; potential factor variables are introduced to represent driving styles, and strategy personification is enhanced in combination with explicit rule punishment and implicit confrontation awards; and a highly anthropomorphic multi-background vehicle sequence cooperative control instruction is generated through multiple rounds of iteration. According to the method and the system, the adversarial imitation learning is iteratively optimized and generated by utilizing potential factor variables and reward enhanced imitation learning, and a decision with diversity, compliance and personification is generated, so that the reliability of an automatic driving simulation test is improved in multiple aspects.

The invention relates to the technical field of automatic driving and intelligent control, in particular to a multi-transportation-equipment cooperative motion control method based on multi-stage mixed learning and a storage medium, and the method comprises the steps: generating expert demonstration data through employing a single-vehicle motion control algorithm based on an expert rule, performing initialization training on the strategy network through online iterative supervised learning to obtain a pre-training strategy model; the multi-vehicle cooperative motion control method comprises the following steps: selecting a multi-vehicle cooperative motion model, loading parameters of the model into an Actor strategy network of multi-agent reinforcement learning, performing interactive training on a plurality of agents in a simulation environment by adopting a centralized training and decentralized execution normal form, and performing online iterative optimization on the strategy based on a composite reward function and generalized advantage estimation to obtain a multi-vehicle cooperative motion control strategy. According to the method, the complementary advantages of imitation learning and reinforcement learning are exerted, the training efficiency, the strategy performance and the collaborative operation capability and robustness of the system in a complex scene are improved, and the method can be directly applied to collaborative scheduling and control of transportation equipment groups in scenes such as surface mines and ports.

The invention discloses a robot gait training method and system based on reinforcement learning, and relates to the technical field of electric vehicle charging, and the method comprises the steps: carrying out the simulation learning initialization of a strategy based on a reference video; selecting a plurality of evaluation indexes to design a dynamic reward function so as to guide the initial strategy network to optimize the gait performance; through an environment difficulty scheduling mechanism, the training environment difficulty is automatically adjusted according to strategy performance; performing optimization training on the strategy network by using a PPO algorithm; and judging the optimized gait strategy by constructing a cognitive load index, and outputting a final gait strategy. According to the method, imitation learning, self-adaptive reward modeling, multi-stage scheduling, reinforcement learning optimization and cognitive feedback are organically fused, and a robot gait training framework with generalization ability, stability and social adaptability is constructed. According to the method, the naturalness, the stability and the man-machine friendliness of the gait of the robot can be remarkably improved under complex terrains and interaction scenes, and the method has wide application prospects and popularization value.

The invention discloses a double-arm robot imitation learning method and system based on action blocking and force perception, and the method comprises the steps: setting the pose of the tail end of a mechanical arm as an action, carrying out the action blocking, and collecting the action blocking and multi-mode observation information in a process that an expert operates robot teaching to complete a specified task; inputting the action blocks and the multi-modal observation information into a pre-established imitation learning model, constructing a mapping relationship between the action blocks and the multi-modal observation information, generating a predicted action sequence, and completing imitation learning model training; and deploying the trained imitation learning model to a real environment, obtaining real-time multi-modal observation information, dynamically generating corresponding action blocks according to the multi-modal observation information, outputting a robot action sequence according to the action blocks, and completing a specified task. According to the method, the accuracy of the position and force in the operation process can be ensured, fine and smooth operation skills can be learned only through a small amount of demonstration, and the task requirements of an unstructured environment and diversity of operation objects are met.

The invention discloses a self-adaptive regulation and control method for an air conditioning system based on information matching. The self-adaptive regulation and control method mainly comprises the following steps that firstly, a data set used for building cold load prediction is divided into a low-completeness data set and a high-completeness data set through a proposed information completeness judgment method; and for the high-completeness data set, regulating and controlling the air conditioning system by adopting a model prediction control method according to the load prediction model and the air conditioning equipment model. And for a low-completeness data set, a proposed migration imitation learning regulation and control method based on deep reinforcement learning is adopted. According to the method, the influence of data quality on the model prediction precision and the regulation and control effect is considered, the influence of fluctuation of historical data quality on the regulation and control effect can be overcome, the relative advantages of the model prediction control method and the reinforcement learning method are integrated, compared with a single regulation and control method, the energy-saving and cost-saving effects can be improved, and good robustness is achieved.

The invention discloses a mechanical arm trajectory planning method and device based on generative adversarial imitation learning, and relates to the technical field of mechanical arms. The method comprises the following steps: acquiring joint state data of the mechanical arm; inputting the mechanical arm joint state data into a trained improved generative adversarial imitation learning model to obtain a mechanical arm action, and circularly executing data acquisition-input of the improved generative adversarial imitation learning model to finally obtain a mechanical arm track; wherein in the improved generative adversarial imitation learning model, the reward function is a dynamic reward function, and parameters in the dynamic reward function are dynamically adjusted according to the task completion degree. According to the technical scheme, the learning efficiency can be improved. Compared with the prior art, the mechanical arm trajectory planning method has the advantages that the stability is obviously improved, expert trajectories can be better simulated, the success rate and accuracy of mechanical arm trajectory planning based on generative adversarial imitation learning are greatly improved, and a solid foundation is laid for practical application of mechanical arms in the fields of material synthesis and the like.

Systems and methods train a transformer-based policy network and Generative Adversarial Network (GAN) by initializing a transformer-based policy network to model action sequences by encoding temporal dependencies within sensor data. Multi-head self-attention mechanisms process sequential sensor inputs by being pre-trained on a labeled dataset having sensor data from known low-risk action sequences. A generator within the GAN is trained to produce generated action sequences, which mimic behavior of low-risk action sequences. A discriminator within the GAN is concurrently trained to differentiate between action sequences derived from the labeled dataset and synthetic action sequences produced by the generator. A feedback loop is employed to adjust parameters to produce sequences indistinguishable from real low-risk action sequences. Risk scores are generated and low-risk action sequences are identified upon reaching a predetermined threshold for accuracy in distinguishing between real and synthetic action sequences.

The invention relates to the technical field of automatic driving, in particular to a special end-to-end planning control method and system for an unmanned logistics vehicle, and the method employs an end-to-end design, integrates the functions of sensing, decision making, planning, control and the like into a unified model, simplifies the system architecture, reduces the development complexity, and improves the development efficiency. And the real-time performance and the consistency of the system are improved. According to the invention, two methods of imitation learning and reinforcement learning are innovatively combined. Simulation learning is used for model initialization, so that the model has a safe and mild driving style quickly; and reinforcement learning is used for strategy optimization, so that the model can exceed the level of a human driver, and a better driving strategy is found. The collaborative optimization mode gives consideration to both training efficiency and performance. The method is specially designed for the unmanned logistics vehicle, and the particularity of the unmanned logistics vehicle in an urban distribution scene is fully considered, so that the unmanned logistics vehicle can better adapt to the application requirements of the unmanned logistics vehicle.

The invention discloses an air combat intelligent decision-making method combining imitation learning and reinforcement learning, and belongs to the field of air combat, and the method comprises the steps: processing battlefield situation information through an air combat intelligent decision-making model, and obtaining a decision-making result used for guiding a local machine. The pre-trained air combat intelligent decision-making model is obtained through the following steps: for sparse expert strategy data with coarse granularity, using a behavior cloning algorithm to train a neural network architecture for imitation learning and reinforcement learning to obtain a strategy network Q1; taking the strategy network Q1 as an initial network in a generative adversarial imitation learning algorithm, and performing imitation learning on a fine-granularity dense expert strategy to obtain a strategy network Q2; and taking the strategy network Q2 as an initial network of a decision network in the reinforcement learning algorithm framework, and carrying out network training by adopting a strategy gradient method until convergence to obtain an air combat intelligent decision model. According to the invention, the sample utilization rate can be effectively improved, and accumulative errors are reduced.