305 results about "Imitation learning" patented technology

The invention provides a mechanical arm grabbing method and system based on multi-modal information fusion, and belongs to the technical field of robot intelligent control. Comprising the steps that the conversion relation between a camera coordinate system and a mechanical arm base coordinate system is established through camera calibration, a deep learning neural network is used for conducting grabbing pose estimation on an obtained RGB-D image, and multiple candidate grabbing poses are determined; analyzing a natural language instruction input by a user based on a multi-modal large model, and recognizing a target object region from the RGB-D image by combining a target detection and image segmentation technology; based on the obtained candidate grabbing poses and the target object area, an optimal grabbing pose is screened through a scoring mechanism and mapped to a mechanical arm base coordinate system; and then a dynamic grabbing path is generated by adopting an imitation learning algorithm, and the mechanical arm is controlled to execute grabbing operation. Through multi-modal semantic understanding, accurate grabbing of the mechanical arm in a complex environment can be achieved.

The invention discloses a visual language navigation method based on memory driving, and the method comprises the following steps: S1, constructing an MDQT model which comprises a panoramic encoder, a text embedding layer, a Q-Former, an action prediction module and a memory updating module; s2, training an MDQT model by using three pre-training tasks of image-concerned mask language modeling, instruction track matching and instruction track contrast learning; and S3, performing fine tuning on the MDQT model by using imitation learning and reinforcement learning. According to the invention, a learnable memory vector with a fixed length is used to encode historical information. In each navigation step, the memory vector interacts with the extracted panoramic information and instruction information, and visual information most relevant to the current instruction information is extracted according to the current memory state of the robot for decision making. The memory of the robot to the historical navigation steps is effectively maintained under limited resources, and the navigation success rate and efficiency of the robot are improved.

The disclosed concepts relate to training a machine learning model to provide help sessions during a video game. For instance, prior video game data from help sessions provided by human users can be filtered to obtain training data. Then, a machine learning model can be trained using approaches such as imitation learning, reinforcement learning, and / or tuning of a generative model to perform help sessions. Then, the trained machine learning model can be employed at inference time to provide help sessions to video game players.

A robot control method based on tactile prediction pre-training comprises the following steps: acquiring and generating a human playing data set consisting of three-channel image tensors in an offline stage, and training a constructed conditional diffusion model comprising a tactile encoder, a tactile decoder and an action and visual encoder; in the online stage, the trained conditional diffusion model is integrated into a standard imitation learning strategy network, and an action instruction of the robot is generated according to the state of the robot, the current visual features and the tactile feature vectors extracted by the imitation learning strategy network. According to the method, a specific agent task is completed by training a deep neural network model, that is, a future tactile signal sequence is predicted according to historical information and future action intentions; the model is enabled to characterize generic haptic features contacting physical dynamic laws for further migration into downstream robot control tasks.

The invention provides a new energy power grid prospective scheduling method and device, and relates to the technical field of electric power system intraday economic scheduling. The method comprises the following steps: firstly, constructing an opportunity constraint optimal power flow model of prospective scheduling, analyzing the influence of new energy uncertainty on opportunity constraint, establishing a constrained Markov decision process of prospective scheduling, and then utilizing a risk evaluator network fitting risk function probability distribution and an actuator network considering extreme scene performance to determine the opportunity constraint optimal power flow model of prospective scheduling. The processing capability of the intelligent agent on a prospective scheduling scene containing a new energy extreme climbing event is enhanced; and finally, the training of the intelligent agent is accelerated by utilizing an imitation learning technology in a power grid prospective scheduling off-line simulation environment. According to the method, the solving speed and the strategy robustness and safety of the double-layer robust optimization model of the look-ahead scheduling can be considered.

The invention relates to the technical field of automatic driving and intelligent control, in particular to a multi-transportation-equipment cooperative motion control method based on multi-stage mixed learning and a storage medium, and the method comprises the steps: generating expert demonstration data through employing a single-vehicle motion control algorithm based on an expert rule, performing initialization training on the strategy network through online iterative supervised learning to obtain a pre-training strategy model; the multi-vehicle cooperative motion control method comprises the following steps: selecting a multi-vehicle cooperative motion model, loading parameters of the model into an Actor strategy network of multi-agent reinforcement learning, performing interactive training on a plurality of agents in a simulation environment by adopting a centralized training and decentralized execution normal form, and performing online iterative optimization on the strategy based on a composite reward function and generalized advantage estimation to obtain a multi-vehicle cooperative motion control strategy. According to the method, the complementary advantages of imitation learning and reinforcement learning are exerted, the training efficiency, the strategy performance and the collaborative operation capability and robustness of the system in a complex scene are improved, and the method can be directly applied to collaborative scheduling and control of transportation equipment groups in scenes such as surface mines and ports.

The invention relates to the technical field of automatic driving, in particular to a special end-to-end planning control method and system for an unmanned logistics vehicle, and the method employs an end-to-end design, integrates the functions of sensing, decision making, planning, control and the like into a unified model, simplifies the system architecture, reduces the development complexity, and improves the development efficiency. And the real-time performance and the consistency of the system are improved. According to the invention, two methods of imitation learning and reinforcement learning are innovatively combined. Simulation learning is used for model initialization, so that the model has a safe and mild driving style quickly; and reinforcement learning is used for strategy optimization, so that the model can exceed the level of a human driver, and a better driving strategy is found. The collaborative optimization mode gives consideration to both training efficiency and performance. The method is specially designed for the unmanned logistics vehicle, and the particularity of the unmanned logistics vehicle in an urban distribution scene is fully considered, so that the unmanned logistics vehicle can better adapt to the application requirements of the unmanned logistics vehicle.

Implementations relate to training a generative model. Some of those implementations include performing offline supervised fine-tuning (SFT) of a base generative model using an imitation learning dataset of successful task episodes. The SFT training utilizes both a behavioral cloning loss function, which compares offline predictions to ground truth data, and a reinforcement reward loss function, comparing predicted rewards to actual episode rewards. Subsequently, online reinforcement learning (RL) is performed on an instance of the base generative model. This online RL processes online data using the model instance to generate predictions and uses the SFT-trained model to generate online predicted rewards. These online predicted rewards are then used to train the instance of the base generative model, improving its performance for various robot and application control tasks.

The invention discloses a liquid drop routing method under fluid constraint in a digital micro-fluidic biochip based on reinforcement learning and imitation learning, and aims to solve the technical problems of low planning efficiency, low success rate and weak generalization ability of a traditional liquid drop routing algorithm under fluid constraint. The method adopts a strategy combining imitation learning and reinforcement learning and comprises the following steps: firstly, generating expert data by utilizing an expert algorithm meeting fluid constraint after transformation, and pre-training a network model through imitation learning to enable the network model to quickly grasp a basic routing strategy; the model and environment interaction are autonomously optimized through reinforcement learning, and the adaptability to the dynamic environment is improved. And finally fusing the weights to form a final model. Through the hybrid training framework, the success rate and planning efficiency of droplet routing are remarkably improved, and the generalization ability of the model to chip environments with different sizes is enhanced.

The invention provides a unified supervision fine tuning and reinforcement learning training method based on dynamic weight fusion, and belongs to the field of artificial intelligence. According to the unified supervision fine tuning and reinforcement learning training method based on dynamic weight fusion, knowledge imitation and strategy exploration are balanced based on unified SFT and RL training targets, and therefore the purpose of improving training stability is achieved. Comprising the following implementation steps: step (1), data preparation; step (2), performing double-path parallel processing; step (2.1), an SFT path is established; (2.2) an RL path; step (3), a dynamic weight fusion mechanism; the loss weights of the SFT path and the RL path are dynamically adjusted through the global coefficient mu, and progressive transition from imitation learning to exploration learning is achieved.

The invention provides an air combat strategy generation method based on lightweight sequence modeling imitation learning. According to the training method, an air combat decision model based on rules can be quickly migrated into a parameter model in a neural network form. The method comprises the following steps: firstly, constructing a rule-based air combat decision model, and generating high-quality training data through adversarial simulation; and then, the training data is converted into a state-action sequence pair, and the state-action sequence pair is input into a lightweight sequence modeling network based on a Transform architecture for learning and training. According to the method, decision knowledge of a rule model is converted into a neural network model through an imitation learning method, and efficient utilization and flexible generalization of expert knowledge in air combat decision model construction are realized; the model designed by the method can accurately reproduce the decision behavior of the rule model through a small amount of training, and compared with a reinforcement learning method, the strategy construction speed is higher and the strategy performance is more stable.

The invention relates to a deep learning technology, and discloses a flow matching and dynamic reward scheduling-based imitation learning method, which comprises the following steps of: acquiring an expert demonstration track, and extracting an expert feature sequence in the expert demonstration track by utilizing a flow matching model; obtaining a strategy state sequence of task execution of the agent strategy network and converting the strategy state sequence into strategy features; constructing a reward function by utilizing the expert feature sequence and the strategy feature; calculating a reward result and optimizing the median function network of the agent strategy network; according to the optimized value function network, an intelligent agent strategy network is optimized, and then an action sequence of a current strategy is sampled; and generating a track fragment according to the action sequence, and iteratively optimizing the intelligent agent strategy network by taking the track fragment as a training sample, and obtaining a target intelligent agent strategy network after iteration optimization is completed. The invention also provides an imitation learning device and equipment based on stream matching and dynamic reward scheduling, and a storage medium. According to the method, the state modeling efficiency in imitation learning and the stability of the reward structure can be improved.

The invention discloses an intelligent learning method for grabbing of a three-finger dexterous hand for multiple types of workpieces, and belongs to the technical field of intelligent grabbing application of manipulators. A pre-defined rule and simulation expert double-layer mode is provided, a high-quality demonstration data set is automatically generated, a simulation grabbing scene of a mechanical arm and a three-fingered dexterous hand is constructed, recognition and classification of multiple types of workpieces are achieved by analyzing workpiece contour features, grabbing hand shapes are matched according to the recognition and classification, strategy pre-training is carried out through simulation learning, and the recognition and classification efficiency is improved. And optimizing the strategy based on a PPO algorithm and a composite reward function. Aiming at the problem of lack of a standardized three-fingered dexterous hand parameter model, a modeling method based on parameterized URDF and automatic script generation is used to realize efficient configuration and optimization of the three-fingered dexterous hand in simulation. Through performance evaluation, stable and high-quality grabbing of multiple types of workpieces can be achieved.

The invention relates to the field of humanoid robots, and provides a control method for biped robot step-by-step gait strategy fusion in order to solve the problems that a biped robot is large in training difficulty, unnatural in gait, poor in adaptability and the like. Training various different terrain categories through an imitation learning algorithm and a reinforcement learning algorithm to obtain a gait strategy model; s2, performing pairwise fusion gait strategy switching between the gait strategy models corresponding to various different terrain categories; s3, the terrain category of the current position is recognized in real time according to an information collection unit of the biped robot, a corresponding gait strategy model is obtained through a strategy execution control unit, and the biped robot is controlled to walk in the corresponding motion state; and S4, when the terrain category of the biped robot is changed, smooth transition between the two gait strategy models is achieved through gait strategy switching, and control over gait strategy fusion of the biped robot is completed.

The invention discloses a robot imitation learning method based on a diffusion model, and belongs to the technical field of robot learning and body intelligence. Comprising the following steps: carrying out standardization processing on input image observation data and a robot state, and carrying out image enhancement by adopting gray scale transformation, random erasure and Gaussian blur; based on the enhanced image and the robot state, training a diffusion model to generate an action sequence, including forward diffusion, conditional feature fusion, noise prediction and joint loss optimization; in real-time control, features are extracted through an image enhancement network, sampling is accelerated through DDIM to generate an action sequence, noise scheduling coefficients are dynamically adjusted based on visual feature differences, and closed-loop optimization is achieved. According to the method, the image enhancement technology and the diffusion strategy are deeply fused, joint optimization of visual features and action sequences is achieved, the action generation accuracy and robustness of the robot under the complex visual interference and small sample conditions are remarkably improved, and the success rate is improved by 18% compared with a base line under 40 sample sizes.

This disclosure provides a method, an apparatus, a device, a medium, and a program product for training a decision model. The method includes: determining a first policy using a supervised learning model and a second policy using a reinforcement learning model within the decision model based on training data; determining an imitation learning loss based on a difference between the first policy and the second policy; and training the decision model based on both the imitation learning loss and a reinforcement learning loss corresponding to the second policy. By combining the imitation learning loss and the reinforcement learning loss, a human-like decision model with excellent performance may be obtained, leveraging the expert data utilization capability of supervised learning and the strong generalization capacity of reinforcement learning. In some embodiments, the trained model is applied to autonomous driving for tasks such as lane-changing.

The invention discloses a hybrid reinforcement learning-based autonomous navigation decision-making method for a cavity surgical robot and a related device. The method comprises the following steps: acquiring a historical operation data set of a doctor to train a PPO model for imitation learning; an environment state input model is acquired in real time in the operation, and discrete actions and continuous PID parameters are output; a control instruction is generated through the dynamic weight fusion layer, and voice or eye movement intervention coverage model output is supported; the intervention content is stored in a playback buffer area according to the priority, and the model is updated. According to the method, the complex perception understanding ability is improved through a hierarchical attention mechanism, the safety is enhanced by utilizing a risk adaptive fusion strategy, and continuous evolution is realized by virtue of human intervention and priority experience playback, so that the autonomous navigation precision, safety and clinical friendliness of the cavity surgical robot in a narrow dynamic environment are improved.

The invention discloses a chain type hybrid deep reinforcement learning quadruped robot control system. The system comprises a sensing module, a data processing module, a chain type hybrid reinforcement learning model module, a feedback module and the like. The invention further discloses a control method of the chain type hybrid deep reinforcement learning quadruped robot control system, and the method comprises the steps: S1, collecting bionic motion data, robot state data and environment data, and training based on an attention mechanism space-time diagram neural network and antagonistic imitation learning to obtain a reference gait model containing motion prior; and S2, constructing a three-level parallel simulation environment, setting a progressive course target, training the progressive course target by adopting an improved PPO algorithm, and optimizing the gait stability, the movement efficiency and the environmental adaptability. By means of high-precision data acquisition with the sampling frequency larger than or equal to 240 Hz and antagonistic imitation learning, motion priori knowledge is introduced, the training period can be shortened by 50% or above, abnormal gaits are avoided, and the training process is controllable.

The invention belongs to the technical field of bioengineering, and particularly relates to a biomass fermentation safety reinforcement learning control method and system, and the method comprises the steps: training an initial strategy network through imitation learning by employing historical data, and providing safety weight initialization for a reinforcement learning agent; and constructing a virtual training environment fusing the mechanism model and the data-driven residual error correction network, applying domain randomization, training an agent in the environment to maximize long-term accumulated rewards, and obtaining a robust control strategy. A training strategy is deployed to a real fermentation system, a control instruction is corrected in real time through an independent safety layer according to a hard constraint rule to ensure operation safety, and meanwhile online fine tuning and updating are conducted on the strategy based on actual operation data. According to the method, safe and smooth transition from virtual training to practical application is realized, and the final ethanol concentration and the overall control performance in the fermentation process are remarkably improved.

The invention discloses a target-driven navigation method and device based on context awareness and imitation learning, and the method comprises the steps: recognizing an object instance of interest in an image based on a target detector DETR, and constructing an object graph; based on context perception graph reasoning, in the navigation process, dynamic context information such as images, actions and memories serves as guidance, object features are projected to hyperplanes of corresponding contexts by means of a TransH method at each time step, the object relation is dynamically learned, and an intelligent agent can better understand the complex environment. Based on visual representation of Transform, visual features and graph features are fused, and spatial semantic information of the environment is better captured. Based on generative adversarial imitation learning, a new dynamic reward function is designed, and an intelligent agent is helped to avoid a deadlock state in combination with environment rewards. Based on a standard asynchronous dominant actor-commentator algorithm, an effective navigation strategy is trained by using a new reward function, and the navigation success rate and efficiency of the intelligent agent in an unfamiliar environment are improved.

The invention discloses a touch enhancement and reinforcement learning collaborative long-task robot operation learning method. The method comprises the following steps: a, completing language task splitting and visual landing by using a large language model LLAMA3 and a visual language model CLIP; b, calculating a tactile continuous index to obtain a unified progress score which is used as a reward signal for strategy input, judgment stopping and reinforcement learning; c, carrying out simulation learning in the unified vision-language-action strategy model through expert demonstration; and d, carrying out two-stage collaborative optimization on the strategy, and writing the operation data into a playback buffer area. The tactile signals directly intervene in decision making and reward, and the one-time success rate and robustness of a contact scene are remarkably improved; the language splitting and unified stopping rule supports step-by-step execution of a long task; a single strategy head and three supervision losses are matched with an explicit reinforcement learning reward template, the structure is simple, and the project landing cost is low; and the method has good generalization ability of cross-object class, cross-task and cross-equipment.

The invention discloses a vacuum coating process parameter adaptive adjustment method and system, the system comprises a data acquisition layer, a DRL intelligent agent and an execution layer, the DRL intelligent agent comprises a strategy network and an evaluation network, the DRL intelligent agent adopts a double-loop learning mechanism, and the DRL intelligent agent comprises an offline imitation learning module and an online reinforcement learning module. According to the technical scheme, through DRL decoupling multi-roller coupling, the tension fluctuation control precision is obviously improved compared with traditional PID control; factors such as roller diameter thermal deformation and environmental disturbance are automatically compensated through an online learning mechanism, the dynamic adaptability is enhanced, and manual adjustment is not needed; in addition, closed-loop control of visual defect detection is fused in the technical scheme, so that the film thickness uniformity is greatly improved, and the wrinkle defect rate is obviously reduced.

The invention relates to the technical field of intelligent equipment. A trajectory diffusion imitation learning method based on single human video demonstration comprises the steps that a demonstration video is analyzed to obtain an operation body posture motion trajectory and an object pose motion trajectory, and the demonstration video is a video of a human or machine operation object; respectively carrying out interpolation calculation on the positions and postures of the operation body posture motion trail and the object pose motion trail to obtain a training data set; using the training data set to train a diffusion strategy model to obtain a target diffusion strategy model; and during deployment, the real-time pose of an end effector of the robot serves as observation input of the target diffusion type strategy model, an operation body pose movement track and an object pose movement track in the deployment stage are formed, and a joint control instruction of a target operation body is obtained. And the task success rate is kept, and the practicability and the expandability of the imitation learning method are improved at the same time.

The invention relates to the technical field of traffic simulation, in particular to an automatic driving traffic flow simulation method and device based on closed-loop reinforcement learning, and the method comprises the steps: obtaining a real driving data set based on a pre-constructed traffic flow simulation frame with an automatic driving vehicle as the center; performing imitation learning pre-training according to the real driving data set to obtain an initial trajectory generation model; and constructing a closed-loop reinforcement learning fine-tuning target function according to the initial trajectory generation model, and updating the initial trajectory generation model by using the closed-loop reinforcement learning fine-tuning target function to obtain a final trajectory generation model. Therefore, the problems that in the prior art, authenticity and controllability cannot be considered at the same time, or covariant offset, mode collapse, unstable training and the like are faced in closed-loop deployment are solved.

The invention provides a humanoid robot trajectory enhancement method and system oriented to a multi-stage continuous task, and belongs to the field of robot intelligent control and imitation learning modeling, and the method comprises the steps: inputting a splicing trajectory and a real-time image sequence into a pre-trained improved conditional variation auto-encoder model, obtaining a predicted trajectory of a humanoid robot, and obtaining the predicted trajectory of the humanoid robot; wherein the pre-trained improved conditional variation auto-encoder model comprises the following steps: constructing an encoder with an attention mechanism, dividing a potential space of the conditional variation auto-encoder into a plurality of layers, each layer of the hierarchical potential space having a potential variable, the feature of each layer of the hierarchical potential space is described by adopting the mean value and the variance of the image potential variables, the mean value and the variance of the image potential variables of each layer are optimized by adopting the loss function, and for long-time, multi-stage and high-dynamic-change imitation learning, the prediction precision is improved by the method, and the prediction efficiency is improved. The problems of unstable training and missing key detail expression are solved.

The invention discloses a robot generative adversarial self-imitation learning method based on large language model feedback, and belongs to the technical field of computer systems based on specific calculation models, and the method comprises the steps: (1) initializing a strategy network, a discriminator network and a demonstration pool; (2) constructing and training a large language model feedback network: obtaining a training sample of the large language model feedback network, and training the large language model feedback network according to the training sample; and (3) an iterative learning step (33): updating the strategy network according to the reward calculated by the new cost function. According to the robot generative adversarial self-imitation learning method based on large language model feedback, the advantages of the large language model in the aspects of semantic comprehension, reasoning and knowledge organization are used as a potential award priori source, and a stable award signal can be provided when the demonstration data quality is poor, so that a strategy exceeding original demonstration is learned.

The invention relates to a multi-unmanned aerial vehicle path planning method with expert experience migration capability, and belongs to the technical field of multi-unmanned aerial vehicle system path planning. Aiming at the problems of slow convergence, low multi-agent cooperative control consistency and weak dynamic environment adaptability in a strategy initialization stage in the prior art, the method provides an innovative solution. The method comprises the following steps of: firstly, formally modeling a path planning task through a partial observable Markov decision process; secondly, performing behavior clone pre-training by utilizing an expert knowledge base in an imitation learning stage, and then introducing a near-end strategy optimization combined generalized advantage estimation mechanism continuous optimization strategy in a reinforcement learning stage; finally, each unmanned aerial vehicle independently decides to generate a combined action in a distributed execution stage, the technical effects comprise strategy initialization efficiency innovation, cooperative control capability breakthrough, dynamic environment adaptation evolution and system deployment efficiency jump, and the reliability of the multi-unmanned aerial vehicle system in the urban complex environment is remarkably improved.

The invention relates to the technical field of bidding and tendering, and discloses a bidding and tendering file deep understanding and multi-rule review management system based on a large model, and the system comprises an environment state representation module which is used for encoding a bidding and tendering file into a structured state vector; the multi-agent decision-making module comprises a plurality of professional agents and is used for executing a review action according to the state vector; the dynamic risk transmission module is used for generating a risk broadcast vector when a professional agent identifies the risk, and adjusting subsequent actions of other agents through a cross-agent attention mechanism; the strategy training module is used for training the strategy of the multi-agent decision-making module by utilizing an expert review track through a generative adversarial imitation learning mode; and the report generation module is used for outputting a dynamic review path report. According to the method, the dynamic optimization of the review path is realized by simulating an expert strategy, the risk collaborative review across professional fields is realized through a risk transmission mechanism, and the traceability of the review process is ensured at the same time.

The invention discloses a low-cost robot imitation learning method and system based on a human video. The method comprises the following steps: S1, data acquisition; s2, data extraction and physical alignment are carried out to eliminate man-machine physical differences; the step is divided into two parallel processing modules of action space alignment and visual space alignment; s3, data set construction: mixing the aligned human data with real robot teleoperation data, carrying out balanced sampling, and constructing a mixed data set Dmix; and S4, cooperative training: constructing a strategy network based on diffusion Transform for training. According to the method, data can be acquired only through the monocular RGB camera, expensive robot teleoperation data are replaced with cheap and easily available human videos, and the data acquisition threshold is greatly reduced. Through a visual alignment strategy of random color grid rendering, a network can learn neglect skin color textures and pay attention to geometric structures without a complex generative model, so that the robot can be seamlessly migrated to robots in different forms.

The invention discloses a robust self-adaptive multi-skill learning method for a quadruped robot based on key frame guidance. The robust self-adaptive multi-skill learning method comprises the following steps: determining a target key frame set of multi-skill learning; in a flat terrain environment, probability sampling is carried out on the key frames according to skill task values to generate skill instructions, the quadruped robot is controlled to execute actions, and training tracks are collected and stored in a strategy learning experience playback pool and a self-simulation learning experience playback pool respectively. Training the strategy network and the value function network through strategy learning circulation; a self-mimicking learning discriminator network is trained through a self-mimicking learning loop to extract a high quality trajectory. And after the flat terrain training is completed, entering a complex terrain environment, and carrying out adaptive training under the guidance of the trained strategy network and the self-simulation learning discriminator network so as to realize the migration of the skill primitives to the complex environment. According to the method, multi-skill learning and terrain adaptive capacity can be effectively considered, and the robustness, flexibility and multi-task execution performance of the quadruped robot are improved.