56 results about "Imitation learning" patented technology

This invention relates to the field of spacecraft orbit control technology, and particularly to a spacecraft approach control method based on the fusion of imitation learning and reinforcement learning. The method includes: S1: generating initial and target states, solving for the optimal fuel transfer trajectory offline using a direct method, and constructing an expert dataset; S2: based on the expert dataset, training a neural network controller through imitation learning to obtain a pre-trained model; S3: loading the weights of the pre-trained model, constructing a composite reward function, and training the policy network using a near-end policy optimization algorithm to obtain an optimized policy network model; S4: acquiring relative state data, optimizing the policy network model, and outputting the control quantity for the optimal fuel consumption trajectory transfer in real time. This invention rapidly masters an approximately optimal policy through imitation learning and surpasses the performance of expert data through reinforcement learning, completing control command solutions in milliseconds. It balances terminal control accuracy and fuel consumption optimization, offering advantages such as high solution efficiency, excellent control accuracy, and strong generalization ability.

The embodiment of the application provides a motion trajectory generation method and device based on imitation learning, equipment and medium, belonging to the field of artificial intelligence. The method comprises the following steps: mixing the synthetic samples and the preset real samples to obtain mixed samples; performing trajectory discrimination on the target samples in the mixed samples by a target trajectory discriminator to obtain adversarial reward data, performing value discrimination on the target samples by a target value discriminator to obtain state value data, and calculating target reward data according to the adversarial reward data and the state value data; updating the network parameters of the original action generation network according to the target reward data to obtain a target action generation network; and performing action generation on the preset target environment state through the target action generation network to obtain a target motion trajectory generation action, and performing motion trajectory generation through the target motion trajectory generation action, so that the accuracy of the motion trajectory generation can be improved.

The application discloses a kind of behavior modeling method and system based on generation confrontation imitation and interactive representation, wherein the method comprises: collecting the multi-modal driving and perception data of each traffic subject, and processing the data to obtain state input data in the form of uniform input sequence;State input data is input into behavior prediction model to obtain predicted driver driving behavior.The application establishes driver behavior prediction model, outputs implicit scene coding that fuses the historical motion state of ego vehicle, candidate trajectory and neighbor vehicle interaction information, converts scene information into output vector suitable for imitation learning, can highly summarize current traffic scene from decision-related perspective, thereby significantly reducing state space dimension and improving modeling stability;And implicit scene coding is input into generation confrontation imitation structure, optimized by minimizing the maximum value of sample discrimination loss function of discriminator network, improve the accuracy and reliability of driver behavior prediction.

This invention relates to a personalized autonomous driving method and system based on a large language model. It constructs an automated annotation engine based on a large language model to annotate collected natural driving trajectory data in style, obtaining a driving style dataset, and trains a multi-style generative adversarial learning model. The trained model is then input with driving style, vehicle state, and environmental information to obtain a basic driving strategy. Based on this basic driving strategy, candidate strategies are generated using a personalized reward mechanism and reinforcement learning algorithms. A trusted evolutionary gating mechanism determines whether to update the candidate strategy to the current driving strategy. If so, the vehicle is controlled according to the driving actions output by the current driving strategy. The system is implemented based on this method. This invention overcomes the data annotation bottleneck, achieves low-cost personalization, preserves the driving styles of human experts, leverages the generalization advantages of reinforcement learning in exploring unknown scenarios, and significantly improves the safety and stability of autonomous driving systems in practical deployments.

The present application relates to the field of low-altitude flight control and digital twin technology, and in particular to a flight path intelligent optimization decision support system based on a low-altitude digital twin model, which comprises a multi-source data perception fusion module, a low-altitude digital twin modeling module, an intelligent path optimization decision module, and a virtual-real linkage control feedback module.A multi-scale dynamic digital twin model is constructed to realize real-time high-precision mapping of static terrain and dynamic environment.A three-stage path optimization algorithm combining imitation learning and reinforcement learning is proposed to complete closed-loop training in combination with a digital twin simulation environment.A distributed federated reinforcement learning collaboration mechanism is designed to ensure the safety and privacy of multi-aircraft path collaborative optimization.A space-time evolution type risk assessment model is established to realize early warning of dynamic risks and adaptive adjustment of paths.The present application can significantly improve the safety, economy and real-time performance of low-altitude flight paths.

This invention relates to the field of robotics and discloses a deep reinforcement learning-based ultrasound language imitation learning control method and system. Through lightweight model architecture design, community-driven data training, and asynchronous inference optimization, this invention achieves high success rate and high precision in ultrasound scanning tasks, while lowering the hardware deployment threshold and promoting the widespread application of intelligent ultrasound technology in primary healthcare institutions. It effectively solves the problems of excessively large model size, high deployment costs, weak generalization ability, and insufficient real-time performance of existing vision, language, and motion models in ultrasound robot applications.

Imitation learning, or artificial intelligence-based learning from demonstration, aims to acquire an agent policy by observing and mimicking the behavior demonstrated in expert demonstrations. Imitation learning can be used to generate reliable and robust learned policies in a variety of tasks involving sequential decision-making, such as autonomous driving and robotics tasks. However, existing methods that use next-token-prediction (NTP) models, where the policy reduces to a classifier over a discrete set of trajectory tokens, suffer from covariate shift due to their open-loop training a closed-loop execution. The present disclosure provides closed-loop fine tuning of autonomous agent policies in a manner that can mitigate covariate shift.

The application provides a robot mechanical arm cooperative control method based on imitation learning. The method comprises the following steps: constructing a robot historical state data set, performing feature extraction on the robot historical state data set through an encoder structure to obtain a latent feature representation; obtaining a color image and a depth image of a target operated by a robot to be predicted through a binocular camera, performing feature extraction processing on the color image and the depth image through a mode classification module and a visual feature extraction network to obtain a mode feature and a visual feature vector; inputting joint state information, the latent representation, the operation mode feature and the fused visual feature into a decoder to obtain an action block prediction result, performing weighted averaging on action blocks at multiple continuous time points based on a time series integration weighting mechanism to obtain a continuous control trajectory of the robot mechanical arm. The application solves the multi-stage strategy discontinuity and error accumulation problems existing in the traditional control algorithm, and realizes smooth action, stable task and double-arm cooperative control.

To reduce user load while improving the reproducibility of simulation images. [Solution] An information processing device (100) that performs adversarial imitation learning using a generator and a classifier, comprising: an acquisition unit (11) that acquires a reference image of a region including an object; a generation unit (12) that performs processing as a generator, and generates a generated image according to a policy model that is updated according to a reward function; an extraction unit (14) that inputs the reference image to an autoencoder to extract latent variables corresponding to the reference image and also inputs the generated image to an autoencoder to extract latent variables corresponding to the generated image; a calculation unit (15) that inputs each extracted latent variable to a classifier that identifies whether it corresponds to a latent variable in the reference image or the generated image, and updates the reward function according to the output result of the classifier; and a learning unit (17) that updates the policy model according to the updated reward function.

The present application relates to the technical field of metal plate intelligent repair, and specifically discloses a stainless steel cathode plate self-adaptive repair method and system simulating manual operation. The method comprises: obtaining current three-dimensional topography data of the cathode plate to be repaired; inputting the data into a first machine learning model trained based on multi-modal expert data to generate an initial repair action sequence; controlling a repair execution unit to execute the sequence, and online obtaining intermediate three-dimensional topography of the cathode plate as a state during the repair process; driving a reinforcement learning algorithm based on the difference between the state and a target state to dynamically adjust subsequent repair actions; and data of successful cases will be used to update a training database and retrain the first machine learning model. The method can solidify expert experience through imitation learning, and realize online adaptive decision-making in combination with reinforcement learning, thereby achieving individualized high-precision repair of various complex deformations, and the flatness after repair can reach below 0.5 mm, and the system has continuous optimization capability.

Imitation learning, or AI-based demonstration-based learning, aims to derive a policy for an agent by observing and imitating the behavior shown in expert demonstrations. Imitation learning can be used to generate reliable and robust learned policies for a variety of tasks requiring sequential decision-making, such as autonomous driving and robotics. However, existing methods that use next-token prediction (NTP) models, where the policy is reduced to a classifier over a discrete set of trajectory tokens, suffer from covariate shifting due to their open-loop training and closed-loop execution.The present disclosure provides a fine-tuning of the policies of autonomous closed-loop agents in a manner that can mitigate covariate shifting.

The application discloses a wearable human-machine operation data acquisition system and method based on relative visual inertial measurement, which is used for robot imitation learning. The system comprises a chest reference node and a hand operation node, each node comprising a camera and an inertial measurement unit, and the poses of the trunk and the hand in a global coordinate system are obtained through a visual inertial odometer respectively. A central processing unit calculates the relative pose of the hand relative to the chest in real time, effectively decouples the overall movement of the human body and the fine action of the hand, and introduces a rotation compensation mechanism based on the angular velocity of the chest reference node, so as to eliminate the false displacement caused by body rotation. The system outputs a strictly time-synchronized, standardized format relative pose sequence, multi-view images and inertial data, which can be directly used for imitation learning algorithm training. The application solves the problems of poor mobility, motion coupling and complex deployment of the traditional teaching system, and realizes high-precision and high-robustness data acquisition in a free-moving scene.

The embodiment of the application provides a kind of based on gait capture and counteracting imitation learning exoskeleton assisting method and system, belong to exoskeleton robot intelligent control technical field.The exoskeleton assisting method includes: obtaining the gait original data that human lower extremity IMU and plantar pressure sensor are synchronously collected in free walking state, obtain multi-modal natural gait original feature data;Multi-modal natural gait original feature data is input into human lower extremity dynamics model, and human original joint torque sequence is obtained;Natural gait expert demonstration set of "state-action" pair is constructed;The exoskeleton assisting control model of counteracting imitation learning is constructed, and the model of training completion is deployed to exoskeleton embedded real-time controller.The application improves the assisting efficiency of exoskeleton robot, wear comfort, individual adaptation speed and complex scene generalization ability, provides safe, efficient, high adaptability man-machine collaborative control method for load assisting, movement function rehabilitation training and other scenes.

The application discloses a kind of MU-MIMO system wireless resource allocation method and system based on condition diffusion strategy enhancement, for MU-MIMO system downlink transmission, method includes: with the total capacity of the MU-MIMO wireless communication system as target, jointly optimize resource block allocation and transmission power, construct optimization problem;Condition diffusion model is trained based on expert dataset;Multiple candidate solutions are generated using the trained model, and the optimal solution of the optimization problem is obtained by further optimizing the multiple candidate solutions through policy enhancement;By constructing a condition diffusion model, imitation learning is performed on the expert dataset;And introduce resampling mechanism, select the optimal solution from multiple candidate solutions, while ensuring to meet system constraints, significantly improve system throughput and reduce computational complexity.

The application provides a method and device for optimizing the motion trajectory of a robot, equipment, a medium and a product, and relates to the technical field of artificial intelligence. The application realizes fine evaluation of the quality of the motion trajectory, and accurate positioning of the abnormal range and abnormal motion state. The target function is used to locally optimize the abnormal motion state of the abnormal range, ensuring that the optimized motion trajectory meets the standard intention of expert behavior and guarantees the task target, while improving the fluency and environmental safety of the motion trajectory. The application realizes a closed-loop optimization mechanism from quality quantization to abnormal positioning to smooth repair of the motion trajectory, which can effectively reduce the abnormal trajectory segments in the existing generated motion trajectory of the robot. The final output of the optimized motion trajectory conforms to the high-quality data required by the robot imitation learning training field, providing reliable technical support for breaking through the bottleneck of insufficient training data in the imitation learning field.

The application discloses a knowledge and data fusion driven cloud control type network connected vehicle cooperative cruise control method, and belongs to the field of automobile intelligent safety and automatic driving. A cloud control type vehicle cooperative cruise control strategy is provided, and a strategy learning process comprises the following steps: 1) a model guided method for realizing multi-agent cooperative control is realized based on model prediction control (MPC), a safety distance is added to avoid collision, and demonstration data for training a strategy network is generated, and the strategy network is pre-trained by using imitation learning; and 2) on the basis of the imitation learning, the strategy is improved by using reinforcement learning, a reinforcement learning model and a heuristic reward function are designed according to the idea of knowledge guided learning, traffic road information is fused in the reward function, a collision avoidance strategy is added to the model, and the training of the reinforcement model is accelerated to improve the learning efficiency, so that a completely distributed controller is obtained, and the cloud control type vehicle cooperative cruise control is realized. The strategy considers vertical comfort degree of rough road, and can obviously improve problems such as traffic congestion, passenger comfort degree and safety hidden danger.

This invention discloses a robotic arm grasping method and apparatus based on a multi-scale sequence model using imitation learning. The method includes: collecting disordered grasping robotic arm motion trajectory data through a robotic arm hardware platform to establish a dataset; establishing a sequence multi-scale module based on an attention mechanism to extract multi-scale features from the dataset; integrating the extracted multi-scale features based on a teach-and-learn strategy to obtain a final corrected feature vector; establishing a motion generation strategy model based on a conditional autoencoder to obtain prediction results at different scales; and integrating the prediction results at different scales to obtain the final corrected predicted motion. This invention improves the efficiency of robotic arm motion trajectory planning. Furthermore, the addition of the multi-scale module allows the model to better learn information from different dimensions, increasing the model's generalization ability. Therefore, compared with traditional imitation learning algorithms, this method is more flexible and can better adapt to different control tasks.

The application discloses a kind of embodied intelligent robot electric core loading system and method based on imitation learning, comprising: active arm, for receiving the remote operation input of operator and generating control instruction sequence;Passive arm, for executing electric core grabbing action and feedback joint state sequence;RGB camera is installed at the end of robot, for collecting the RGB image sequence of working environment;Data acquisition module is electrically connected with active arm, passive arm and RGB camera respectively, for synchronously collecting control instruction sequence, joint state sequence and RGB image sequence, and adding time stamp for control instruction sequence, joint state sequence and RGB image sequence respectively;Data processing module is used for time stamp alignment and missing data interpolation to the collected control instruction sequence, joint state sequence and RGB image sequence, generates continuous alignment multimodal teaching data set.The system and method of the application can realize the flexible grabbing and loading of electric core, significantly improve production efficiency and product qualification rate.

This invention discloses a highway compliance behavior decision-making method based on large language model imitation learning, relating to the field of autonomous driving technology. It includes: S1: Extracting traffic regulations and expert driving trajectories to construct a compliance instruction dataset, and combining kinematic safety masks to correct potentially dangerous actions; S2: Obtaining a law-enhanced large language model through supervised fine-tuning and hyperparameter optimization; S3: Inputting the environmental state vector into multi-branch parallel decision processing, outputting action Q-values ​​by a deep Q-network, outputting compliance scores by the law-enhanced large language model, and outputting dynamic efficiency and safety weights by the SE-Net mechanism; S4: Performing multi-source weighted fusion of Q-values ​​and compliance scores, selecting the optimal action with the highest comprehensive score for execution; S5: Monitoring and controlling rounds through a composite termination criterion. This invention achieves a dynamic balance between driving safety and traffic efficiency, effectively suppresses model illusions, and improves compliance and robustness in complex scenarios.

The present application provides a kind of anthropomorphic virtual test scene multi-background car cooperative decision-making method, method includes the following steps: based on graph structure, the dynamic traffic graph of multi-vehicle is built, the generation of antithetical imitation learning framework of parameter sharing is built, the node feature after optimization of graph attention network is input to reinforcement learning network to generate serialized control instruction, the state-action pair of expert data is compared with the control instruction generated using the discriminator of shared scoring parameter, to iteratively optimize control instruction generator;Introduce latent factor variable to represent driving style, combine explicit rule penalty and implicit adversarial reward, enhance strategy anthropomorphism;Through multiple iterations, highly anthropomorphic multi-background car sequence cooperative control instruction is generated.According to the present application, latent factor variable and reward reinforcement imitation learning are used to iteratively optimize generative adversarial imitation learning, to generate decisions with diversity, compliance and anthropomorphism, thereby improving the reliability of autonomous driving simulation test from multiple perspectives.

The application provides a new energy power grid forward scheduling method and device, and relates to the technical field of power system daily economic scheduling. The application firstly constructs an opportunity constraint optimal power flow model of forward scheduling, analyzes the influence of new energy uncertainty on the opportunity constraint, and establishes a constraint Markov decision process of forward scheduling, then uses a risk evaluator network fitting the probability distribution of a risk function and an actor network considering the performance of extreme scenarios to enhance the processing capacity of the agent for the forward scheduling scene of extreme ramping events containing new energy, and finally uses the imitation learning technology to accelerate the training of the agent in the offline simulation environment of the power grid forward scheduling. The application can balance the solution speed, strategy robustness and safety of the double-layer robust optimization model of forward scheduling.

A computer-implemented method for training a neural network model for behavioral imitation using generative adversarial Imitation Learning (GAIL) is disclosed.The procedure comprises obtaining demonstration data representing the behavior of a demonstrator for a task, wherein the demonstration data comprises a sequence of state-action pairs, where a state corresponds to a condition for decision-making and an action corresponds to the decision made by the demonstrator; updating a neural network discriminator to maximize a first objective function, wherein the first objective function is modified by a first term to make the discriminator indistinguishable between the behavior of the demonstrator and state-action pairs generated by a neural network policy generator; and updating the policy generator to minimize a second objective function, wherein the second objective function is modified by a second term to make the policy generator converge towards the behavior of the demonstrator.Several additional aspects have been revealed.

This system for providing a predictive temporal ensemble technique for enhancing imitation learning-based robot task speed comprises a memory storing instructions and a processor, wherein the instructions, when executed by the processor, cause the system to: receive, as inputs, ambient environment data and robot movement data collected through teleoperation; learn a policy of a robot through an imitation learning algorithm by using the collected data, wherein the policy is a function that receives the current state as an input and outputs a future action sequence; generate a control input for the robot by ensembling, in a predictive temporal manner, actions after the current time point in a future action sequence predicted from the policy; and control the robot by using the generated control input.

This application provides an autonomous driving method and system based on scene adaptive recognition. The method includes: acquiring environmental information and previous operation information in the driving scene; a path planning module determining a path planning trajectory in the current driving scene based on the previous operation information; determining the scene complexity based on the parameter space of the path planning trajectory; an imitation learning module deriving the uncertainty distribution of the decision from a neural network based on the environmental information; determining the scene anomaly degree based on the uncertainty distribution; and a decision module determining the autonomous driving method based on the scene complexity and scene anomaly degree. This solution improves the real-time performance and accuracy of autonomous driving.

This invention relates to the field of ultrasonic robot technology, providing a method and acquisition device for reproducing expert scanning actions of an autonomous ultrasonic robot. The reproduction method acquires multimodal expert scanning action data with a unified reference coordinate system and time axis. Based on a force-position hybrid control framework, the robot's end-effector pose is recursively updated to reproduce the robot's scanning actions. The acquisition device uses a multimodal time synchronization unit to align data from the ultrasonic image acquisition unit, probe pose acquisition unit, and contact force / torque acquisition unit with a unified coordinate system and time, constructing multimodal temporal action data that consistently represents expert scanning behavior. This invention constructs an integrated technical solution for acquisition, modeling, and reproduction, which can simultaneously serve multiple application scenarios such as scanning action reproduction, robot control verification, and imitation learning or strategy learning, significantly improving the reusability and engineering transformation value of expert scanning experience in autonomous ultrasonic robots.

The application discloses a dynamic high-precision docking control method and system for a fluid loading and unloading arm in a non-steady environment based on imitation learning, and the method comprises the following steps: initializing the end position of the fluid loading and unloading arm to a position at which complete image information can be collected; collecting a ship receiving device marker point image in real time, and calculating a visual feature error vector; inputting the visual feature error vector into a Gaussian mixture model regression model which is trained in advance by using an expert demonstration data set, and outputting a dynamic compensation speed vector; calculating a control instruction for minimizing docking deviation according to the visual feature error and the compensation speed; and driving a hydraulic execution system of the fluid loading and unloading arm to perform real-time adjustment according to the control instruction until the visual error converges, and high-precision docking of the end of the fluid loading and unloading arm and the ship receiving device is completed. The application is suitable for cm-level accurate docking of a fluid loading and unloading arm in a non-steady marine environment coupled by strong wind and sea waves, and significantly improves the safety and reliability of dangerous fluid conveying operations.

The application discloses a human-like driving behavior control method and system based on scene driving and imitation learning, and relates to the technical field of intelligent driving. The method comprises the following steps: classifying typical driving scenes in a driving data set, modeling a historical driving behavior sequence by using a time sequence neural network to extract the evolution trend features of driving behavior with scene changes; structurally coding the human-vehicle-environment elements in the driving scene, and modeling the interaction between the scene features and the traffic participant features based on an attention mechanism to obtain interaction features; fusing the evolution trend features and the interaction features to form a scene feature vector as a behavior cognitive representation of scene driving; inputting a real-time perceived scene feature vector into a human-like control strategy network pre-trained through deep imitation learning, and outputting vehicle control actions including longitudinal acceleration and steering angle by the network. The application aims to realize human-like autonomous decision of an unmanned vehicle in a complex driving scene and improve the rationality of driving behavior.

This application relates to the field of robot control technology, and in particular to a method and apparatus for processing robot operation imitation learning trajectories based on physical features. The method includes: using a trajectory saliency detector (TSD) to align the original taught trajectory dataset of the target robot to obtain an aligned trajectory dataset; using spatial entropy and centripetal acceleration features to identify fine target segments and target maneuver segments in the aligned trajectory dataset; and then performing target compression and target augmentation processing on the original taught trajectory dataset to obtain a processed robot imitation learning trajectory dataset to determine the target robot's operation trajectory. This solves the problems of low data acquisition efficiency and inability to guarantee data quality in robot operation imitation learning in related technologies, which reduce the efficiency and resource utilization of imitation learning model training.

The application discloses an aircraft intelligent evasion guidance method and system based on a generative adversarial imitation learning. Firstly, a three-dimensional aircraft evasion simulation environment is constructed, and evasion demonstration data is generated based on program maneuver or differential game expert strategy. Secondly, a GAIL framework including a strategy network, a value network and a double-branch discriminator is constructed, and the strategy network is pre-trained by using behavior cloning. Then, fusion GAIL training is performed through mechanisms such as progressive GAIL reward introduction, PPO strategy optimization, adaptive supervision decay and EMA strategy smoothing. Finally, the trained strategy model is deployed in an online guidance system, and evasion control instructions are output according to the real-time state of a red aircraft. The GAIL imitation learning framework is adopted, and strategy learning is guided relying on expert demonstration data, so that a complex multi-objective reward function does not need to be artificially designed, and the ability of strategy autonomous optimization and surpassing the performance of experts is reserved.

The application discloses an imperfect demonstration imitation learning method and device and a storage medium, relates to the technical field of imitation learning, and comprises the following steps: acquiring a sparse expert data set and a suboptimal data set in an imitation learning scene; training a diffusion model based on the sparse expert data set, performing distribution-level correction on the suboptimal data set through the diffusion model, and obtaining a corrected data set; training a discriminator based on the sparse expert data set and the corrected data set, performing dynamics consistency screening on the corrected data set by using the discriminator, and obtaining a high-quality enhanced data set; and fusing the sparse expert data set and the high-quality enhanced data set, training a policy network through a behavior cloning algorithm, and outputting a target policy network suitable for the sparse expert data scene. The application realizes purification and enhancement of poor data through diffusion correction and discrimination screening, and effectively alleviates the problems of cumulative error and insufficient generalization ability caused by the scarcity of expert samples in traditional imitation learning.