42 results about "Reinforcement learning control" patented technology

The invention provides an underwater robot multi-task rapid adaptive control method based on meta reinforcement learning. The method comprises the following steps: constructing an underwater robot dynamic model and a thrust distribution strategy matrix; three reinforcement learning controllers, namely, a non-overshoot position controller, an overshoot allowing position controller and a propeller flexible control controller, are respectively designed according to diversified task requirements; introducing a meta-learning mechanism to build a meta-training platform, and training the three reinforcement learning controllers to obtain a group of optimal initialization parameters; and deploying the obtained optimal initialization parameters and the subtask reinforcement learning controller to the underwater robot, and carrying out two-stage training according to different tasks. Finally, when the controller is deployed in engineering practice, the output of the propeller can be intelligently adjusted according to the relative distance and speed information, calculated in real time, between the controller and the target position, and it is ensured that accurate position control can be achieved in various task scenes. The method aims at meeting the requirement for rapid self-adaption of multiple tasks of the underwater robot in the complex and changeable underwater environment, and accurate and flexible response to position control is achieved through the control method based on meta reinforcement learning when the task requirements change.

This invention discloses a wind-field-based energy-efficiency optimization navigation method for unmanned aerial vehicles (UAVs), comprising the following steps: Step 1, constructing a coupled model of UAV dynamics, wind field, and energy consumption; Step 2, global spatiotemporal planning and task decomposition; Step 3, constructing a reinforcement learning decision model; Step 4, local decision-making and flight control. This invention employs a spatiotemporal task decomposition method based on the 4D A* algorithm, introducing a time dimension into the three-dimensional space, breaking through the limitations of static environments in traditional planning. By decomposing long-endurance tasks into short-range sub-tasks, it solves the problems of sparse rewards and convergence difficulties in end-to-end reinforcement learning, significantly improving training efficiency. The reinforcement learning control strategy using acceleration output eliminates redundant trajectory optimization steps, significantly improving response speed under sudden wind field changes. Combined with a composite reward function that includes wind field utilization and terrain constraints, it balances low energy consumption and high safety during local flight.

This invention provides a multi-agent cooperative underwater logistics system and method based on reinforcement learning, applicable to the fields of underwater robot cooperative control and intelligent logistics technology. The system integrates six modules: task management, multimodal communication, cooperative localization, path planning, energy management, and fault handling. Through a dynamic switching mechanism in multimodal communication, it adaptively selects between acoustic, optical, and magnetic induction modes to ensure communication reliability in underwater environments. A distributed cooperative localization method based on dynamic weights is employed, optimizing weight allocation by considering communication quality, battery power, and computing power to achieve cooperative localization. A multi-agent reinforcement learning controller with a multi-branch attention mechanism is introduced to achieve real-time obstacle avoidance, formation maintenance, and task-oriented cooperative path planning. An intelligent charging strategy based on multi-level battery thresholds and task urgency supports continuous system operation. A sliding window fault detection and adaptive task reassignment mechanism enhance the system's fault tolerance and robustness.

The application discloses a kind of reinforcement learning batch process control methods based on improved AC algorithm, it is related to the field of deep reinforcement learning and batch process control field.The method will be based on reinforcement learning method The batch process control is modeled as an optimal control problem on the basis of Markov decision process;Control action constraint is introduced in the reward function of reinforcement learning controller, the number of effective reward samples is increased to improve the learning rate of reinforcement learning controller, and the control cycle is shortened.Priority sampling method is introduced in the Actor-Critic algorithm of deep reinforcement learning, and a soft actor-critic algorithm with priority sampling is proposed to improve the sampling efficiency in the experience replay pool.The present application does not depend on prior knowledge and process model, and can realize model-free control of batch process.

The application discloses a soybean seed radio frequency drying intelligent monitoring system based on computer vision and deep reinforcement learning, and relates to the technical field of agricultural products processing. The data processing hub is connected with a drying execution module, a visual monitoring module and a reinforcement learning control module, is used for receiving data of the modules and outputting control instructions, and realizes phenotype monitoring and temperature control in a soybean seed radio frequency drying process. The application can accurately identify the embryo cracks of soybean seeds, accurately monitor color characteristics, solves the problem of low precision of traditional phenotype monitoring methods, and provides reliable data support for the regulation and control of the drying process.

This invention provides a reinforcement learning-based data augmentation method for small target detection. It involves acquiring images of small targets and dividing them into training and validation sets based on image quantity and classification. Hyperparameters of the reinforcement learning controller are set to determine a set of potential image augmentation operations. A two-stage decoupled search is employed: the search space is decomposed into individual parameter search and policy combination search; the individual parameter search space is determined; the individual parameter search is performed; the policy combination search space is determined; within the policy combination search space, the reinforcement learning controller continues the search to obtain the optimal policy combination; and the optimal policy combination is used for data augmentation. This method significantly reduces search time, enabling reinforcement learning to be effectively used in data augmentation, enriching the dataset, and improving the accuracy of small target detection by selecting the optimal policy combination.

A method and system for controlling engine cooling in a vehicle includes receiving temperature data from a temperature sensor (104) that monitors the engine temperature. A control unit (106) processes the data using a reinforcement learning model (108) comprising a probabilistic network (110) and a deterministic network (112). The probabilistic network (110) determines a probability distribution of cooling actions based on the temperature data. Specific cooling actions are selected to control a cooling actuator (114). The deterministic network (112) evaluates the effectiveness of the actions and updates the probability distribution. Continuous control signals are generated to operate the cooling actuator (114) and maintain the engine temperature within a target range.The system enables efficient, adaptable control of engine cooling using reinforcement learning techniques.

This invention provides a method for optimizing BUCK circuit control based on adaptive reward weighting (DQN), comprising the following steps: Step 1, constructing a reinforcement learning control environment for the BUCK circuit; Step 2, designing a reward function based on output voltage error; Step 3, optimizing the reward weight parameters in the reward function using a particle swarm optimization (PSO) algorithm to obtain the optimal reward weight parameters and applying them to the reward function; Step 4, introducing external disturbances into the BUCK circuit control environment and training the DQN agent using the optimized reward function. This invention does not require a precise mathematical model and can achieve adaptive optimization control of the BUCK circuit under different operating conditions, demonstrating significant engineering application value.

This invention discloses a self-optimization method for an intelligent controller based on reinforcement learning, comprising the following steps: S1, constructing the state space, action space, and environmental feedback function of the control system; S2, initializing the improved TD3 control algorithm structure; S3, forming experience tuples and storing them in an experience replay buffer; S4, randomly sampling a batch of experience tuples from the experience replay buffer; S5, if the delayed update condition is met, updating the parameters of the main policy network once; S6, synchronously updating the parameters of the target policy network and the target Q network; S7, deploying the trained main policy network to the intelligent controller and outputting the optimal control action; S8 continuously collecting interaction data and appending it to the experience replay buffer, periodically triggering the policy incremental training process. This invention constructs a reinforcement learning control structure, integrates policy evaluation and online training mechanisms, and achieves adaptive optimization of the controller.

This invention proposes a method for enhancing the stability of power hardware-in-the-loop systems based on DDPG-RP. First, an equivalent circuit model of the system containing grid fault disturbances is established, its dynamic interaction characteristics are analyzed, and the impedance matching conditions for stable operation are determined. Based on this, a phase-compensated and enhanced repetitive-voltage feedforward composite controller is designed. An upper-level intelligent controller is constructed by combining this with a deep deterministic policy gradient algorithm. These two controllers are deployed in the bottom and upper-level control units, respectively, forming a hierarchical collaborative control architecture for grid fault simulation. A test platform is built based on this architecture. After offline training and online deployment of the deep reinforcement learning controller, a three-phase short-circuit fault condition is simulated using a high-power grid-connected interface device to verify its dynamic response and stability. This invention, relying on hierarchical collaboration and intelligent optimization, effectively solves the problem of traditional control systems struggling to balance stability and performance under complex faults, significantly improving system adaptability and robustness.

This invention discloses a reinforcement learning control method for quadruped robots oriented towards simulation-to-real-world transfer, belonging to the field of intelligent control technology. This invention constructs a time-series observation using a historical sliding window, employing only non-privileged state information from the robot's own sensors as input to the policy network; it dynamically adjusts the complexity of the simulated terrain based on the robot's horizontal walking ability assessment results, injecting seven types of multi-source uncertainties into the parallel simulation environment; it calculates discounted cumulative reward and advantage estimation using state transition data from the parallel environment, iteratively updating the policy and value network parameters through deep reinforcement learning; during the deployment phase, it loads the convergence network to generate control commands, while simultaneously recording operational data for subsequent optimization. This invention integrates non-privileged observation modeling, curriculum learning, and multi-source perturbation injection, significantly improving the stability, robustness, and simulation-to-real-world transfer performance of quadruped robots in non-ideal environments, reducing deployment costs, and enhancing long-term operational feasibility.

This invention discloses a distribution network voltage control method based on safety constraints and online hyperparameter optimization. The invention constructs a distribution network voltage state transition model under no-communication conditions; derives a stability criterion based on Banach's fixed-point theorem and solves for the stability safety constraints of the action gain of each controlled node; configures an independent reinforcement learning control agent for each controlled node and applies stability safety constraints and equipment capacity constraints; trains the agent using a group relative policy optimization algorithm, updating the policy network parameters based on the relative advantage value of the trajectory within the group; employs a hyperparameter optimization method based on a multi-armed slot machine to optimize the reinforcement learning training hyperparameters online; and achieves cross-scenario migration of the reinforcement learning control agent through parameter remapping and incremental training when the distribution network scenario changes. This invention can realize distribution network voltage control, reduce the risk of voltage instability, reduce the computational overhead of hyperparameter optimization, and achieve rapid cross-scenario migration and adaptation.

The application discloses a water channel pipe network robot control method based on reinforcement learning, relates to the technical field of robot intelligent control, and comprises the following steps: collecting multi-source data of a water network pipeline and preprocessing the multi-source data to generate basic observation data; based on robot motion state data and driving feedback data, a fluid disturbance prediction model is constructed to inversely obtain fluid characteristic disturbance information by fluid action, and the stability state of the robot is constructed in combination with robot posture data; the basic observation data, the fluid disturbance characteristic information and the stability state are adaptively fused to construct an enhanced environment state, the pipe structure features are identified based on the enhanced environment state, the pipe network topology memory structure is updated in a graph structure learning mode, and path guide information is generated. Through the multi-target collaborative analysis of the reinforcement learning control driven based on the enhanced environment state, the control stability and adaptability of the robot in the complex pipe network environment are effectively improved.

This invention relates to the field of power systems and energy storage technology. It discloses a method, system, equipment, and medium for constructing a power storage system based on new energy operation. The method includes: classifying the operating conditions of energy storage battery clusters according to operating data; identifying the state transition cost of each energy storage battery cluster using a state transition model based on the classified operating conditions and operating data; performing rolling optimization with the goal of minimizing the total system operating cost, wherein the total system operating cost includes a cost item calculated based on the state transition cost, and outputting a reference power command for each energy storage battery cluster; and generating the final power control signal for each energy storage battery cluster using a reinforcement learning controller based on the reference power command and the state transition cost. This method can quantify the internal electrochemical state transition cost of the battery and deeply integrate it with system-level economic dispatch and device-level intelligent control to maximize the value of the energy storage system throughout its entire lifecycle.

This invention relates to the field of intelligent control technology for industrial processes, specifically disclosing a reinforcement learning control method for wet leaching processes based on feedback enhancement and operating condition guidance. By defining extended state variables to establish an interactive environment conforming to Markov properties, the theoretical adaptability of the reinforcement learning method is ensured. By introducing control error, control error integral, and control error derivative into the state space to construct a feedback-enhanced state space, cross-scenario generalization control performance is improved. By designing a comprehensive reward function covering steady-state performance, dynamic performance, and dynamic-steady-state balance performance, an effective balance between the dynamic and steady-state performance requirements of the control system is achieved. Through an action range selection mechanism guided by operating condition knowledge, multi-operating-condition control is achieved with a single reinforcement learning agent, avoiding the high training costs and fragmented operating-condition data problems associated with integrating multiple reinforcement learning controllers.

The present application relates to the technical field of health management of ship power system, and in particular to a ship diesel engine control and diagnosis collaborative method based on double-agent reinforcement learning, comprising the following steps: running state data acquisition and observation space construction, local feature coding and intention vector generation, two-way intention communication and feature fusion: each agent carries out cross-level information interaction, exchanges each other's intention vector, collaborative action generation and environment interaction, based on the fused features, two agents respectively output the decision action of the current time step through the action mapping head, collaborative action generation and environment interaction, and the network parameters are optimized through continuous interaction in the simulation environment; online deployment and execution of the control and diagnosis collaborative strategy, compared with the traditional control and diagnosis separation mode and single reinforcement learning control algorithm, the present application can not only effectively realize active fault tolerance and inhibit control performance degradation, but also improve the diagnosis accuracy rate of early weak faults and system robustness.

This invention relates to the field of fuel cell control technology, specifically a method and storage medium for real-time optimization of hydrogen fuel cell operating efficiency. The method includes: collecting multi-sensor operating data such as stack voltage and current, gas path pressure and flow rate, temperature, and system load requirements; performing data synchronization alignment and outlier cleaning to construct a high-fidelity time-series operating feature sequence; accurately extracting key features corresponding to three types of polarization and auxiliary features for thermal and water management; relying on an improved deep deterministic strategy gradient reinforcement learning control model; and continuously interacting with the operating environment in real time to uncover the dynamic mapping relationship between operating parameters and system efficiency. Based on the model's real-time output decision commands, the method dynamically adjusts the hydrogen circulation pump speed, air compressor power, back pressure valve opening, and cooling fan duty cycle. This method can precisely sense internal battery operating losses and changes in thermal and humidity conditions, achieving coordinated dynamic control of multiple component parameters and stabilizing the fuel cell's continuous and efficient operating state.

The present application relates to the technical field of energy storage control, in particular to a hybrid energy storage device cooperative control method and system based on machine learning. First, the expected workload spectrum is obtained according to historical operation data and service period, then the dominant control mode is determined and the mode health degree is generated according to the expected workload spectrum and the current health state of the system, the real-time state characteristics are extracted to input the reinforcement learning controller branch to obtain power distribution instructions, the life consumption increment and the comprehensive quality coefficient are calculated, and finally the effectiveness of the instructions is evaluated according to the quality coefficient threshold to realize control mode switching or controller parameter updating. By implementing the present application, the optimal balance of asset health and comprehensive benefit of the mobile storage and charging device hybrid energy storage system in the whole life cycle can be realized.

The application discloses a ship reinforcement learning control method based on motion prediction and belongs to the technical field of ship autonomous control; the method comprises the following steps: constructing a limited time domain motion prediction module trained by a sequence regressor as a prior component, inputting a current state and a candidate action sequence, and outputting a short-time prediction trajectory; combining a prediction module output trajectory and a task cost function into a multi-step value evaluator, realizing short-domain cumulative cost estimation of a state-action sequence, and performing online training of the sequence regressor; designing a sequence type strategy network and training the sequence type strategy network with the goal of multi-step value maximization, and outputting an action sequence with a length of N; only the first step action of the action sequence is issued during online control, so that the low delay requirement is met, and interactive data is written back to an experience replay pool to continuously optimize the model. The application improves the utilization rate of prior sea trial data, reduces the cumulative risk of estimation bias, realizes robust control under a ship black box model, and is suitable for ship track keeping, dynamic positioning and other autonomous control scenes.

The application relates to a robot reinforcement learning control method, system, device and storage medium. The method comprises the following steps: obtaining real interaction data of a robot control task, constructing an environment dynamics model based on a probability neural network set by using the real interaction data, and capturing the uncertainty of a real environment and generating virtual data by using the environment dynamics model; combining the virtual data with the real interaction data to generate mixed training samples, training a parameterized policy network and a value network by using the mixed training samples; performing model-based reinforcement learning on the policy network and the value network, introducing a dynamic relative entropy regularization mechanism in the policy optimization process of the reinforcement learning, and generating an optimal policy of the robot control task. By introducing the dynamic relative entropy regularization constraint mechanism in the model-based policy optimization framework, the cumulative influence of the environment model prediction error is inhibited, and the training instability problem caused by the model bias is effectively alleviated.

The present application relates to the technical field of intelligent control of hydroelectric power generation, in particular to a water turbine speed regulation adaptive control system based on multi-source data, comprising: data acquisition, feature extraction, coupling relationship analysis, water head prediction, state fusion and instruction generation module. The system collects multiple types of real-time monitoring data of the water turbine, extracts the unit operation stability features by using the harmonic analysis algorithm adaptive to the tracking optimization of the water turbine frequency and its multiple frequency, constructs the hydraulic-mechanical dynamic coupling feature matrix and generates the short-term water head prediction sequence. The multiple features and the prediction sequence are fused to form a comprehensive control state representation, and the speed regulation control instruction is output by the deep reinforcement learning control strategy generator. The system can accurately identify the unit vibration state, match the hydraulic-mechanical coupling change law, adapt to the water head fluctuation trend, make the speed regulation instruction more suitable for the unit real-time working condition, and improve the dynamic adaptation effect of speed regulation.

The application belongs to the technical field of bionic robot control and relates to a reinforcement learning-based bionic mechanical fish prediction rectification navigation control method and system, which comprises the following steps: constructing a state vector required for reinforcement learning control; updating a next state according to environment feedback, calculating an instant reward, a termination flag and task stage information to obtain a state transition sample; storing the state transition sample into an experience sample set, sampling training data, updating a control strategy; when a task starts a World Model, predicting a subsequent state change trend, designing a reward according to a task type and performing prediction rectification; repeatedly performing the steps of state acquisition, action output, strategy updating and prediction rectification until a converged control model of a target navigation task is obtained, and outputting multi-target navigation, dynamic target tracking and stable circling results, and expanding from two dimensions to three-dimensional obstacle environment. The application can maintain basic navigation performance and improve stability in a long-time domain task.

This invention relates to the field of intelligent control and discloses an adaptive PID control method incorporating nonlinear compensation and disturbance feedforward. The method involves acquiring feedback signals, actuator states, and disturbance signals; establishing a dynamic nonlinear mapping model of the actuator using a recurrent neural network; achieving lightweighting and online updates through knowledge distillation and elastic weight consolidation; calculating the feedforward compensation amount based on the disturbance state using a multi-mode feedforward compensation model; obtaining the basic control quantity through a deep reinforcement learning controller using control error, feedforward compensation amount, historical control quantity, and actuator state as inputs; inputting the basic control quantity into an explicit inverse model, correcting it through a residual compensation network, and then outputting it to the actuator; and incrementally learning and updating the model and controller using actual response data. This invention achieves nonlinear compensation, disturbance suppression, and adaptive optimization, improving control accuracy, response speed, and reliability.

This invention belongs to the field of robot control and reinforcement learning technology, specifically disclosing a unified reinforcement learning control method and system for modular multi-morphological legged robots. The method includes: constructing legged robot configurations with different numbers of legs, establishing a unified action space and a unified observation space; constructing a topology graph based on the mechanical connection relationships of the target robot joints, using the joint positions and joint velocities as node features, and generating target robot morphological feature vectors through a graph neural network; inputting the morphological feature vectors and the unified observation vectors into a policy network to generate unified joint action control vectors; and combining the action mask of the target robot configuration to generate joint action control vectors for the target robot configuration. This invention enables policy learning for robots with different configurations within a unified training framework, allowing robots with different configurations to complete their respective action outputs under the same policy network.

The application discloses a trajectory tracking control method for an unmanned system without an initial stabilizing learning strategy, relates to the technical field of unmanned systems, and can overcome the dependence of an initial stabilizing control strategy on a system model and ensure that an unmanned vehicle system realizes trajectory tracking of a leader under a designed reinforcement learning controller. First, an unmanned system model is established; a trajectory tracking controller of a follower and a corresponding cost function are designed to realize optimal trajectory tracking of the follower; an offset factor is introduced on the basis of an augmented system of the unmanned system to construct a new closed-loop system; system data are generated according to the new closed-loop system, and the system data are collected; and a policy iteration learning algorithm is used to find an optimal control strategy to realize model-free trajectory tracking control without dependence on an initial stabilizing control strategy.

Systems and methods for training rate control neural networks through reinforcement learning. During training, reward values for training examples are generated from the current performance of the rate control neural network in encoding the video in the training example and the historical performance of the rate control neural network in encoding the video in the training example.

This invention discloses an adaptive robust control method and system for auxiliary equipment systems in thermal power plants under varying operating conditions, specifically relating to the field of auxiliary equipment system control technology in thermal power plants. The method includes the following steps: S1, constructing a dual-network architecture; S2, using the prediction model network to perform multi-step prediction of the controlled variables of the auxiliary equipment system, obtaining a predicted state sequence; S3, inputting the current operating condition characteristics, the controlled variable deviation, and the predicted state sequence into the controller network, and the controller network outputting the original control quantity. This invention achieves accurate prediction and adaptive control of the dynamic characteristics of the auxiliary equipment system; ensures the stability of the reinforcement learning controller output, overcoming the insufficient security of traditional black-box models; utilizes a covariance matrix adaptive evolution strategy to perform offline optimization and incremental updates of controller parameters, achieving continuous self-optimization of the controller; and simultaneously realizes equipment degradation perception and active compensation, ensuring continuous operation of the system under sensor failure conditions.