102 results about "Sequential decision" patented technology

An exemplary system and method that generate a finite state automaton of a controller for sequential decision-making of a computing device for a user-requested task, the generation using automaton-based representations derived from outputs of a large-scale generative language model. The exemplary system and method receive a user command and (i) construct the finite state automaton by parsing GLM responses to prompts / queries for candidate steps and sub-steps for the task, and (ii) verify the finite state automaton, e.g., for logical errors, safety specification, performance specification, and intent of the user, and / or update the finite state automaton to meet such specifications. The state machine can be synthesized on the fly for real-time operation via the GLM outputs and evaluated in the same pipeline operation against physical models, safety models, etc., that can be combined to iteratively modify the state machine until it meets the verification requirements.

Provided are a reinforcement learning method and system based on sequential decision-making, a device, and a medium. The method includes: preprocessing historical trajectory data of reinforcement learning to generate preprocessed historical trajectory data to train a Transformer network model, and transforming the reinforcement learning into a language conversion model task by using a text conversion mechanism in the Transformer network model, to generate a trained Transformer network model, where the Transformer network model is used to predict action information at a next time point in a historical environment, and determine a maximum target reward value in a historical environmental state to obtain a complete trajectory in the historical environmental state; and predicting action information at the next time point in a real environmental state by using the trained Transformer network model, to obtain a complete trajectory in the real environmental state.

The invention provides a near space communication network relay unmanned aerial vehicle intelligent trajectory planning method, which comprises the following steps: step 1, designing a generation method of a communication network topological graph: constructing a directed graph called as a full connection graph, adding from a base station node to each relay unmanned aerial vehicle, and carrying out trajectory planning between the relay unmanned aerial vehicles; and a directed edge E from each relay unmanned aerial vehicle to each ground user, wherein the edge between the relay unmanned aerial vehicles has two directions; 2, converting the trajectory planning process of the stratospheric airship into a sequential decision process, and constructing a multi-agent Markov decision process model; 3, designing a strategy network and a value network, and constructing a relay unmanned aerial vehicle trajectory planning virtual simulation environment; and step 4, based on a soft actor commentator algorithm, training the intelligent agent to find an optimal strategy. According to the method, the position of the relay unmanned aerial vehicle can be planned in real time, the communication link is adjusted, and the network connectivity is kept.

The invention relates to a dexterous two-hand cooperative control method based on multi-agent reinforcement learning, and the method comprises the steps: initializing a neural network and an environment of dexterous two hands, and collecting the data during environment interaction; based on data during environment interaction, the joints perform sequential decision actions according to a greedy strategy; evaluating the decision action through a Q-value distribution network, and calculating a reward return at the same time; on the basis of reward return, entropy regularization is introduced, a strategy gradient optimization method is used, and a joint strategy is updated; and based on a preset condition, ending updating of the joint strategy, and outputting an optimal cooperative control strategy. The method is suitable for complex scenes such as grabbing, rotating and assembling, and has high cooperation efficiency and environmental adaptability.

Imitation learning, or artificial intelligence-based learning from demonstration, aims to acquire an agent policy by observing and mimicking the behavior demonstrated in expert demonstrations. Imitation learning can be used to generate reliable and robust learned policies in a variety of tasks involving sequential decision-making, such as autonomous driving and robotics tasks. However, current imitation learning solutions are limited in their ability to generalize states or goals unseen from the expert's demonstrations. The present disclosure integrates a diffusion model into generative adversarial imitation learning, which, in terms of prior solutions, can provide superior performance in generalizing to states or goals unseen from the expert's demonstrations, provide data efficiency for varying the amounts of available expert data, and capture more robust and smoother rewards.

Techniques are provided for dynamic prediction-based regression optimization. In one embodiment, the techniques involve determining, via a process model, a variable state of the process model, wherein the variable state includes a first input state variable, a first output state variable, and a first control parameter, generating, via a short-term prediction module, a first prediction of a first update of the variable state, generating, via a terminal value prediction module, a second prediction of a second update to the variable state, generating, via a control optimization module, a second control parameter based on the first prediction and the second prediction, and controlling, via a processor, a production process of the process model based on the second control parameter.

The invention relates to the technical field of data processing, and particularly discloses a matrix pre-sequencing method and system based on reinforcement learning and a storage medium, which are used for modeling a matrix pre-sequencing problem into a sequence decision process, and learning an optimal sequencing strategy by utilizing a reinforcement learning agent through interaction with a matrix environment. The state of the agent is defined by the structural features of the matrix, and the action corresponds to the selection of nodes that have not been eliminated. The reward function aims at minimizing the ratio of the number of filling elements to the matrix dimension in the calculation process. The performance of the method is compared with that of a traditional heuristic algorithm in an experiment, and the result shows that in specific test cases such as a large-scale circuit model, the pre-sequencing method based on reinforcement learning has the advantages in the aspects of reducing the number of filling elements and shortening the calculation time.

A method and system for performing a task requiring a plurality of sequential decision-making operations by an autonomous agent are disclosed. The method includes acquiring a portion of offline data from a database, and discovering a set of separate skills. The method further includes segmenting the offline data and segmenting the offline data according to skill, and abstracting the segmented offline data into reusable functions. From the abstracted reusable functions, distilling primitives for each skill and distilling reusable tips in various functions. Modifying skill based text-based and code-based policies for augmenting the autonomous agent, and processing a task requiring a set of sequential decision-making operations using the augmented autonomous agents.

Embodiments of the present disclosure are directed to systems and methods to display a plurality of items in a spatial layout, monitor an eye gaze of the user while making sequential decisions about the plurality of items, determine that a drift has occurred, automatically regularize a plurality of alternative choice options, learn and identify an upper boundary and a lower boundary based on whether the user chooses or rejects at least one alternative choice, model the drift using machine learning based at least in part on a history of user decision responses, predict a drift direction of the drift and a relevance impact of different attributes into a drift state, classify the drift states into a risk-averse classification and a risk-seeking classification by clustering drift parameters over different behavioral measures to learn an attentive drift of the user, and output a desirable behavior model for machine learning.

Vehicle decision-making is analyzed and can be used to modify a decision-making process. For subsets of features comprising a vehicle operational scenario, a first value is generated that quantifies behavior of an artificial intelligence (AI) agent as the AI agent performs a sequence of actions within a first world model based on a complete set of observations for the subset of features. A first world model is a copy of a second world model for sequential decision making. A second value is generated that quantifies behavior of an AI agent as the AI agent performs a sequence of actions in the second world model based on an incomplete set of observations for the subset of features. A difference between the first and second values determines the impact of individual features on the AI agent within the second world model. A decision-making process of the AI agent can be updated.

The invention belongs to the technical field of power systems, and more specifically relates to a power grid intra-day scheduling optimization method considering photovoltaic output uncertainty in severe weather, and the method employs an A2C algorithm framework and a knowledge migration technology to solve and optimize a power grid scheduling problem of photovoltaic output uncertainty in severe weather. The method comprises the steps of firstly determining a sequential decision process of power grid intra-day scheduling and establishing a mathematical model, and then determining a power grid intra-day scheduling optimization target and constraint conditions; secondly, using a CGAN scene generation method to generate a training sample, and using an A2C algorithm framework to optimize a scheduling agent capable of adapting to various source load scenes; and finally, optimization of a specific intra-day scheduling agent is improved by using transfer learning, so that the problem that the accuracy of prediction data in a large-scale photovoltaic access power system causes great influence on stable operation of a power grid is solved, and the power grid is ensured to operate in a relatively economical manner on the premise of safety.

The invention discloses a dynamic multi-interval charging and discharging voltage threshold value adjusting method based on deep reinforcement learning by adopting a multi-train super-capacitor energy storage system aiming at the problem of traction network voltage safety and stability caused by high power and high energy of an urban rail transit multi-train energy storage system circuit model. According to a super-capacitor double-cascade control part of charge and discharge voltage thresholds, the charge and discharge voltage thresholds of a multi-train super-capacitor energy storage system (SC-ESSs) are dynamically adjusted according to intervals in real time by adopting a deep reinforcement learning (Soft Actor-Critic SAC) algorithm, and online strategy updating and sequential decision execution are further designed, so that the online strategy updating and sequential decision execution are realized. On-line strategy learning and optimization are carried out by combining'trial and error 'and'feedback' mechanisms of the agent, so that the training efficiency and optimization accuracy of the agent are improved, the energy-saving and voltage-stabilizing characteristics of the traction network are effectively improved, the charging and discharging current of the ground super capacitor can be stably controlled, the over-charging and over-discharging phenomena are prevented, and the service life of an energy storage element of the super capacitor is prolonged.

The invention provides a sequence decision strategy acquisition method and system for high-neuroplasm crowds. The method comprises the following steps: establishing a full-screen drawing canvas, and loading a task set comprising a decision map type, an emotion titer ratio, a reward parameter, a random seed and a sequence control parameter; collecting user gender selection and binding a corresponding emotion picture library; generating a decision map based on a Markov decision principle under the control of the parameter set, defining nodes and connecting edges, randomly scheduling and laying emotion pictures according to a titer ratio, and assigning rewards to form an emotion and reward coupling environment; based on the environment, recording a mouse position and a mouse path in an execution exploration stage, recording key pressing time, an action direction and an award in an execution stage, and establishing step sequence association according to a timestamp to form sequence decision data; a Latin square balance control task sequence is adopted, and the process is propelled under the trigger of a user; and finally, filing the data in a binary system and a text file for subsequent analysis. According to the invention, the authenticity and repeatability of data are improved.

The invention provides a receiving-end power grid load shedding control method considering a transient voltage recovery demand, belongs to response driving type load shedding control, and solves the technical problem of insufficient system voltage regulation capability caused by large-scale access of new energy to a receiving-end power grid and mass replacement of a traditional fossil energy synchronous unit. According to the technical scheme, the method comprises the following steps: (1) establishing a Markov sequential decision process model for transient voltage control; (2) designing states, actions and rewards in the Markov process; (3) establishing a reinforcement learning model based on the Double DQN; (4) performing off-line centralized learning and training of corresponding agents; and (5) forming a distributed self-adaptive load shedding control method. The method has the beneficial effects that the load shedding controller can be comprehensively deployed at each load station, and a closed-loop distributed load shedding control strategy of'monitoring-judgment-decision-control 'is realized, so that the overall voltage recovery of a power grid is promoted.

The invention relates to the technical field of shield tunneling machine cutter maintenance, and discloses a shield tunneling machine cutter maintenance decision-making method and device, a medium and a computer program product, and the method comprises the steps: initializing a state; defining a sequence decision process in a simulation environment: according to a maintenance action and an updated state calculation moment award, storing an experience tuple into an experience playback buffer area; randomly sampling a batch of empirical tuples from an empirical playback buffer area, minimizing a loss function by using a gradient descent method, and updating policy network parameters of an inspection agent and a maintenance agent; and repeating the above process until a preset termination condition is reached. According to the method, the problem of passive agents in multi-agent reinforcement learning is solved by constructing a sequential simulation environment and adopting a value decomposition network architecture, and the inspection interval can be dynamically adjusted according to different degradation conditions of the shield tunneling machine cutter, so that the maintenance strategy is optimized, and the maintenance cost is reduced.

The invention discloses a laboratory intelligent material scheduling system and method based on artificial intelligence. The system comprises an AI-driven intelligent prediction and decision-making system, an automatic scheduling and ordering execution system, a solvent recovery optimization system and a data and service support layer. According to the method, inventory prediction, replenishment decision and recovery process parameter optimization are uniformly modeled into a sequential decision problem by adopting an SAC reinforcement learning algorithm, so that the system can perform collaborative optimization and dynamic adjustment based on multi-dimensional state information; through fusion of time sequence analysis, graph database association mining and a multi-objective optimization model, material scheduling meets experimental requirements; by constructing a closed-loop autonomous system of'perception-prediction-decision-execution-optimization 'and a micro-service architecture, the system has continuous online learning and self-evolution capabilities, and finally, the intelligent level, the operation efficiency and the economic and environmental protection benefits of laboratory material management are comprehensively improved.

This invention belongs to the field of anti-spoofing and interference technology for satellite navigation and inertial navigation integrated systems. It provides a GNSS / INS tightly coupled adaptive anti-spoofing method based on generalized likelihood ratio and sequential decision-making, applicable to multi-satellite spoofing interference and micro-ramp spoofing scenarios in autonomous driving. The method includes: first, constructing a GNSS / INS tightly coupled model; second, constructing a GNSS spoofing and interference model; third, constructing a generalized likelihood ratio statistic and designing an iterative isolation strategy, improving detection accuracy through sequential hypothesis testing; and fourth, embedding the detection results into a filter and constructing an adaptive factor to achieve anomaly weighting and prediction adjustment. This method integrates spoofing detection and spoofing suppression, significantly improving the anti-spoofing capability of the navigation system.

The application provides a time sequence consistent evidence organization method for time limit question answering, comprising: obtaining time limit question answering text; obtaining a candidate evidence set; obtaining priori scores corresponding to each candidate evidence; constructing action scoring references corresponding to the candidate evidence based on the priori scores, and modeling an evidence selection process as a sequential decision process; in each round of candidate evidence selection, constructing a current state according to a current selected evidence set, an unselected candidate set, a time slot coverage state, a semantic abstract state and a remaining budget, calculating action probabilities corresponding to candidate selection actions and a stop action through a strategy network, and selecting a target candidate evidence to join the selected evidence set based on the action probabilities, or selecting the stop action based on the action probabilities; when the strategy network selects the stop action, or the current remaining budget is less than a preset budget threshold, or the current selection round number reaches a preset maximum selection round number, outputting the selected evidence set and generating a corresponding question answering result.

The application relates to a single-chip intelligent adjustment control method and system based on linkage of environmental parameters. The method realizes real-time calculation of an environmental deviation vector through a lightweight atlas of a preset device mutual exclusion relationship and an environmental influence relationship of a single-chip microcomputer, and obtains a device operation urgency through weighted accumulation after influence direction matching, thereby estimating a required operation time length and expected power to form a unified request. Then, future environmental states are deduced in a prediction time domain by using the influence relationship, hard boundaries are formed by using mutual exclusion constraints and an upper limit of household power, and a sequential decision is made on all requests to generate an execution sequence and implement control, so that real-time resolution of device mutual exclusion conflicts, peak shaving and valley filling of household power loads and prospective linkage maintenance of indoor comfort are realized under extremely low computing resources.

A dynamic task scheduling method for heterogeneous multi-machine systems based on multi-agent reinforcement learning includes the following steps: 1. Inputting task information and decomposing all tasks into multiple atomic tasks with different capability requirements that can be executed by a single robot; 2. Establishing an adhesion utility evaluation model and heterogeneous priorities to create a mathematical optimization model; 3. Establishing a local perception domain model and a Markov decision model based on the sequential decision-making process of scheduling decision points; 4. Building a heterogeneous multi-machine system scheduling environment and combining multiple advanced training strategies to learn a deep reinforcement learning solver for heterogeneous multi-machine task scheduling through the interaction between the environment and the agents; 5. Using the deep reinforcement learning solver, sequentially outputting actions for each scheduling decision point for the scheduling instance to obtain a dynamic scheduling solution. This method can handle the uncertainty in the task execution process by inserting dynamic scheduling decision points for dynamic events, thereby achieving fast and robust dynamic task scheduling.

The invention discloses a multi-agent-based micro-grid energy management method and device containing flexible resources, and a storage medium. The method comprises the following steps: step 1, determining each main body of a micro-grid; 2, constructing a state space, an action space and a reward function of each main body in the three main bodies, thereby constructing a sequential decision model of the micro-grid intelligent body under a GPRO algorithm framework; 3, determining an objective function and constraint conditions; step 4, generating a training sample, and performing multiple rounds of training on the sequential decision model of the micro-grid intelligent agent under a GPRO algorithm framework through the training sample to obtain a micro-grid energy management model; and 5, inputting the current state of each main body in the micro-grid to be managed into the micro-grid energy management model to realize micro-grid energy management. The device and the storage medium are used for implementing the method. According to the method, flexible resources on the load side are fully considered, memory occupation is reduced, and training resources are remarkably reduced.

The present application discloses an optimization control method, device and medium for unmanned aerial vehicle (UAV) information collection, which includes: defining the information collection method of the UAV, establishing a set of UAV flight actions, and constructing an optimization control problem for UAV information collection; obtaining the action information and location information taken by the UAV at a certain moment, and constructing a model for the total amount of information collected by the UAV; constructing a sequential optimization decision model based on the model for the total amount of information collected by the UAV, and modeling the UAV information collection optimization control problem as a problem of finding the optimal strategy for the sequential optimization decision problem; and reconstructing the sequential decision problem as a steady-state reinforcement learning problem and establishing a robust reinforcement learning model; based on the robust reinforcement learning model, using a robust Q-value learning algorithm to solve the problem, and obtaining the optimal control strategy for information collection, which can collect information quickly and effectively, adapt to various scenarios and environments, and have good scalability.

The invention relates to a multi-agent cooperative communication method based on dynamic priority scheduling, and the method comprises the steps: determining an initial priority through priority prediction and distributed auction, carrying out the local optimization based on dependence conflicts, carrying out the real-time adjustment for tiny dependence changes, and achieving the dynamic priority distribution; then, the intelligent agents make a sequence decision according to a priority sequence, an autoregressive action decoder is used for carrying out action inference, a message generation network and a message reconstruction network are combined, information transmission between the intelligent agents is optimized, and the overall performance is improved in a mode that all modules are trained in a centralized mode; through the design, the problems of limitation of a fixed priority sequence, lack of dynamic adjustment and difficulty in coordination in multi-agent communication are solved, the adaptability and flexibility of a multi-agent system in a dynamic environment are improved, the efficiency of information transmission and decision-making processes among the agents is improved, and the communication efficiency is improved. Therefore, the cooperation efficiency and decision-making ability in the multi-agent environment are finally improved, and efficient cooperation of the multi-agents is realized.

The invention discloses a structural grid quality optimization method based on deep reinforcement learning, and relates to the field of computer-aided engineering and computational fluid mechanics pretreatment. The problem of position optimization of singular points in a structural grid is modeled as a sequential decision process; grid features are extracted through a state sensing module, an intelligent agent strategy network outputs singular point position movement actions, grids are updated after the environment is executed, reward values based on quality changes are calculated, and the strategy network is updated after interaction experience is stored. A pre-trained expert strategy network is introduced to accelerate training convergence, and suggested actions of the expert strategy network are introduced into a reward function to guide an intelligent agent exploration direction. After training is completed, the fixed strategy network can be rapidly deployed to optimization of a new geometric model. According to the method, automatic global optimization of singular point layout is realized, the grid quality and optimization efficiency are remarkably improved, and the trained model has good generalization ability and can be widely applied to structured grid generation.

A computer-implemented method, computer program product, and computer system for automated model predictive control. The computer system trains multiple step look-ahead regression models, using historical states and historical actions for a to-be-optimized system, for each timestep of a past time horizon. Regression models may be either linear or nonlinear in order to capture process dynamics and nonlinearity. The computer system generates optimization constraints for each timestep of a future time horizon. The computer system generates optimization variables, based on the multiple step look-ahead regression models, for each timestep of the future time horizon. The computer system constructs a mixed integer linear programming based optimization model that includes an objective function, the optimization constraints, and the optimization variables. Nonlinear regression models are converted into piecewise linear approximation functions. The computer system solves the optimization model to produce actions for the to-be-optimized system, over the future time horizon, and recommend commitment-look-ahead actions.

The invention discloses a privacy protection method and system based on data shuffling in a sequential decision, which are efficient in calculation and are used for solving the dual technical problems of user data privacy leakage and overhigh algorithm calculation complexity when reinforcement learning is adopted to carry out the sequential decision. According to the method, on the basis of introducing a credible shuffling device to guarantee data privacy, a brand-new staged strategy iteration framework which does not need to traverse a strategy space is provided. According to the framework, through an embedded adaptive dichotomy strategy search module and an experimental design method, the exponential traversal search on an active strategy set and the dependence on an external calculation oracle machine are thoroughly avoided. According to the method, the optimal regret boundary equivalent to or even better than centralized differential privacy can be realized in a scene with extremely high privacy protection requirements, meanwhile, the performance of a traditional local differential privacy model is remarkably superior, and the actual deployment feasibility and the calculation efficiency of a privacy protection scheme are greatly improved.

The invention belongs to the technical field of computers, provides a fault risk sequential decision optimization method for a high-availability system, and solves the problems that the missed report / false report cost of a static model is asymmetric and a dynamic decision is missing. Extracting features from logs and performance data in a multi-scale manner, and encoding the features into a state vector z; the modeling comprises MDP, state space, action, '1' table early warning and '0' table no risk, and designing an asymmetric reward function, underreporting penalty and false reporting penalty; based on DQN training, layered playback and balanced sampling are adopted to relieve sample sparsity so as to minimize the TD error of the signal, and iteration is optimal; and inputting, selecting and outputting a judgment result in real time. The method realizes dynamic risk quantitative decision and improves system toughness.

The present invention provides an intelligent trajectory planning method for a near-space communication network relay unmanned aerial vehicle, which includes: Step 1, designing a method for generating a communication network topology graph: constructing a directed graph called a fully connected graph, and adding directed edges E from the base station node to each relay unmanned aerial vehicle, between each relay unmanned aerial vehicle, and from each relay unmanned aerial vehicle to each ground user, where the edges between each relay unmanned aerial vehicle have two directions; Step 2, converting the trajectory planning process of the stratospheric airship into a sequential decision-making process and constructing a multi-agent Markov decision-making process model; Step 3, designing a policy network and a value network and constructing a virtual simulation environment for the trajectory planning of the relay unmanned aerial vehicle; Step 4, training the agent based on the soft actor-critic algorithm to find the optimal policy. The method of the present invention can plan the position of the relay unmanned aerial vehicle in real time, adjust the communication link, and maintain network connectivity.

Vehicle decision-making is analyzed and can be used to modify a decision-making process. For subsets of features comprising a vehicle operational scenario, a first value is generated that quantifies behavior of an artificial intelligence (AI) agent as the AI agent performs a sequence of actions within a first world model based on a complete set of observations for the subset of features. A first world model is a copy of a second world model for sequential decision making. A second value is generated that quantifies behavior of an AI agent as the AI agent performs a sequence of actions in the second world model based on an incomplete set of observations for the subset of features. A difference between the first and second values determines the impact of individual features on the AI agent within the second world model. A decision-making process of the AI agent can be updated.

The invention relates to an unmanned cluster reconnaissance rescue task allocation method based on Monte Carlo tree search, and belongs to the technical field of task planning. According to the method, the task allocation problem in reconnaissance and rescue is expressed as a sequential decision process, heuristic information of the problem is fully utilized, a state space and an action space are designed, the action completeness and the action number are balanced, and calculation resources are saved while the allocation effect is guaranteed. Besides, a self-adaptive simulation number selection mechanism is provided, simulation times can be automatically selected according to the tasks to be distributed and the number of the unmanned aerial vehicles, and a large amount of repeated simulation is avoided, so that the calculation amount of an algorithm is reduced, and the calculation efficiency of task distribution is further improved. According to the method, the task allocation effectiveness of the unmanned system can be enhanced, and the task efficiency of the unmanned system is improved.