186 results about "Action selection" patented technology

The invention discloses an internet constellation dynamic routing optimization method and system, and the method comprises the steps: setting orbit plane parameters and satellite parameters, and carrying out the modeling of an orbit and a constellation; performing link visualization and time-varying graph generation according to ephemeris data and the constructed constellation model; defining a related space, a network and a function based on a deep reinforcement learning algorithm; and based on an MADDPG framework, introducing a multi-agent collaborative decision-making mechanism, and outputting action selection and execution results after collaborative optimization. The system comprises a model construction module, an embedding module, a reinforcement learning design module and a collaborative optimization module. By using the method, the dynamic change of the low-orbit satellite network can be effectively dealt with, satellite resources are fully utilized, and efficient and reliable communication is realized. The method can be widely applied to the field of wireless communication.

The invention provides a deadline constraint-based DAG application dynamic scheduling optimization method and system, and the method comprises the steps: converting an application deadline into an instant reward of each task scheduling through employing a DAG structured encoder, a Transform encoding network based on gating feature fusion, and a multi-action selection deep reinforcement learning task scheduling method based on a pointer network, in combination with a dynamic mask scheme, the mobility of a DAG application and the dynamic nature of edge resources are dealt with, then priority subtask selection and real-time decision of the scheduling position of the priority subtask selection are made, and the completion time and execution energy consumption of the application are reduced. In order to stabilize and accelerate DRL scheduler training, task encoder training and reinforcement learning training are decoupled, and a DAG encoder is pre-trained based on self-supervised learning.

The invention discloses a compilation optimization pass combination optimization method based on deep reinforcement learning. The method comprises the following steps: 1) collecting a reference program data set for various general function tests; 2) constructing a deep reinforcement learning-based compilation optimization pass-combination optimization strategy model for a compilation optimization pass-combination optimization process, wherein the compilation optimization pass-combination optimization strategy model comprises agent D3QN construction and definition of a reward space, a state space and an action space; 3) introducing a composite program feature combined with basic features such as a control flow embedding vector extracted by the graph neural network, an instruction number, a basic block number and the like into the expression of the state space, enriching the expression ability of the state, optimizing the pass based on the state selection and acting on the intermediate expression by the D3QN, generating an award, and obtaining the state space; and repeating the loop of feature extraction-action selection-reward generation until the maximum action upper limit is reached, and generating an optimized pass combination adapted to the program features. The problem that in existing compiling optimization, the compiling optimization effect on part of programs is poor is solved.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network used to select an action to be performed by an agent interacting with an environment. In one aspect, a method includes: receiving a latent representation characterizing a current state of the environment; generating a trajectory of latent representations that starts with the received latent representation; for each latent representation in the trajectory: determining a predicted reward; and processing the state latent representation using a value neural network to generate a predicted state value; determining a corresponding target state value for each latent representation in the trajectory; determining, based on the target state values, an update to the current values of the policy neural network parameters; and determining an update to the current values of the value neural network parameters.

The invention provides a coal bulk loading and unloading efficiency optimization method and system based on machine learning, and relates to the technical field of coal bulk loading and unloading, and the method comprises the steps: recognizing key factors and quantitative association rules between the key factors and the coal bulk loading and unloading efficiency through a large language model; the storage yard decision topological graph is input into a reinforcement learning model, a coal bulk cargo loading and unloading scheme is determined by executing the process of state space coding, action selection and reward function calculation, and a reward function is obtained by taking a predicted value of the coal bulk cargo loading and unloading efficiency in the current period as a target and being constructed according to key factors and quantitative association rules; after the coal bulk cargo loading and unloading scheme is completed, the actual value of the coal bulk cargo loading and unloading efficiency in the current period is collected, the efficiency difference value is calculated, and the coal bulk cargo loading and unloading efficiency in the next period is optimized based on the efficiency difference value. According to the method, the efficiency can be effectively improved while various necessary conditions of loading and unloading operation are accurately met.

The invention relates to a resource allocation and process planning optimization method under complex constraint conditions, which comprises the following steps of: representing various constraint conditions of a process planning process by utilizing a double-layer directed acyclic graph, and generating and constructing an initial feasible solution set meeting all the constraint conditions; initializing a state space, wherein the state space comprises state variables of a process number, an average process length, reward value fluctuation and fitness; noise is collected and coupled to a Q network, action selection is carried out, and low-level heuristic algorithm options are output; generating a new taboo sequence by using the selected low-level heuristic algorithm, and optimizing the current feasible solution; switching a low-level heuristic algorithm according to the improvement amplitude and the execution duration; and outputting an optimal processing feature sequence and process division combination after multiple times of training. Compared with the prior art, the method has the advantages that multiple process decisions can be optimized at the same time, sub-optimal results caused by separate optimization processes are avoided, processing time and resource consumption are remarkably reduced, and production efficiency and scheme adaptability are improved.

The invention discloses an interpretable high-speed ramp intelligent driving vehicle confluence control method. The method comprises the steps that a state space is constructed, wherein the state space comprises the position and speed of a vehicle, lane information and the relative position and speed of adjacent vehicles; human-like decision guidance: training a human-like behavior discriminator based on real driving data by using a random forest model, screening action selection of reinforcement learning, and constructing a sample set containing state vectors and human-like action labels; reinforcement learning control; and carrying out interpretability analysis. Through an SHAP analysis method, an explainable DQN longitudinal acceleration control model in an automatic driving vehicle ramp merging process is provided, and an explainable automatic driving vehicle ramp merging control model is constructed in combination with an merging decision-making model constructed based on a random forest. According to the explainable high-speed ramp automatic driving vehicle confluence control method based on the confluence urgency degree and the environmental historical information, traffic conflicts during highway confluence can be relieved, and therefore the road passing efficiency is effectively improved.

The invention discloses a task execution method and device for a vehicle-mounted environment, equipment and a storage medium, and relates to the technical field of vehicle-mounted interaction, and the method comprises the steps: obtaining a current task target and current screen state information for controlling a vehicle machine screen; based on a pre-constructed candidate action space, the current task target and the current screen state information, constructing verification prompt information conforming to a preset template; inputting the verification prompt information into a target verification model for action verification to obtain an action verification score; determining a target action from the candidate action space according to the action verification score; and executing the target action to complete the current task target, reducing the delay of the task by changing the large language model from generating a long sequence to outputting an action score, and selecting an optimal action by evaluating all possible actions, thereby reducing illusion and errors when the large language model is directly generated, and improving the success rate of the task.

The invention discloses a multi-target reinforcement learning method based on quasi-measurement and planning, which comprises the steps of constructing a representation function, a quasi-measurement value function and a hidden space dynamics model, and performing joint training by utilizing transfer data sampled from environment interaction and a target state, the quasi-measurement value function being optimized to fit the cost from the state to the target, and the quasi-measurement value function being optimized to fit the target state; the distance measurement standard is used for optimizing the hidden space dynamic model; during decision making, action selection optimization is carried out by utilizing an MPC framework and combining with a CEM, in the process, a future trajectory is predicted by utilizing a hidden space dynamics model, the cost of the predicted trajectory is evaluated by using a quasi-metric function, and a single action applied to a current state is determined according to a lowest-cost planning result obtained by CEM optimization. According to the method, by learning the quasi-metric cost function and the hidden space model and combining model prediction control planning, sparse rewards can be effectively dealt with, and the sample efficiency and final performance of multi-target reinforcement learning are remarkably improved.

The invention discloses a heavy-load freight direct-current electric locomotive on-board platform and a train operation curve calculation method. Step-by-step iterative optimization is carried out through a dynamic programming algorithm. Presetting operation steps and step lengths, and defining a selectable action set in each step length; and with the current state of the train as a starting point, evaluating each action in the action set in each iteration step, and calculating the next train state which may be transferred to. And evaluating all possible train state changes by adopting a target cost function, sorting according to the cost values from small to large, and preferentially selecting the action with the minimum cost value for subsequent calculation. And if all the action cost values of the current step exceed the limit, backtracking to the previous iteration step for reselection. And after iteration to the maximum step number, storing all train state changes, and finally obtaining an optimal train operation curve under the constraint of the target function. The method optimizes a control strategy through a dynamic programming algorithm, improves prediction precision, optimizes action selection, accelerates convergence, reduces redundancy, and improves operation stability, safety and algorithm efficiency.

The invention discloses a fall and turn-over control method and device of a foot type robot and a storage medium. The fall and turn-over control method comprises the steps of firstly obtaining fall posture information of the foot type robot; calling an action selection model to perform multi-stage iterative turnover action prediction based on the falling posture information to obtain an optimal turnover action of each iteration stage, controlling the foot type robot to execute the optimal turnover action of the current iteration stage in each iteration stage, and controlling the foot type robot to execute the optimal turnover action of the current iteration stage after the foot type robot executes each optimal turnover action. The foot-type robot is in a stable posture when the foot-type robot turns over, and the foot-type robot turns over successfully after executing the optimal turning-over action of the last iteration stage. Wherein when the action selection model is called to carry out multi-stage iterative turning-over action prediction based on the falling posture information, the action selection model predicts the optimal turning-over action of the next iteration stage according to the stable posture of the foot robot in the current iteration stage. According to the embodiment, the manual design cost can be reduced, and the applicability is high.

The invention discloses a cold start strategy optimization method and device in no-service computing, equipment and a medium, and relates to the technical field of cloud computing, and the method comprises the steps: determining a reward function of a target cold start strategy model according to a state space and an action space; predicting the number of times of function calling in a future time period to obtain a prediction result; determining action selection of a target deep reinforcement learning algorithm, and determining a target value based on the action selection and a reward function; determining a comprehensive loss function based on the Q value loss function, the Huber loss function and the target loss function; updating the strategy network by using the action change difference value, obtaining an updated network, optimizing the value network through the comprehensive loss function, obtaining an optimized network, determining a target cold start strategy model through the updated network and the optimized network, and optimizing the cold start strategy without service calculation by using the target cold start strategy model. Therefore, the response speed and the resource utilization efficiency of the system are improved.

The invention belongs to the technical field of computer network, artificial intelligence and network flow optimization, and discloses a Q-Learning network path planning method based on multi-dimensional reward and adaptive exploration, which comprises the steps of 1, network environment modeling and parameter initialization, 2, multi-strategy action selection, and 3, multi-strategy action selection. Step 3, calculating a multi-dimensional reward according to a multi-dimensional reward rule in a data packet transmission process; step 4, performing adaptive exploration adjustment, monitoring the search success rate in a predetermined period in real time, and dynamically adjusting the exploration intensity epsilon according to the search success rate; step 5, performing adaptive pruning optimization to finally obtain a candidate action set; and step 6, iteratively updating and finally outputting an optimal routing path and a Q value table. According to the invention, the routing exploration efficiency can be improved, dense learning signals are provided, and the calculation overhead is reduced.

The invention provides a network slice selection method and system based on deep reinforcement learning of a gated cycle unit, and relates to the technical field of wireless communication. The method comprises the following steps: constructing a simulation vehicle-mounted network scene and modeling the simulation vehicle-mounted network scene into a Markov decision process; a dynamic attenuation action selection strategy is adopted for action selection, a time sequence state, actions, rewards and an updated state form a tetrad to be stored in a time sequence experience pool, and the experience pool is managed through a first-in first-out and stratified sampling mechanism; and training a neural network containing a GRU layer based on a sample in the time sequence experience pool, taking a mean square error of a target Q value and a predicted Q value as a loss function optimization parameter, and outputting an optimal slice selection result by utilizing the trained network. According to the invention, through dynamic attenuation strategy balance exploration and utilization, the unbalance problem of a traditional fixed probability mechanism is solved; state time sequence association is reserved by means of a time sequence experience pool, and time sequence breakage caused by isolated samples is avoided; state long-term dependence is effectively captured by using the GRU network, the Q value estimation accuracy is improved, and an efficient solution is provided for slice selection in a vehicle-mounted network environment.

The invention provides a maze robot brain-like cognitive system based on an emotion and memory mechanism, and the system comprises a perception and emotion generation module which obtains the environment state of a robot, and calculates the reflective fear value and cognitive fear value of the robot; the memory module is used for recording short-time memory, working memory and long-time memory of the robot in combination with an anti-thought memory mechanism; the behavior decision-making module is used for comparing the evaluation function based on the cognitive fear value with the energy margin to obtain a behavior decision-making result; the learning module is used for exploring and learning a target node and an energy node to obtain an updated target guide table and an updated energy guide table; and the action selection module is used for updating the target guide table and the energy guide table according to the behavior decision result, and outputting the next action. According to the invention, the survival rate of the robot in the environment learning process can be improved, and long-time learning is realized; and the robot can be prevented from repeatedly exploring in an invalid area, the learning efficiency is improved, the energy consumption is reduced, and self-survival is ensured.

The invention discloses a rail transit intelligent customer service expert mixed large language model, and relates to the technical field of large language models. Step 2, interaction and data sampling; step 3, discrete action training: using a deep Q network to process discrete action selection; 4, performing continuous action training, and optimizing a continuous strategy by using a strategy gradient method; step 5, joint training; according to the method, a deep Q network is used for processing discrete action selection in discrete action training, a strategy gradient method is used for optimizing a continuous strategy in continuous action training, then joint training is matched, collaborative optimization of the two methods is achieved, and the performance of a model in a complex decision problem is improved; in the joint training, through sharing reward signals and state features, data acquisition is carried out on three dimensions of user behavior habits, family identities and consumption habits, so that more accurate training output of the model to the user can be improved.

The invention discloses a multi-target flow dynamic allocation method based on deep learning, which comprises the following steps: collecting network topology data, link operation data and service flow request data to form a network state sequence; a multi-target constraint set is obtained and written into a target constraint description table; constructing a state vector, and dividing the state vector into multiple groups of input sub-vectors; carrying out conjoint analysis on the service flow, and constructing a candidate allocation action set; constructing an improved CMAC network model, and outputting an action value vector; constructing an anti-multi-arm bandit decision-making module, and outputting action selection distribution; and executing multi-target feasible region projection processing to generate a feasible traffic distribution decision. According to the method, the improved CMAC network model is introduced, and the anti-multi-arm bandit decision-making method is combined, so that stable and self-adaptive dynamic distribution of the network traffic under the multi-target constraint and complex network state is realized.

The invention discloses a VVC code rate control algorithm based on deep reinforcement learning, and the algorithm comprises the following steps: importing a video sequence into an encoder, and enabling the video sequence to enter initial frame coding; after the encoder completes the default encoding of the first two frames, the subsequent frame prediction firstly extracts the encoding state information of the previous prediction frame, and a greedy strategy is adopted to perform action selection; an overall reward value is obtained through CTU-level code rate control and an actual coding process in sequence; observing the next state and the last reward, performing TD iteration on the Q value, and adding the Q value into the Q value network after TD iteration; setting an experience playback pool; after the capacity of the experience playback pool reaches a threshold value, randomly sampling from the experience playback pool in batches; resetting the Q value network; and carrying out coding test based on the Q value network obtained by training. The VVC code rate control algorithm based on deep reinforcement learning has a good capability of guiding code rate control coding, and compared with standard code rate control in VTM13.0, the VVC code rate control algorithm based on deep reinforcement learning can bring high rate distortion performance and improve code control precision.

The invention discloses a power system dispatching auxiliary strategy generation method in an extreme scene, which comprises the following steps: (1) constructing a dynamic index system fusing an extreme scene model and a power grid model, and realizing dynamic coupling of three types of indexes through a dynamic weight adjustment mechanism to output a comprehensive risk index; (2) generating a scheduling strategy based on an improved deep Q network, taking a comprehensive risk index as an input, processing a continuous and discrete mixed action space through a dual-network architecture of a decoupling action selection network and a value evaluation network, capturing long-term evolution of a power grid state by adopting a time sequence dependence processing module, and determining a value of the power grid state; physical information constraints are embedded in the model to ensure that the strategy conforms to the physical law of the power system; and (3) on the basis of the scheduling strategy, driving strategy optimization through a reward function including a key load power supply rate, an equipment damage degree and recovery time, combining antagonism training and transfer learning verification, and adapting to an extreme scene to form a closed-loop optimized scheduling strategy.

The invention discloses a multi-agent distributed path searching method and system and computer equipment. The method comprises the following steps: firstly, acquiring surrounding small-range environment information of each agent through a sensor; path planning is realized by using heuristic search, static cost (based on a grid map topological structure) and dynamic cost (based on an updatable observation value of a single agent) are calculated in a grid map, and a path with low cost is selected; strategy generation is realized by using a neural network, effective training can be carried out without setting reward parameters and additional planning, the application universality of the method is effectively improved, and a single agent is controlled by using a distributed method, that is, action selection plans of the single agent are different; finally, after action decoding, action distribution of a single agent is output, and therefore generation of the obstacle avoidance action is achieved. The problem that a traditional method is low in efficiency of searching a group of conflict-free paths is solved, and the method has better generalization ability and better robustness.

The invention discloses a cost-aware workflow task scheduling method based on deep reinforcement learning. The method comprises the following steps: step (1): constructing a workflow task time delay model; step (2): constructing a workflow task cost model; (3) calculating to obtain workflow task embedding; step (4), constructing a deep reinforcement learning task scheduling environment; and step (5), constructing a task scheduling decision network, and training an iterative task scheduling decision network by a reward function through a current task scheduling environment and workflow task embedding until iteration is completed. According to the method, the graph neural network is adopted to obtain task embedding, so that the high-dimensional features of the workflow task are represented; according to the method, the action selection strategy and the priority experience playback strategy are optimized, the CA-DRL can dynamically optimize the execution position of the task according to the task characteristics, the model can achieve more effective balance between task delay and the cost target, and then efficient and economical task scheduling performance is achieved.

The invention provides an unmanned aerial vehicle cluster distributed region search method and system based on role differentiation. The method comprises the following steps: initializing an unmanned aerial vehicle collaborative search task environment; the information unmanned aerial vehicle updates a cluster search cognitive map; task allocation modeling of unmanned aerial vehicle cluster collaborative search; unmanned aerial vehicle tasks are pre-allocated based on CBAA information; updating a cluster topology connection state of the information subgroup; relay unmanned aerial vehicle task allocation based on connectivity maintenance; unmanned aerial vehicle waypoint tracking control based on collision time cooperative guidance; adjusting an information-relay subgroup proportion based on a region coverage rate; and outputting an unmanned aerial vehicle cluster collaborative search result. According to the method, the CBAA algorithm supports the communication of the unmanned aerial vehicle cluster to keep a distributed decision through improved strategies such as removal of disconnected individuals, an action selection tabu table and mobile base station switching; meanwhile, heterogeneous cooperation between information-relay subgroups is utilized, the action space of the unmanned aerial vehicle is released, the persistent exploration capability of a later cluster is enhanced, and local optimum of a search process is avoided.

A system includes one or more processors to receive a first machine learning model from a first computing device (e.g., a neural network, support vector machine, random forest, etc.) and a second machine learning model from a second computing device; execute the first machine learning model to generate a first recommendation and the second machine learning model to generate a second recommendation; adjust one or more weights or parameters of the second machine learning model; receive a request for one or more potential actions at a first user interface presented on a display of the client device; execute the second machine learning model using an account identifier of a user account being used to access the application; and generate a second user interface on the display of the client device comprising the one or more potential actions.

The invention discloses an intelligent routing flow management method and system based on reinforcement learning, and the method comprises the following steps: building a lightweight motion selection model at each network node, and mapping a communication link which can be used by the node into a plurality of forwarding options; the current network state is formed by collecting the delay, packet loss rate and utilization condition of each link in real time; and the system distributes priorities of different forwarding options by utilizing a probability strategy, and performs dynamic adjustment in combination with the change of the network state. And after each round of forwarding, the system updates and optimizes the strategy according to the communication effect, so that a routing decision mechanism which does not depend on central control and can be adaptively adjusted is realized. The method is suitable for a complex and frequently-changing network environment, and the overall transmission efficiency and the system stability are improved.

The invention discloses an intelligent segment routing method based on deep reinforcement learning, which uses an SAC algorithm under an AC framework as a core framework, and uses a traffic matrix of global network state information, combined with network topology and SR label stack depth as a state space of an intelligent agent. The intelligent agent not only considers an action selection strategy of selecting a next switch node by path planning, but also designs an action selection strategy of determining whether a newly added switch node is selected as an adhesion switch node or not. And a reward function form which considers a forwarding path optimization factor and a flow table issuing time overhead factor of the controller to the adhered switch nodes is designed. According to the method, the optimization model capable of solving the path planning scheme and the optimal segment routing adhesion switch node selection strategy at the same time is established under the SDN architecture, the time overhead of the controller for issuing the optimal adhesion switch node flow table is considered, and the flow table issuing speed and the segment routing performance are improved to the maximum extent.

Systems and methods for providing a simulated browsing session by instantiating an AI controlled agent, receiving task data identifying an online task, autonomously determining an online action for advancing the online task using the AI controlled agent and based on the task data, selecting, based on the online action, one or both of a cloud browser or an API for executing the action, executing the online action using one or both of the cloud browser or API to generate a result for advancing the task, determining subsequent online actions for advancing the online task based on one of the task data, frame data generated as the result of executing the action, or raw data generated as the result of executing the online action, and selecting, based on the subsequent online actions, one or both of the cloud browser or the API for executing the subsequent simulated online action.

The application provides a robot dance automatic generation method and system based on music feature analysis, and relates to the technical field of robot control. First, a music feature spectrum covering rhythm, energy and other dimensions is constructed to convert abstract music into structured information understandable by machines, and then a dance semantic label sequence bound to a time axis is generated to provide accurate basis for action selection. By combining with a forward-looking window to predict future music features, the current and future candidate actions are filtered through a multi-objective model to realize the dual matching of action and music micro-rhythm and macro-structure. The finally generated control instruction sequence directly drives the physical robot, which breaks the dependence on video materials, realizes the automatic generation of dance with creativity and artistic expression, and improves the generation efficiency and adaptive flexibility.

This invention discloses a multi-agent distributed path-finding method, system, and computer device. First, sensors are used to acquire information about the small-scale surrounding environment of each agent. Path planning is achieved using heuristic search, calculating static costs (based on the grid map topology) and dynamic costs (based on the updateable observations of individual agents) in a grid map, and selecting low-cost paths. Neural networks are used to generate strategies, and effective training can be performed without setting reward parameters or additional planning, effectively improving the method's applicability. A distributed approach is used to control individual agents, meaning that each agent's action selection plan is unique. Finally, after action decoding, the action distribution of each agent is output, thereby enabling the generation of obstacle avoidance maneuvers. This method overcomes the inefficiency of traditional methods for finding a set of conflict-free paths and exhibits improved generalization and robustness.

To suggest an action executable during a user's free time by extracting the action according to difference in transit time due to a transit method, to a user.SOLUTION: An action proposal system 1 includes: a free time recognition unit 12 which recognizes free time of a user U; a free time starting site recognition unit 13 which recognizes a free time starting point being a site where the user is present at the starting time point of the free time; a candidate action selection unit 15 which selects, as a candidate action to be suggested to the user U, an action executable by the user U while using the free time, and containing a movement between the free time starting site and a destination site by any of a plurality of movement methods and a prescribed activity at the destination site; and a candidate action information providing unit 16 which allows a display device 50 used by the user U to display an action proposal window to display required time period for moving between the free time starting site and the destination site for each of the plurality of movement methods, and contents of the prescribed activity.SELECTED DRAWING: Figure 1