307 results about "Action selection" patented technology

A video localization system localizes actions in videos based on a classification model and an actionness model. The classification model is trained to make predictions of which segments of a video depict an action and to classify the actions in the segments. The actionness model predicts whether any action is occurring in each segment, rather than predicting a particular type of action. This reduces the likelihood that the video localization system over-relies on contextual information in localizing actions in video. Furthermore, the classification model and the actionness model are trained based on weakly-labeled data, thereby reducing the cost and time required to generate training data for the video localization system.

A video localization system localizes actions in videos based on a classification model and an actionness model. The classification model is trained to make predictions of which segments of a video depict an action and to classify the actions in the segments. The actionness model predicts whether any action is occurring in each segment, rather than predicting a particular type of action. This reduces the likelihood that the video localization system over-relies on contextual information in localizing actions in video. Furthermore, the classification model and the actionness model are trained based on weakly-labeled data, thereby reducing the cost and time required to generate training data for the video localization system.

The invention belongs to the field of artificial intelligence and natural language processing, and provides a method and a system for determining a dynamic retrieval decision scheme based on Monte Carlo tree search in order to solve the problems of redundant retrieval and incapability of multi-step reasoning and dynamic planning of a traditional retrieval enhancement generation method. The dynamic retrieval decision scheme determination method based on Monte Carlo tree search comprises the following steps: receiving an initial query problem and taking the initial query problem as a starting point of a reasoning process to trigger and call a large language model operation; calling a large language model to execute sub-query generation and action selection operation, searching, iteratively and exploring inference actions of sub-query problems based on a Monte Carlo tree, and dynamically screening inference paths by considering self-confident degree scores and honesty degree scores of the large language model to obtain a complete inference track set; and on the basis of the complete reasoning track set, a preference pair is constructed for each sub-query problem, and a reasoning retrieval strategy is optimized, so that the optimal retrieval opportunity of the large language model is autonomously judged. According to the method, invalid retrieval and multi-step reasoning dynamic planning can be reduced.

The invention discloses an intelligent customer service method for adaptively adjusting an interaction strategy by using reinforcement learning, and the method comprises the following steps: S1, obtaining a natural language text inputted by a user, and generating a multi-dimensional state vector; s2, grouping the users by using a real-time dynamic clustering algorithm, and updating a clustering result; s3, generating an action space in a specific state, and enriching the action space in combination with a rule-based strategy and a learning-based strategy; s4, aggregating neighbor node features through a graph convolutional network, and extracting local features; s5, weighting adjacent node features by using a graph attention network, and dynamically allocating attention weights; s6, carrying out interaction strategy optimization by adopting a spiral ladder type deep exploration network, and optimizing action selection; and S7, combining online learning with an experience playback mechanism, continuously updating an interaction strategy, and making a quick response to dynamically changing user demands. According to the method, reinforcement learning, the graph neural network and the sentiment analysis technology are combined, and intelligent customer service interaction strategy self-adaptive adjustment and optimization are achieved.

The invention discloses a mechanical arm autonomous grabbing system based on a visual language model and a control method of the mechanical arm autonomous grabbing system. The method comprises the steps that firstly, a mechanical arm is initialized, and a color image and a depth image of a working interval are collected; secondly, based on the color image and the natural language description, generating a target object bounding box and confidence, and judging whether a target object exists in the scene or not; and when the target object does not exist, instance segmentation and capture feasibility detection are carried out on the visible object in the current scene. And then the depth image and the object binary mask passing through grabbing feasibility detection are sent to an interaction object selection strategy and an action selection strategy to obtain an object to be interacted and a pushing action direction. And finally, center coordinates of the to-be-interacted object and center coordinates of the mechanical arm are obtained, the obstacle is moved in the pushing action direction until the target object appears and can be grabbed, and the mechanical arm grabs the target object and places the target object at a designated position. According to the method, the object specified by the human natural language can be accurately recognized and grabbed.

The invention provides a method and system for optimizing deep reinforcement learning based on composite priority experience playback. The method comprises the following steps: acquiring a current training network and a target network; the current training network comprises a current strategy network and a current value network, and the target network comprises a target strategy network and a target value network; acquiring an experience tuple based on an action selection strategy of the current strategy network; calculating a composite priority of the experience tuple, and enabling a life cycle pointer to update an experience playback area according to the experience tuple and the composite priority of the experience tuple; and sampling from the experience playback area to update the network parameters of the current training network, and updating the network parameters of the current training network to a target network. According to the method, the time sequence difference error of the instant reward and the experience is introduced as the priority evaluation standard of the new experience, the priority of the new experience is improved through the playback times, and the replacement efficiency of the experience playback area is improved through the lifetime pointer.

The invention relates to a self-adaptive plug-in architecture optimization method and device based on Q-Learning, and the method comprises the steps: defining a state space, an action space and a reward function of a target object, and initializing a mapping table of states and action values; obtaining a preset action selection strategy, and determining and executing a target execution action according to the action selection strategy; obtaining running state information after the target object executes the target execution action, and determining an instant reward value according to the running state information and the reward function; and acquiring a preset action value updating strategy, and iteratively determining the updated action value according to the action value updating strategy, the instant reward value, the state space and the action space until the change amplitude of the updated action value is not greater than a preset amplitude threshold. By adopting the method, the requirement of plug-in configuration can be automatically adjusted according to the change of the system state and the service requirement, so that the stability and the reliability of the system are improved.

The invention discloses a hierarchical dynamic scheduling method based on a strategy optimization DDQN algorithm, which is used for solving a dynamic scheduling problem of a reentrant hybrid flow shop with a batch processor, and comprises the following steps: firstly, determining an objective function and a constraint condition of the scheduling problem; secondly, by introducing a self-attention mechanism, providing a layered structure based on the self-attention mechanism DDQN, and constructing two agents, namely a batching agent and a scheduling agent, to respectively solve batching sub-problems and scheduling sub-problems in the problem; besides, in order to cope with the multi-stage batch processing and reentry scheduling characteristics in the problem, a Markov decision process considering the characteristics of the two agents is designed, and the Markov decision process comprises states, actions and rewards. Furthermore, an action selection strategy and a soft start target network updating strategy based on a mask combined with an epsilon-greedy strategy are provided, so that the efficiency and generalization ability are improved. The method has a remarkable effect on solving the dynamic scheduling problem of the reentrant hybrid flow shop with the batch processor.

The invention provides a distributed multi-agent traffic signal cooperative control method, and belongs to the field of intelligent traffic control. The method comprises the steps of performing initialization operation on a road network environment, constructing an adjacency list according to a topological structure of an urban road network, and regarding each traffic signal lamp control intersection in the urban road network as an independent intelligent agent; for each intelligent agent, acquiring a local observation state of the intersection where the intelligent agent is located; performing action selection through sampling according to the action probability distribution; sending the selected action to the traffic simulator by using the intelligent agent, updating the state of the traffic signal lamp, and calculating and returning the intelligent agent information of the next time step by using the traffic simulator; sending the action probability distribution to all neighbor agents; and parameter updating is carried out until a preset training round is reached, and cooperative control of the traffic signals is completed. By adopting the distributed multi-agent reinforcement learning framework, the exploration efficiency is improved, and the collaboration among the agents is promoted.

The invention discloses a path planning method based on an improved DQN algorithm. The path planning method aims at selecting a collision-free path to reach an ideal position. Path planning needs to solve the problems of target reaching, obstacle bypassing and the like, and an environment map needs to be preset in a traditional method. The deep reinforcement learning is used for solving a dynamic path planning problem, and a common path planning method is separated from online. DRL learns an optimal strategy through trial and error and environment interaction, and a DQN algorithm is applied to path planning but has the problems of slow convergence and the like. According to the improved DQN path planning algorithm and system provided by the invention, the problem is replayed and solved by improving a reward function and an action selection strategy and introducing prior knowledge and prior experience. Comprising the steps of constructing a two-dimensional grid network environment map, constructing, improving and optimizing a DQN path planning model based on the two-dimensional grid network environment map and a Markov decision process, improving strategy covering reward modeling, designing an attenuated epsilon-greedy strategy, and introducing prior knowledge and prior experience for replaying.

The invention discloses an internet constellation dynamic routing optimization method and system, and the method comprises the steps: setting orbit plane parameters and satellite parameters, and carrying out the modeling of an orbit and a constellation; performing link visualization and time-varying graph generation according to ephemeris data and the constructed constellation model; defining a related space, a network and a function based on a deep reinforcement learning algorithm; and based on an MADDPG framework, introducing a multi-agent collaborative decision-making mechanism, and outputting action selection and execution results after collaborative optimization. The system comprises a model construction module, an embedding module, a reinforcement learning design module and a collaborative optimization module. By using the method, the dynamic change of the low-orbit satellite network can be effectively dealt with, satellite resources are fully utilized, and efficient and reliable communication is realized. The method can be widely applied to the field of wireless communication.

The invention relates to the technical field of robot control, and discloses a robot continuous control-oriented triple optimization SAC reinforcement learning method, which comprises the following steps that a robot executes an initial strategy in a real environment to collect empirical data; initializing a strategy network, a value network and a target network, and performing experience playback and training updating on the strategy network, the value network and the target network through a triple optimization SAC reinforcement learning method; and continuously circulating the network updating process to obtain an optimal control sequence of task execution of the robot. The problem of underestimation is relieved by improving a strategy updating mechanism and stable value function learning, the problem that the robot cannot execute the optimal strategy is solved, underestimation deviation in the strategy is effectively relieved, the accuracy of robot action selection is improved, and more accurate guidance is provided for the robot to adopt correct actions.

The invention discloses an intelligent reinforcement learning training method based on self-gaming, particularly relates to the field of heterogeneous agent collaborative decision-making, is used for solving the problems of real-time performance and strategy stability in a high-dimensional heterogeneous action space, and reduces the scale of network parameters through typed splitting and shared feature extraction of the action space. Due to the application of the dynamic binary masks, illegal actions are filtered in real time, the hysteresis and exploration deviation depending on a reward punishment mechanism are overcome, and the compliance and high efficiency of the decision making process are ensured; a priority score matrix constructed by fusing tactical stability and environmental complexity is combined with layered importance sampling, so that an action selection strategy is optimized, and the decision stability and environmental adaptability are improved; through introduction of the cross attention network and the collaborative gain coefficient, tactical association between agents is effectively captured and utilized, and the overall efficiency and robustness of collaborative decision making are enhanced.

The invention provides a deadline constraint-based DAG application dynamic scheduling optimization method and system, and the method comprises the steps: converting an application deadline into an instant reward of each task scheduling through employing a DAG structured encoder, a Transform encoding network based on gating feature fusion, and a multi-action selection deep reinforcement learning task scheduling method based on a pointer network, in combination with a dynamic mask scheme, the mobility of a DAG application and the dynamic nature of edge resources are dealt with, then priority subtask selection and real-time decision of the scheduling position of the priority subtask selection are made, and the completion time and execution energy consumption of the application are reduced. In order to stabilize and accelerate DRL scheduler training, task encoder training and reinforcement learning training are decoupled, and a DAG encoder is pre-trained based on self-supervised learning.

The invention discloses a compilation optimization pass combination optimization method based on deep reinforcement learning. The method comprises the following steps: 1) collecting a reference program data set for various general function tests; 2) constructing a deep reinforcement learning-based compilation optimization pass-combination optimization strategy model for a compilation optimization pass-combination optimization process, wherein the compilation optimization pass-combination optimization strategy model comprises agent D3QN construction and definition of a reward space, a state space and an action space; 3) introducing a composite program feature combined with basic features such as a control flow embedding vector extracted by the graph neural network, an instruction number, a basic block number and the like into the expression of the state space, enriching the expression ability of the state, optimizing the pass based on the state selection and acting on the intermediate expression by the D3QN, generating an award, and obtaining the state space; and repeating the loop of feature extraction-action selection-reward generation until the maximum action upper limit is reached, and generating an optimized pass combination adapted to the program features. The problem that in existing compiling optimization, the compiling optimization effect on part of programs is poor is solved.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training an action selection policy neural network used to select an action to be performed by an agent interacting with an environment. In one aspect, a method includes: receiving a latent representation characterizing a current state of the environment; generating a trajectory of latent representations that starts with the received latent representation; for each latent representation in the trajectory: determining a predicted reward; and processing the state latent representation using a value neural network to generate a predicted state value; determining a corresponding target state value for each latent representation in the trajectory; determining, based on the target state values, an update to the current values of the policy neural network parameters; and determining an update to the current values of the value neural network parameters.

The invention discloses a closed-loop supply chain network optimization method for improving a non-dominated sorting genetic algorithm, and relates to the technical field of calculation methods. The method comprises the following steps: establishing an optimization model; setting parameters; generating an initial solution; a fitness function value is calculated, and rapid non-dominated sorting is carried out; updating the population; judging whether the current generation is greater than the maximum number of iterations; calculating a comprehensive state value; calculating a reward function value; performing action selection and parameter adjustment; and outputting a result. Aiming at the problems that a traditional meta-heuristic algorithm is insufficient in solution set diversity, prone to falling into a local optimal solution, sensitive in parameter and complex in adjustment when solving a multi-objective optimization problem of a closed-loop supply chain network, the method is improved on the basis of an NSGA-II algorithm, an intelligent learning mechanism is introduced to enhance the adaptability and dynamic response capability of the algorithm, the optimization process is accelerated, and the optimization efficiency is improved. Therefore, the overall solution quality is improved, the local optimal risk is reduced, and the calculation efficiency is improved.

The invention discloses an entrance ramp fleet confluence method based on reinforcement learning, and the method comprises the steps: carrying out the scene modeling according to an actual scene, and designing a plurality of fleet scales and vehicle speeds; vehicle acceleration and deceleration are designed as action selection, and the running speeds of main road vehicles and ramp vehicles are changed, so that the ramp vehicles select proper confluence opportunities; the change of the total travel time reflects the motorcade afflux efficiency, a reward function is established according to the change of the vehicle speed, the change of the time headway and the queuing waiting time of a ramp vehicle on an acceleration lane, an action mask is designed to improve the training efficiency, an intelligent agent is trained by using a PPO algorithm, and a trained deep reinforcement model is taken as a vehicle afflux model. The method gives consideration to the better passing efficiency of the main line and the shorter queuing waiting time of the vehicles on the ramp, enables the passing of the main line to be smoother, reduces the waiting time of the vehicles on the ramp as much as possible, and reduces the interference on the main line caused by the convergence of the vehicles on the ramp.

The invention provides a model training method, a treatment planning method, equipment and a medium, and relates to the technical field of medical treatment, in particular to the technical field of radiotherapy. The deep reinforcement learning model is configured to comprise an action selection network and an evaluation network. The method comprises the following steps: acquiring initial dose distribution state data of a target region; determining output data of a plurality of sub-threads in parallel by using the plurality of sub-threads based on the initial dose distribution state data of the target region and the current strategy data of the action selection network and the evaluation network; updating the current strategy data of the action selection network and the evaluation network based on the output data of each sub-thread in the plurality of sub-threads in sequence, and completing the current training of the deep reinforcement learning model; and iterating the training process until the number of times of training the deep reinforcement learning model reaches a preset number of times, and obtaining the trained deep reinforcement learning model.

The invention provides a coal bulk loading and unloading efficiency optimization method and system based on machine learning, and relates to the technical field of coal bulk loading and unloading, and the method comprises the steps: recognizing key factors and quantitative association rules between the key factors and the coal bulk loading and unloading efficiency through a large language model; the storage yard decision topological graph is input into a reinforcement learning model, a coal bulk cargo loading and unloading scheme is determined by executing the process of state space coding, action selection and reward function calculation, and a reward function is obtained by taking a predicted value of the coal bulk cargo loading and unloading efficiency in the current period as a target and being constructed according to key factors and quantitative association rules; after the coal bulk cargo loading and unloading scheme is completed, the actual value of the coal bulk cargo loading and unloading efficiency in the current period is collected, the efficiency difference value is calculated, and the coal bulk cargo loading and unloading efficiency in the next period is optimized based on the efficiency difference value. According to the method, the efficiency can be effectively improved while various necessary conditions of loading and unloading operation are accurately met.

The invention relates to a resource allocation and process planning optimization method under complex constraint conditions, which comprises the following steps of: representing various constraint conditions of a process planning process by utilizing a double-layer directed acyclic graph, and generating and constructing an initial feasible solution set meeting all the constraint conditions; initializing a state space, wherein the state space comprises state variables of a process number, an average process length, reward value fluctuation and fitness; noise is collected and coupled to a Q network, action selection is carried out, and low-level heuristic algorithm options are output; generating a new taboo sequence by using the selected low-level heuristic algorithm, and optimizing the current feasible solution; switching a low-level heuristic algorithm according to the improvement amplitude and the execution duration; and outputting an optimal processing feature sequence and process division combination after multiple times of training. Compared with the prior art, the method has the advantages that multiple process decisions can be optimized at the same time, sub-optimal results caused by separate optimization processes are avoided, processing time and resource consumption are remarkably reduced, and production efficiency and scheme adaptability are improved.

The invention discloses an interpretable high-speed ramp intelligent driving vehicle confluence control method. The method comprises the steps that a state space is constructed, wherein the state space comprises the position and speed of a vehicle, lane information and the relative position and speed of adjacent vehicles; human-like decision guidance: training a human-like behavior discriminator based on real driving data by using a random forest model, screening action selection of reinforcement learning, and constructing a sample set containing state vectors and human-like action labels; reinforcement learning control; and carrying out interpretability analysis. Through an SHAP analysis method, an explainable DQN longitudinal acceleration control model in an automatic driving vehicle ramp merging process is provided, and an explainable automatic driving vehicle ramp merging control model is constructed in combination with an merging decision-making model constructed based on a random forest. According to the explainable high-speed ramp automatic driving vehicle confluence control method based on the confluence urgency degree and the environmental historical information, traffic conflicts during highway confluence can be relieved, and therefore the road passing efficiency is effectively improved.

The invention provides a resource constraint workshop scheduling method and system based on a multi-target graph neural network, and belongs to the technical field of workshop scheduling. Constructing a resource-constrained workshop scheduling model according to the obtained workpiece set and workshop scheduling resources; performing nodal representation on the workshop scheduling model by using a disjunction graph, and expressing the state, action and reward value of the workshop scheduling model after nodal representation by using a Markov decision-making method; based on a multi-target graph neural network, determining the optimal action of the next step from the action space shown by the disjunction graph according to the current state; and executing workshop scheduling according to the determined optimal action. According to the method, high efficiency and high precision of optimal action selection can be kept on the basis of comprehensively considering all constraint conditions, and then reasonable resource constraint workshop scheduling is achieved.

The invention discloses a method for intelligently controlling a digital human through an action tag, and relates to the technical field of intelligent control. The method comprises the following steps: S1, determining the range and purpose of actions, classifying the actions, and determining which specific application scenes and target user groups are to be used by an action library; s2, designing a technical label system used for describing various action features and providing a basis for technical action recognition and analysis; s3, establishing a behavior control model of the digital character according to the tagged action description; and S4, through cross validation, dividing the data set into a plurality of subsets for multiple times of training and verification, and adjusting the structure of the model according to an evaluation result. According to the method, the naturalness and fluency of digital character expression can be improved, the flexibility and customization capability of the technology are enhanced, and intelligent action selection and transition are realized, so that better user experience and technical support are provided for various application scenes.

The invention discloses a task execution method and device for a vehicle-mounted environment, equipment and a storage medium, and relates to the technical field of vehicle-mounted interaction, and the method comprises the steps: obtaining a current task target and current screen state information for controlling a vehicle machine screen; based on a pre-constructed candidate action space, the current task target and the current screen state information, constructing verification prompt information conforming to a preset template; inputting the verification prompt information into a target verification model for action verification to obtain an action verification score; determining a target action from the candidate action space according to the action verification score; and executing the target action to complete the current task target, reducing the delay of the task by changing the large language model from generating a long sequence to outputting an action score, and selecting an optimal action by evaluating all possible actions, thereby reducing illusion and errors when the large language model is directly generated, and improving the success rate of the task.

A user equipment (UE) at a vehicle may receive a set of positioning signals from a set of positioning signal transmission devices (e.g., global positioning satellites (GPSs)). The UE may calculate a set of positioning attributes of the UE using a positioning device (e.g., a global navigation satellite system (GNSS) device) based on the received set of positioning signals. The UE may calculate an initial state of the positioning device based on the calculated set of positioning attributes and a set of environmental inputs associated with the UE. The UE may calculate an expected reward value for each of a set of potential positioning actions for the positioning device based on the calculated initial state. The UE may perform a positioning action of the set of potential positioning actions on the positioning device based on the calculated expected reward value for each of the set of potential positioning actions.

The invention discloses a multi-target reinforcement learning method based on quasi-measurement and planning, which comprises the steps of constructing a representation function, a quasi-measurement value function and a hidden space dynamics model, and performing joint training by utilizing transfer data sampled from environment interaction and a target state, the quasi-measurement value function being optimized to fit the cost from the state to the target, and the quasi-measurement value function being optimized to fit the target state; the distance measurement standard is used for optimizing the hidden space dynamic model; during decision making, action selection optimization is carried out by utilizing an MPC framework and combining with a CEM, in the process, a future trajectory is predicted by utilizing a hidden space dynamics model, the cost of the predicted trajectory is evaluated by using a quasi-metric function, and a single action applied to a current state is determined according to a lowest-cost planning result obtained by CEM optimization. According to the method, by learning the quasi-metric cost function and the hidden space model and combining model prediction control planning, sparse rewards can be effectively dealt with, and the sample efficiency and final performance of multi-target reinforcement learning are remarkably improved.

The invention discloses a heavy-load freight direct-current electric locomotive on-board platform and a train operation curve calculation method. Step-by-step iterative optimization is carried out through a dynamic programming algorithm. Presetting operation steps and step lengths, and defining a selectable action set in each step length; and with the current state of the train as a starting point, evaluating each action in the action set in each iteration step, and calculating the next train state which may be transferred to. And evaluating all possible train state changes by adopting a target cost function, sorting according to the cost values from small to large, and preferentially selecting the action with the minimum cost value for subsequent calculation. And if all the action cost values of the current step exceed the limit, backtracking to the previous iteration step for reselection. And after iteration to the maximum step number, storing all train state changes, and finally obtaining an optimal train operation curve under the constraint of the target function. The method optimizes a control strategy through a dynamic programming algorithm, improves prediction precision, optimizes action selection, accelerates convergence, reduces redundancy, and improves operation stability, safety and algorithm efficiency.

The invention discloses a fall and turn-over control method and device of a foot type robot and a storage medium. The fall and turn-over control method comprises the steps of firstly obtaining fall posture information of the foot type robot; calling an action selection model to perform multi-stage iterative turnover action prediction based on the falling posture information to obtain an optimal turnover action of each iteration stage, controlling the foot type robot to execute the optimal turnover action of the current iteration stage in each iteration stage, and controlling the foot type robot to execute the optimal turnover action of the current iteration stage after the foot type robot executes each optimal turnover action. The foot-type robot is in a stable posture when the foot-type robot turns over, and the foot-type robot turns over successfully after executing the optimal turning-over action of the last iteration stage. Wherein when the action selection model is called to carry out multi-stage iterative turning-over action prediction based on the falling posture information, the action selection model predicts the optimal turning-over action of the next iteration stage according to the stable posture of the foot robot in the current iteration stage. According to the embodiment, the manual design cost can be reduced, and the applicability is high.

The invention discloses a cold start strategy optimization method and device in no-service computing, equipment and a medium, and relates to the technical field of cloud computing, and the method comprises the steps: determining a reward function of a target cold start strategy model according to a state space and an action space; predicting the number of times of function calling in a future time period to obtain a prediction result; determining action selection of a target deep reinforcement learning algorithm, and determining a target value based on the action selection and a reward function; determining a comprehensive loss function based on the Q value loss function, the Huber loss function and the target loss function; updating the strategy network by using the action change difference value, obtaining an updated network, optimizing the value network through the comprehensive loss function, obtaining an optimized network, determining a target cold start strategy model through the updated network and the optimized network, and optimizing the cold start strategy without service calculation by using the target cold start strategy model. Therefore, the response speed and the resource utilization efficiency of the system are improved.