135 results about "Inverse reinforcement learning" patented technology

The invention relates to the technical field of multivariable analysis, and discloses a method and a device for improving the strength of flow-state solidified soil based on multivariable analysis, and the method comprises the following steps: collecting a material proportion parameter matrix, a stirring parameter matrix and an environment monitoring data matrix in flow-state solidified soil equipment, and executing feature extraction and feature integration; obtaining a target feature vector; performing intensity prediction of the time sequence on the target feature vector to obtain an intensity prediction value of the flow-state solidified soil; sensitivity analysis is carried out based on the intensity predicted value, and an intensity improvement potential index, a parameter adjustment stability index and an energy consumption influence index are obtained; according to the strength improvement potential index, the parameter adjustment stability index and the energy consumption influence index, adaptive parameter optimization is carried out through discount inverse reinforcement learning, and a target control parameter set is obtained. And the strength and the quality uniformity of the flow-state solidified soil are effectively improved while the operation stability is guaranteed.

The invention relates to a data-driven intersection vehicle complex interaction behavior strategy simulation method and system. The method comprises the following steps: acquiring real intersection vehicle traffic trajectory data, constructing a plurality of interaction scenes, extracting observation information of each time step, and obtaining real vehicle trajectory data; taking each vehicle as an intelligent agent, based on a Gaussian strategy network pre-constructed according to the vehicle type, calculating a reward by using a reward based on a target guiding gain and a reward estimator from a discriminator, and generating simulated vehicle trajectory data; based on the real vehicle trajectory data and the simulated vehicle trajectory data, respectively extracting observed value-action pair information, updating a discriminator of an intelligent agent by taking distinguishing different attribute information to the greatest extent as a target, and updating a Gaussian strategy network and a value estimator network in a generator by taking confusing the discriminator to the greatest extent as a target, so as to obtain the real vehicle trajectory data and the simulated vehicle trajectory data. And vehicle complex interaction behavior strategy simulation based on multi-agent countermeasure reverse reinforcement learning is realized.

The invention provides an interactive perception automatic driving method based on dual-mode game and deep inverse reinforcement learning, and belongs to the field of automatic driving. A Stackelberg game and a Bayesian Stackelberg game are adopted to comprehensively judge whether V2V is available or not, a leader declares a lane changing intention and a follower responds to a decision-making means of the leader according to a driving style of the follower, and then a human-like track is planned through deep inverse reinforcement learning, so that the use aims of better fitting the reality and improving safety and efficiency are achieved. The decision is divided into two modes of a Stackelberg game and a Bayesian Stackelberg game based on the integrity of the information; if all related vehicles provide interaction intentions through the V2V, a Stackelberg game is adopted; if part of the vehicle intentions are not transparent, opponent behaviors need to be speculated through a sensor and a probability model, and the Bayesian Stackelberg game is triggered.

The invention discloses a smart home energy management method based on deep countermeasure reverse reinforcement learning. The method comprises the following steps: (1) modeling a smart home energy cost minimization problem and designing an environment state and an action corresponding to a Markov decision process; (2) constructing n groups of random parameter historical data based on a group of random parameter historical data and a rolling generation method; (3) solving the minimization problem by using n groups of random parameter historical data and an optimization algorithm and obtaining n expert tracks; (4) training the reward network by the discriminator based on the expert trajectory and the trajectory generated by the generator agent; under the guidance of the reward network, training a generator agent by using an experience tuple and a near-end strategy optimization algorithm; (5) repeating the step (4) until an intelligent agent strategy with stable training performance is obtained; and (6) deploying the intelligent agent strategy obtained by training in an actual environment for operation. Compared with an existing method, the method can effectively reduce energy cost and improve user comfort.

The invention relates to an afflux area main road vehicle driving behavior modeling method based on deep inverse reinforcement learning. The method comprises the following steps: constructing a main road vehicle expert trajectory set; constructing a simulation environment model, wherein the input of the environment model comprises a current state feature provided by a main road vehicle expert track set D and an action feature provided by a main road vehicle strategy network control main road vehicle in a PPO algorithm; the state transfer function is used for returning the state feature of the next time step according to the current state feature and the action feature; the reward network is used for giving an instant reward for each time step according to the state characteristics of the main road vehicle; constructing a framework formed by a maximum entropy depth inverse reinforcement learning algorithm and a PPO algorithm; then training is carried out, and a PPO algorithm is used for training to generate a prediction track of the main road vehicle in a simulation environment; and inputting the predicted trajectory and the main road vehicle expert trajectory into a maximum entropy depth inverse reinforcement learning algorithm, calculating a loss function, and updating reward network parameters until convergence, thereby solving the problem of strategy misalignment existing in a traditional single method.

The invention relates to the technical field of intelligent traffic, in particular to an On-Ramp scene trajectory planning method based on deep inverse reinforcement learning, and the method comprises the following steps: S100, candidate trajectory generation: generating a plurality of candidate trajectories based on vehicle state information and traffic environment information; s200, track authenticity filtering, wherein the generated candidate tracks are evaluated and screened through a track authenticity filtering framework, so that abnormal tracks are eliminated, and credible tracks are reserved; and S300, reward function training: utilizing reward function optimization of deep inverse reinforcement learning to realize continuous adjustment and improvement of a trajectory evaluation standard so as to carry out quantitative analysis on rationality and optimality of the trajectory. By constructing a trajectory authenticity filtering framework, optimizing a reward function design and training strategy, and introducing a multi-dimensional scoring mechanism, the trajectory authenticity filtering framework is optimized, so that the accuracy of the trajectory authenticity filtering framework is improved. According to the invention, efficient, safe and dynamically adaptive trajectory planning is realized, and reliable support is provided for real-time decision making of an automatic driving vehicle in a complex scene.

The invention discloses a water microorganism risk analysis method based on reinforcement learning. The method comprises the following steps: S1, generating a standardized observation data set; s2, constructing a dynamic water body map based on the standardized observation data set; s3, executing dynamic graph representation learning for the dynamic water body graph structure, and generating a space-time risk heat matrix on the basis of the node vector representation matrix; s4, constructing prior distribution of a Bayesian reward function, initializing the reward function and establishing a reward function parameter set; s5, performing incremental learning on the award function parameter set by adopting a maximum posterior reverse reinforcement learning model, and outputting a first propagation potential field; s6, obtaining an optimal diffusion path sequence; and S7, performing path reconstruction error analysis on the optimal diffusion path sequence and the standardized observation data set to obtain a dynamic feedback index. According to the method, pollution diffusion response delay caused by emergencies in actual measurement is greatly shortened, and the emergency modeling capability of the system in an emergent pollution scene is remarkably improved.

The invention discloses a trajectory planning inverse reinforcement learning method, system and product, belongs to the technical field of robots, and solves the technical problems of weak strategy generalization ability and insufficient environmental state perception when facing a more complex compliant placement task in the prior art. Initializing an execution environment and execution parameters of the mechanical arm; loading a current state, and collecting demonstration data; extracting hierarchical coordinate sensing neural network features according to the current state; constructing a strategy function, and generating a current execution action; calculating a current total reward based on the current state, the current execution action and the demonstration data; calculating a state value, and combining the current total reward to obtain a value advantage of the current execution action; and after the corresponding strategy parameters and value parameters are updated through the total loss, the mechanical arm is adopted to execute the current action, whether the current action reaches the target position or not is judged, and the optimal strategy is output. The mechanical arm is used for achieving humanoid fruit bunching and smooth placement.

The invention provides a motor health management method and system based on full life cycle monitoring, and the method comprises the steps: firstly collecting motor multi-source sensor data, and mapping the motor multi-source sensor data into a coordinate point in a low-dimensional health state space through a manifold learning model, so as to represent the comprehensive health state of the motor multi-source sensor data; and on the basis of the expert strategy database, a health cost function associated with the health coordinates is constructed by adopting inverse reinforcement learning and is used for quantifying health loss of different operations. And a decision model is constructed, and the model is trained through an optimal control algorithm, so that an operation strategy capable of minimizing the total health cost of the whole life cycle of the motor can be found out. In actual operation, the system determines current health coordinates in real time, generates an optimal operation strategy by using a decision model in combination with future task requirements, and analyzes the optimal operation strategy into specific control parameters to be issued and executed. According to the method, active health decline path management and closed-loop control of the motor can be realized, so that dynamic optimal balance between the performance and the service life of the motor is realized.

The invention discloses an intersection right-turn vehicle and pedestrian interaction trajectory prediction method and computer equipment. The method comprises the following steps: acquiring a human-vehicle historical trajectory and initial state information in intersection right-turn vehicle and pedestrian interaction; a confrontation reinforcement learning structure is improved through KL regularization, and combined learning of a reward function and a strategy is achieved; a Nash Q-learning algorithm with a KL constraint is utilized to carry out joint optimization; and inputting the optimized strategy model into a behavior simulator, driving a generated prediction trajectory by an initial state, comparing the prediction trajectory with a real trajectory, and evaluating a prediction result based on a specified index. According to the method, the performance of the model in the aspects of prediction precision, behavior consistency and environment generalization ability can be effectively improved, the stability of a reward function and a strategy network is further improved, the generalization ability of the model in different traffic scenes subsequently is enhanced, and the learned strategy has better migration potential.

The invention discloses an injection molding production optimization method based on multi-agent cooperation, and the method comprises the following steps: decomposing a plurality of targets in an injection molding production process, and constructing a layered multi-agent structure; collecting data in the injection molding production process in real time, and constructing a global expert behavior track and a local expert behavior track; based on the global target and each local target, obtaining initial parameters of a global reward function and a local reward function; adopting an inverse reinforcement learning method to obtain an optimal global reward function and an optimal local reward function; obtaining an optimal decision strategy of each agent through a hierarchical collaboration mechanism by each hierarchical agent; when detecting that a conflict exists between the targets, dynamically adjusting the reward weight of the reward function by adopting a conflict coordination mechanism; and continuously collecting new expert behavior data, and updating the optimal decision strategy of each agent. According to the invention, a layered multi-agent inverse reinforcement learning method is adopted, and multi-target dynamic collaborative optimization of injection molding production is realized.

The invention relates to a defensive intelligent driving aid decision-making method based on inverse reinforcement learning, and the method comprises the steps: simulating and reproducing a typical dangerous scene, and collecting the defensive driving behavior data of a driver; based on a generative adversarial reinforcement learning AIRL method, training by using the collected defensive driving behavior data to obtain a defensive driving decision model with defensive driving ability; and the driving risk of the current scene is evaluated by adopting the risk index, and intelligent driving assistance operation including active defense driving behavior suggestion and passive risk avoiding driving behavior intervention is executed by the defensive driving decision-making model according to the driving risk classification. Compared with the prior art, the method can effectively learn defensive driving behaviors from human drivers, accurately understand and cope with complex and variable traffic scenes, improve the reliability of defensive driving aid decision making, and realize preventive driving safety.

The invention belongs to the technical field of intelligent task-based dialogue, and particularly relates to a task-based dialogue strategy learning method and system based on reinforcement learning, which adopts a soft actor commentator algorithm to be combined with behavior cloning and maximum entropy inverse reinforcement learning to relieve a cold start problem. According to the method, maximum entropy inverse reinforcement learning is utilized, a reward value is calculated, and a user target in a dialogue is accurately deduced according to maximum log likelihood estimation of a human dialogue and a simulated dialogue; a potential reward function is extracted from a successful track through inverse reinforcement learning, manual reward design is replaced, accumulated reward maximization is pursued during strategy optimization, diversity and exploratory performance of the strategy are encouraged, and the dialogue strategy is prevented from being converged to a single mode too early.

The invention discloses an interaction-considered automatic driving decision planning method and system and terminal equipment, and relates to the technical field of automatic driving. The method comprises the following steps: designing based on safety cost, passing cost and comfort cost to obtain a joint cost function, and carrying out inverse number processing on the joint cost function to obtain a joint reward function; training the joint reward function by adopting a maximum entropy inverse reinforcement learning algorithm until the joint reward function is converged; sampling candidate joint trajectories according to the states of the own vehicle and the interactive vehicle, and calculating the rewards of the candidate joint trajectories of the own vehicle and the interactive vehicle by using the converged joint reward function; and selecting the candidate joint trajectory with the maximum reward as a planning result and outputting the planning result. According to the method, the integrated prediction framework is constructed, the interactive game relationship between the automatic driving vehicle and other traffic vehicles is fully considered, and the joint reward function of the integrated prediction framework is automatically calibrated according to expert demonstration data of human drivers through the maximum entropy inverse reinforcement learning algorithm.

The invention provides an enterprise management system intelligent optimization method and system based on digital twinning, and the method comprises the following steps: constructing an organization structure diagram with a path weight by taking a post as a node, taking a superior-subordinate relationship as a directed edge, and taking a KPI (Key Performance Indicator) target vector as a node attribute; constructing an instruction conduction path model through breadth-first traversal based on the organization structure chart to generate an effective conduction path set, and calculating a target propagation vector of each path; a post is regarded as a reinforcement learning agent, an optimal post behavior strategy is recovered through inverse reinforcement learning, and a target vector matrix of each post adjusted according to the strategy is obtained; constructing an anti-fact management strategy optimization instruction conduction path, and generating an optimized target matrix and path set; and comparing the optimized target matrix with actual execution data, and calculating a target deviation as an execution effect evaluation value. According to the method, the defects of an existing system in the aspects of multi-level management collaboration, behavior deviation modeling and strategic execution consistency guarantee are overcome.

The invention relates to the technical field of vehicle cooperative interaction control, in particular to an intelligent network connection vehicle social cooperative interaction control method and system in a mixed traffic scene, and the method comprises the steps: obtaining a spatial feature vector and a time sequence feature vector; obtaining a comprehensive feature vector through the spatial feature vector and the time sequence feature vector; performing linear combination according to the comprehensive feature vectors to obtain a reward function; obtaining an optimal weight vector through inverse reinforcement learning according to the reward function; the SVO angle value of the HDV is estimated through the optimal weight vector, and the SVO angle estimated value of the CAV is obtained; sVO parameters are optimized and dynamically adjusted through the Stackelberg game according to the SVO angle estimation value of the CAV, and optimal response parameters of the HDV are obtained; and according to the IDM model and the optimal response parameter of the HDV, simulating social perception and response of a human driver so as to adjust the driving behavior. According to the invention, the accuracy of vehicle cooperative interaction control is improved.

This invention provides a method for optimizing the hybridization and selection of high-iron strains in black fungus based on big data analysis. It relates to the fields of big data analysis and black fungus variety breeding. The method includes: Step S1: Collecting basic data of strains, hybridization process data, culture environment parameters, and external reference data, and performing standardization and structuring processing; Step S2: Encoding and optimizing hybridization combinations using a biased random bond genetic algorithm, combined with a dual-elite evolutionary mechanism and dynamic parameter adjustment strategy to generate the optimal hybridization scheme; Step S3: Constructing a dynamic environmental model using a model-based hybrid inverse reinforcement learning algorithm, searching for the optimal culture strategy that maximizes iron content in a virtual environment; Step S4: Constructing a multi-index comprehensive scoring system based on the analytic hierarchy process (AHP), identifying abnormal samples and triggering model backtracking adjustments, outputting high-performance strains for the next round of breeding. This invention significantly improves breeding efficiency and the accuracy of strain selection.

The embodiment of the invention relates to the field of multi-modal emotion analysis, and discloses a multi-modal emotion fusion method based on inverse reinforcement learning. Dynamic weighted fusion is carried out through modal features such as voice, facial expressions and texts, an optimal modal fusion strategy is derived from known emotion label data by adopting inverse reinforcement learning, and the contribution weight of each modal in emotion analysis is automatically adjusted, so that efficient, accurate and stable emotion recognition is realized. Through the technology of the invention, the sentiment analysis system can adaptively optimize modal fusion in various sentiment situations, the generalization ability of the sentiment analysis system in different scenes is significantly improved, the dependence on a large amount of annotated data is reduced, and the sentiment recognition precision is improved at the same time. The method has a wide application prospect and can be widely applied to the fields of intelligent customer service, emotion calculation and the like. The method and the device can be at least used for solving the problems of fixed modal weight and low emotion recognition accuracy in the prior art.

The invention relates to a traffic scene closed site cluster simulation method and device, equipment and a medium. The method comprises the following steps: establishing a traffic participant game utility matrix; introducing the traffic participant game utility matrix into a maximum entropy inverse reinforcement learning model, and constructing a multi-traffic participant traffic behavior model; training the model through a traffic participant interaction data set to obtain an inverse reinforcement learning traffic participant dynamic game model; building a real object scene, a virtual simulation scene and a data interaction link; and performing traffic behavior simulation on the plurality of second traffic participants through an inverse reinforcement learning traffic participant dynamic game model in a virtual simulation scene by adopting a virtual-real combination method, and controlling action tracks of the plurality of first traffic participants through a data interaction link. According to the technical scheme of the invention, the method can achieve the vivid simulation of the interactive game traffic behaviors of a plurality of traffic participants in a real object scene through the target object, achieves the precise testing of an automatic driving system, and improves the performance of the automatic driving system.

A recommendation model training method is applied to content recommendation in the technical field of artificial intelligence. According to the method, based on the interaction process of the user and the recommendation system, a satisfaction prediction model is trained through inverse reinforcement learning, that is, the satisfaction change condition of the user in the interaction process of the user and the recommendation system is learned through the inverse reinforcement learning, and then the satisfaction of the user in various interaction states can be captured through the satisfaction prediction model. Therefore, in the process of training the recommendation model, the user satisfaction corresponding to each piece of training data is predicted through the satisfaction prediction model, so that the recommendation model can learn and recommend contents with high user satisfaction as much as possible when the recommendation model is trained, and the recommendation accuracy of the recommendation model obtained through training is improved.

The invention discloses a scheduling method and system matched with air luggage change information, and relates to the technical field of air transportation intelligent scheduling, and the method comprises the steps: collecting luggage change information, passenger travel data and airport equipment state data in an air transportation scene in real time, and outputting an original data stream; an airport operation digital twinborn body is constructed in the element universe, a collaborative transfer path is imported into a digital twinborn environment for space-time preview simulation, luggage flow conflict points are detected in real time, and a physical constraint conflict area coordinate set is marked; inverse solution is carried out on conflict events and traceability strategy defect features in rehearsal simulation through an inverse reinforcement learning algorithm, a reward function space is reconstructed, and gradient compensation parameters are output; and fusing the physical constraint conflict region coordinate set and the gradient compensation parameters, generating a three-dimensional space scheduling instruction set, and executing instruction issuing. Based on the space-time rehearsal simulation method, the security boundary of the luggage transfer path is accurately identified, and the physical constraint overrun risk is effectively reduced.

The invention discloses a supply chain fund flow intelligent scheduling method based on reinforcement learning, and the method comprises the steps: S1, collecting multi-dimensional heterogeneous data, constructing a dynamic digital twin environment, and carrying out the fusion to generate a standardized input tensor; s2, aggregating node features by using a heterogeneous graph attention network, and outputting a high-dimensional state feature graph; s3, analyzing market fluctuation and risk through a high-level agent, and outputting a global scheduling guidance strategy; s4, outputting a refined scheduling instruction set by using a DPPO algorithm in combination with tail risk minimization; s5, based on the improved MAPPO algorithm, noise is injected to cover privacy, and Nash equilibrium optimization is executed through inverse reinforcement learning; and S6, fund allocation is executed, and online iterative optimization is carried out. According to the invention, supply chain fund scheduling efficiency and security are improved, and individual and collective benefits are effectively balanced.

The invention discloses an unmanned system multi-intention recognition method and system based on inverse reinforcement learning, and belongs to the technical field of artificial intelligence and autonomous decision making. According to the system, a deep multi-intention inverse reinforcement learning framework is provided for solving the problems of multi-modality, fuzziness and dynamics of unmanned system decision-making intentions in a complex dynamic environment. By designing a multi-scale intention encoder based on Transform, state-intention combined dynamic reasoning is realized; constructing an end-to-end intention perception reward network, and adaptively fusing multi-intention features through an attention mechanism; and proposing an expectation maximization optimization strategy to realize collaborative optimization of intention reasoning and strategy learning. The system can automatically separate and identify various potential intentions from expert demonstration of mixed intentions, and learn corresponding reward functions and decision strategies. According to the method, the intention recognition accuracy and the strategy reproduction capability in a multi-intention scene are remarkably improved, and an effective solution is provided for behavior understanding and autonomous decision making of an unmanned system in a complex environment.

The invention discloses a decision model training method and device and a driving decision method and device, and the method comprises the steps: inputting each first candidate decision behavior corresponding to a first traffic scene into a to-be-trained decision model, and obtaining the decision probability of each first candidate decision behavior; based on the decision probability of each first candidate decision behavior, performing inverse reinforcement learning on each first candidate decision behavior to obtain an inverse reinforcement learning loss value of the first traffic scene; performing imitation learning on the first optimal decision behavior with the maximum decision probability in the first candidate decision behaviors to obtain an imitation learning loss value of the first traffic scene; based on the inverse reinforcement weight and the imitation weight, performing weighted fusion on the inverse reinforcement learning loss value and the imitation learning loss value to obtain a weighted loss value of the first traffic scene; and based on the weighted loss value, optimizing model parameters of the decision model to be trained to obtain a trained decision model. The reliability of the decision model can be improved.

The invention discloses a parameter optimization method and device for oil reservoir development, equipment and a storage medium, and relates to the technical field of oil reservoir development. Comprising the steps that oil field geological parameters, well location and operation parameters, injection parameters and economic evaluation parameters are obtained and input into a numerical simulator to complete initialization, and oil reservoir state change data and development effect indexes are calculated; generating a first data set representing the corresponding relationship among the parameters, the oil reservoir state and the development effect by combining a particle swarm optimization algorithm; a second data set containing multiple expert tracks is obtained through screening and data enhancement processing, and the expert tracks reflect time sequence evolution of the oil reservoir state under the action of the injection parameters; inputting the expert track into a preset inverse reinforcement learning model, and screening a target expert track according to the strategy network update amplitude and the discriminator probability value; and finally, optimizing the input parameters based on the target expert trajectory to obtain optimal parameters. Intelligent optimization of oil reservoir parameters is realized, optimization efficiency and suitability are improved, and development benefits are guaranteed.

The invention discloses a task-oriented dialogue system reward function optimization method and system, and belongs to the technical field of natural language processing. The method comprises the following steps: acquiring an expert dialogue track from a dialogue system data set, extracting a state-action-reward triple, training an initial reward function by using a maximum entropy inverse reinforcement learning framework, and initializing a strategy network; an Actor-Critic reinforcement learning algorithm is adopted to train a strategy network, and a suboptimal trajectory is collected to dynamically update a reward function; and taking the dynamic reward function as a unified evaluation signal, and optimizing the strategy network to form a dialogue strategy model. Through dynamic reward function optimization, manual rule dependence is reduced, the generalization ability, the task completion rate and the stability of a dialogue system are improved, and the method is suitable for complex dialogue scenes in multiple fields.

Provided is a system and a method for scheduling. The method includes training a first scheduling policy based on a first weight and training a second scheduling policy based on a second weight that is different from the first weight, calculating a first evaluation result based on the first scheduling policy, calculating a second evaluation result based on the second scheduling policy, determining, by a processing circuit, a third scheduling policy based on inputting the first evaluation result and the second evaluation result into a policy-combination algorithm, calculating a third evaluation result based on the third scheduling policy, based on the third evaluation result, determining a third weight for the third scheduling policy by inverse reinforcement learning or by Bayesian optimization, training the third scheduling policy based on the third weight, and controlling a scheduling process based on the third scheduling policy.

The invention discloses a security and protection unmanned vehicle path planning method fusing a tactical risk map and game prediction, and relates to the technical field of intelligent security and protection, and the method comprises the steps: receiving a task path issued by a remote control security and protection unmanned vehicle, carrying out the semantic perception of an environment, and constructing a static tactical risk map; acquiring real-time observation track sequences of potential opponents, inferring target intentions of the opponents on line through a Bayesian inverse reinforcement learning BIRL model, and generating a high-risk region thermodynamic diagram; generating a speed and angular speed control instruction through a path tracking controller based on the balanced deterrent path; and executing a speed and angular speed control instruction, collecting a real-time observation track sequence of a potential opponent, and performing online fine adjustment of a Bayesian inverse reinforcement learning BIRL model to generate an optimized security unmanned vehicle path. According to the invention, the unmanned vehicle can continuously generate an optimized path with both safety and tactical initiative in a dynamic confrontation environment.