323 results about "Decision process" patented technology

The application relates to a multi-unmanned aerial vehicle (UAV) assisted edge computing task offloading method and system, and belongs to the technical field of mobile edge computing task offloading. The method comprises the following steps: constructing a multi-UAV assisted edge computing task offloading system model and an optimization problem model; based on the above model, a Markov decision process under a multi-agent environment is constructed, a reward function taking queue stability and information age minimization as targets is defined, a deep deterministic policy gradient algorithm is adopted, the optimization problem is solved through interaction between the multi-agent and the environment, and an optimal offloading strategy of a ground communication device computing task is obtained. The system comprises a ground communication device and a UAV, the UAV is in communication connection with the ground communication device, and an edge computing server arranged on the UAV is used for realizing the above method. The application not only guarantees the stability of a long-term data queue, but also obtains an optimal information age in an online mode, and solves problems such as communication congestion and poor user experience quality.

This application relates to the field of intelligent identification technology in the iron and steel metallurgy industry, and discloses a method and system for identifying the slag formation state of a steelmaking furnace based on vision-sonar fusion. The method includes: simultaneously acquiring flame images at the furnace opening and sonar signals inside the furnace, extracting anti-interference visual features and acoustic features respectively; innovatively employing a bidirectional cross-modal correlation attention mechanism for feature-level deep fusion to uncover the physical correlation between heterogeneous signals; based on this, a decision-level arbitration method based on dynamic confidence correction and a metallurgical rule engine is proposed, adjusting modal weights according to real-time signal quality, and intelligently resolving multimodal judgment conflicts based on process knowledge such as the blowing stage. This invention effectively solves the problem of poor reliability of single or static fusion sensing systems under strong dynamic interference, achieving highly robust and accurate identification and early warning of key slag formation states such as "re-drying" and "splashing," and the decision-making process is interpretable, possessing significant industrial application value.

This invention discloses a method for optimizing the topology of a 3D on-chip network (SoC) for high-speed data acquisition systems. Addressing the problem that general-purpose 3D SoC architectures are difficult to adapt to the communication characteristics of high-speed data acquisition systems, leading to high transmission latency and redundant link resources, this method models the network topology adjustment process as a Markov decision process and employs deep reinforcement learning to achieve adaptive optimization of the topology. The method uses a general-purpose 3D SoC as the initial architecture, constructing a state representation that includes node connection features, average network latency, longest path latency, and link area; defining a pruning and regrowth action space; designing a reward function that integrates changes in average latency, longest path latency, and link area; and using a deep Q-learning network with a multilayer perceptron structure for policy training. To improve learning efficiency, an action candidate set is constructed and combined with a two-layer greedy policy to achieve fast and effective search. Through iterative learning and topology updates, a 3D SoC topology that better suits the communication load of high-speed data acquisition systems can be obtained, reducing average transmission latency and link area overhead while ensuring connectivity.

The application relates to a kind of distributed unmanned aerial vehicle ad hoc network routing methods based on deep reinforcement learning, comprising;According to Markov decision process, the deep reinforcement learning architecture of unmanned aerial vehicle communication network is built;Running Dijkstra algorithm sends original data packet from source node to destination node and generates original training data according to the routing process of original data packet to pre-train the deep reinforcement learning architecture;The coordinates of the destination node D of target data packet are input, the next hop node B of current node A is obtained using the pre-trained deep reinforcement learning architecture, and target training data is generated, and the deep reinforcement learning architecture is retrained according to the original training data and target training data;Next hop node B is used as starting node, until the next hop node is the destination node, the routing of target data packet is completed, and the application can enhance the robustness of network and improve the life of unmanned aerial vehicle communication network.

Systems, methods, and computer program products are presented for receiving and evaluating financial data of a client, creating classes for the financial data, and generating financial planning recommendations and financial plans including calculations defining an allocation of funds between the classes to maintain or achieve the client's financial goals. The system includes a server and user devices, each having a processor and memory. The user devices communicate with the server over a network. The server and the user devices cooperate to calculate and provide visibility to a client's finances, permit simulation of performance when an attribute of the client's financial data is revised and to provide a measurement tool of performance and of client's decision-making process. The system employs graphical user interfaces that allow an interactive review and modification of a client's financial data during meetings between the client and one or more of their financial planning advisers.

The application provides an extreme weather wind power prediction method based on reinforcement learning adaptive sampling, and relates to the field of wind power prediction. The method comprises the following steps: obtaining meteorological time series data and wind power data of a wind power station site, constructing a training set, a validation set and a test set, and respectively extracting a training subset, a validation subset and a test subset corresponding to extreme weather; designing a training framework based on reinforcement learning to obtain a Markov decision process component, build a parameterized sampling strategy network and a wind power prediction model; iteratively performing a cooperative optimization process of the sampling strategy network and the prediction model until the training converges, and outputting an optimized target prediction model and a target sampling strategy network. Through the reinforcement learning adaptive sampling method, the sampling strategy network and the prediction model form an optimized closed loop, effectively improving the accuracy of wind power prediction under extreme weather, and ensuring the prediction effect under normal weather.

The application discloses a kind of computing power network green space-time task scheduling and mixed energy collaborative optimization method.The steps are as follows: one, establish the computing power network system model of fusing computing resources, network routing resources and mixed energy, construct multi-objective joint optimization problem;Two, model the problem as a Markov decision process, define the state space containing task, network and energy state, and the action space composed of computing power node selection, routing path selection and forwarding time;Three, use the integrated deep reinforcement learning framework, configure multiple intelligent agents with different preference weights for parallel training, to achieve the best balance between quality of service and carbon emissions;Four, according to the integrated strategy of complete training, combined with the real-time input task flow and carbon emission intensity, output the optimal space-time scheduling decision and energy storage charging and discharging strategy.

This invention proposes a method for deploying and offloading edge network services in the field of vehicle-to-everything (V2X) computing. The invention constructs a V2X edge computing system. Based on task request information from vehicles, a central base station formulates a contract set, which is then broadcast to potential service-oriented vehicle-mounted drones for contract selection. An optimization model is constructed using the objective function of maximizing the minimum task completion rate, combined with constraints. Solving this optimization model is transformed into a Markov decision process, and deep reinforcement learning is used to train the service deployment and task offloading model. Based on the trained model, the optimal service deployment and task offloading decisions are made according to the current task request. This invention can fully utilize the computing resources of service nodes, reduce task latency, alleviate the burden on roadside units, and improve task completion rate while protecting privacy.

The application discloses a kind of based on layered multi-agent deep reinforcement learning's unmanned aerial vehicle cluster task planning algorithm and its evaluation method, belong to unmanned aerial vehicle technical field, solve the current unmanned aerial vehicle cluster collaborative task planning problem solving algorithm generally exists online solving efficiency is low, large amount of computation, poor stability, difficult to apply to complex dynamic scene etc.The application first establishes the cluster task allocation and flight path planning problem part Markov decision process model respectively;Second, a layered deep reinforcement learning framework based on multi-agent deep reinforcement learning is proposed, through coupled training, hierarchical solving method, the task allocation and flight path planning associated coupling problem is solved simultaneously;Also based on UE4 and Airsim build unmanned aerial vehicle cluster combat simulation environment, design evaluation index, the result shows that the application has good performance in cluster task planning, can effectively improve the intelligentization and actual combat level of cluster combat.

This invention discloses a heterogeneous robot collaborative scheduling method for multiple modules of a space station. It employs a hierarchical distributed collaborative architecture based on a semi-Markov decision process. Physical constraints are transformed into filtering conditions through a pre-defined physical constraint sub-mask that includes area access, resource capacity, deadline reachability, and skill matching. A task residual update mechanism mitigates the waiting and overhead issues caused by explicit synchronization negotiation under communication constraints. A task commitment mechanism is introduced, dynamically updating the task residual based on the confidence level of the preceding robot's commitment to the target task. Subsequent robots then fill in, replace, or re-match based on the updated task residual. When local matching enters an oscillating state or local matching convergence stalls, a skill gap tension vector reflecting the gap status of each skill dimension is generated and fed back to the upper layer. The upper layer determines the cause based on the gap type and degree of different skill dimensions and takes appropriate measures for collaborative scheduling.

The application discloses a multi-mode non-sensing resource migration method for an electric power IMS virtualized core network, and is used for solving the problems of service interruption, load imbalance and low resource utilization in multi-mode resource migration. The method comprises the following steps: establishing an NFV system model, defining an utility index based on SLA violation, and establishing optimization targets for three modes of virtual machine migration, container migration and application backup respectively; calculating resource utilization and migration deviation in a single mode and a multi-mode fusion environment, and constructing a multi-mode non-sensing migration utility function as a weighted average target; modeling the problem as a Markov decision process model, and converting the migration decision into a strategy; and solving the model by using a deep deterministic policy gradient algorithm to obtain an optimal migration strategy. The application realizes unbiased and non-sensing multi-mode dynamic load balancing, and improves the resource utilization and service reliability of the electric power IMS virtualized core network.

The application discloses a kind of multi-agent collaborative resource allocation methods based on deep reinforcement learning, belong to resource allocation technical field.The method includes: S1, the task scene containing multiple execution unit cluster is initialized configuration;S2, resource allocation decision task is modeled as sequence decision process, constructs single execution unit cluster level and global level optimization target;S3, each execution unit cluster is regarded as independent agent, constructs multi-agent collaborative resource allocation decision model based on deep reinforcement learning and carries out parameter initialization;S4, at each time step, each agent generates resource allocation decision according to its own observation state, and stores interactive experience;S5, data is sampled from experience cache, and loss function is constructed by introducing collaborative credibility, and model parameters are updated;S6, after training, the model is used to guide multiple execution unit cluster to implement collaborative resource allocation.The application improves the collaborative decision-making capability and overall processing efficiency of multiple execution unit cluster in complex operational environment.

The application discloses a kind of reinforcement learning batch process control methods based on improved AC algorithm, it is related to the field of deep reinforcement learning and batch process control field.The method will be based on reinforcement learning method The batch process control is modeled as an optimal control problem on the basis of Markov decision process;Control action constraint is introduced in the reward function of reinforcement learning controller, the number of effective reward samples is increased to improve the learning rate of reinforcement learning controller, and the control cycle is shortened.Priority sampling method is introduced in the Actor-Critic algorithm of deep reinforcement learning, and a soft actor-critic algorithm with priority sampling is proposed to improve the sampling efficiency in the experience replay pool.The present application does not depend on prior knowledge and process model, and can realize model-free control of batch process.

The application discloses a lunar polar sun-synchronous path planning method and system based on deep reinforcement learning, relates to the technical field of path planning, and comprises the following steps: acquiring dynamic continuous illumination map data, performing definition processing on the dynamic continuous illumination map data based on a preset Markov decision process, obtaining a state multi-dimensional vector of a rover, inputting the state multi-dimensional vector of the rover into a pre-established rover agent, performing optimal path planning strategy learning based on a preset comprehensive reward function and a rover navigation control model, and outputting a trained rover agent; acquiring a real-time sensing state vector of the rover, inputting the real-time sensing state vector of the rover into the trained rover agent, outputting a global optimal action value, controlling the rover to perform based on the global optimal action value, and realizing the planning of a sun-synchronous path.

The application discloses a kind of medium and long term random production simulation method, system, medium and equipment, obtain the basic technical data of power system containing renewable energy and the predicted distribution data of renewable energy;Based on the basic technical data of power system and predicted distribution data, to realize system power supply safety, renewable energy consumption, reduce the target of system total cost, in the case where complex constraints are considered Markov decision process is constructed;Approximate dynamic programming method is used to iteratively train Markov decision process, and an optimal set of decisions is searched as a medium and long term operation mode.The application searches the optimal operation mode through the distribution of random factors, avoids the interference of subjective factors, reduces the total cost of the system under the condition of ensuring power supply safety and renewable energy consumption in extreme scenarios, and finally obtains a medium and long term operation mode and evaluation results that perform better than traditional solutions, providing a reference for the safe operation of power systems.

This invention provides a long-term strategy constraint method for integrated air-space-ground networks, comprising: constructing an integrated air-space-ground network model; calculating the structural importance weights of each ground node based on ground-side traffic and backhaul communication load; constructing a node interference cost and budget constraint model based on the weights and the total budget; modeling budget scheduling as a Markov decision process, using reinforcement learning to output the budget allocation ratio for each cycle to obtain the available budget for the current cycle; selecting ground nodes in descending order of unit structural benefit ratio using a reverse auction mechanism to obtain an intervention node set and update the remaining budget; calculating the structural influence factor based on the intervention node set to construct a repeated game model between the interferer and the user; and adjusting parameters using a zero-determinant strategy to limit the user's long-term benefit within a target range. This method can accurately identify key ground nodes, easily allocate interference resources rationally in multi-cycle operation, and continuously adjust the user's long-term benefit without significantly affecting the normal operation of the network.

A method for detecting the cliques of evaluators in a decision-making process includes collecting a data matrix Sij to a computer system using an automatic input interface. The elements of the matrix Sij are the real numbers in a predefined range. Each of the elements of the matrix Sij represent a numerical evaluation provided by an evaluator j for an evaluated entity i. For each pair of evaluators j, where j1 and j2 (j1≠j2) from the data matrix Sij, calculating by the computer system the Pairwise Adjusted Distances EDj<sub2>1< / sub2>j<sub2>2< / sub2>. Applying by the computer system a nonlinear transformation to the Pairwise Adjusted Distances EDj<sub2>1< / sub2>j<sub2>2< / sub2>. Identifying cliques of evaluators by comparing by the computer system the transformed distancesEDj1⁢j2*to a robust lower threshold.

This disclosure relates to a user perception optimization method, apparatus, device, medium, and product in the field of edge computing technology. The method includes: constructing a user perception optimization system model with the goal of enhancing the perception of all users through task processing mode selection; transforming the user perception optimization system model into a user perception quality maximization problem based on Markov decision processes; and employing a proximity policy optimization method within a deep reinforcement learning framework to obtain the optimal task scheduling strategy for user perception in the Markov decision process user perception quality maximization problem. This disclosure establishes a dynamic task scheduling model for user perception in edge computing networks, achieving an accurate description of the user perception problem. Based on the Markov decision process user perception quality maximization problem, it uses a proximity policy optimization method within a deep reinforcement learning framework to intelligently derive the optimal task scheduling strategy, ensuring the perception experience of each user throughout a continuous period.

The present application relates to the technical field of real-time monitoring, in particular to a real-time monitoring system for road and bridge construction progress, which comprises a dynamic relationship analysis module, an event response decision module, a structure health monitoring module, a risk management and prediction module, a progress prediction and adjustment module, a resource optimization configuration module, an environmental impact assessment module and an overall progress management module.In the present application, the dynamic relationship in construction is captured through the graph neural network, the graph attention network and the dynamic graph convolution network, the construction model is updated in real time, the accuracy of progress analysis is ensured, the event response decision module quickly optimizes the construction plan through the predictive control algorithm, the strategy flexibility is improved, the structure health monitoring and risk management module combines the Markov decision process and the Monte Carlo tree search, effectively prevents risks and ensures safety, the progress prediction, resource optimization and environmental impact assessment module improves resource efficiency and environmental adaptability, and the application of comprehensive evaluation method realizes efficient and high-quality construction management.

The application discloses a kind of memory hub methods and systems for multi-agent cognitive decision-making, comprising: obtaining the perception information, behavior result or decision context generated in the running process of cognitive system;The perception information is structured modeling, and memory unit with time attribute, access attribute and evolution attribute is generated;The memory unit is stored and managed in life cycle;Before or in the process of cognitive decision generation, according to the current decision context, select relevant historical experience from the memory unit to participate in decision formation;Based on the use effect of memory unit in the decision process, update the hierarchical state or call strategy of the memory unit, and construct the continuously evolving cognitive decision process based on historical experience feedback. Unified management and monitoring can be carried out on memory access behavior, permission configuration, life cycle state and abnormal conditions, and parameter configuration and state viewing can be supported during system operation.

The invention discloses a dispatching optimization method and system for a mine micro-grid, and relates to the technical field of micro-grids. The method comprises the steps of constructing a system model of the mining area comprehensive energy system; a mining area comprehensive energy operation model is constructed based on the system model, and constraint functions of the mining area comprehensive energy operation model comprise a net income maximization function, a renewable energy consumption maximization function and a carbon emission minimization function; the model is converted into a Markov decision process for determining a scheduling strategy for performing day-ahead scheduling control on the mining area integrated energy system, and the scheduling strategy comprises an output power range and an equipment state of each equipment in a day-ahead scheduling period; and under the limitation of the agreed range of the scheduling strategy and the constraint condition of the mining area comprehensive energy operation model, with the purpose of minimizing the operation cost, performing intra-day scheduling control by adopting mixed integer linear programming. Therefore, the problem that a scheduling optimization method in the prior art cannot be applied to the mine micro-grid or has many defects during application is solved.

This invention relates to a scalable industrial vision task offloading method and scheduling device based on energy consumption awareness, belonging to the field of task scheduling and resource management technology. To address the problems of current task offloading methods failing to simultaneously consider reliability, latency, and energy consumption, and lacking flexibility in adjusting latency and energy consumption, this invention first introduces energy consumption concern weights to establish a hierarchical transmission and computation energy consumption model, and constructs a utility function that comprehensively considers reliability, latency, and energy consumption. Then, task offloading scheduling is modeled as a Markov decision process using deep reinforcement learning, with the utility function as the immediate reward. Through the interaction between the deep reinforcement learning agent and the environment, a policy network is trained to learn the optimal joint decision on execution location and offloading layer under different channel and computing power conditions, thereby maximizing long-term average utility and satisfying latency constraints. Finally, intelligent scheduling is achieved based on deep reinforcement learning.

The application discloses a kind of fusion multi-model intelligent reasoning's cause of death chain automatic generation and radical cause of death determination support method and equipment.The method comprises: obtaining and preprocessing electronic medical record data;Adopt four-stage series screening mechanism including semantic matching, context adaptation, timing verification and statistical optimization, map clinical text into candidate ICD coding;Adopt search enhancement generation framework and dynamic multi-round controllable reasoning engine to generate candidate cause of death chain;Adopt the mixed probability model of timing-cause dual constraint to evaluate candidate cause of death chain, calculate comprehensive probability to determine radical cause of death, and the result is visualized and presented.The application can realize the automation of cause of death chain generation and radical cause of death determination, significantly improve accuracy and efficiency, enhance the explainability of decision-making process, and have continuous learning ability.

This invention relates to the field of computer-aided design technology, and more particularly to a method and system for constructing wind dams for concentrated wind energy capture. The method includes the following steps: acquiring multi-source wind field data; processing the multi-source wind field data to obtain wind field structure data; generating a guiding structure from the wind field structure data to obtain wind dam geometric structure data; performing fluid-structure interaction (FSI) simulation based on the wind field structure data and the wind dam geometric structure data to obtain FSI data; and performing multi-constraint dynamic optimization on the FSI data to obtain construction control strategy data. This invention achieves proactive guidance and efficient matching of the wind dam structure to airflow convergence behavior, improving the efficiency of concentrated wind energy capture and the reliability of structural design. By using a surrogate model and multi-constraint dynamic optimization, the highly complex simulation process is transformed into a rapidly reasoning-based decision-making process, effectively reducing computational costs.

The application discloses a multi-unmanned aerial vehicle task unloading and resource scheduling method and device, relates to the technical field of communication, and comprises the following steps: taking maximizing load fairness, minimizing overall energy consumption and minimizing total time delay of a computing task as targets, adopting an unmanned aerial vehicle flight model, a task processing time delay model, an energy consumption model and an unmanned aerial vehicle load balancing model to construct a target optimization function, and combining a decision constraint condition to construct a multi-target joint optimization problem; converting the multi-target joint optimization problem into a Markov decision process and correspondingly constructing a multi-agent deep reinforcement learning model; acquiring task decision state data associated with a to-be-processed computing task, and inputting the multi-agent deep reinforcement learning model for decision analysis, and outputting task decision action data.

The application discloses a network application firewall rule configuration method and device, equipment and medium, and relates to the technical field of network security, comprising: after accessing the network application firewall, detecting attacks on network requests received in a preset time period based on the sub-rules of each level in each protection rule under the current environment, determining the detection result and the time consumed by the sub-rule detection; determining the protection rule hit type distribution in the preset time period by analyzing the detection result; triggering the sub-rule level update based on the protection rule hit type distribution and the time consumed by the sub-rule detection, determining the target protection rule combination under the current environment by solving the Markov decision process constructed based on the update result; when a misjudgment interception behavior occurs based on the configured target protection rule combination, analyzing based on a preset expert decision interface to determine whether the rule correction trigger condition is met at present. The application can dynamically adjust the WAF protection rule combination based on the current environment.

This invention discloses an ecological management zoning method and system based on static and dynamic supply and demand matching, belonging to the field of ecological management technology. The invention constructs a node feature matrix by calculating a comprehensive static supply and demand matching index and a comprehensive dynamic change trend index, then generates a dynamic spatiotemporal adjacency matrix to construct a spatiotemporal physical connectivity graph. This graph is then input into a spatiotemporal graph neural network for processing, outputting predicted static supply and demand matching indices and predicted dynamic change trend indices. Based on the zero-value boundaries of these indices, the study area is divided into initial four-level basic management zones. A Markov decision process is then constructed, and a Pareto optimal solution set is obtained through reinforcement learning algorithms to fine-tune the spatial boundaries of the initial four-level basic management zones. The resulting refined ecological management zoning map and management priority sequence are output, making the zoning results more targeted for management. This solves the problem that existing ecological management zoning results lag behind actual changes and are difficult to effectively support adaptive management.

The application provides a large model NL2SQL evaluation method and device based on database exploration, and relates to the technical field of data processing. The method constructs the semantic correctness evaluation process of NL2SQL as a multi-step decision process driven by a large model agent. The large model evaluation agent module generates a probing query by calling a read-only database execution tool module, iteratively updates the context according to the dynamically returned execution results, and outputs a semantic equivalence judgment. The present scheme changes the traditional "static text matching" evaluation paradigm to "dynamic execution verification". Through the closed-loop feedback mechanism of "proposing a hypothesis - executing verification - updating evidence", the evaluation bias caused by a single reference SQL is effectively solved, complex cases such as "accidental consistency of execution results but semantic error" or "inconsistency of execution results but semantic correctness" can be identified, and the execution accuracy of the evaluation and the robustness of the semantic discrimination are significantly improved.