Disposal scheme generation method and device, equipment and medium

Abstract

The invention discloses a processing scheme generation method, device and equipment and a medium, and relates to the technical field of scheme generation, and the method combines the main advantages of a random forest and a PPO algorithm in the respective characteristics and complementarity of the random forest and the PPO algorithm. The system not only can process simple conditions with clear rules, but also can cope with dynamic and complex environments, the robustness and reliability of the system are improved, and under the condition that the simple rules cannot cover, the PPO algorithm can comprehensively consider various factors and provide a finer and more personalized evacuation scheme. Through the combination of the random forest and the PPO algorithm, the system can better deal with a complex decision scene, the accuracy and reliability of decision are improved, and a more stable solution is provided for dealing with various complex conditions. Along with use in the later period, reinforcement learning can continuously generate schemes towards the direction which is considered to be good, and the more used schemes are, the more easily generated schemes conform to user preferences. And the process of readjustment by the user after generation is omitted.

Description

Technical Field

[0001] The present invention relates to the field of solution generation technology, and in particular to a method, device, equipment and medium for generating a disposal solution based on random forest and reinforcement learning. Background Art

[0002] With the development of society and advancements in science and technology, users are increasingly demanding new technologies. The application of nuclear, biological, and chemical technologies is playing an increasingly important role in addressing energy crises and other areas. However, the harm caused by nuclear, biological, and chemical hazards already accounts for a significant proportion of sudden disasters worldwide. Toxic and harmful gases, radioactive contamination, and other hazards pose a serious threat to the human living environment and also threaten human life.

[0003] It can be seen that in the application scenarios of nuclear, biological, and chemical related technologies, in order to ensure the safety of humans and property after sudden safety accidents, decision makers of relevant departments need to be able to formulate relevant emergency response plans at any time.

[0004] Traditional emergency response plans are often developed by experts based on prior knowledge. While this approach can yield effective solutions, it also has numerous drawbacks. For example, relying solely on experts to develop emergency response plans based on prior knowledge requires extensive data calculations, placing a high level of labor intensity on decision-makers. Furthermore, manual planning of emergency response plans is time-consuming and cannot quickly generate optimal solutions when emergencies occur.

[0005] Therefore, how to provide a disposal plan generation method that can provide decision makers with key event information, predicted trends and available rescue and support force information, thereby assisting decision makers in formulating disposal plans, is a technical problem that urgently needs to be solved by technical personnel in this field. Summary of the Invention

[0006] In view of the above problems, the present invention provides a treatment plan generation method, apparatus, device and medium for overcoming the above problems or at least partially solving the above problems.

[0007] The present invention provides the following solutions:

[0008] A method for generating a disposal plan, comprising:

[0009] Get the preset feature vector of the current event;

[0010] A random forest decision-making model is used to make predictions based on the feature vectors, with each decision tree classifying the input feature vectors and providing a recommendation result; the recommendation results of all decision trees are voted on or averaged to obtain a candidate disposal plan; the candidate disposal plan includes at least a candidate evacuation order and a candidate time and interval;

[0011] Determine whether the random forest decision-making model is unable to clarify the decision path or faces high-dimensional complexity issues;

[0012] After determining that the random forest decision-making auxiliary model has a clear decision path or does not face high-dimensional complexity problems, determining the candidate disposal solution as the target disposal solution;

[0013] After determining that the random forest decision-making assistance model cannot clearly define the decision path or faces high-dimensional complexity problems, the high-dimensional data corresponding to the current event is input into the reinforcement learning algorithm; the high-dimensional data at least includes a preset evacuation order, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation order and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

[0014] Preferably, the data processing flow of the reinforcement learning algorithm includes:

[0015] Using a strategy network to generate new actions based on the preset evacuation sequence, preset time and interval, speed, wind direction, and traffic conditions, and evaluating the effectiveness of different disposal plans;

[0016] Perform multiple trajectory sampling to record the state, action, reward, and next state;

[0017] The advantage function is calculated, and the optimization strategy network is updated through a restriction strategy to optimize the candidate evacuation sequence and the candidate time and interval included in the candidate treatment plan to obtain the target treatment plan.

[0018] Preferably, the goal of the reinforcement learning algorithm includes maximizing the following objective function L CLIP (θ):

[0019]

[0020] Where: θ represents the cost function parameter, represents the ratio of the current policy to the old policy, a t Indicates action, s t Indicates status, represents the advantage function, ∈ represents the hyperparameter, E t Represents policy parameters.

[0021] Preferably, a task understanding and generation model is used to obtain a preset feature vector of the current event.

[0022] Preferably: the task understanding and generation model is used to collect, organize and analyze information and data related to the disposal, and collect data from various sensors, monitoring equipment, and personnel reports, and perform data cleaning, integration and analysis to form situation analysis results.

[0023] Preferably: use the pandas library and numpy module for data cleaning, integration, and analysis.

[0024] Preferably, a task simulation deduction model is used to simulate and deduce the target disposal plan to evaluate the feasibility and effectiveness of the target disposal plan.

[0025] A treatment plan generating device, used to execute the above-mentioned treatment plan generating method, the device comprising:

[0026] A feature acquisition unit, used to acquire a preset feature vector of the current event;

[0027] a candidate disposal plan generating unit, configured to use a random forest decision-making model to make predictions based on the feature vectors, wherein each decision tree classifies the input feature vectors and provides a recommendation result; and to obtain a candidate disposal plan by voting or averaging the recommendation results of all decision trees; the candidate disposal plan at least includes a candidate evacuation order and a candidate time and interval;

[0028] A judgment unit, configured to judge whether the random forest decision-making auxiliary model is unable to clarify a decision path or faces a high-dimensional complexity problem;

[0029] a target disposal solution determination unit, configured to determine that the candidate disposal solution is the target disposal solution after determining that the random forest decision-making auxiliary model has a clear decision path or does not face high-dimensional complexity problems;

[0030] A reinforcement learning unit is used to input the high-dimensional data corresponding to the current event into a reinforcement learning algorithm after determining that the random forest decision-making auxiliary model cannot clearly define the decision path or faces a high-dimensional complexity problem; the high-dimensional data at least includes a preset evacuation sequence, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation sequence and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

[0031] A treatment plan generating device, comprising a processor and a memory:

[0032] The memory is used to store program code and transmit the program code to the processor;

[0033] The processor is configured to execute the above-mentioned treatment plan generating method according to the instructions in the program code.

[0034] A computer-readable storage medium is used to store program code, and the program code is used to execute the above-mentioned treatment plan generation method.

[0035] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0036] The embodiment of the present application provides a method, device, equipment and medium for generating a disposal plan. The main advantage of combining the use of random forest and PPO algorithms lies in their respective characteristics and complementarity. The system can handle simple situations with clear rules, and can also cope with dynamic and complex environments, improving the robustness and reliability of the system. In cases where simple rules cannot cover them, the PPO algorithm can comprehensively consider multiple factors (such as wind speed, wind direction, traffic conditions, etc.) to provide more refined and personalized evacuation plans. By combining random forest and PPO algorithms, the system can better cope with complex decision-making scenarios, improve the accuracy and reliability of decisions, and provide a more robust solution for dealing with various complex situations. Later, with use, reinforcement learning can continuously generate plans in the direction that is considered good (user preferred direction, user: commander, troop staff, etc.). The more it is used, the easier it is to generate plans that meet user preferences. The process of user adjustment after generation is eliminated.

[0037] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort.

[0039] Figure 1 This is a flow chart of a method for generating a treatment plan provided by an embodiment of the present invention;

[0040] Figure 2 This is a business flow chart of the auxiliary decision model operation provided by an embodiment of the present invention;

[0041] Figure 3 This is an overall flow chart of a sample construction scheme provided by an embodiment of the present invention;

[0042] Figure 4 This is a random forest algorithm diagram provided by an embodiment of the present invention;

[0043] Figure 5 is a schematic diagram of a treatment plan generating device provided by an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of a treatment plan generating device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of the present invention.

[0046] See also Figure 1 , is a method for generating a treatment plan provided by an embodiment of the present invention, such as Figure 1 As shown, the method may include:

[0047] S101: Obtain a preset feature vector of the current event; in specific implementation, the embodiment of the present application may provide a task understanding and generation model to obtain a preset feature vector of the current event.

[0048] Furthermore, the task understanding and generation model is used to collect, organize, and analyze information and data related to the action. It also collects data from various sensors, monitoring equipment, and personnel reports, and cleans, integrates, and analyzes the data to produce situation analysis results. The pandas library and numpy modules are used for data cleaning, integration, and analysis.

[0049] S102: Using a random forest decision-making model to make predictions based on the feature vector, each decision tree classifies the input feature vector and provides a recommendation result; voting or averaging the recommendation results of all decision trees to obtain a candidate disposal plan; the candidate disposal plan includes at least a candidate evacuation order and a candidate time and interval;

[0050] S103: Determine whether the random forest decision-making support model is unable to clarify the decision path or faces high-dimensional complexity problems;

[0051] S104: After determining that the random forest decision-making auxiliary model has a clear decision path or does not face high-dimensional complexity problems, determine the candidate disposal solution as the target disposal solution;

[0052] S105: After determining that the random forest decision-making auxiliary model cannot clearly define the decision path or faces a high-dimensional complexity problem, the high-dimensional data corresponding to the current event is input into the reinforcement learning algorithm; the high-dimensional data at least includes a preset evacuation order, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation order and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

[0053] In specific implementation, the embodiment of the present application may provide a data processing flow of the reinforcement learning algorithm including:

[0054] Using a strategy network to generate new actions based on the preset evacuation sequence, preset time and interval, speed, wind direction, and traffic conditions, and evaluating the effectiveness of different disposal plans;

[0055] Perform multiple trajectory sampling to record the state, action, reward, and next state;

[0056] The advantage function is calculated, and the optimization strategy network is updated through a restriction strategy to optimize the candidate evacuation sequence and the candidate time and interval included in the candidate treatment plan to obtain the target treatment plan.

[0057] The goal of the reinforcement learning algorithm includes maximizing the following objective function L CLIP (θ):

[0058]

[0059] Where: θ represents the cost function parameter, represents the ratio of the current policy to the old policy, a t Indicates action, s t Indicates status, represents the advantage function, ∈ represents the hyperparameter, E t Represents policy parameters.

[0060] It can be understood that the final determined target disposal plan can be directly displayed to the decision maker for the decision maker to analyze and judge on his own to achieve the purpose of assisting decision-making. In actual applications, in order to deduce the feasibility of the formed target disposal plan, the embodiment of the present application can also provide a task simulation deduction model to simulate and deduce the target disposal plan to evaluate the feasibility and effectiveness of the target disposal plan.

[0061] The following takes the setting simulation deduction step as an example to introduce the method for generating a disposal solution provided by the embodiment of the present application in detail. Figure 2 shown.

[0062] First, the construction process of the intelligent decision-making support model (including the random forest decision-making support model and the reinforcement learning algorithm) provided in the embodiment of the present application is introduced.

[0063] 1. Under the guidance of experts, establish an expert system, form a plan library for emergency response action plans, and initially form a system for linking plan elements;

[0064] Second, based on the comprehensive judgment conclusions of various events and the intentions of superiors, we extract the characteristics of elements, quickly and intelligently match them with the plan library, build models such as "numerical attribute similarity", "text attribute similarity", and "fuzzy attribute similarity", and calculate the similarity of element attributes;

[0065] 3. Based on the constraints related to achieving the action objectives and predetermined goals, guarantee capabilities and risks, etc., assign element matching and overall matching scores, and form an action plan document through optimized combination.

[0066] 4. Based on professional simulation and deduction tools, different action plans are simulated and verified to identify their respective advantages and disadvantages. Combined with expert experience, humans are in the loop to adjust and optimize the plans, assist in scientific decision-making, and ensure the scientific nature and effectiveness of the plans.

[0067] In order to better train the model, in actual applications, several sample solutions can be pre-established to form a solution library. The existing solution database is an indispensable foundation for artificial intelligence to generate decision-making solutions. It not only provides rich information resources for artificial intelligence, but also ensures the scientificity, effectiveness and innovation of decision-making. Specifically, the method of using artificial intelligence to generate decision-making solutions relies on existing solution databases, because these databases provide artificial intelligence with rich historical data, expert knowledge and lessons learned, enabling artificial intelligence to quickly generate personalized and efficient decisions through pattern recognition, risk assessment and multi-dimensional analysis. The information accumulated in the database supports artificial intelligence in simulation, prediction and continuous learning, thereby optimizing the decision-making process, reducing human bias, and improving the consistency and standardization of decisions. The process of constructing sample solutions is as follows: Figure 3 shown.

[0068] Furthermore, the database can stimulate AI's innovative thinking, helping it quickly adapt to complex and changing environments and generate effective response strategies, ensuring the objectivity and reliability of its decisions. Furthermore, the solution library provides quick access to preset solutions, allowing users to make decisions quickly in emergency situations and reduce reaction time. The solution library ensures that all actions follow standardized processes, reducing inconsistencies and errors caused by individual differences. The solution library can provide a variety of response strategies based on different environments and types, enhancing flexibility in responding to various situations. The solution library can help users more effectively allocate limited resources, such as manpower, supplies, and time, to respond to emergencies. Effective evacuation, protection, and rescue plans can minimize casualties among soldiers and civilians. In short, existing solution databases are an indispensable knowledge base and information source for AI to generate decision-making solutions.

[0069] After the intelligent decision-making assistance model is established, the system is divided into the following three parts: task understanding and generation, automatic generation of action plans, and task simulation and deduction.

[0070] a. Task understanding and generation. Responsible for collecting, organizing, and analyzing information and data related to the response to form a comprehensive understanding of the information and obtain the characteristic vector of the current event. Data from various sensors, monitoring equipment, and personnel reports is collected, cleaned, integrated, and analyzed to form a situational analysis. Based on the document rule model, the system rapidly identifies and converts superior intent to form a coarse-grained task list. Based on the task generation model and the nuclear, biological, and chemical (NBC) emergency knowledge base, tasks are finely decomposed and a visual task list is quickly generated to assist personnel in decision-making.

[0071] b. Automatic generation of action plans. Based on the results of situation analysis and preset goals, task plans and action plans for disposal are formulated. According to resources, laws and regulations, and actual needs, the personnel, equipment, and materials involved in the disposal are determined, and the priority and time sequence of task execution are arranged to achieve efficient combat response. The automatic generation of action plans uses situation analysis results and task lists to assist in decision-making, intelligently generate action plans, and push them to the system platform in the form of charts. Each time the intelligent decision-making model obtains a different feature vector, it will generate a set of target disposal plans. In actual applications, users can call the intelligent decision-making model multiple times according to actual needs, and each call of the intelligent decision-making model can generate a set of disposal plans, and finally generate multiple sets of disposal plans for decision makers' reference.

[0072] c. Mission simulation and deduction. Use models and algorithms to simulate and deduce disposal plans to evaluate their feasibility and effectiveness. By simulating various scenarios and the execution of disposal measures, predict the results of defensive disposal, evaluate the feasibility and effectiveness of the plan, provide reference and improvement suggestions, and further optimize the decision-making process. When implemented, receive action plan input, simulate different scenarios, and output simulation and deduction results for subsequent evaluation and optimization. The mission simulation and deduction module receives the plan input provided by the decision support, runs the simulation model to deduce and evaluate the plan, and outputs the simulation and deduction results. Human intervention in the loop is required for adjustment, and the adjustment results are fed back after adjustment to regenerate the results.

[0073] The intelligent decision-making support model provided in the embodiment of the present application includes a random forest decision-making support model and reinforcement learning. The two parts are introduced in detail below.

[0074] Reinforcement learning is a machine learning technique that trains software to make decisions to achieve optimal outcomes. It mimics the trial-and-error learning process humans use to achieve goals. Software actions that contribute to the goal are reinforced, while actions that deviate from the goal are ignored.

[0075] Reinforcement learning algorithms use a reward-and-penalty model when processing data. These algorithms learn from the feedback from each action and independently discover the optimal path to achieve a final outcome. These algorithms are also capable of delayed gratification. The optimal overall strategy may require short-term sacrifices, so the optimal approach they discover may include some penalties or detours along the way.

[0076] Advantages of using reinforcement learning methods: ① Reinforcement learning algorithms can be used in complex environments with many rules and dependencies. In the same environment, even if humans know the environment very well, they may not be able to determine the best path. Reinforcement learning algorithms can quickly adapt to changing environments and find new strategies to optimize results. ② In traditional machine learning algorithms, humans must guide the algorithm by labeling data pairs. When using reinforcement algorithms, there is no need for manual labeling. Such algorithms can learn on their own. At the same time, such algorithms also provide mechanisms for integrating human feedback, allowing the system to be debugged based on human preferences, expertise, and corrections. ③ Reinforcement learning essentially focuses on maximizing long-term rewards, so it is suitable for scenarios where actions can have long-term consequences. It is particularly suitable for real-world situations where feedback is not immediate at every step because it can learn from delayed rewards.

[0077] Proximal Policy Optimization (PPO) is an advanced reinforcement learning algorithm that combines policy gradient methods with value function approximation, aiming to maximize cumulative rewards by optimizing policies. PPO is a policy gradient-based algorithm and a variant of Trust Region Policy Optimization (TRPO). PPO maintains policy stability by clipping the objective function to limit the magnitude of policy updates at each step. Its key features include: clipped probability ratios to prevent the policy from changing too much during updates; multiple updates can be safely performed on a single data batch, especially in high-dimensional inputs and real-time learning environments; simple implementation, which does not require complex mathematical operations to ensure the security of policy updates; and balanced exploration and exploitation: PPO attempts to strike a balance between learning stability and sufficient exploration to avoid local optimality and improve policy performance.

[0078] The goal of PPO is to maximize the following objective function L CLIP (θ):

[0079]

[0080] Where: θ represents the cost function parameter, represents the ratio of the current policy to the old policy, a t Indicates action, s t Indicates status, represents the advantage function, ∈ represents the hyperparameter, E t Represents policy parameters.

[0081] This objective function is designed to prevent the policy from being updated too quickly, thereby ensuring the stability of the policy.

[0082] Random forest learning algorithms, such as Figure 4 As shown:

[0083] Random forests may not be effective when dealing with complex environmental conditions, and a single decision-making method often struggles to cope with all possible battlefield variables. The random forest model, with its simple and intuitive rule design and fast computing capabilities, can effectively make preliminary decisions. However, when faced with high-dimensional data and the interaction of multiple factors, random forests may encounter performance bottlenecks and decision-making difficulties. Random forests may not be able to fully model and provide accurate decisions. In addition, when multiple factors (such as wind speed, wind direction, distance, etc.) influence decision-making simultaneously and have complex interactions with each other, a single random forest will have difficulty effectively handling these complex interactions, making it difficult to generate the best decision-making plan. In this case, it is necessary to combine the PPO algorithm with reinforcement learning to launch new response plans by continuously optimizing the strategy.

[0084] In practical decision-making support for nuclear, biological, and chemical (NBC) emergency response plans, random forests can help decision-makers comprehensively consider multiple factors and make more accurate decisions. They can analyze key indicators such as the concentration of characteristic aerogels, mortality rate, and morbidity in NBC emergencies and recommend the optimal response plan based on these indicators. Random forests also offer advantages such as resistance to overfitting, ability to handle high-dimensional data, and robustness.

[0085] The Random Forest algorithm (Random Forest Algorithm) supports decision-making by constructing multiple decision trees and combining their outputs to provide more robust and accurate predictions. This algorithm leverages the advantages of ensemble learning to reduce the bias and variance that can be introduced by a single decision tree, making the relationship between event characteristics and classification results more reliable. The Random Forest algorithm clearly demonstrates decision logic through its tree-like structure. When a new task description and situation information are input, the Random Forest model gradually determines and classifies the situation based on its learned rules, ultimately recommending the solution number that best matches the current situation.

[0086] Random forest is an ensemble learning algorithm that uses a voting algorithm to construct multiple decision trees to classify samples. Its core concept is to aggregate the classification results of each decision tree and ultimately assign the sample to the category with the most votes. Therefore, compared to general classification algorithms, it is less susceptible to overfitting, is not limited by data dimensionality, and can handle high-dimensional data. A random forest can be viewed as a mapping f:X→Y, where X represents the input and Y represents the output. f implements the random forest by passing X input to each decision tree for voting, ultimately outputting the category Y.

[0087] When a new nuclear, biological, or chemical (NBC) incident occurs, the data is fed into the trained random forest model. The model then makes predictions based on the input feature vector. Each decision tree classifies the input feature vector and generates a recommendation. The random forest model then votes or averages the predictions of all decision trees to arrive at a final recommendation. This means that if a majority of decision trees favor a particular evacuation plan, that plan is considered the final recommendation. Finally, based on the voting results or average, a label or ID for the recommended evacuation plan is determined. This label or ID serves as the output of the recommendation system to aid the decision-making process.

[0088] The main advantage of combining the random forest and PPO algorithms lies in their respective characteristics and complementary nature. The random forest model has a relatively simple structure, a clear decision path, and is easy to understand and interpret. It can quickly perform preliminary screening and provide initial evacuation plans based on predefined rules and conditions, making it suitable for most common situations. It also has low computational cost and can quickly produce results when dealing with simpler problems with clear rules. The PPO algorithm excels in handling high-dimensional data and complex nonlinear decision-making problems. It can make optimized decisions in dynamic and complex environments. Through continuous learning and optimization, it adapts to changing environments and conditions, improving the system's decision-making capabilities. Through trajectory sampling and cumulative reward calculation, it optimizes decision-making strategies, making the generated evacuation plans more reasonable and effective overall.

[0089] In some cases, the random forest model may not be able to effectively handle complex environmental data and environmental changes, resulting in an unclear decision path or difficulty handling high-dimensional data and nonlinear relationships. In this case, the Proximal Political Optimization (PPO) algorithm will be called to provide optimized decisions.

[0090] When a random forest model encounters a dilemma where it can't clearly define a decision path, it's usually due to conflicting information or high uncertainty in the input data. In these situations, the random forest model can't provide a clear decision solution. The PPO algorithm, however, uses a policy network to generate a new decision-making strategy that can handle dynamic and uncertain environments.

[0091] Furthermore, when the input data is high-dimensional and contains complex nonlinear relationships between variables, the random forest model may not be able to adequately model and generate reasonable decisions. For example, when on-site information includes multiple sources of danger, complex wind direction and speed variations, and diverse traffic conditions, the random forest model struggles to process this high-dimensional data and generate effective evacuation plans. In such cases, the PPO algorithm continuously optimizes its decision-making strategy through trajectory sampling and cumulative reward calculation. This effectively handles high-dimensional data and complex nonlinear relationships, generating more reasonable and effective evacuation plans.

[0092] In summary, the main advantage of the disposal plan generation method provided by this application, which combines the use of random forest and PPO algorithms, lies in their respective characteristics and complementarity. The system can handle simple situations with clear rules, and can also cope with dynamic and complex environments, improving the robustness and reliability of the system. In cases where simple rules cannot cover them, the PPO algorithm can comprehensively consider multiple factors (such as wind speed, wind direction, traffic conditions, etc.) to provide more refined and personalized evacuation plans. By combining random forest and PPO algorithms, the system can better cope with complex decision-making scenarios, improve the accuracy and reliability of decisions, and provide a more robust solution for dealing with various complex situations. In the later stage, as it is used, reinforcement learning can continuously generate plans in the direction that is considered good (the direction of user preference, user: commander). The more it is used, the easier it is to generate plans that meet user preferences. The process of user adjustment after generation is eliminated.

[0093] See also Figure 5 , the embodiment of the present application can also provide a treatment plan generating device, such as Figure 5 As shown, the apparatus for executing the above-mentioned treatment plan generating method may include:

[0094] A feature acquisition unit 501 is used to acquire a preset feature vector of a current event;

[0095] The candidate disposal plan generating unit 502 is configured to use a random forest decision-making model to make predictions based on the feature vectors. Each decision tree classifies the input feature vectors and provides a recommendation result. The recommendation results of all decision trees are voted or averaged to obtain a candidate disposal plan. The candidate disposal plan includes at least a candidate evacuation sequence and a candidate time and interval.

[0096] A judgment unit 503 is used to judge whether the random forest decision-making auxiliary model cannot clarify the decision path or faces high-dimensional complexity problems;

[0097] A target solution determination unit 504 is configured to determine that the candidate solution is the target solution after determining that the random forest decision-making model has a clear decision path or does not face high-dimensional complexity issues;

[0098] The reinforcement learning unit 505 is used to input the high-dimensional data corresponding to the current event into the reinforcement learning algorithm after determining that the random forest decision-making auxiliary model cannot clearly define the decision path or faces a high-dimensional complexity problem; the high-dimensional data at least includes a preset evacuation order, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation order and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

[0099] The present application may also provide a treatment plan generating device, the device comprising a processor and a memory:

[0100] The memory is used to store program code and transmit the program code to the processor;

[0101] The processor is configured to execute the steps of the above-mentioned treatment plan generating method according to the instructions in the program code.

[0102] like Figure 6 As shown, a treatment plan generating device provided by an embodiment of the present application may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, the memory 11, and the communication interface 12 all communicate with each other via the communication bus 13.

[0103] In the embodiment of the present application, the processor 10 may be a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit, a digital signal processor, a field programmable gate array, or other programmable logic devices.

[0104] The processor 10 may call a program stored in the memory 11 . Specifically, the processor 10 may execute operations in an embodiment of the treatment plan generating method.

[0105] The memory 11 is used to store one or more programs. The program may include program code, and the program code includes computer operating instructions. In the embodiment of the present application, the memory 11 stores at least a program for implementing the following functions:

[0106] Get the preset feature vector of the current event;

[0107] A random forest decision-making model is used to make predictions based on the feature vectors, with each decision tree classifying the input feature vectors and providing a recommendation result; the recommendation results of all decision trees are voted on or averaged to obtain a candidate disposal plan; the candidate disposal plan includes at least a candidate evacuation order and a candidate time and interval;

[0108] Determine whether the random forest decision-making model is unable to clarify the decision path or faces high-dimensional complexity issues;

[0109] After determining that the random forest decision-making auxiliary model has a clear decision path or does not face high-dimensional complexity problems, determining the candidate disposal solution as the target disposal solution;

[0110] After determining that the random forest decision-making assistance model cannot clearly define the decision path or faces high-dimensional complexity problems, the high-dimensional data corresponding to the current event is input into the reinforcement learning algorithm; the high-dimensional data at least includes a preset evacuation order, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation order and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

[0111] In one possible implementation, the memory 11 may include a program storage area and a data storage area, wherein the program storage area can store an operating system and application programs required for at least one function (such as a file creation function, a data reading and writing function), etc.; the data storage area can store data created during use, such as initialization data, etc.

[0112] In addition, the memory 11 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0113] The communication interface 12 may be an interface of a communication model, used for connecting to other devices or systems.

[0114] Of course, it needs to be explained that Figure 6 The structure shown does not constitute a limitation on the treatment plan generating device in the embodiment of the present application. In actual application, the treatment plan generating device may include Figure 6 More or fewer components than shown, or combinations of certain components.

[0115] An embodiment of the present application may also provide a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the steps of the above-mentioned treatment plan generation method.

[0116] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0117] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present application.

[0118] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for generating a treatment plan, characterized in that: include: Get the preset feature vector of the current event; A random forest decision-making model is used to make predictions based on the feature vectors. Each decision tree classifies the input feature vectors and gives a recommendation result. The recommendation results of all decision trees are voted or averaged to obtain a candidate disposal plan. The selected disposal plan at least includes a selected evacuation order and a selected time and interval; Determine whether the random forest decision-making model is unable to clarify the decision path or faces high-dimensional complexity issues; After determining that the random forest decision-making auxiliary model has a clear decision path or does not face high-dimensional complexity problems, determining the candidate disposal solution as the target disposal solution; After determining that the random forest decision-making assistance model cannot clearly define the decision path or faces high-dimensional complexity problems, the high-dimensional data corresponding to the current event is input into the reinforcement learning algorithm; the high-dimensional data at least includes a preset evacuation order, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation order and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

2. The method for generating a treatment plan according to claim 1, wherein: The data processing flow of the reinforcement learning algorithm includes: Using a strategy network to generate new actions based on the preset evacuation sequence, preset time and interval, speed, wind direction, and traffic conditions, and evaluating the effectiveness of different disposal plans; Perform multiple trajectory sampling to record the state, action, reward, and next state; The advantage function is calculated, and the optimization strategy network is updated through a restriction strategy to optimize the candidate evacuation sequence and the candidate time and interval included in the candidate treatment plan to obtain the target treatment plan.

3. The method for generating a treatment plan according to claim 1, wherein: The goal of the reinforcement learning algorithm includes maximizing the following objective function L CLIP (θ): Where: θ represents the cost function parameter, represents the ratio of the current policy to the old policy, a t Indicates action, s t Indicates status, represents the advantage function, ε represents the hyperparameter, E t Represents policy parameters.

4. The method for generating a treatment plan according to claim 1, wherein: The task understanding and generation model is used to obtain the feature vector of the preset current event.

5. The method for generating a treatment plan according to claim 4, wherein: The task understanding and generation model is used to collect, organize and analyze information and data related to disposal, and collect data from various sensors, monitoring equipment, and personnel reports, and perform data cleaning, integration and analysis to form situation analysis results.

6. The method for generating a treatment plan according to claim 5, wherein: Use the pandas library and numpy module for data cleaning, integration and analysis.

7. The method for generating a treatment plan according to claim 1, wherein: The task simulation deduction model is used to simulate and deduce the target disposal plan to evaluate the feasibility and effectiveness of the target disposal plan.

8. A treatment plan generating device, characterized in that: The device is used to execute the treatment plan generation method according to any one of claims 1 to 7, comprising: A feature acquisition unit, used to acquire a preset feature vector of the current event; a candidate disposal plan generating unit, configured to use a random forest decision-making model to make predictions based on the feature vectors, wherein each decision tree classifies the input feature vectors and provides a recommendation result; and to obtain a candidate disposal plan by voting or averaging the recommendation results of all decision trees; the candidate disposal plan at least includes a candidate evacuation order and a candidate time and interval; A judgment unit, configured to judge whether the random forest decision-making auxiliary model is unable to clarify a decision path or faces a high-dimensional complexity problem; a target disposal solution determination unit, configured to determine that the candidate disposal solution is the target disposal solution after determining that the random forest decision-making auxiliary model has a clear decision path or does not face high-dimensional complexity problems; A reinforcement learning unit is used to input the high-dimensional data corresponding to the current event into a reinforcement learning algorithm after determining that the random forest decision-making auxiliary model cannot clearly define the decision path or faces a high-dimensional complexity problem; the high-dimensional data at least includes a preset evacuation sequence, a preset time and interval, a speed, a wind direction, and traffic conditions; the reinforcement learning algorithm is used to use the high-dimensional data to optimize the candidate evacuation sequence and the candidate time and interval included in the candidate disposal plan to obtain the target disposal plan.

9. A treatment plan generating device, characterized in that: The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the treatment plan generating method according to any one of claims 1 to 7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium is used to store program code, and the program code is used to execute the treatment plan generation method according to any one of claims 1 to 7.

Images