Automatic scheduling method based on reinforcement learning
Through the automatic scheduling method based on reinforcement learning, a pipeline system model is built and resource allocation is dynamically adjusted, which solves the problem of transportation plan deviation in automatic scheduling of long-term pipelines, and achieves more efficient and intelligent transportation management.
Patent Information
- Application Number
- CN202411580590.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-07
- Publication Date
- 2024-12-06
AI Technical Summary
The existing automatic scheduling algorithm for long-distance pipelines is difficult to adaptively generate transportation plans based on actual needs, resulting in deviations from transportation plans and actual needs.
Automatic scheduling method based on reinforcement learning is adopted to obtain pipeline system status information, build system models, optimize models, generate transportation plans, and continuously adjust to adapt to changes, dynamically adjust resource allocation, and prioritize emergency material transportation tasks.
A more accurate and flexible transportation plan has been achieved, the intelligent level of transportation management has been improved, and it can adapt to various disturbances and changes in actual situations, ensuring transportation efficiency and resource utilization.
Smart Images

Figure CN119090153A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of scheduling and dispatching, and in particular to an automatic scheduling method based on reinforcement learning. Background Art
[0002] Long-distance pipelines play an important role in the transportation of oil and natural gas. They are characterized by large diameter, long distance, large transmission volume, and are equipped with various supporting auxiliary projects and independent pipeline monitoring and management systems. The task of long-distance pipelines is to transport liquid or gas products such as crude oil, refined oil, natural gas, etc. over long distances. They are of great significance in meeting the requirements of both domestic and foreign supply and demand parties and ensuring safe, economical and fast transportation.
[0003] With the advancement of science and technology, automation technology has been gradually applied to long-distance pipeline transportation, improving transportation efficiency and safety. The automation system of the oil pipeline includes multiple links such as data acquisition, monitoring, and control, which can realize remote monitoring and intelligent scheduling. Automatic scheduling technology is one of the specific applications of automation technology in long-distance pipeline transportation. It aims to optimize pipeline transportation plans through algorithms and improve transportation efficiency and resource utilization. This technology can automatically generate the optimal transportation plan based on the actual situation and transportation needs of the oil pipeline, including transportation volume, transportation time, transportation route, etc.
[0004] Therefore, the automatic scheduling technology of long-distance pipelines requires algorithms that can adapt to different pipeline characteristics and transportation needs. However, existing algorithms are difficult to adapt scheduling according to actual needs, resulting in deviations between the generated transportation plans and actual needs. Summary of the invention
[0005] In order to make the system's transportation plan better meet actual needs, this application provides an automatic scheduling method based on reinforcement learning.
[0006] The present application provides an automatic scheduling method based on reinforcement learning, which adopts the following technical solutions: In a first aspect, the present application provides an automatic scheduling method based on reinforcement learning, comprising the following steps: Acquire pipeline system status information, wherein the pipeline system status information includes flow distribution data, pipeline pressure data, environmental data, pipeline characteristic data, and transportation demand data; According to the pipeline system state information, a system model is constructed, wherein the system model is used to evaluate the specific conditions of the pipeline system facing deterministic disturbances and uncertain disturbances; Solving the system model to obtain corresponding evaluation results; Optimizing the model according to the evaluation results; Applying the optimized system model to automatic pipeline scheduling to generate a transportation plan based on the pipeline system status information; Continuously collect feedback data and continuously adjust and optimize the system model.
[0007] Furthermore, the step of applying the optimized system model to the automatic scheduling of the pipeline and generating a transportation plan according to the pipeline system status information specifically includes: Obtaining priority setting factors from a preset database, wherein the priority setting factors include transportation importance and urgency; According to the transportation demand data, the pipeline transportation tasks are prioritized, and the priorities of the pipeline transportation tasks are specifically divided into key material transportation tasks, conventional material transportation tasks and non-emergency material transportation tasks from high to low; By dynamically adjusting resource allocation, transportation planning is performed according to the urgency, and a transportation planning form is generated, wherein the transportation planning form includes all pipeline transportation tasks that are waiting and transportation time periods corresponding to the pipeline transportation tasks; According to the urgency and the transportation planning form, the urgency of the key material transportation task is matched with the transportation time period of the key material transportation task; If the transportation time period of a critical material transportation task cannot meet the urgency requirements of the critical material transportation task, idle pipeline resources will be allocated to high-priority tasks first to ensure that high-priority tasks can be completed on time.
[0008] Furthermore, if the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the step of preferentially allocating idle pipeline resources to high-priority tasks specifically includes: Prioritize key material transportation tasks according to the key material transportation tasks and their urgency; Obtain the estimated completion time of the key material transportation task; Get the preset interval time from the preset database; Calculating an early completion time interval, where the early completion time interval is the difference between the estimated completion time of the key material transportation task and the urgency of the key material transportation task; If there is a critical material transportation task whose early completion time interval is greater than the preset interval time, and the early completion time intervals corresponding to all critical material transportation tasks after the critical material transportation task are greater than the preset interval time, then according to the early completion time interval and the transportation planning form, the most urgent conventional material transportation task or non-urgent material transportation task shall be arranged within the first early completion time interval greater than the preset interval time; The step of calculating the early completion time interval is repeatedly executed.
[0009] Furthermore, after the step of allocating idle pipeline resources to high-priority tasks first if the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the method further includes: Regenerate and update the transportation planning form, and determine whether the updated transportation planning form can meet the urgency requirements of key material transportation tasks; If the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the ongoing material transportation task will be called up; If the urgency of the critical material transportation task is higher than that of the ongoing material transportation tasks, the transportation plan will be adjusted to terminate the ongoing material transportation tasks with lower urgency and give priority to the critical material transportation tasks with higher urgency. Re-update the transport planning form; If the transportation time period of the critical material transportation task still cannot meet the urgency requirements of the critical material transportation task, the ongoing material transportation task with a higher urgency level will be terminated, and the critical material transportation task with a higher urgency level will be performed first; Repeat the steps to re-update the transportation planning form.
[0010] Furthermore, after the step of applying the optimized system model to the automatic scheduling of the pipeline and generating a transportation plan according to the pipeline system status information, the method further includes: If an emergency material transportation task with a higher priority than the critical material transportation task is received, the transportation resources required to execute the emergency material transportation task are immediately calculated, and the execution of non-emergency material transportation tasks is suspended or adjusted first to release pipeline capacity and pump station power resources; Adjust the execution plan of some routine material transportation tasks to reduce the impact on emergency material transportation tasks; After the urgent material transportation task is completed, the system immediately resumes the execution plan of other tasks and regenerates the transportation planning form.
[0011] Furthermore, the step of constructing a system model according to the pipeline system status information specifically includes: Based on the one-dimensional flow in a rigid pipe, the gas dynamics equation in the pipe is used for derivation, and the dynamics equation of the control body is combined with the gas state equation to form an equation group; ; Where m is the mass flow rate of the gas, in kg / s; p is the pressure of the gas, in Pa; A is the cross-sectional area of the pipeline, in m2 ; ρ is the density of the gas, in kg / m 3 ; D is the inner diameter of the pipe, in m; g is the acceleration due to gravity, in m / s 2 ; Δh is the vertical height of the pipeline and the horizontal plane, in m; L is the length of the pipeline, in m; R is the gas constant, in kJ / (kg·K); λ is the hydraulic friction coefficient of the pipeline; Z is the gas compressibility factor; c is the wave velocity of the gas, in m / s; x is the pipeline position variable, in m; t is the time variable, in s; When the gas in the pipeline runs in the square resistance area, the friction coefficient λ is calculated by the following formula; ; When the gas is determined, the compressibility factor Z is a function of the gas pressure and temperature and is calculated using the following formula: ; Where Re is the Reynolds number of the pipe; r is the absolute equivalent roughness of the pipe wall, in m; p c Critical pressure of gas, in Pa; T c The critical temperature of the gas, in K; The transfer function of the pipe section is derived on the basis of the equations, and the frequency domain expressions of the upstream pressure, flow rate and the downstream pressure, flow rate are obtained; The dynamic equations of the pipe section are linearized and Laplace transformed to obtain a set of ordinary differential equations in the frequency domain; assuming that the inlet pressure and outlet flow rate of the pipeline are specified as boundary conditions, a high-order transfer function expression is obtained.
[0012] Furthermore, the step of solving the system model to obtain the corresponding evaluation result specifically includes: Solving the system model using an iterative algorithm; Starting from an initial point, iterate according to the given rules to obtain a sequence. The iteration terminates within a finite number of steps, and the last point is the solution of the evaluation result. The convergence criterion is used to ensure that the iteration stops at the solution of the evaluation result. The convergence criterion is: ; Among them, ε1, ε2 are given small positive numbers; express ; is the minimum value of function f, Known or calculated in some way; Represents the gradient of function f at point x. If function f is a smooth function, the gradient at the local optimal point is a zero vector; The final point satisfies: ; Among them, ε3 and ε4 are small positive numbers used to characterize x k feasibility, ε3 is used for constraints that need to be strictly satisfied, and ε4 is used for inequality constraints, which allow a certain degree of violation but still need to be kept in a small range; c i represents the i-th constraint function; x k represents the kth candidate solution or iteration point in the optimization process; c i (x k ) thus means that at x k The value of the i-th constraint function at position; m represents the number of equality constraints; l represents the number of inequality constraints; m+l is the total number of constraints; Consider x k The distance between the optimal solution and the optimal solution. Since the optimal solution is generally unknown in advance, when the optimal solution is unique, the benchmark algorithm is used to obtain an approximate optimal solution. , and then calculate its difference with x k distance to evaluate the performance of the algorithm.
[0013] In a second aspect, the present application provides an automatic scheduling system based on reinforcement learning, comprising: A status information acquisition module, used to acquire pipeline system status information, wherein the pipeline system status information includes flow distribution data, pipeline pressure data, environmental data, pipeline characteristic data, and transportation demand data; A system model building module, used to build a system model according to the pipeline system state information, wherein the system model is used to evaluate the specific conditions of the pipeline system facing deterministic disturbances and uncertain disturbances; A solution module, used for solving the system model to obtain corresponding evaluation results; An optimization module, used for optimizing the model according to the evaluation result; A transportation plan generation module, used to apply the optimized system model to the automatic scheduling of pipelines and generate a transportation plan based on the pipeline system status information; The feedback module is used to continuously collect feedback data and continuously adjust and optimize the system model.
[0014] In a third aspect, the present application provides an intelligent terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above-mentioned automatic scheduling method based on reinforcement learning.
[0015] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned automatic scheduling method based on reinforcement learning.
[0016] In summary, compared with the prior art, the above technical solution has the following beneficial effects: The automatic scheduling method based on reinforcement learning described in the present application can obtain comprehensive and accurate pipeline system status information, build a system model that can reflect the actual situation of the pipeline system and face various disturbances, provide a basis for subsequent solution and optimization, obtain the optimal transportation strategy under various disturbances, provide guidance for subsequent automatic scheduling, form a closed-loop feedback and optimization mechanism, enable the system model to continuously adapt to changes in actual conditions, and improve the level of intelligence of transportation management. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flowchart of an automatic scheduling method based on reinforcement learning in an embodiment of the present application; Figure 2 It is a schematic diagram of a process of obtaining priority setting factors from a preset database in an embodiment of the present application; Figure 3 It is a schematic diagram of the process of re-updating the transportation planning form in an embodiment of the present application; Figure 4 It is a flowchart of adjusting the execution plan of some medium-priority tasks (conventional material transportation tasks) in an embodiment of the present application. DETAILED DESCRIPTION
[0018] The present application is further described in detail below in conjunction with all the accompanying drawings.
[0019] The present application embodiment discloses an automatic scheduling method based on reinforcement learning, referring to Figure 1 , an automatic scheduling method based on reinforcement learning includes: S101. Obtain pipeline system status information.
[0020] Specifically, the scheduling system obtains pipeline system status information, which includes flow distribution data, pipeline pressure data, environmental data, pipeline characteristic data, and transportation demand data. In this embodiment, the scheduling system collects flow distribution data and pipeline pressure data of the pipeline system in real time through a sensor network; obtains environmental data such as temperature, humidity, wind speed, etc. using external data sources such as meteorological stations and environmental monitoring stations; obtains pipeline characteristic data from a preset database, and the data in the preset database is entered by the user according to actual conditions, such as pipeline material, diameter, wall thickness, etc.; determines transportation demand data according to market demand, contract requirements, etc., including transportation volume, transportation time, transportation priority, etc., and then obtains comprehensive and accurate pipeline system status information, providing reliable data support for subsequent system modeling and solving.
[0021] S102: construct a system model according to the pipeline system status information.
[0022] Specifically, the scheduling system constructs a system model based on the pipeline system status information, and the system model is used to evaluate the specific situation of the pipeline system facing deterministic disturbances and uncertain disturbances; the scheduling system regards the pipeline system as a complex dynamic system, and uses reinforcement learning theory to build a system model, considering the impact of deterministic disturbances (such as climate change, holidays) and uncertain disturbances (such as sudden disasters, price fluctuations, pipeline defects) in the model. Design appropriate state space and action space to reflect the actual operation of the pipeline system and the optimization goals. By building a system model that can reflect the actual situation of the pipeline system and face various disturbances, it is helpful to provide a basis for subsequent solutions and optimization.
[0023] S103: Solve the system model to obtain corresponding evaluation results.
[0024] Specifically, the scheduling system solves the system model to obtain corresponding evaluation results; uses a reinforcement learning algorithm to solve the system model; uses historical data and simulation data for training so that the algorithm can learn the optimal transportation strategy and obtain the optimal transportation strategy under various disturbances to provide guidance for subsequent automatic scheduling.
[0025] S104: Optimize the model according to the evaluation results.
[0026] Specifically, the scheduling system optimizes the model according to the evaluation results; uses the evaluation results obtained by solving to analyze the performance of the model when facing different disturbances. According to the analysis results, the model parameters are adjusted, the algorithm structure is optimized, or new characteristic variables are introduced, and the optimized model is solved and evaluated again to ensure the optimization effect. This is conducive to improving the accuracy and robustness of the model, so that it can better adapt to various disturbances and actual conditions.
[0027] S105: Apply the optimized system model to the automatic scheduling of the pipeline, and generate a transportation plan according to the pipeline system status information.
[0028] Specifically, the scheduling system applies the optimized system model to the automatic scheduling of the pipeline, and generates a transportation plan according to the pipeline system status information; the optimized system model is embedded in the pipeline transportation management system, and the model is used to generate the optimal transportation plan according to the real-time pipeline system status information and transportation demand data, and the generated transportation plan is sent to each pipeline site and execution agency to realize automatic scheduling and dispatching, which is conducive to realizing automatic scheduling and dispatching of the pipeline and improving transportation efficiency and accuracy.
[0029] S106. Continuously collect feedback data and continuously adjust and optimize the system model.
[0030] Specifically, during the pipeline transportation process, the scheduling system continuously collects actual operation data and feedback information, uses the collected data to verify and evaluate the system model, discovers problems and deficiencies in the model, and adjusts and optimizes the model again based on the evaluation results to improve the adaptability and accuracy of the model. This forms a closed-loop feedback and optimization mechanism, allowing the system model to continuously adapt to changes in actual conditions and improve the level of intelligence in transportation management.
[0031] In another embodiment, S102 specifically includes the following sub-steps: Based on the one-dimensional flow in a rigid pipe, the gas dynamics equation in the pipe is used for derivation, and the dynamics equation of the control body is combined with the gas state equation to form an equation group; ; Where m is the mass flow rate of the gas, in kg / s; p is the pressure of the gas, in Pa; A is the cross-sectional area of the pipeline, in m 2 ; ρ is the density of the gas, in kg / m 3 ; D is the inner diameter of the pipe, in m; g is the acceleration due to gravity, in m / s 2 ; Δh is the vertical height of the pipeline and the horizontal plane, in m; L is the length of the pipeline, in m; R is the gas constant, in kJ / (kg·K); λ is the hydraulic friction coefficient of the pipeline; Z is the gas compressibility factor; c is the wave velocity of the gas, in m / s; x is the pipeline position variable, in m; t is the time variable, in s; When the gas in the pipeline runs in the square resistance area, the friction coefficient λ is calculated by the following formula; ; When the gas is determined, the compressibility factor Z is a function of the gas pressure and temperature and is calculated using the following formula: ; Where Re is the Reynolds number of the pipe; r is the absolute equivalent roughness of the pipe wall, in m; p c Critical pressure of gas, in Pa; T c The critical temperature of the gas, in K; The transfer function of the pipe section is derived on the basis of the equations, and the frequency domain expressions of the upstream pressure, flow rate and the downstream pressure, flow rate are obtained; The dynamic equations of the pipe section are linearized and Laplace transformed to obtain a set of ordinary differential equations in the frequency domain; assuming that the inlet pressure and outlet flow rate of the pipeline are specified as boundary conditions, a high-order transfer function expression is obtained.
[0032] In this embodiment, the one-dimensional flow of a rigid pipeline is used as the basis because the one-dimensional flow model can simplify the problem while retaining the main characteristics of the pipeline flow. The control body is derived based on the gas dynamics equation in the pipe. These equations usually include the mass conservation equation, the momentum conservation equation and the energy conservation equation. The gas state equation (such as the ideal gas state equation) is combined with the dynamics equation of the control body to form a complete set of equations. The set of equations can fully describe the flow state of the gas in the pipeline, including the changes in key parameters such as pressure, flow rate, density, etc., and provides a solid theoretical basis for subsequent model solving and optimization. By calculating the friction coefficient λ and the gas compression factor Z, the accuracy of the equation group in describing the dynamic behavior of the pipeline system is improved.
[0033] In order to simplify the problem and facilitate the solution, the dynamic equations of the pipe section are linearized, and the linearized equations are Laplace transformed to convert them from the time domain to the frequency domain, which helps to analyze the stability and frequency response characteristics of the system; in the frequency domain, the Laplace transformed equations are used to derive the transfer function expression of the pipe section. It is assumed that the inlet pressure and outlet flow of the pipeline are specified as boundary conditions, which helps to determine the unknown parameters and variables in the transfer function. The transfer function expression of the pipe section is derived, which provides a basis for the subsequent model solution and frequency response analysis. By specifying the boundary conditions, the unknown parameters and variables in the transfer function are determined, which improves the accuracy and practicality of the model.
[0034] By solving the derived equations using numerical methods (such as iteration method, matrix solution, etc.), the dynamic response of the pipeline system under different working conditions can be obtained; the solution results are compared and verified with the actual situation to ensure the accuracy and reliability of the model, which usually involves the comparison and analysis of experimental data or historical data. According to the verification results, the model is optimized and improved, including adjusting the parameters of the equation group, improving the numerical solution method or introducing a new physical model, etc. The dynamic response characteristics of the pipeline system are obtained through numerical solution, which provides a basis for subsequent pipeline transportation management. Through model verification and optimization, the accuracy and reliability of the model are improved, so that it can better adapt to changes in actual conditions.
[0035] In another embodiment, S103 specifically includes the following sub-steps: Solving the system model using an iterative algorithm; Starting from an initial point, iterate according to the given rules to obtain a sequence. The iteration terminates within a finite number of steps, and the last point is the solution of the evaluation result. The convergence criterion is used to ensure that the iteration stops at the solution of the evaluation result. The convergence criterion is: ; Among them, ε1, ε2 are given small positive numbers; express ; is the minimum value of function f, Known or calculated in some way; Represents the gradient of function f at point x. If function f is a smooth function, the gradient at the local optimal point is a zero vector; The final point satisfies: ; Among them, ε3 and ε4 are small positive numbers used to characterize x k feasibility, ε3 is used for constraints that need to be strictly satisfied, and ε4 is used for inequality constraints, which allow a certain degree of violation but still need to be kept in a small range; c i represents the i-th constraint function; x k represents the kth candidate solution or iteration point in the optimization process; c i (x k ) thus means that at x k The value of the i-th constraint function at position; m represents the number of equality constraints; l represents the number of inequality constraints; m+l is the total number of constraints; Consider x k The distance between the optimal solution and the optimal solution. Since the optimal solution is generally unknown in advance, when the optimal solution is unique, the benchmark algorithm is used to obtain an approximate optimal solution. , and then calculate its difference with x k distance to evaluate the performance of the algorithm.
[0036] Specifically, the scheduling system selects a suitable iterative algorithm based on the characteristics of the system model and the solution requirements. Before the algorithm starts, it is necessary to select an initial point as the starting point of the iteration. This initial point can be randomly selected or selected based on the background knowledge or experience of the problem. Starting from the initial point, it is iterated according to the rules given by the algorithm. In each iteration, the next candidate point is calculated based on the information of the current point (such as gradient, Hessian matrix, etc.). The iterative algorithm can gradually approach the solution of the system model, and by continuously iteratively updating the candidate points, an approximate solution close to the true solution is finally obtained. The choice of the initial point has a certain impact on the performance of the iterative algorithm, but the appropriate algorithm can usually find a better solution within a certain range.
[0037] In order to determine whether the iteration is terminated, it is necessary to set the convergence criterion. Convergence criteria usually include the amount of change in the function value, the norm of the gradient, etc. For each parameter in the convergence criterion (such as ε1, ε2, ε3, ε4), it is necessary to set appropriate thresholds. These thresholds determine when the iteration stops and the accuracy of the solution. After each iteration, check whether the current point meets the convergence criterion. If it does, the iteration terminates; if it does not, the iteration continues. The setting of the convergence criterion can ensure that the iterative algorithm stops when a sufficiently accurate solution is found, avoiding unnecessary calculations. The setting of the threshold has an important impact on the accuracy of the solution and the performance of the algorithm, and needs to be adjusted according to the actual problem.
[0038] After each iteration, calculate the value of each constraint function at the current point. According to the value of the constraint function, check whether the current point satisfies all constraints. For equality constraints, the value of the constraint function is required to be zero; for inequality constraints, the value of the constraint function is required to be less than or equal to zero (or greater than or equal to zero, depending on the direction of the constraint). According to the result of the constraint check, judge whether the current point is feasible. If feasible, continue to iterate; if not feasible, adjust the algorithm or reselect the initial point. The feasibility check of the solution can ensure that the obtained solution satisfies all constraints and meets the requirements of the actual problem. If an infeasible solution is found, the algorithm can be adjusted in time or the initial point can be reselected to avoid falling into the local optimal solution or failing to find a feasible solution.
[0039] After the iterative algorithm terminates, an approximate optimal solution is obtained. , this solution is the global optimal solution or the local optimal solution, depending on the performance of the algorithm and the complexity of the problem; use some distance metric (such as Euclidean distance, Manhattan distance, etc.) to calculate the approximate optimal solution With the current candidate solution x k The distance between the two systems; the performance of the algorithm is evaluated based on indicators such as the size of the distance and the number of iterations. If the distance is small and the number of iterations is small, the algorithm performance is good; otherwise, the algorithm performance is poor. Algorithm performance evaluation can objectively reflect the performance of the iterative algorithm in solving the system model. By comparing the performance of different algorithms, you can select an algorithm that is more suitable for the current problem or optimize the algorithm.
[0040] In another embodiment, referring to Figure 2 , S105 specifically includes the following sub-steps: S105.11. Obtain priority setting factors from a preset database.
[0041] Specifically, the scheduling system obtains priority setting factors from a preset database, and the priority setting factors include the importance and urgency of transportation; a database is preset in the system to store various priority setting factors, which are entered by the user according to actual conditions. These factors usually include the importance of the transported materials (such as key production raw materials, auxiliary materials, non-production materials, etc.) and the urgency of the transportation task, and the urgency is specifically manifested as the final deadline of the transportation task.
[0042] Users need to regularly update and maintain the priority setting factors in the database to ensure their accuracy and timeliness. This includes adjusting the importance and urgency of materials according to changes in business needs. By establishing a priority database, priority setting factors can be easily obtained and updated, providing reliable data support for subsequent priority division. The accuracy and timeliness of priority setting factors are crucial to ensuring the rationality and efficiency of transportation plans.
[0043] S105.12. Prioritize pipeline transportation tasks based on the transportation demand data.
[0044] Specifically, the scheduling system prioritizes pipeline transportation tasks according to the transportation demand data, and pipeline transportation tasks are specifically divided into key material transportation tasks (high priority), conventional material transportation tasks (medium priority) and non-urgent material transportation tasks (low priority).
[0045] The scheduling system analyzes the current transportation demand data based on the pipeline system status information and business needs, including the type, quantity, destination and other information of the transported materials. Pipeline transportation tasks are prioritized based on the transportation demand data and the set factors in the priority database. Specifically, they are divided into key material transportation tasks (high priority), conventional material transportation tasks (medium priority) and non-emergency material transportation tasks (low priority). Each transportation task is assigned a priority identifier so that it can be quickly identified and processed in subsequent transportation planning and resource allocation. By prioritizing pipeline transportation tasks, it can ensure that key materials and important transportation tasks are given priority, improving transportation efficiency and service quality. The introduction of priority identifiers makes transportation planning and resource allocation clearer and more efficient.
[0046] S105.13. By dynamically adjusting resource allocation, transportation planning is performed according to the degree of urgency, and a transportation planning form is generated.
[0047] Specifically, the scheduling system dynamically adjusts resource allocation, performs transportation planning according to the urgency, and generates a transportation planning form, which includes all pipeline transportation tasks waiting and the transportation time periods corresponding to the pipeline transportation tasks. The scheduling system monitors the resource status of the pipeline system in real time, including pipeline capacity, transportation speed, available time, etc. At the same time, the availability and allocation of resources are evaluated based on historical data and the needs of current transportation tasks.
[0048] The scheduling system generates a transportation planning form based on the results of resource monitoring and evaluation, as well as the priority division of transportation tasks. The form includes all pipeline transportation tasks that are waiting and the transportation time periods corresponding to these tasks. Based on the transportation planning form, dynamic adjustments are made according to the urgency of the transportation tasks and resource availability. This includes adjusting the transportation sequence, allocating pipeline resources, adjusting the transportation time periods, etc. By dynamically adjusting resource allocation and transportation planning, efficient utilization of pipeline system resources can be ensured while meeting the urgency requirements of various transportation tasks. The generation of transportation planning forms makes the formulation and execution of transportation plans clearer and more controllable.
[0049] S105.14. According to the urgency and transportation planning form, the urgency of the key material transportation task is matched with the transportation time period of the key material transportation task.
[0050] Specifically, the scheduling system matches the urgency of the key material transportation task with the transportation time period in the transportation planning form according to the urgency and transportation planning form, ensuring that the key material transportation task with high urgency can be processed within the appropriate transportation time period, and then knowing whether the key material transportation task can be completed within the required time.
[0051] S105.15. If the transportation time period of the key material transportation task cannot meet the urgency requirements of the key material transportation task, the idle pipeline resources will be allocated to high-priority tasks first.
[0052] Specifically, if the transportation time period of a critical material transportation task cannot meet its urgency requirements, then according to the priority adjustment strategy, idle pipeline resources will be allocated to high-priority tasks first, and at the same time, low-priority tasks will be appropriately adjusted or delayed.
[0053] S105.16. Prioritize key material transportation tasks based on the key material transportation tasks and their urgency.
[0054] Specifically, the scheduling system sorts all critical material transportation tasks by urgency to ensure that the most urgent tasks are handled first. Ensure that urgent tasks can be arranged as soon as possible to reduce losses caused by delays. By matching the urgency of critical material transportation tasks with the transportation time period, it can ensure that high-urgency tasks are handled in a timely manner, improving transportation efficiency and service quality. The implementation of priority adjustment strategies can ensure that high-priority tasks can be completed on time while minimizing the impact on low-priority tasks.
[0055] S105.17. Obtain the estimated completion time of the key material transportation task.
[0056] Specifically, the scheduling system estimates the time for each key material transportation task based on the current transportation resources and task allocation, providing basic data for the subsequent calculation of the early completion time interval, which helps to more accurately evaluate the rationality of task scheduling.
[0057] S105.18. Obtain a preset interval time from a preset database.
[0058] Specifically, the scheduling system obtains the preset interval time from a preset database; the preset interval time is a reasonable preset interval time set by the user based on historical data and experience, and is used to determine whether the time interval between tasks is sufficient.
[0059] S105.19. Calculate the early completion time interval.
[0060] Specifically, the scheduling system calculates the early completion time interval, which is the difference between the estimated completion time of the key material transportation task and the urgency of the key material transportation task; the estimated completion time of the key material transportation task is compared with the urgency requirement, and the early completion time interval is calculated to provide a time reference for task scheduling, which helps to optimize task allocation and improve resource utilization efficiency.
[0061] S105.110. If there is a key material transportation task whose early completion time interval is greater than the preset interval time, and the early completion time intervals corresponding to all key material transportation tasks after this key material transportation task are greater than the preset interval time, then according to the early completion time interval and the transportation planning form, the most urgent conventional material transportation task or non-urgent material transportation task shall be arranged within the first early completion time interval that is greater than the preset interval time.
[0062] Specifically, if there is a key material transportation task whose early completion time interval is greater than the preset interval time, and the early completion time intervals of all key material transportation tasks after this task are greater than the preset interval time, it means that this key material transportation task and the key material transportation tasks after it have been completed prematurely and can be appropriately postponed for a period of time. At this time, the scheduling system will arrange the most urgent conventional material transportation task or non-urgent material transportation task within the first early completion time interval that is greater than the preset interval time based on the early completion time interval and the transportation planning form.
[0063] According to the urgency of the task, the scheduling system arranges the most urgent routine material transportation task or non-urgent material transportation task in the first longer early completion time interval. During the transportation task, the scheduling system continuously updates data to determine whether the subsequent key material transportation task can be completed within the specified period. When the early completion time interval is less than the preset interval, the scheduling system stops the current transportation task and restarts the key material transportation task. Under the premise of ensuring that the key material transportation task is not affected, the idle resources are reasonably used to improve the overall transportation efficiency.
[0064] S105.111. Repeat the step of calculating the early completion time interval.
[0065] Specifically, the scheduling system repeatedly executes the step of calculating the early completion time interval. After each task is scheduled, the early completion time interval of the remaining tasks is recalculated to ensure that the scheduling of subsequent tasks still meets the preset conditions. Through continuous iteration and optimization, the efficiency and flexibility of the entire transportation plan are ensured.
[0066] Among them, as another implementation, S105.16 to S105.111 of the embodiment of the present application are optional.
[0067] Reference Figure 3 After S105.15, the transportation planning form is updated according to the urgency of the material transportation task, which includes the following steps: S105.21. Regenerate and update the transportation planning form, and determine whether the updated transportation planning form can meet the urgency requirements of the key material transportation tasks.
[0068] Specifically, the scheduling system regenerates and updates the transportation planning form, and determines whether the updated transportation planning form can meet the urgency requirements of key material transportation tasks; after allocating idle pipeline resources to high-priority tasks first, the transportation planning form is regenerated and updated according to the new task allocation. Ensure that the transportation planning form reflects the latest task allocation and resource status, and provide a basis for subsequent decision-making.
[0069] S105.22. If the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the ongoing material transportation task shall be retrieved.
[0070] Specifically, the scheduling system analyzes the updated transportation planning form to check whether the transportation time period of the key material transportation task still cannot meet its urgency requirements. Timely identification of problems provides a basis for subsequent adjustment decisions. If the updated transportation planning form still cannot meet the urgency requirements of the key material transportation task, the information of the currently ongoing material transportation task is retrieved to provide necessary data support for the evaluation and adjustment of the ongoing task.
[0071] S105.23. If the urgency of the critical material transportation task is higher than the urgency of the ongoing material transportation task, adjust the transportation plan.
[0072] Specifically, the scheduling system compares the urgency of the key material transportation task with the urgency of the ongoing material transportation task. If the key material transportation task has a higher urgency, the transportation plan will be adjusted to terminate the material transportation task with a lower urgency and give priority to the key material transportation task with a higher urgency. This ensures that the key material transportation task is handled in a timely manner, reduces losses caused by delays, and optimizes resource utilization by adjusting the transportation plan.
[0073] S105.24. Update the transportation planning form.
[0074] Specifically, the scheduling system re-updates the transportation planning form; after adjusting the transportation plan, the transportation planning form is re-updated again according to the new task allocation. This ensures that the transportation planning form is consistent with the actual situation and provides accurate guidance for subsequent operations.
[0075] S105.25. If the transportation time period of the critical material transportation task still cannot meet the urgency requirement of the critical material transportation task, the ongoing material transportation task with a higher level of urgency shall be terminated, and the critical material transportation task with a higher level of urgency shall be executed first.
[0076] Specifically, if after the above adjustments, the transportation time period of the critical material transportation task still cannot meet its urgency requirements, then continue to evaluate and suspend the material transportation tasks with a higher urgency, give priority to the critical material transportation tasks with a higher urgency, and provide execution conditions for the critical material transportation tasks with a higher urgency by suspending some of the ongoing material transportation tasks. At this time, the urgency of the suspended material transportation task is higher than that of the critical material transportation task, but it is the lowest urgency among the ongoing material transportation tasks. Through continuous evaluation and adjustment, ensure that the critical material transportation task is given the greatest priority.
[0077] S105.26. Repeat the step of re-updating the transportation planning form.
[0078] Specifically, the scheduling system repeats the steps of re-updating the transportation planning form. After each adjustment of the transportation plan, the transportation planning form is re-updated to reflect the latest task allocation and resource status. After completing the adjustment work in the previous step, the scheduling system re-judges whether the adjusted transportation planning form can ensure that all key material transportation tasks are completed on time. If they cannot be completed on time, the scheduling system will continue to evaluate and terminate the material transportation tasks with a higher degree of urgency, and give priority to the key material transportation tasks with a higher degree of urgency, so as to ensure the real-time and accuracy of the transportation planning form and provide a basis for subsequent decision-making and operations.
[0079] Reference Figure 4 Further, after S105, as another implementation mode, the embodiment of the present application may further include the following steps: S201. If an urgent material transportation task (very high priority) is received, immediately calculate the transportation resources required to execute the urgent material transportation task.
[0080] Specifically, if an emergency material transportation task (extremely high priority) is received, the scheduling system will immediately calculate the transportation resources required to execute the emergency material transportation task, and prioritize suspending or adjusting the execution of low-priority tasks (non-emergency material transportation tasks) to release pipeline capacity and pump station power resources; when the system receives an emergency material transportation task, it will first immediately trigger the resource calculation module.
[0081] The scheduling system calculates the required transportation resources such as pipeline capacity and pump station power based on the characteristics of the emergency task (such as material type, transportation distance, time requirements, etc.). The system also evaluates the current availability of pipeline and pump station resources to determine whether existing tasks need to be adjusted immediately, quickly respond to emergency tasks, ensure accurate calculation of resource requirements, and provide data support for subsequent resource adjustments.
[0082] The scheduling system prioritizes these tasks and, based on the needs of emergency tasks, suspends or adjusts low-priority tasks that occupy more resources. Suspended or adjusted tasks will be marked as pending recovery so that they can be rescheduled after the emergency tasks are completed, releasing the necessary pipeline capacity and pump station power resources to meet the needs of emergency tasks, ensuring that emergency tasks can be executed as quickly as possible and reducing delays.
[0083] S202. Adjust the execution plan of some medium-priority tasks (routine material transportation tasks).
[0084] Specifically, the scheduling system will adjust the execution plan of some medium-priority tasks (conventional material transportation tasks) to reduce the impact on urgent material transportation tasks; based on the resource calculation results, the system will adjust the execution plan of some medium-priority tasks (conventional material transportation tasks) on the premise of meeting the resource requirements of urgent tasks. The adjustment may include changing the transportation time, adjusting the transportation route, or reallocating transportation resources. The scheduling system will minimize the impact on urgent tasks while maintaining the feasibility of medium-priority tasks. While ensuring that urgent tasks are executed first, the normal operation of medium-priority tasks is maintained as much as possible. Improve the overall flexibility and response speed of the system.
[0085] S203. After the urgent material transportation task is completed, the system immediately resumes the execution plan of other tasks and regenerates the transportation planning form.
[0086] Specifically, after the emergency material transportation task is completed, the scheduling system immediately resumes the execution plan of other tasks and regenerates the transportation planning form. After the emergency material transportation task is completed, the system immediately triggers the recovery module, which checks the previously suspended or adjusted tasks and reschedules the execution plan of these tasks based on the current availability of pipeline and pump station resources. The system also regenerates the transportation planning form to reflect the latest task arrangement and resource allocation, ensuring that the system can quickly return to normal and continue to execute other tasks. Improve the stability and reliability of the system and reduce the overall impact caused by emergency tasks.
[0087] Based on the above method, the embodiment of the present application also discloses an automatic scheduling system based on reinforcement learning. Specifically, an automatic scheduling system based on reinforcement learning includes: A status information acquisition module, which is used to acquire pipeline system status information, wherein the pipeline system status information includes flow distribution data, pipeline pressure data, environmental data, pipeline characteristic data, and transportation demand data; A system model building module, the system model building module is used to build a system model according to the pipeline system state information, and the system model is used to evaluate the specific situation of the pipeline system facing deterministic disturbances and uncertain disturbances; A solution module, the solution module is used to solve the system model to obtain corresponding evaluation results; An optimization module, the optimization module is used to optimize the model according to the evaluation result; A transportation plan generation module, which is used to apply the optimized system model to the automatic scheduling of the pipeline and generate a transportation plan according to the pipeline system status information; Feedback module, the feedback module is used to continuously collect feedback data and continuously adjust and optimize the system model.
[0088] An embodiment of the present application further discloses an intelligent terminal, which includes a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes the above-mentioned automatic scheduling method based on reinforcement learning.
[0089] The embodiment of the present application also discloses a computer-readable storage medium. The computer-readable storage medium stores a computer program that can be loaded by a processor and executes the above-mentioned automatic scheduling method based on reinforcement learning. The computer-readable storage medium includes, for example, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0090] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on these embodiments, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field can still combine, add, delete or make other adjustments to the features in the various embodiments of the present invention according to the circumstances without conflict, without making creative work, so as to obtain different other technical solutions that do not deviate from the concept of the present invention in essence, and these technical solutions also belong to the scope of protection of the present invention.
Claims
1. An automatic scheduling method based on reinforcement learning, characterized in that: The following steps are involved: Acquire pipeline system status information, wherein the pipeline system status information includes flow distribution data, pipeline pressure data, environmental data, pipeline characteristic data, and transportation demand data; According to the pipeline system state information, a system model is constructed, wherein the system model is used to evaluate the specific conditions of the pipeline system facing deterministic disturbances and uncertain disturbances; Solving the system model to obtain corresponding evaluation results; Optimizing the model according to the evaluation results; Applying the optimized system model to automatic pipeline scheduling to generate a transportation plan based on the pipeline system status information; Continuously collect feedback data and continuously adjust and optimize the system model.
2. The automatic scheduling method based on reinforcement learning according to claim 1, characterized in that: The step of applying the optimized system model to the automatic scheduling of the pipeline and generating a transportation plan according to the pipeline system status information specifically includes: Obtaining priority setting factors from a preset database, wherein the priority setting factors include transportation importance and urgency; According to the transportation demand data, the pipeline transportation tasks are prioritized, and the priorities of the pipeline transportation tasks are specifically divided into key material transportation tasks, conventional material transportation tasks and non-emergency material transportation tasks from high to low; By dynamically adjusting resource allocation, transportation planning is performed according to the urgency, and a transportation planning form is generated, wherein the transportation planning form includes all pipeline transportation tasks that are waiting and transportation time periods corresponding to the pipeline transportation tasks; According to the urgency and the transportation planning form, the urgency of the key material transportation task is matched with the transportation time period of the key material transportation task; If the transportation time period of a critical material transportation task cannot meet the urgency requirements of the critical material transportation task, idle pipeline resources will be allocated to high-priority tasks first to ensure that high-priority tasks can be completed on time.
3. The automatic scheduling method based on reinforcement learning according to claim 2, characterized in that: If the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the steps of allocating idle pipeline resources to high-priority tasks first include: Prioritize key material transportation tasks according to the key material transportation tasks and their urgency; Obtain the estimated completion time of the key material transportation task; Get the preset interval time from the preset database; Calculating an early completion time interval, where the early completion time interval is the difference between the estimated completion time of the key material transportation task and the urgency of the key material transportation task; If there is a critical material transportation task whose early completion time interval is greater than the preset interval time, and the early completion time intervals corresponding to all critical material transportation tasks after the critical material transportation task are greater than the preset interval time, then according to the early completion time interval and the transportation planning form, the most urgent conventional material transportation task or non-urgent material transportation task shall be arranged within the first early completion time interval greater than the preset interval time; The step of calculating the early completion time interval is repeatedly executed.
4. The automatic scheduling method based on reinforcement learning according to claim 2 or 3, characterized in that: After the step of allocating idle pipeline resources to high-priority tasks first if the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the method further includes: Regenerate and update the transportation planning form, and determine whether the updated transportation planning form can meet the urgency requirements of key material transportation tasks; If the transportation time period of the key material transportation task cannot meet the urgency requirement of the key material transportation task, the ongoing material transportation task will be called up; If the urgency of the critical material transportation task is higher than that of the ongoing material transportation tasks, the transportation plan will be adjusted to terminate the ongoing material transportation tasks with lower urgency and give priority to the critical material transportation tasks with higher urgency. Re-update the transport planning form; If the transportation time period of the critical material transportation task still cannot meet the urgency requirements of the critical material transportation task, the ongoing material transportation task with a higher urgency level will be terminated, and the critical material transportation task with a higher urgency level will be performed first; Repeat the steps to re-update the transportation planning form.
5. The automatic scheduling method based on reinforcement learning according to claim 2, characterized in that: After the step of applying the optimized system model to the automatic scheduling of the pipeline and generating a transportation plan according to the pipeline system status information, the method further includes: If an emergency material transportation task with a higher priority than the critical material transportation task is received, the transportation resources required to execute the emergency material transportation task are immediately calculated, and the execution of non-emergency material transportation tasks is suspended or adjusted first to release pipeline capacity and pump station power resources; Adjust the execution plan of some routine material transportation tasks to reduce the impact on emergency material transportation tasks; After the urgent material transportation task is completed, the system immediately resumes the execution plan of other tasks and regenerates the transportation planning form.
6. The automatic scheduling method based on reinforcement learning according to claim 1, characterized in that: The step of constructing a system model according to the pipeline system status information specifically includes: Based on the one-dimensional flow in a rigid pipe, the gas dynamics equation in the pipe is used for derivation, and the dynamics equation of the control body is combined with the gas state equation to form an equation group; ; Where m is the mass flow rate of the gas, in kg / s; p is the pressure of the gas, in Pa; A is the cross-sectional area of the pipeline, in m 2 ; ρ is the density of the gas, in kg / m 3 ; D is the inner diameter of the pipe, in m; g is the acceleration due to gravity, in m / s 2 ; Δh is the vertical height of the pipeline and the horizontal plane, in m; L is the length of the pipeline, in m; R is the gas constant, in kJ / (kg·K); λ is the hydraulic friction coefficient of the pipeline; Z is the gas compressibility factor; c is the wave velocity of the gas, in m / s; x is the pipeline position variable, in m; t is the time variable, in s; When the gas in the pipeline runs in the square resistance area, the friction coefficient λ is calculated by the following formula; ; When the gas is determined, the compressibility factor Z is a function of the gas pressure and temperature and is calculated using the following formula: ; Where Re is the Reynolds number of the pipe; r is the absolute equivalent roughness of the pipe wall, in m; p c Critical pressure of gas, in Pa; T c The critical temperature of the gas, in K; The transfer function of the pipe section is derived on the basis of the equations, and the frequency domain expressions of the upstream pressure, flow rate and the downstream pressure, flow rate are obtained; The dynamic equations of the pipe section are linearized and Laplace transformed to obtain a set of ordinary differential equations in the frequency domain; assuming that the inlet pressure and outlet flow rate of the pipeline are specified as boundary conditions, a high-order transfer function expression is obtained.
7. The automatic scheduling method based on reinforcement learning according to claim 1, characterized in that: The step of solving the system model to obtain the corresponding evaluation result specifically includes: Solving the system model using an iterative algorithm; Starting from an initial point, iterate according to the given rules to obtain a sequence. The iteration terminates within a finite number of steps, and the last point is the solution of the evaluation result. The convergence criterion is used to ensure that the iteration stops at the solution of the evaluation result. The convergence criterion is: ; Among them, ε1, ε2 are given small positive numbers; express ; is the minimum value of function f, Known or calculated in some way; Represents the gradient of function f at point x. If function f is a smooth function, the gradient at the local optimal point is a zero vector; The final point satisfies: ; Among them, ε3 and ε4 are small positive numbers used to characterize x k feasibility, ε3 is used for constraints that need to be strictly satisfied, and ε4 is used for inequality constraints, which allow a certain degree of violation but still need to be kept in a small range; c i represents the i-th constraint function; x k represents the kth candidate solution or iteration point in the optimization process; c i (x k ) thus means that at x k The value of the i-th constraint function at position; m represents the number of equality constraints; l represents the number of inequality constraints; m+l is the total number of constraints; Consider x k The distance between the optimal solution and the optimal solution. Since the optimal solution is generally unknown in advance, when the optimal solution is unique, the benchmark algorithm is used to obtain an approximate optimal solution. , and then calculate its difference with x k distance to evaluate the performance of the algorithm.
8. An automatic scheduling system based on reinforcement learning, characterized in that: include: A status information acquisition module, used to acquire pipeline system status information, wherein the pipeline system status information includes flow distribution data, pipeline pressure data, environmental data, pipeline characteristic data, and transportation demand data; A system model building module, used to build a system model according to the pipeline system state information, wherein the system model is used to evaluate the specific conditions of the pipeline system facing deterministic disturbances and uncertain disturbances; A solution module, used for solving the system model to obtain corresponding evaluation results; An optimization module, used for optimizing the model according to the evaluation result; A transportation plan generation module, used to apply the optimized system model to the automatic scheduling of pipelines and generate a transportation plan based on the pipeline system status information; The feedback module is used to continuously collect feedback data and continuously adjust and optimize the system model.
Citation Information
Patent Citations
Permanent magnet motor layered robust optimization design method based on parameter sensitive domain
CN113627000A
APS dynamic production plan scheduling system
CN114298426A
Electric power system inertia continuous evaluation method considering uncertainty system identification
CN115764976A
Intra-day rolling scheduling method and system considering analytic reliability evaluation
CN117239736A
MES-based production workshop intelligent scheduling system
CN117709617A
Cited By
Crude oil storage and transportation safety state dynamic monitoring method for pressure pipeline
CN120256979A
