Multi-energy supply and demand balance intelligent scheduling method for carrying equipment hot processing workshop

Abstract

The invention provides a multi-energy supply and demand balance intelligent scheduling method for a carrying equipment hot working workshop, and belongs to the technical field of hot working process management and control. Comprising the following steps: setting a scheduling period and a time step, and carrying out multi-energy mechanism energy consumption model modeling on energy consumption equipment; establishing a supply and demand balance equation of electric energy, gas energy and water quantity at each time step according to the time scale of the scheduling cycle and the multi-energy mechanism energy consumption model; based on a multi-energy mechanism energy consumption model, associating batches of heat treatment procedures of a heat processing workshop to form a schedulable task set; modeling a multi-energy supply and demand balance scheduling problem into a Markov decision process, and defining a state vector, an action space and a reward function; constructing a strategy network and a value network, and performing joint training on the strategy network and the value network by adopting a deep reinforcement learning algorithm to obtain a trained water-electricity-gas multi-energy supply and demand balance scheduling strategy of the carrying equipment hot processing workshop; and applying the balance scheduling strategy.

Description

Technical Field

[0001] This invention relates to the field of thermal processing control technology, and in particular to an intelligent scheduling method for balancing the supply and demand of multiple energy sources in a thermal processing workshop for transport equipment. Background Technology

[0002] Key components of transportation equipment in aviation, aerospace, and automotive industries commonly employ hot forging and heat treatment processes. Typical production lines include: gas-fired or electric heating furnaces, free forging or die forging, and heat treatment. These processes consume natural gas, coal gas, electricity, and cooling water to varying degrees, forming a complex, coupled water-electricity-gas multi-energy system. The multi-energy supply method in forging-heat treatment workshops has the following characteristics: electricity is typically supplied by the plant's substation at time-of-use pricing, resulting in demand-based electricity charges and peak-valley price differences; natural gas and coal gas are mostly supplied using contracted capacity or daily quotas; the cooling water system consists of cooling towers, circulating pumps, spray systems, or quenching tanks, consuming both water and the electricity required for circulating pumps. Simultaneously, the hot processing technology is highly sensitive to temperature and time, and production plans are subject to uncertainties such as order insertions, model changes, and equipment maintenance, causing significant fluctuations and coupling in the timing distribution of multi-energy supply and demand.

[0003] Traditional workshop energy management often employs rule-based or experience-based start-up and shutdown strategies, or optimization scheduling methods based on linear programming, mixed-integer programming, or model predictive control. These methods typically require forecasting future electricity prices, loads, and production cycles, or assume that disturbances have a known statistical distribution; therefore, scheduling effectiveness heavily depends on forecast accuracy and model precision. Existing research on multi-energy system scheduling largely focuses on single-energy scenarios or simple situations, with relatively few studies on the coordinated scheduling of high-power furnaces, mechanical presses / hydraulic presses, and order-driven production constraints in thermal processing workshops.

[0004] Therefore, it is essential to provide an intelligent scheduling method for multi-energy supply and demand balance in the thermal processing workshop of transportation equipment. Under the premise of repeatedly considering production process constraints, equipment operating characteristics, and multi-energy supply constraints of water, electricity, and gas, this method can adaptively learn the multi-energy supply and demand balance of the workshop in an uncertain environment without the need for precise prediction of future disturbances, thus meeting the energy efficiency optimization and operational safety constraints of the thermal processing workshop of transportation equipment. Summary of the Invention

[0005] In view of this, the present invention proposes a multi-energy supply and demand balance intelligent scheduling method for a heat treatment workshop of transport equipment, which meets the constraints of forging and heat treatment processes and equipment safety constraints, and achieves the goal of multi-energy balance scheduling while coordinating energy consumption.

[0006] This invention provides a method for intelligent scheduling of multi-energy supply and demand balance in a thermal processing workshop for transportation equipment, comprising the following steps: S1. Set the scheduling cycle, model the energy consumption of energy-consuming equipment using a multi-energy mechanism, and divide the scheduling cycle into several equal time steps; S2. Based on the time scale of the scheduling cycle and the energy consumption model of the multi-energy mechanism, establish the supply and demand balance equations of electricity, gas and water at each time step, as well as the multi-energy operating cost model with constraints, to obtain the multi-energy operating cost and the penalty for constraint violation. S3. Based on the multi-energy mechanism energy consumption model, the batch-level tasks of the heat treatment process in the heat processing workshop are balanced and associated with the electricity, gas and water consumption to form a set of schedulable tasks. S4. Model the multi-energy supply and demand balance scheduling problem as a Markov decision process. Based on the set of schedulable tasks, define the state vector and action space. The action space includes continuous actions and discrete actions. Combine the multi-energy operating cost and constraint violation penalty to construct the reward function. S5. Construct a policy network with state vector as input and continuous action-discrete action scheduling as output, and a value network with state vector as input and expected reward under given state vector as output. Use deep reinforcement learning algorithm to jointly train the policy network and value network to obtain the trained water-electricity-gas multi-energy supply and demand balance scheduling strategy for the hot processing workshop of the transportation equipment. S6. Input the current workshop status into the trained multi-energy supply and demand balance scheduling strategy for water, electricity and gas in the hot processing workshop of the transport equipment, obtain scheduling actions, and send them to the energy management system and production execution system to realize real-time supply and demand balance scheduling of water, electricity and gas.

[0007] Based on the above technical solutions, preferably, step S1 involves: setting the scheduling period. T The time step is 24 hours, and the length of each time step Δt is 15 minutes. The batch of the energy-consuming equipment corresponding to the process is determined. A multi-energy mechanism energy consumption model is established based on the energy consumption balance relationship of each batch of energy-consuming equipment. The theoretical values ​​of electricity consumption, gas consumption and water consumption of each energy-consuming equipment are calculated through the multi-energy mechanism energy consumption model. Combined with the number of time steps required for the process of the energy-consuming equipment, the theoretical values ​​of electricity consumption, gas consumption and water consumption of each time step are obtained.

[0008] Preferably, the processes of the energy-consuming equipment include heating, forging, and heat treatment processes.

[0009] More preferably, the step of establishing a multi-energy mechanism energy consumption model based on the energy consumption balance relationship of each batch of energy-consuming equipment, and then calculating the theoretical values ​​of electricity consumption, gas consumption, and water consumption for each energy-consuming equipment through the multi-energy mechanism energy consumption model, specifically includes: For heating and heat treatment processes, the quality of workpieces loaded into the furnace via energy-consuming equipment... m Initial temperatureT in Target temperature T set Specific heat capacity c Calculate theoretical heat demand Q th , Q th = m × c ×( T set - T in And consider the necessary energy losses of the energy-consuming equipment. Q loss Overall efficiency of energy-consuming equipment or f and the equivalent heat of energy Q fuel / e To obtain the actual heat requirement: Q total = Q th + Q loss = or f × Q fuel / e ,in Q total To account for the actual heat demand after considering necessary energy losses, the heating process and the temperature rise heat treatment process use natural gas or electric heating; if the energy-consuming equipment uses natural gas, the theoretical gas consumption is [value missing]. Q total / ( or f × H u ), H u This is the lower heating value of natural gas; if the energy-consuming equipment uses electrical energy, the theoretical value of the electricity consumption is... Q total / or f ; For the quenching and cooling heat treatment process, let the quenching quality be... m w1 Quenching start temperature T q Average temperature of the workpiece after water discharge T out specific heat capacity of the workpiece c w Specific heat of cooling water c water Cooling water temperature rise △ T w The heat released by the workpiece is Qcool = m w1 × c w ×( T q - T out The theoretical value of cooling water consumption is... Q cool / ( r w × c water ×△ T w ), r w This is the density of the cooling water.

[0010] Further preferred, step S2 is: let the time step... k Grid power P grid ( k This is the sum of the electrical power of all energy-consuming devices plus line losses; based on the time step... k The gas or water consumption of each energy-consuming device is summed up, and necessary losses are taken into account, to obtain... k Gas consumption of all energy-consuming devices at all times V gas,tot ( k Or water consumption V w,tot ( k Define time step k Multi-energy operating costs for: , c e ( k ), c g ( k ), c w ( k () are time steps k The unit prices of electricity, natural gas, and water are calculated; the operating costs at each time step of the scheduling cycle are accumulated, and penalty terms for constraint violations are considered to obtain the multi-energy operating cost model within the scheduling cycle. , C pen To constrain violations and penalties; the constraints include a peak power constraint on the power grid, meaning the purchased power must not exceed the maximum value stipulated in the contract. , The maximum peak power of the grid under contractual constraints, and the gas contractual constraints. , The planned natural gas consumption for each scheduling cycle This is a penalty for exceeding the planned natural gas consumption within the dispatch cycle.

[0011] More preferably, step S3 includes: for batch-level tasks, providing the earliest start time and the latest finish time; setting continuity constraints for processes, ensuring the sequence of heating, forging, and heat treatment processes is continuous; setting the longest waiting time constraint to prevent the workpiece from becoming too cold or too aged; and binding the energy demand of the energy-consuming equipment with the batch and process through the above constraints to form a set of schedulable tasks.

[0012] Furthermore, in a more preferred embodiment, the state vector defined in step S4 is: [the state vector is defined by the time step]. k state vector s k Including electricity unit price c e ( k ), natural gas unit price c g ( k ), grid power P grid ( k ) and the historical maximum power of the power grid State subvectors of furnace-type equipment Forging equipment state sub-vector Batch production schedule sub-vector Constraint margins for daily cumulative electricity, gas, and water consumption and current time step k The sequence number is used to rewrite the elements in the schedulable task set into state vector form.

[0013] Furthermore, the motion space described in step S4 is preferred. a k This includes the installation of energy-consuming equipment for heating processes and heat treatment processes. u furn ( k ), Setting of energy-consuming equipment in the forging process u press ( k ), Setting up energy-consuming equipment for quenching and cooling heat treatment processes u water ( k ), Batch task start and delay decisions u schedule ( k Action space a k The expression is Each element in the action space includes continuous actions and / or discrete actions.

[0014] Furthermore, in step S4, the construction of the reward function aims to minimize both cost and penalty for breach of contract, defining a single time step. k Rewards ,in For time step k Multi-energy operating costs, For penalties exceeding the contractual obligations, Penalties for delayed processes or excessive waiting time. The penalty is applied for temperature deviations from the process window or overloading of energy-consuming equipment; the scheduling problem is defined as maximizing the cumulative discounted reward. , c It is the discount rate, with a value range of [0,1]. π It is a mapping from state to action. E π Indicating in strategy π The theoretical expectation is such that the reward function with a negative sign is maximized, ensuring that the total cost is minimized.

[0015] More preferably, step S5, which involves jointly training the policy network and value network using a deep reinforcement learning algorithm to obtain the trained multi-energy supply and demand balance scheduling strategy for the hot processing workshop of the transportation equipment, applies a deep reinforcement learning framework based on policy gradients to the structure of the policy network and value network; and constructs a measurement network. Used for time steps k state vector s k As input, output the mean and variance of each continuous action, and the probability of each discrete action; for the parameters of the policy network... i The objective function of the multi-energy supply and demand balance scheduling strategy is to perform iterative updates. ,in Represents time step k Expectation on, policy ratio function , This represents the policy network before the parameter update under the old policy, in the state vector. s k Take action when inputting a k probability, For the estimation of the advantage function, The shear coefficient; clip Functions are used to restrict the values ​​in an array to a specified range; Value Network Used to estimate a given state vector s kThe expected return is designed for the policy distribution. For continuous actions, a multidimensional Gaussian distribution is used for modeling, and for discrete actions, a Bernoulli / classification distribution is used for modeling. The value network uses mean squared error loss to learn and update the parameters φ of the value network, so as to achieve unified decision-making in the action space.

[0016] The present invention provides an intelligent scheduling method for balancing the supply and demand of multiple energy sources in a thermal processing workshop for transport equipment, which has the following advantages compared to the prior art: 1. This application establishes a multi-energy mechanism energy consumption model of water, electricity and gas, which optimizes the consumption and cost of energy from different sources within the same framework. This facilitates a global perspective, comprehensively considers the unit price of energy at different times, and organically combines it with production planning to carry out cross-energy coordinated scheduling.

[0017] 2. Directly link batch-level tasks in the heat treatment process with energy consumption, set process constraints, and ensure that the processes in the heat treatment workshop can be completed sequentially while meeting the process constraints, in accordance with the actual process flow and quality requirements.

[0018] 3. Incorporate grid peak power constraints, gas contract constraints, and process parameters into the model. When designing the reward function, introduce a penalty term for constraint violation to guide the model to actively avoid default behavior during the learning process. Unify the handling of continuous actions such as power setting of energy-consuming equipment, as well as discrete actions such as equipment start-up and shutdown and process sequencing, to ensure that energy-consuming equipment operates under reasonable load, effectively reduce comprehensive energy costs and peak load, and minimize the amount of penalty.

[0019] 4. The multi-energy supply and demand balance scheduling problem is modeled as a Markov decision process and solved using deep reinforcement learning. The resulting scheduling strategy is an intelligent agent that can respond quickly to the current state variables. It can respond to dynamic disturbances such as grid price fluctuations and temporary changes in production plans in real time, and achieve rapid rescheduling and optimization based on time steps, thereby improving the robustness of the method in uncertain scenarios such as order insertion and workpiece change. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a multi-energy supply and demand balance intelligent scheduling method for a thermal processing workshop of transport equipment according to the present invention; Figure 2This is a schematic diagram of the multi-energy system structure of the intelligent scheduling method for balancing the supply and demand of multiple energy sources in a thermal processing workshop of a transport equipment according to the present invention. Figure 3 This is a schematic diagram of the deep reinforcement learning strategy network and value network structure of a multi-energy supply and demand balance intelligent scheduling method for a thermal processing workshop of a transport equipment according to the present invention. Figure 4 This is a comparison diagram of the peak power reduction effect of the intelligent scheduling method for balancing the supply and demand of multiple energy sources in the thermal processing workshop of a transportation equipment according to the present invention. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0023] Traditional workshop energy management often employs rule-based or experience-based start-stop strategies, or optimization scheduling methods based on linear programming, mixed-integer programming, or model predictive control. These methods typically require forecasting future electricity prices, loads, and production cycles, or assume that disturbances have a known statistical distribution; therefore, scheduling effectiveness heavily depends on forecast accuracy and model precision. Existing research on multi-energy system scheduling largely focuses on single-energy scenarios or simple situations, with relatively few studies addressing multi-energy mixed scenarios in thermal processing workshops, or on coordinated scheduling under order-driven production constraints. In view of this, such as Figure 1 and Figure 2 As shown, this invention provides a multi-energy supply and demand balance intelligent scheduling method for a thermal processing workshop of transportation equipment, comprising the following steps:

[0024] S1. Set the scheduling cycle, model the energy consumption of energy-consuming equipment using a multi-energy mechanism, and divide the scheduling cycle into several equal time steps.

[0025] Step S1 is: Set the scheduling period T If the time period is 24 hours and the length of each time step Δt is 15 minutes, then each scheduling cycle has 96 time steps. (Time step index...) k =1,2,…,96. Determine the batch of the process corresponding to the energy-consuming equipment, establish a multi-energy mechanism energy consumption model based on the energy consumption balance relationship of each batch of energy-consuming equipment, and then calculate the theoretical values ​​of electricity consumption, gas consumption and water consumption of each energy-consuming equipment through the multi-energy mechanism energy consumption model. Combined with the number of time steps required for the process of the energy-consuming equipment, obtain the theoretical values ​​of electricity consumption, gas consumption and water consumption for each time step.

[0026] Specifically, a multi-energy mechanism energy consumption model is established based on the energy consumption balance of each batch of energy-consuming equipment. Then, the theoretical values ​​of electricity, gas, and water consumption for each piece of energy-consuming equipment are calculated using this model. This includes: For heating and heat treatment processes, the quality of workpieces loaded into the furnace via energy-consuming equipment... m Initial temperature T in Target temperature T set Specific heat capacity c Calculate theoretical heat demand Q th , Q th = m × c ×( T set - T in And consider the necessary energy losses of the energy-consuming equipment. Q loss Overall efficiency of energy-consuming equipment or f and the equivalent heat of energy Q fuel / e To obtain the actual heat requirement: Q total = Q th + Q loss = or f × Q fuel / e ,in Q total To account for the actual heat demand after considering necessary energy losses, the heating process and the temperature rise heat treatment process use natural gas or electric heating; if the energy-consuming equipment uses natural gas, the theoretical gas consumption is [value missing]. Q total / ( or f × H u ), H u This is the lower heating value of natural gas; if the energy-consuming equipment uses electrical energy, the theoretical value of the electricity consumption is... Q total / or f ; For the quenching and cooling heat treatment process, let the quenching quality be... m w1 Quenching start temperature T q Average temperature of the workpiece after water dischargeT out specific heat capacity of the workpiece c w Specific heat of cooling water c water Cooling water temperature rise △ T w The heat released by the workpiece is Q cool = m w1 × c w ×( T q - T out The theoretical value of cooling water consumption is... Q cool / ( r w × c water ×△ T w ), r w This is the density of the cooling water.

[0027] Taking the landing gear steel hot processing workshop of an aerospace company as an example, this workshop mainly processes large steel forgings. The multi-energy supply side mainly consists of electricity, natural gas, and hydropower, supplied by the factory's 10kV substation. The workshop is equipped with a 4MVA transformer. The time-of-use electricity price for the factory includes off-peak hours (0:00AM-7:00AM): 0.45 yuan / kWh; flat hours (7:00AM-6:00PM, 10:00PM-0:00AM): 0.75 yuan / kWh; peak hours (6:00PM-10:00PM): 1.20 yuan / kWh. The maximum contracted demand is 3.5MW; electricity exceeding the maximum contracted demand is subject to a penalty of 1.5 times the electricity price. When using natural gas supply, the quota for each dispatch cycle is 6000 Nm³. 3 The excess portion per Nm 3 An additional fine of 0.4 yuan will be imposed, and the price of natural gas will be 2.4 yuan / Nm³. 3 Hydropower is charged based on actual usage.

[0028] The main energy-consuming equipment includes one gas-fired heating furnace with a rated combustion power of 3MW, one electric heating tempering furnace with a rated electric power of 1.5MW, one free forging hydraulic press with a rated electric power of 800kW, one die forging hydraulic press with a rated electric power of 600kW, and two sets of quenching water tanks and circulating water pumps, with a single circulating pump power of 55kW.

[0029] The production processes for energy-consuming equipment typically include heating, forging, and heat treatment. The heat treatment process further includes heating, quenching, cooling, and aging processes. This embodiment simplifies the production task, ensuring that the production of four batches of landing gear forgings all include heating-free forging-die forging and tempering stages, as detailed in the table below.

[0030]

[0031] Taking the B1 heating process as an example, the calculation mechanism of the gas energy consumption of the heating furnace is as follows: Batch B1 uses a gas-fired heating furnace for pre-forging heating, and the furnace loading quality... m =1800kg, initial temperature T in The target temperature is 20℃. T set Specific heat capacity of steel at 1200℃ c =0.6kJ / (kg·K), necessary energy loss of energy-consuming equipment Q loss Equivalent to 30% of theoretical heat demand Q th Furnace efficiency or f =0.7, lower heating value of natural gas H u =34MJ / Nm 3 Theoretical heat increase Q th =1800×0.6×(1200-20)=1274400KJ, the actual heat demand after considering the furnace body's heat storage and heat dissipation losses is Q total =1.3× Q th =1656720KJ≈1656.72MJ, theoretical natural gas consumption is Q total / ( or f × H u ≈69.6 Nm 3 If the heating time for this furnace is 2 hours, then the average equivalent gas consumption per hour is 34.8 Nm³. 3 / h.

[0032] Taking the quenching and tempering furnace heating of batch B1 as an example, electric heating is used, and the quality of the quenched and tempered workpieces... m =1800kg, initial temperature T in The target temperature for tempering and heating is 20℃. T setThe specific heat capacity of steel at 650℃ c =0.6kJ / (kg·K), necessary energy loss of energy-consuming equipment Q loss Equivalent to 20% of theoretical heat demand Q th quenching and tempering furnace efficiency or f =0.6, then the theoretical heat rise of the quenching and tempering furnace is... Q th =1800×0.6×(650-20)=453600KJ, the actual heat demand after considering energy loss is Q total =1.2× Q th =544320KJ=544.32MJ, the corresponding equivalent electricity consumption is Q total / or f =907.2MJ, which, converted to 1kWh=3.6MJ, corresponds to 252kWh of electricity. If the conditioning process lasts for 3 hours, the average hourly electricity consumption is 84kWh. By reasonably selecting the start time of the process, without violating process constraints, and in conjunction with the peak and off-peak electricity price distribution periods, it is possible to smooth out electricity price fluctuations and reduce electricity costs.

[0033] In addition, regarding the quenching heat treatment process, taking the water quenching of batch B1 as an example, the quality of the quenched workpiece... m =1800kg, quenching start temperature T q =850℃, average temperature of the water-exit workpiece T out Specific heat capacity of steel at 80℃ c w =0.6kJ / (kg·K), specific heat of cooling water c water =4.2kJ / (kg·K), cooling water temperature rise Δ T w =15℃, the workpiece releases heat of Q cool =1800×0.6×(850-80)=831600KJ, cooling water consumption is Q cool / ( r w × c water ×△ T w )=13.21m 3 The theoretical value of the equivalent water consumption per hour can be obtained by averaging it over the total time of the quenching heat treatment process.

[0034] Divide the theoretical values ​​of gas consumption, electricity consumption, and water consumption mentioned above by the number of corresponding time steps to obtain the theoretical values ​​of electricity consumption, gas consumption, and water consumption for each time step.

[0035] S2. Based on the time scale of the scheduling cycle and the energy consumption model of the multi-energy mechanism, establish the supply and demand balance equations of electricity, gas and water at each time step, as well as the multi-energy operating cost model with constraints, to obtain the multi-energy operating cost and the penalty for constraint violation.

[0036] Step S2 is as follows: Set the time step... k Grid power P grid ( k This is the sum of the electrical power of all energy-consuming devices and line losses. P loss ( k (Accumulated) , P i ( k () represents the electrical power of different energy-consuming devices. i For different energy-consuming devices, serial numbers; based on time steps k The gas or water consumption of each energy-consuming device is summed up, and necessary losses are taken into account, to obtain... k Gas consumption of all energy-consuming devices at all times V gas,tot ( k Or water consumption V w,tot ( k ): , V j ( k () indicates the gas consumption of different energy-consuming devices. , V l ( k () indicates the water consumption of different energy-consuming devices. V loss,w ( k ( ) represents water consumption loss. Considering the consumption of the above three types of energy, the time step is defined. k Multi-energy operating costs for: , c e ( k ), c g ( k ), c w ( k () are time steps kThe unit prices of electricity, natural gas, and water are calculated; the operating costs at each time step of the scheduling cycle are accumulated, and penalty terms for constraint violations are considered to obtain the multi-energy operating cost model within the scheduling cycle. , C pen To constrain violations and penalties; the constraints include a peak power constraint on the power grid, meaning the purchased power must not exceed the maximum value stipulated in the contract. , The maximum peak power of the grid under contractual constraints, and the gas contractual constraints. , The planned natural gas consumption for each scheduling cycle This is a penalty for exceeding the planned natural gas consumption within the dispatch cycle.

[0037] S3. Based on the multi-energy mechanism energy consumption model, the batch-level tasks of the heat treatment process in the heat processing workshop are balanced and associated with the electricity, gas and water consumption to form a set of schedulable tasks.

[0038] For batch-level tasks, the earliest start time and latest finish time are given; the continuity constraints of the process are set, and the order of heating, forging, and heat treatment processes is continuous; the longest waiting time constraint is set to avoid the workpiece being too cold or too aged; through the above constraints, the energy demand of the energy-consuming equipment is bound to the batch and process to form a set of schedulable tasks.

[0039] S4. Model the multi-energy supply and demand balance scheduling problem as a Markov decision process. Based on the set of schedulable tasks, define the state vector and action space. The action space includes continuous actions and discrete actions. Combine the multi-energy operating cost and constraint violation penalty to construct the reward function.

[0040] Markov Decision Process (MDP) is a mathematical framework used to model and solve sequential decision problems. Its core concepts include state vector, action space, transition probability, and reward function. The state vector represents the state of the energy-consuming device at a certain moment; the action space represents the operations that the energy-consuming device can choose; the transition probability represents the probability of transitioning from one state vector to another after performing an action; and the reward function is the immediate reward obtained by performing an action under a specific state vector.

[0041] The state vector defined here is: let the time step k state vector s k Including electricity unit price c e ( k ), natural gas unit price c g ( k ), grid power Pgrid ( k ) and the historical maximum power of the power grid State subvectors of furnace-type equipment Such as furnace number, remaining heating / holding time, and current temperature parameters for gas-fired furnaces and tempering furnaces; forging equipment state sub-vectors. Such as the start / stop status of free forging hydraulic presses and die forging hydraulic presses, the current processing batch and remaining processing time; batch production progress sub-vectors. This includes the completion markers for each batch (B1-B4) and the remaining time until the delivery deadline; and the constraint margins for the cumulative daily electricity, gas, and water consumption. and current time step k The sequence number is used to rewrite the elements in the schedulable task set into a state vector form. Time step k corresponding state vector s k for: .

[0042] The action space here a k This corresponds to the time step. k The decision variables include the setting of energy-consuming equipment in the heating process and the temperature rise process. u furn ( k Such as the operating modes and power settings of each controllable heating furnace / heat treatment furnace; the setting of energy-consuming equipment in the forging process. u press ( k Such as the start-up, shutdown, and cycle adjustment of various hydraulic presses and mechanical presses; the setting of energy-consuming equipment corresponding to quenching, cooling, and heat treatment processes. u water ( k Examples include the start / stop of cooling water system circulating pumps and the combination of process water valves; and decisions regarding the start and delay of batch tasks. u schedule ( k For example, rearranging the sequence within the process operation window; motion space. a k The expression is The elements of the action space include continuous actions and / or discrete actions. Continuous actions include power settings and flow settings, while discrete actions include equipment start / stop and process window reordering.

[0043] The construction of the reward function described in step S4 aims to minimize both cost and penalty for default, defining a single time step. k Rewards ,in For time step k Multi-energy operating costs, For penalties exceeding the contractual obligations, Penalties for delayed processes or excessive waiting time. This is a penalty for temperature deviations from the process window or for overloading of energy-consuming equipment.

[0044] Taking time step series 40 as an example, if the electricity price is at the flat rate at this time... c e (40) = 0.75 yuan / kWh, grid power at time step 40 P grid (40) = 1209kW, gas consumption V gas,tot (40) is 20 Nm 3 Water consumption V w,tot (40) = 0.8m 3 Then the operating cost of multiple energy sources in time step 40 The reward for this step is 277.09 yuan. If the grid power does not exceed the contract demand, the gas volume does not exceed the daily quota, and there are no delays or process violation penalties, the reward for this step is -277.09 yuan. If the grid power exceeds the contract demand, the gas volume exceeds the daily quota, or there are delays or process violation penalties, the penalty for this step is 277.09 yuan.

[0045] The scheduling problem is defined as maximizing the cumulative reward of discounts: , c It is the discount rate, with a value range of [0,1]. π It is a mapping from state to action. E π Indicating in strategy π The theoretical expectation is such that the reward function with a negative sign is maximized, ensuring that the total cost is minimized.

[0046] S5. Construct a policy network with state vector as input and continuous action-discrete action scheduling as output, and a value network with state vector as input and expected reward under a given state vector as output. Use deep reinforcement learning algorithm to jointly train the policy network and the value network to obtain the trained water-electricity-gas multi-energy supply and demand balance scheduling strategy for the hot processing workshop of the transportation equipment.

[0047] like Figure 3 As shown, step S5, which involves using a deep reinforcement learning algorithm to jointly train the policy network and the value network to obtain the trained multi-energy supply and demand balance scheduling strategy for the hot processing workshop of the transportation equipment, employs a deep reinforcement learning framework based on policy gradients for the structure of the policy network and the value network; and constructs a measurement network. Used for time steps k state vector sk As input, output the mean and variance of each continuous action, and the probability of each discrete action; for the parameters of the policy network... i The objective function of the multi-energy supply and demand balance scheduling strategy is to perform iterative updates. ,in Represents time step k Expectation on, policy ratio function , This represents the policy network before the parameter update under the old policy, in the state vector. s k Take action when inputting a k probability; For the estimation of the advantage function, The shear coefficient is... clip Functions are used to restrict the values ​​in an array to a specified range.

[0048] Value Network Used to estimate a given state vector s k The expected return is designed for the policy distribution. For continuous actions, a multidimensional Gaussian distribution is used for modeling, and for discrete actions, a Bernoulli / classification distribution is used for modeling. The value network uses mean squared error loss to learn and update the parameters φ of the value network, so as to achieve unified decision-making in the action space.

[0049] E π Indicating in strategy π The theoretical expectation is a theoretical value, which is actually obtained by acquiring time steps from limited historical data. k Expectations As E π This is used to solve the scheduling problem.

[0050] Value networks, through their structure of pruning, policy gradients, and value baselines, are beneficial for achieving stable training in scenarios where continuous and discrete actions coexist.

[0051] S6. Input the current workshop status into the trained multi-energy supply and demand balance scheduling strategy for water, electricity and gas in the hot processing workshop of the transport equipment, obtain scheduling actions, and send them to the energy management system and production execution system to realize real-time supply and demand balance scheduling of water, electricity and gas.

[0052] During the training phase, parameters are repeatedly sampled and updated in the simulation environment based on historical production plans, actual execution records, and energy metering data. i After convergence with φ, the policy network is solidified and deployed in the workshop energy management system to receive the current state vector in real time. sk Output scheduling action a k The data is then sent to the equipment layer or the MES / PCS system for execution, enabling real-time supply and demand balance scheduling of multiple energy sources, including water, electricity, and gas.

[0053] It should be noted that, in addition to the forged workpieces mentioned above, this method is also applicable to automotive aluminum and steel forging-heat treatment workshops; the production process includes at least heating in an electric or gas-fired furnace, billet preparation, pre-forging and final forging, edge trimming, and tempering of steel or solution treatment and aging of aluminum alloys, and the data processing is basically similar.

[0054] The following detailed explanation uses an example. Based on the definitions of the state space, action vector, and reward function, a simulation environment is constructed using production plans and energy consumption data from seven consecutive days. Each training round corresponds to one day, or 96 time steps, and random simulations are performed for disturbances such as batch arrival time, order insertion, and short-term equipment downtime. The policy network and value network adopt a three-layer fully connected neural network structure, with the state vector as the input. s k The output is a hybrid continuous-discrete action parameter. During training, a policy gradient algorithm with a policy ratio cutoff of 0.98 is used. The network parameters are updated every 10 training epochs. After approximately 1000 training epochs, the cumulative reward essentially converges, resulting in a stable multi-energy scheduling strategy. On a representative production day, the method of this invention is compared with the workshop's existing experience-based scheduling strategy, and the statistical results are as follows.

[0055]

[0056] As can be seen from the table, under the premise of ensuring that all batches are delivered on time and without violating the process window, the method of the present invention achieves an overall energy cost reduction of about 10% and reduces the maximum grid demand by about 14%, effectively smoothing the load curve.

[0057] As attached Figure 4As shown, on this representative production day, under the experienced dispatch scheme, the grid power showed significant peaks between 10:00 AM and 11:00 AM and between 7:00 PM and 8:00 PM, with a maximum power of approximately 3.82 MW and a large peak-to-valley load difference. The method of this invention reduces the peak power to approximately 3.29 MW by shifting some of the quenching and tempering furnace heating and tempering stages to the flat and valley electricity price periods, and by staggering some free forging and die forging processes within the limits of the process, resulting in a smoother curve. It is evident that the method of this invention utilizes energy consumption mechanism models of processes such as heating, tempering, and quenching to finely couple furnace secondary production tasks with the balance of multiple energy sources including water, electricity, and gas. This enables scheduling decisions to be optimized while ensuring process constraints such as temperature and time. The multi-energy scheduling problem is modeled as a Markov decision process, and deep reinforcement learning is used to directly learn scheduling strategies in the state-action-reward interaction. This allows for adaptive optimization in environments with order insertion and disturbances without the need for precise prediction of future loads and electricity prices. Comparative results on representative production days demonstrate that the method provided by this invention achieves significant reductions in overall energy costs and peak power while satisfying process and delivery constraints, verifying the effectiveness of the method in the multi-energy supply and demand balance scheduling of hot processing workshops for transportation equipment.

[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for intelligent scheduling of multi-energy supply and demand balance in a thermal processing workshop for transport equipment, characterized in that, Includes the following steps: S1. Set the scheduling cycle, model the energy consumption of energy-consuming equipment using a multi-energy mechanism model, and divide the scheduling cycle into several equal time steps; S2. Based on the time scale of the scheduling cycle and the energy consumption model of the multi-energy mechanism, establish the supply and demand balance equations of electricity, gas and water at each time step, as well as the multi-energy operating cost model with constraints, and obtain the multi-energy operating cost and constraint violation penalty. S3. Based on the multi-energy mechanism energy consumption model, the batch-level tasks of the heat treatment process in the heat processing workshop are balanced and associated with the electricity, gas and water consumption to form a set of schedulable tasks. S4. Model the multi-energy supply and demand balance scheduling problem as a Markov decision process. Based on the set of schedulable tasks, define the state vector and action space. The action space includes continuous actions and discrete actions. Combine the multi-energy operating cost and constraint violation penalty to construct the reward function. S5. Construct a policy network with state vector as input and continuous action-discrete action scheduling as output, and a value network with state vector as input and expected reward under given state vector as output. Use deep reinforcement learning algorithm to jointly train the policy network and value network to obtain the trained water-electricity-gas multi-energy supply and demand balance scheduling strategy for the hot processing workshop of the transportation equipment. S6. Input the current workshop status into the trained multi-energy supply and demand balance scheduling strategy for water, electricity and gas in the hot processing workshop of the transport equipment, obtain scheduling actions, and send them to the energy management system and production execution system to realize real-time supply and demand balance scheduling of water, electricity and gas.

2. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 1, characterized in that, Step S1 is: Set the scheduling period T The time step is 24 hours, and the length of each time step Δt is 15 minutes. The batch of the energy-consuming equipment corresponding to the process is determined. A multi-energy mechanism energy consumption model is established based on the energy consumption balance relationship of each batch of energy-consuming equipment. The theoretical values ​​of electricity consumption, gas consumption and water consumption of each energy-consuming equipment are calculated through the multi-energy mechanism energy consumption model. Combined with the number of time steps required for the process of the energy-consuming equipment, the theoretical values ​​of electricity consumption, gas consumption and water consumption of each time step are obtained.

3. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 2, characterized in that, The processes involved in energy-consuming equipment include heating, forging, and heat treatment.

4. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 3, characterized in that, The process involves establishing a multi-energy mechanism energy consumption model based on the energy consumption balance of each batch of energy-consuming equipment, and then calculating the theoretical values ​​of electricity, gas, and water consumption for each piece of energy-consuming equipment using the multi-energy mechanism energy consumption model. Specifically, this includes: For heating and heat treatment processes, the quality of workpieces loaded into the furnace via energy-consuming equipment... m Initial temperature T in Target temperature T set Specific heat capacity c Calculate theoretical heat demand Q th And consider the necessary energy losses of the energy-consuming equipment. Q loss Overall efficiency of energy-consuming equipment η f and the equivalent heat of energy Q fuel / e Obtain the actual heat demand Q total The heating process and heat treatment process use natural gas or electric heating; if the energy-consuming equipment uses natural gas, the theoretical gas consumption is [value missing]. Q total / ( η f × H u ), H u This is the lower heating value of natural gas; if the energy-consuming equipment uses electrical energy, the theoretical value of the electricity consumption is... Q total / η f ; For the quenching and cooling heat treatment process, the specific heat of the cooling water is... c water Cooling water temperature rise △ T w The heat released by the workpiece is Q cool The theoretical value of cooling water consumption is Q cool / ( ρ w × c water ×△ T w ), ρ w This is the density of the cooling water.

5. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 3, characterized in that, Step S2 is as follows: Set the time step... k Grid power P grid ( k This is the sum of the electrical power of all energy-consuming devices plus line losses; based on the time step... k The gas or water consumption of each energy-consuming device is summed up, and necessary losses are taken into account, to obtain... k Gas consumption of all energy-consuming devices at all times V gas,tot ( k Or water consumption V w,tot ( k ); k The grid power, gas consumption, and water consumption of all energy-consuming devices at each time step are multiplied by the unit price of that time step and then summed to obtain the time step. k Multi-energy operating costs ; By summing the operating costs at each time step of the scheduling cycle and considering the constraint violation penalty, the multi-energy operating cost model within the scheduling cycle is obtained as follows: , C pen To constrain violations and penalties; the constraints include a peak power constraint on the power grid, meaning the purchased power must not exceed the maximum value stipulated in the contract. , The maximum peak power of the grid under contractual constraints, and the gas contractual constraints. , The planned natural gas consumption for each scheduling cycle This is a penalty for exceeding the planned natural gas consumption within the dispatch cycle.

6. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 5, characterized in that, Step S3 involves providing the earliest start time and latest finish time for batch-level tasks; setting continuity constraints for processes, ensuring the sequential order of heating, forging, and heat treatment processes; setting the longest waiting time constraint to prevent workpieces from becoming too cold or too aged; and binding the energy demand of energy-consuming equipment to batches and processes through the above constraints to form a set of schedulable tasks.

7. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 6, characterized in that, The definition of the state vector mentioned in step S4 is: to let the time step k state vector s k Including electricity unit price c e ( k ), natural gas unit price c g ( k ), grid power P grid ( k ) and the historical maximum power of the power grid State subvectors of furnace-type equipment Forging equipment state sub-vector Batch production schedule sub-vector Constraint margins for daily cumulative electricity, gas, and water consumption and current time step k The sequence number is used to rewrite the elements in the schedulable task set into state vector form.

8. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 7, characterized in that, The action space described in step S4 a k This includes the installation of energy-consuming equipment for heating processes and heat treatment processes. u furn ( k ), Setting of energy-consuming equipment in the forging process u press ( k ), Setting up energy-consuming equipment for quenching and cooling heat treatment processes u water ( k ), Batch task start and delay decisions u schedule ( k Action space a k The expression is Each element in the action space includes continuous actions and / or discrete actions.

9. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 7, characterized in that, The construction of the reward function described in step S4 aims to minimize both cost and penalty for default, defining a single time step. k Rewards ,in For time steps k Multi-energy operating costs, For penalties exceeding the contractual obligations, Penalties for delayed processes or excessive waiting time. The penalty is applied for temperature deviations from the process window or overloading of energy-consuming equipment; the scheduling problem is defined as maximizing the cumulative discounted reward. , γ It is the discount rate, with a value range of [0,1]. π It is a mapping from state to action. E π Indicating in strategy π The theoretical expectation is such that the reward function with a negative sign is maximized, ensuring that the total cost is minimized.

10. The intelligent scheduling method for balancing the supply and demand of multiple energy sources in a heat treatment workshop for transport equipment according to claim 9, characterized in that, Step S5 describes using a deep reinforcement learning algorithm to jointly train the policy network and value network to obtain the trained multi-energy supply and demand balance scheduling strategy for the hot processing workshop of the transportation equipment. This involves applying a deep reinforcement learning framework based on policy gradients to the structure of the policy network and value network; and constructing a measurement network. Used for time steps k state vector s k As input, output the mean and variance of each continuous action, and the probability of each discrete action; for the parameters of the policy network... θ The objective function of the multi-energy supply and demand balance scheduling strategy is to perform iterative updates. ,in Represents time step k Expectation on, policy ratio function , This represents the policy network before the parameter update under the old policy, in the state vector. s k Take action when inputting a k probability, For the estimation of the advantage function, The shear coefficient; clip Functions are used to restrict the values ​​in an array to a specified range; Value Network Used to estimate a given state vector s k The expected return is designed for the policy distribution. For continuous actions, a multidimensional Gaussian distribution is used for modeling, and for discrete actions, a Bernoulli / classification distribution is used for modeling. The value network uses mean squared error loss to learn and update the parameters φ of the value network, so as to achieve unified decision-making in the action space.

Images