A Dryer Group Scheduling Method Based on PPO Algorithm

Abstract

The present invention relates to the field of industrial control and optimization technology, and discloses a scheduling method for a dryer group based on a PPO algorithm, including the following steps: S1, modeling the working state and tank state of the dryer group as a Markov decision process; S2, defining the state space, action space and reward function; S3, using the PPO algorithm to train the policy network and the value network to optimize the scheduling of the dryer group, and the S1 step specifically includes the following steps: S1.1, constructing a tank state table to record whether each tank has completed the regeneration operation and whether the drying operation can be performed; S1.2, constructing a working state table for the dryer group. By optimizing the scheduling of the dryer group, the regeneration operation is arranged during the off-peak period when the electricity price is lower, thereby greatly reducing the total electricity cost expenditure, and the PPO algorithm in deep reinforcement learning is used, combined with the Markov decision process model, and through continuous interaction and learning with the environment, the intelligent agent can autonomously optimize the scheduling strategy and improve the scheduling efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial control and optimization, and in particular to a drying machine group scheduling method based on a PPO algorithm. Background Art

[0002] Drying is an essential technology in industrial manufacturing, aimed at reducing product moisture activity to slow deterioration and maintain quality. Drying accounts for a significant portion of global energy consumption in industries such as food production and the chemical industry. According to statistics, in developed countries, 9-25% of total industrial energy consumption is dedicated to dryer operation. Therefore, reducing dryer production costs, particularly electricity costs, has been a hot research topic.

[0003] At present, domestic and foreign scholars have achieved certain research results in saving electricity costs for dryers. For example, in food drying, the use of continuous renewable energy technology is used to reduce the use of heat and electricity generated by fossil energy and its derivatives; spray dryers use exhaust heat recovery technology to recover the low to medium temperature waste heat generated, effectively improving the energy efficiency of the spray dryer. The literature studied the heat recovery system of the spray dryer, directly mixing the exhaust gas with the feed air, which reduced the energy consumption of the dryer by 14%. For the drying of textiles, mathematical modeling is used to achieve process design and minimize energy costs. For example, a literature has constructed a semi-empirical model for clothing drying, covering the final constant speed period and the decreasing speed period.

[0004] However, these studies primarily focus on energy-saving optimization for individual dryers, while research on scheduling optimization for fleets of dryers remains limited. While optimization methods for individual dryers can reduce electricity costs, in actual industrial production, dryers are typically used in clusters, requiring consideration of coordination and scheduling issues across multiple dryers. Furthermore, existing optimization methods are often based on heuristic algorithms such as linear programming and simulated annealing. While these methods can quickly provide scheduling solutions, they struggle to guarantee a globally optimal solution. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the present invention provides a drying machine group scheduling method based on the PPO algorithm, which solves the problem of how to reduce electricity expenditure and improve production efficiency by optimizing the scheduling of drying machine groups.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a drying machine group scheduling method based on the PPO algorithm, comprising the following steps:

[0007] S1. Model the working state of the dryer group and the state of the tank as a Markov decision process;

[0008] S2. Define the state space, action space and reward function;

[0009] S3. Use the PPO algorithm to train the policy network and value network to optimize the scheduling of the dryer fleet.

[0010] Preferably, the step S1 specifically includes the following steps:

[0011] S1.1. Construct a tank status table to record whether each tank has completed the regeneration operation and whether it can be dried;

[0012] S1.2. Construct a working status table for the dryer group to record the current working status of each tank.

[0013] Preferably, in step S1.2, the current working state of each tank includes drying operation, regeneration operation and standby state.

[0014] Preferably, the step S2 specifically includes the following steps:

[0015] S2.1. Define the state space, including the tank state table and the dryer group working state table;

[0016] S2.2. Define the action space, including the actions that each tank can choose at each time step;

[0017] S2.3. Design reward functions, including action reward function, time reward function and cooperation reward function.

[0018] Preferably, the step S3 specifically includes the following steps:

[0019] S3.1. Initialize the policy network, value network, and environment;

[0020] S3.2. Filter the actions that the tank can perform based on the tank status table and the dryer group working status table;

[0021] S3.3, the drying machine group executes the action and updates the environment;

[0022] S3.4. Determine whether the set working time has been reached. If so, calculate the advantage function.

[0023] S3.5. Determine whether the number of iterations has been reached. If not, update the strategy network and value network according to the formula.

[0024] S3.6. Determine whether the total number of training times has been reached. If so, select the optimal strategy; otherwise, return to step S3.1.

[0025] Preferably, the reward function in step S2.3 is designed as follows:

[0026] The action reward function is:

[0027]

[0028] Preferably, the formula of the advantage function in step S3.4 is:

[0029] Where γ represents the discount factor, λ represents the advantage function discount factor, γ and λ are used to balance the importance of current rewards and future rewards, t represents the current time step, T represents the final time step, δ t It is expressed as an estimate of the advantage function at time step t as follows:

[0030] δ t =R t +γ * V(s t+1 )-V(s t ), where R t represents the reward obtained at time t, V(s t+1 ) means s t+1 The state value estimation, V(s t ) means s t The estimated state value.

[0031] Preferably, in step S3.5, an objective function is used to update the policy network parameters, wherein the objective function is:

[0032] where π θ represents the decision of the new policy network, represents the decision of the old policy network, π θ and The ratio of is called the new and old strategy ratio.

[0033] The present invention provides a method for scheduling a drying machine group based on the PPO algorithm. It has the following beneficial effects:

[0034] 1. The present invention optimizes the scheduling of the dryer group and arranges the regeneration operation during the off-peak period when electricity costs are lower, thereby significantly reducing the total electricity expenditure. In addition, the present invention adopts the PPO algorithm in deep reinforcement learning, combined with the Markov decision process model, and enables the intelligent agent to autonomously optimize the scheduling strategy and improve scheduling efficiency through continuous interaction and learning with the environment.

[0035] 2. The present invention flexibly adjusts the time period of the regeneration operation based on fluctuations in electricity prices and production needs. It is independent of the specific structure of the dryer and the drying object, and has a wide range of applicability. While optimizing scheduling, it ensures that a certain number of tanks are dried at each moment, meeting the production needs of the factory without affecting the normal progress of production tasks.

[0036] 3. The present invention reduces the reliance on manual scheduling through an intelligent scheduling optimization method, improves the automation level of the system, and reduces labor costs and error rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the present invention;

[0038] Figure 2 Schematic diagram of the dryer scheduling optimization process of the present invention;

[0039] Figure 3 This is a working state diagram of the drying machine group of the present invention;

[0040] Figure 4 This is a schematic diagram of different numbers of tanks performing drying operations according to the present invention;

[0041] Figure 5 Schematic diagram of different starting moments of the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solution of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] Example:

[0044] Please see the attached Figure 1 -Attached Figure 2 The embodiment of the present invention provides a drying machine group scheduling method based on the PPO algorithm, comprising the following steps:

[0045] S1. Model the working state of the dryer group and the state of the tank as a Markov decision process;

[0046] S2. Define the state space, action space and reward function;

[0047] S3. Use the PPO algorithm to train the policy network and value network to optimize the scheduling of the dryer fleet.

[0048] Specifically, the present invention provides a scheduling method for a drying machine group based on the PPO algorithm, which significantly improves the operating efficiency and economic benefits of the drying machine group.

[0049] 1. Reduce electricity expenses: By optimizing scheduling, regeneration operations are scheduled during off-peak hours when electricity prices are lower, effectively reducing the total electricity expenses of regeneration operations.

[0050] 2. Improve scheduling efficiency: Using the PPO algorithm in deep reinforcement learning, the intelligent agent continuously interacts and learns with the environment, autonomously optimizes the scheduling strategy, and significantly improves scheduling efficiency.

[0051] 3. Strong adaptability: The method can flexibly adjust the regeneration operation time period based on the fluctuation of electricity prices and production demand. It is independent of the specific dryer structure and drying object and has wide applicability.

[0052] 4. Meet production needs: Ensure that while optimizing scheduling, a certain number of tanks are dried at each moment to meet the production needs of the factory without affecting the normal progress of production tasks.

[0053] 5. Reduce manual intervention: Intelligent scheduling optimization methods reduce dependence on manual scheduling, improve the degree of system automation, and reduce labor costs and error rates.

[0054] Step S1 specifically includes the following steps:

[0055] S1.1. Construct a tank status table to record whether each tank has completed the regeneration operation and whether it can be dried;

[0056] S1.2. Construct a working status table for the dryer group to record the current working status of each tank.

[0057] In step S1.2, the current working status of each tank includes drying operation, regeneration operation and standby state.

[0058] Specifically, step S1 provides a systematic framework to formalize the complex dryer fleet scheduling problem so that the problem can be processed by deep reinforcement learning algorithms.

[0059] Specific solutions:

[0060] S1.1. Construct a tank status table: record whether each tank has completed the regeneration operation and whether it can be dried.

[0061] Solve the problem of inaccurate recording of tank status. Through the status table, the status of each tank is accurately recorded to ensure the accuracy and timeliness of the data.

[0062] Specific steps:

[0063] The state space consists of two key components: the tank state table and the dryer cluster operating state table. These two tables describe the states in the system and play a key role in the tank's ability to sense environmental information.

[0064] Create a tank status table to record the regeneration status (completed or incomplete) and drying status (available or not) of each tank. As shown in Table 1, the tank status table:

[0065]

[0066] Use Boolean values ​​(0 or 1) to represent the status of each tank. For example, c,1=0 means that tank No. 1 of the i-th dryer can be dried, and c,1=1 means that it cannot be dried.

[0067] The tank status table indicates whether a tank is suitable for drying. Specifically, after a tank has completed drying, it must undergo regeneration; otherwise, subsequent drying operations cannot proceed. The tank status table records the tank's status. If a tank has completed regeneration, its status is marked as "dryable," allowing subsequent drying operations. If a tank has not completed regeneration, its status is marked as "undryable," preventing subsequent drying operations. The tank status table is crucial, determining which tanks can undergo drying operations and constantly influencing the decision-making process of the dryer fleet.

[0068] S1.2. Construct a working status table for the dryer group: record the current working status of each tank, including drying operation, regeneration operation and standby state.

[0069] Solve the problem of being unable to monitor the working status of the dryer group in real time. Through the working status table, the working status of each tank is monitored and updated in real time.

[0070] Specific steps:

[0071] To meet the production demands and physical limitations of the dryer cluster, a tank needs to obtain information about the operating status of other tanks during its decision-making process. Table 2 records the operating status of the dryer cluster. By viewing the information in Table 2, a tank can obtain information about the operating status of other tanks, enabling better decision-making.

[0072] Create a dryer group working status table to record the current working status of each tank, including drying operation, regeneration operation and standby.

[0073] Table 2 Drying machine group working status table:

[0074]

[0075] Step S2 specifically includes the following steps:

[0076] S2.1. Define the state space, including the tank state table and the dryer group working state table;

[0077] S2.2. Define the action space, including the actions that each tank can choose at each time step;

[0078] S2.3. Design reward functions, including action reward function, time reward function and cooperation reward function.

[0079] Specifically, the present invention systematically optimizes the scheduling of the drying machine fleet by defining the state space, action space, and reward function, effectively improving operational efficiency and reducing electricity costs. The specific beneficial effects and functions are as follows:

[0080] 1. Improve decision-making efficiency: By precisely defining the state space and action space, the PPO algorithm can efficiently optimize decisions based on a comprehensive understanding of the system state and optional actions.

[0081] 2. Reduce operating costs: A reasonable reward function is designed to guide the tank to perform regeneration operations during periods with lower electricity costs, significantly reducing total electricity costs.

[0082] 3. Ensure production needs: Through the design of cooperative reward functions, ensure that the drying machine group meets the factory's production needs while optimizing scheduling, without affecting the normal progress of production tasks.

[0083] Solve the following problems faced by dryer group scheduling in existing technologies:

[0084] 1. Incomplete status records: Traditional methods make it difficult to fully record the working status of the dryer fleet, resulting in a lack of accurate data support for scheduling decisions.

[0085] 2. Inflexible action selection: The action space is not clearly defined, resulting in inflexible scheduling strategies and difficulty in responding to dynamically changing environments and demands.

[0086] 3. Unclear optimization objectives: The reward function is not designed properly and cannot effectively guide the algorithm to optimize towards reducing costs and meeting production needs.

[0087] Specific solutions

[0088] S2.1. Define the state space, including the tank state table and the dryer group working state table:

[0089] Providing comprehensive status information enables the algorithm to accurately grasp the current status of each tank and the entire dryer fleet.

[0090] Specific solutions:

[0091] Tank Status Table: Records whether each tank has completed regeneration and can be dried. This solves the problem of inaccurate tank status recording.

[0092] Ensure the accuracy and timeliness of data, making the status of each tank clearly visible.

[0093] Dryer Cluster Working Status Table: This records the current working status of each tank, including drying operation, regeneration operation, and standby state. This solves the problem of being unable to monitor the working status of the dryer cluster in real time.

[0094] Monitor and update the working status of each tank in real time, providing comprehensive status information.

[0095] S2.2. Define the action space, including the actions that each tank can take at each time step:

[0096] Providing diverse decision-making options enables the algorithm to flexibly respond to different situations and optimize scheduling strategies.

[0097] Specific solutions:

[0098] Action definition: including drying operation, regeneration operation and standby operation. Solve the problem of unclear action selection.

[0099] The action space consists of the actions that the tank can choose in each time step. The action space is shown as follows:

[0100] A={0 i , 1 i , 2 i}

[0101] Where 0 i Indicates that the No. 1 tank of the i-th dryer selects the regeneration operation, 1 i Indicates that the No. 2 tank of the i-th dryer selects the regeneration operation, 2 i Indicates that the i-th tank selects standby operation.

[0102] Provide flexible scheduling options so that the algorithm can make reasonable decisions based on current status and demand.

[0103] S2.3. Design reward functions, including action reward function, time reward function and cooperation reward function:

[0104] Through a reasonable reward mechanism, the algorithm is guided to optimize towards reducing electricity costs and meeting production needs.

[0105] Specific solutions:

[0106] Action reward function: Encourages the tank to perform regeneration operations. This solves the problem of unclear algorithm optimization direction.

[0107] The reward function is designed as follows:

[0108]

[0109] The guidance algorithm selects the action to perform the regeneration operation to improve the utilization rate of the tank.

[0110] The reward function for determining whether the tank performs regeneration:

[0111]

[0112] Time reward function: guides the tank to perform regeneration operations during off-peak hours to reduce electricity costs.

[0113] Based on the electricity price range, it is used to guide the drying fleet to make decisions in the direction of saving electricity costs.

[0114] By using time rewards, we can guide the tanks to regenerate during periods of lower electricity costs, thus reducing electricity costs. The time reward function is designed based on electricity price ranges, aiming to guide the dryer fleet in making decisions that reduce electricity costs. Time rewards result in rewards or penalties depending on the electricity price range.

[0115] The time reward function is as follows:

[0116]

[0117] Cooperation reward function: While ensuring production needs, guide the tanks to cooperate to complete the task.

[0118] Through cooperative incentives, we ensure that the production needs of the factory are met while optimizing scheduling.

[0119]

[0120] The cooperative reward function is a reward function designed for the production needs of the factory.

[0121] The goal is to encourage cooperation among drying tanks to meet factory production needs. If the number of drying tanks at a given moment is less than the factory's set number of drying tanks, the alarm is set to 1 and the cooperation reward is -1. If the number of drying tanks at a given moment is equal to the factory's set number of drying tanks, the alarm is set to 0 and the cooperation reward is 1.

[0122] In order to reduce the difference between the time reward function and the cooperation reward function and optimize the stability of the algorithm, we adopted a method to normalize the rewards. The normalization formula is shown as follows:

[0123]

[0124] Among them, R is the reward calculated by the reward function, R mean Represents the mean operation of the reward, R std Represents the standard deviation operation of the reward.

[0125] Summarize:

[0126] By defining a state space, an action space, and designing a rational reward function, the proposed dryer fleet scheduling method overcomes several challenges in existing technologies. Specifically, comprehensive state information recording enables the algorithm to accurately grasp system status, flexible action selection improves the adaptability of the scheduling strategy, and a rational reward mechanism ensures a clear optimization direction for the algorithm, effectively reducing electricity costs while meeting production needs.

[0127] The S3 step specifically includes the following steps:

[0128] S3.1. Initialize the policy network, value network, and environment;

[0129] S3.2. Filter the actions that the tank can perform based on the tank status table and the dryer group working status table;

[0130] S3.3, the drying machine group executes the action and updates the environment;

[0131] S3.4. Determine whether the set working time has been reached. If so, calculate the advantage function.

[0132] S3.5. Determine whether the number of iterations has been reached. If not, update the strategy network and value network according to the formula.

[0133] S3.6. Determine whether the total number of training times has been reached. If so, select the optimal strategy; otherwise, return to step S3.1.

[0134] Specifically, the present invention uses the PPO algorithm to train the policy network and value network to optimize the scheduling of the dryer group, significantly improving the operating efficiency and economic benefits of the dryer group. The specific beneficial effects and functions are as follows:

[0135] 1. Improve optimization efficiency: Through adaptive learning of intelligent algorithms, the policy network and value network can continuously improve scheduling strategies, improve optimization efficiency, and reduce manual intervention.

[0136] 2. Reduce electricity costs: The intelligent scheduling strategy can flexibly adjust the regeneration operation time according to the electricity rate period, significantly reducing electricity expenses.

[0137] 3. Meet production needs: Ensure that while optimizing scheduling, the working status of the drying machine group can meet the production needs of the factory and does not affect normal production.

[0138] 4. Reduce manual operations: The automated optimization process reduces manual scheduling operations, improves the automation level of the system, and reduces the possibility of human errors.

[0139] Specific solutions:

[0140] S3.1. Initialize the policy network, value network, and environment:

[0141] Provide initial conditions and environment for the algorithm to ensure the effectiveness of the training process.

[0142] Specific solutions:

[0143] Initialize the parameters of the policy network and value network, set the initial environment state, and ensure a stable foundation when training begins.

[0144] The problem of unreasonable initial training conditions is solved, allowing the algorithm to start learning in a stable environment.

[0145] S3.2. Based on the tank status table and the dryer group working status table, select the actions that the tank can perform:

[0146] Based on the current status, screen out executable actions to ensure the rationality of the scheduling strategy.

[0147] Specific solutions:

[0148] According to the tank state table and the dryer group working state table, determine the actions (drying, regeneration or standby) that each tank can perform at the current time step.

[0149] It solves the problem of ambiguous action selection and makes the decision at each time step more accurate and reasonable.

[0150] S3.3. The drying cluster executes actions and updates the environment:

[0151] Execute the selected action and update the state of the environment to provide a basis for the next decision.

[0152] Specific solutions:

[0153] Execute the filtered actions, update the tank status table and the dryer group working status table to reflect the execution results of the current action.

[0154] Solved the problem of untimely environment status updates, ensuring that the environment status is always synchronized with actual operations.

[0155] S3.4. Determine whether the set working time has been reached. If the set working time has been reached, calculate the advantage function:

[0156] By calculating the advantage function, the pros and cons of the current strategy are evaluated, providing a basis for strategy optimization.

[0157] Specific solutions:

[0158] Determine whether the set working time has been reached. If so, calculate the advantage function to evaluate the effect of the current action.

[0159] Advantage function calculation formula:

[0160] Among them, γ represents the discount factor, λ represents the advantage function discount factor, γ and λ are used to balance the importance of current rewards and future rewards, t represents the current time step, T represents the final time step, δ t It is expressed as an estimate of the advantage function at time step t as follows:

[0161] δ t =R t +γ * V(s t+1 )-V(s t ), where R t represents the reward obtained at time t, V(s t+1 ) means s t+1 The state value estimation, V(s t ) means s t The state value estimates are obtained through the value network calculation.

[0162] The problem of inaccurate strategy evaluation is solved, and the effect of the current strategy is accurately evaluated through the calculation of the advantage function.

[0163] S3.5. Determine whether the number of iterations has been reached. If not, update the strategy network and value network according to the formula:

[0164] Through multiple iterations, the strategy network and value network are continuously optimized to improve the effectiveness of the scheduling strategy.

[0165] Specific solutions:

[0166] Determine whether the set number of iterations has been reached. If not, update the policy network and value network according to the following formula:

[0167] Use the objective function for policy network update, the formula is as follows:

[0168] where π θ represents the decision of the new policy network, represents the decision of the old policy network, π θ and The ratio of is called the new and old strategy ratio, and ε is the shear rate of the shear function.

[0169] The shear function is to limit the ratio of the new and old strategies to the range (1-ε,1+ε) neritic, limiting π θ and The gap between them controls the update amplitude of the decision network.

[0170] The value network loss function is:

[0171]

[0172] The value network loss function is used to train and optimize the value network parameters V(s) represents the current parameters The calculated value network's estimated value for state s, V target Represents the target value network's estimate of state s as follows:

[0173]

[0174] Where t represents the current time step, T represents the final time step, and γ represents the discount factor.

[0175] The problem of slow algorithm convergence was solved. Through multiple iterations and continuous optimization of strategies, the convergence speed and effect were improved.

[0176] S3.6. Determine whether the total number of training times has been reached. If so, select the optimal strategy; otherwise, return to step S3.1:

[0177] By setting the total number of training times, we ensure that the algorithm eventually converges to the optimal strategy.

[0178] Specific solutions:

[0179] Determine whether the set total number of training times has been reached. If so, select the optimal strategy; if not, return to step S3.1 and continue training.

[0180] The problem of insufficient algorithm training is solved by setting a reasonable number of training times to ensure that the algorithm can fully learn and optimize.

[0181] Summarize:

[0182] Through the above steps, the dryer fleet scheduling method of the present invention solves several challenges in the existing technology. Specifically, initialization and environment setup ensure the effectiveness of training; state screening and action execution ensure the rationality of scheduling decisions; advantage functions and network updates ensure continuous strategy optimization; and setting the number of training times ensures sufficient training and ultimate convergence of the algorithm. Ultimately, it achieves the effects of improving scheduling efficiency, reducing electricity costs, meeting production needs, and reducing manual operations.

[0183] The reward function in step S2.3 is designed as follows:

[0184] The action reward function is:

[0185] The time reward function is:

[0186] The cooperation reward function is:

[0187]

[0188] The formula of the advantage function in step S3.4 is:

[0189] Where γ represents the discount factor, λ represents the advantage function discount factor, γ and λ are used to balance the importance of current rewards and future rewards, t represents the current time step, T represents the final time step, δ t It is expressed as an estimate of the advantage function at time step t as follows:

[0190] δ t =R t +γ * V(s t+1 )-V(s t ), where R t represents the reward obtained at time t, V(s t+1 ) means s t+1 The state value estimation, V(s t ) means s t The estimated state value.

[0191] In step S3.5, the objective function is used to update the policy network parameters, where the objective function is:

[0192] where π θ represents the decision of the new policy network, represents the decision of the old policy network, π θ and The ratio of is called the new and old strategy ratio.

[0193] Please see the attached Figure 3 -Attached Figure 5To verify the effectiveness of the optimization algorithm presented in this paper, the algorithm was used to reduce the number of drying tanks and shorten the operating time of the drying cluster to more effectively reduce electricity costs. The following experiment used four of the seven drying tanks simultaneously for drying. After 240 hours of operation, it successfully reduced the electricity costs associated with the regeneration operation by approximately 43.2%. Even if only five tanks were used simultaneously for drying, the regeneration electricity costs could still be reduced by approximately 15.8%. The specific experimental content is as follows:

[0194] Purpose of the experiment:

[0195] This experiment aims to verify the effectiveness of the dryer group scheduling method based on the PPO algorithm, mainly evaluating its performance in reducing electricity expenditure and meeting production needs.

[0196] Experimental steps:

[0197] Experimental preparation:

[0198] Initialize the policy network, value network, and environment.

[0199] Set up a dryer cluster consisting of seven dryers, each with two tanks, and record their initial states.

[0200] Determine the experimental parameters and electricity price range (peak period, flat period, off-peak period), as shown in the following table.

[0201]

[0202] Data collection

[0203] Create a tank status table and a dryer group working status table to record the current working status of each tank (drying operation, regeneration operation, standby).

[0204] According to the experimental time period, the state changes of the tank and the electricity consumption at each time step are recorded.

[0205] Experimental process

[0206] Follow these steps to conduct the experiment:

[0207] S1: Model the working status of the dryer group and the tank status as a Markov decision process

[0208] S1.1: Construct a tank status table to record whether each tank has completed the regeneration operation and whether it can be dried.

[0209] S1.2: Construct a working status table for the dryer group to record the current working status of each tank (drying operation, regeneration operation and standby).

[0210] S2: Define state space, action space, and reward function

[0211] S2.1: Define the state space, including the tank state table and the dryer group working state table.

[0212] S2.2: Define the action space, including the actions that each tank can choose at each time step (drying operation, regeneration operation, standby operation).

[0213] S2.3: Design reward functions, including action reward function, time reward function and cooperation reward function.

[0214] S3: Use the PPO algorithm to train the policy network and value network to optimize the scheduling of the dryer fleet

[0215] S3.1: Initialize the policy network, value network, and environment.

[0216] S3.2: Based on the tank status table and the dryer group working status table, filter the actions that the tank can perform.

[0217] S3.3: The dryer cluster executes the action and updates the environment.

[0218] S3.4: Determine whether the set working time has been reached. If the set working time has been reached, calculate the advantage function.

[0219] S3.5: Determine whether the number of iterations has been reached. If not, update the strategy network and value network according to the formula.

[0220] S3.6: Determine whether the total number of training times has been reached. If so, select the optimal strategy; otherwise, return to step S3.1.

[0221] Data Analysis

[0222] According to the experimental data, the working status and electricity expenditure of the drying machine group in different time periods are analyzed. The working status of the drying machine group is shown in the attached figure. Figure 3 As shown in the figure, the electricity cost comparison when different numbers of tanks are used for drying operation is as follows: Figure 4 As shown, the starting time is set to the 0th time step and the ending time is the 240th time step. In Figure (a), two tanks are set for drying operations, and the results show that the reduction in electricity costs ranges from 70.4% to 84.8%. For Figure (b), this experiment set up three tanks for drying operations, which can reduce electricity costs to between 73.4% and 84.8%. In Figure (c), four tanks are set for drying operations, and the reduction in electricity costs is observed to be between 39.0% and 43.2%. Finally, in Figure (d), five tanks are set for drying operations, and it is found that the reduction in electricity costs ranges from 15.8% to 19.9%.

[0223] To verify the effectiveness of the optimization algorithm at different start times, we selected four different start times for our experiment. The horizontal axis represents the group, and the vertical axis represents the total electricity cost incurred by the dryer fleet during regeneration. We selected one control group and seven experimental groups. The control group, labeled CG, employed conventional factory methods, meaning regeneration was performed immediately after the tanks were dried. The experimental group, labeled EG, employed the optimization algorithm to schedule the dryer fleet. The formula used is as follows:

[0224] ROR=(CG-EG)÷CG×100%

[0225] As an indicator of the degree of electricity cost reduction, the larger the ROR value is, the greater the reduction in electricity cost is, and the more significant the effect of the optimization algorithm is. Set 4 cans for drying operation and take the 240th time step as the end time. Figure 4 shown.

[0226] Figure (a) starts at the 1st time step and reduces the electricity bill by 34.5% to 44.2%. Figure (b) starts at the 11th time step. This experiment reduces the electricity bill by 43.8% to 46.5%. Figure (c) starts at the 14th time step and reduces the electricity bill by 48.2% to 59.2%. Figure (d) starts at the 21st time step and reduces the electricity bill by 52.0% to 60.4%.

[0227] Comparing the electricity expenditure and the completion of drying operations before and after the PPO algorithm optimization,

[0228] Summary of results:

[0229] Organize the experimental data into a table to show the changes in electricity expenditure.

[0230] The experimental results are summarized to evaluate the performance of the PPO algorithm in reducing electricity costs and meeting production needs.

[0231] Experimental summary:

[0232] Through the dryer group scheduling method based on the PPO algorithm, the experimental results show that:

[0233] Significant reduction in electricity costs: Within 240 time steps, the electricity costs of the regeneration operation of the dryer cluster were reduced by approximately 43.2%, verifying the effectiveness of this method in reducing electricity costs.

[0234] Improved scheduling efficiency: Intelligent scheduling strategies ensure efficient operation of the drying fleet in different time periods to meet production needs.

[0235] Reduce manual operations: Through automated scheduling, the reliance on manual intervention is reduced and the degree of automation of the system is improved.

[0236] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A drying machine group scheduling method based on PPO algorithm, characterized in that: The following steps are involved: S1. Model the working state of the dryer group and the tank state as a Markov decision process; S2. Define state space, action space and reward function; The S2 step specifically includes the following steps: S2.

1. Define the state space, including the tank state table and the dryer group working state table; S2.2, define the action space, including the actions that each tank can choose at each time step; S2.

3. Design reward functions, including action reward function, time reward function and cooperation reward function; The reward function in step S2.3 is designed as follows: The action reward function is: The action reward function is used to determine whether the tank performs the regeneration operation; The time reward function is divided according to the electricity price range, which is used to guide the dryer fleet to make decisions in the direction of saving electricity costs; The cooperative reward function is used to ensure production demand while guiding the tanks to cooperate in completing the task; S3, use the PPO algorithm to train the strategy network and value network to optimize the scheduling of the dryer group; The S3 step specifically includes the following steps: S3.

1. Initialize the policy network, value network and environment; S3.

2. Filter the actions that the tank can perform according to the tank status table and the dryer group working status table; S3.3, the dryer group executes the action and updates the environment; S3.4, determine whether the set working time has been reached, and if so, calculate the advantage function; The formula of the advantage function in step S3.4 is: Where γ represents the discount factor, λ represents the advantage function discount factor, γ and λ are used to balance the importance of current rewards and future rewards, t represents the current time step, T represents the final time step, δ t It is expressed as an estimate of the advantage function at time step t as follows: δ t =R t +γ * V(s t+1 )-V(s t ), where R t represents the reward obtained at time t, V(s t+1 ) means s t+1 The state value estimation, V(s t ) means s t Estimation of the state value of ; S3.

5. Determine whether the number of iterations has been reached. If not, update the strategy network and value network according to the formula; S3.

6. Determine whether the total number of training times has been reached. If so, select the optimal strategy; otherwise, return to step S3.

1.

2. A method for scheduling a drying machine group based on a PPO algorithm according to claim 1, characterized in that: The S1 step specifically includes the following steps: S1.

1. Construct a tank status table to record whether each tank has completed the regeneration operation and whether it can be dried; S1.

2. Construct a working status table for the dryer group to record the current working status of each tank.

3. A method for scheduling a drying machine group based on a PPO algorithm according to claim 2, characterized in that: The step S1.2 checks that the current working state of each tank includes drying operation, regeneration operation and standby state.

4. The method for scheduling a drying machine group based on the PPO algorithm according to claim 1, characterized in that: In step S3.5, the objective function is used to update the policy network parameters, where the objective function is: where π θ represents the decision of the new policy network, represents the decision of the old policy network, π θ and The ratio of is called the new and old strategy ratio.

Images