Air compressor cluster optimization control method and device based on reinforcement learning

Abstract

The invention discloses an air compressor cluster optimization control method and device based on reinforcement learning, and the method comprises the steps: receiving target loads in different time periods in a control period, and selecting a plurality of working state sets corresponding to the target loads; wherein the multiple working state sets are working state sets under multiple different working conditions, and each working state set comprises the operation state of each air compressor in the air compressor cluster system; selecting a working state set of each time period in the control period based on a reinforcement learning method; operating the air compressor cluster system based on the working state set; according to the method, the working state sets under different working conditions are constructed, and the working state sets of all the time periods are selected through the reinforcement learning method, so that the control method with the lowest operation energy consumption of the air compressor cluster system can be obtained, and the operation energy consumption of the air compressor cluster system is reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of air compressor cluster control, and in particular relates to an air compressor cluster optimization control method and device based on reinforcement learning. Background Art

[0002] With the development of industrial automation and intelligent manufacturing, air compressors play an increasingly important role in industrial production. Air compressors are widely used in many fields such as industrial production. Cluster systems composed of multiple air compressors often require efficient control strategies to meet the gas demand in production, while reducing energy consumption, reducing equipment wear, improving air compressor energy efficiency, and reducing frequent start and stop of air compressors.

[0003] However, traditional air compressor control systems have problems such as high energy consumption and low operating efficiency. Control methods are often difficult to adapt to complex and changeable gas conditions, and there are problems such as energy waste and low control accuracy. Therefore, how to improve the energy efficiency and operating efficiency of air compressor cluster systems by optimizing control strategies has become a hot topic in current research. Summary of the invention

[0004] The purpose of the present invention is to provide an air compressor cluster optimization control method and device based on reinforcement learning to reduce the operating energy consumption of the air compressor cluster system.

[0005] The present invention adopts the following technical solution: an air compressor cluster optimization control method based on reinforcement learning, comprising the following steps:

[0006] Receive target loads at different time periods in a control cycle, and select multiple working state sets corresponding to the target loads; wherein the multiple working state sets are working state sets under multiple different working conditions, and the working state sets include the operating state of each air compressor in the air compressor cluster system;

[0007] Select the working state set for each period in the control cycle based on the reinforcement learning method;

[0008] Run the air compressor cluster system based on the working status set.

[0009] Furthermore, before selecting a plurality of working state sets corresponding to the target load, the method further includes:

[0010] Divide the load interval of the air compressor cluster system into multiple sub-intervals according to a preset step length;

[0011] The working state set of the air compressor cluster system corresponding to each sub-interval under different working conditions is calculated; wherein the different working conditions are that a predetermined air compressor in the air compressor cluster system is in an on state or an off state.

[0012] Furthermore, selecting a plurality of working state sets corresponding to the target load includes:

[0013] The subinterval to which the target load belongs is determined, and multiple working state sets corresponding to the subinterval are selected.

[0014] Furthermore, the working state set for each period in the control cycle is selected based on the reinforcement learning method, including:

[0015] The working state sets under multiple different working conditions corresponding to each time period are taken as the action space, the working state sets of the air compressor cluster system in different time periods are taken as the state space, and the negative number of the sum of the total electricity cost and the start-stop cost of the air compressor cluster system in the control period is taken as the utility function value to select the working state sets of each time period in the control period.

[0016] Furthermore, the working state set for each period in the control cycle is selected based on the reinforcement learning method and further includes:

[0017] Generate a random number in the range [0,1) each time you select an action;

[0018] When the random number is less than the threshold, an action is randomly selected in the action space.

[0019] Furthermore, when the random number is greater than or equal to a threshold, an action corresponding to the maximum value of the utility function is selected.

[0020] Furthermore, the threshold calculation method is:

[0021]

[0022] Among them, β is the threshold, k is the maximum number of iterations, and epoch is the current number of iterations.

[0023] Furthermore, the working state set of the air compressor cluster system corresponding to each sub-interval under different working conditions is calculated, including:

[0024] The reinforcement learning method is used for calculation; the total energy consumption and penalty term of the air compressor cluster system are used as the efficiency function, and the penalty term is calculated based on the difference between the gas output and the load of the air compressor cluster system.

[0025] Furthermore, the penalty term is calculated as:

[0026]

[0027] Among them, punish represents the penalty term, targe_q represents the load, and q represents the gas production.

[0028] Another technical solution of the present invention: an air compressor cluster optimization control device based on reinforcement learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the above method is implemented when the processor executes the computer program.

[0029] The beneficial effect of the present invention is that the present invention constructs a set of working states under different working conditions and selects the working state sets for each time period through a reinforcement learning method, thereby obtaining a control method with the lowest operating energy consumption of the air compressor cluster system, thereby reducing the operating energy consumption of the air compressor cluster system. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic flow chart of an optimization control method for an air compressor cluster system based on reinforcement learning in an embodiment of the present invention;

[0031] Figure 2 is a schematic diagram of a reinforcement learning method in an embodiment of the present invention;

[0032] Figure 3 This is a framework diagram of the solution process of the reinforcement learning method in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0034] In recent years, reinforcement learning has achieved remarkable results in the field of control, especially in dealing with nonlinear and dynamic systems. The air compressor cluster optimization control method based on reinforcement learning can automatically learn the best control strategy without manually defining rules. Through interaction with the environment, the control strategy can be gradually adjusted to maximize the optimization target. It is more flexible, adaptable and real-time than traditional methods, and provides a more energy-efficient control strategy. Therefore, applying reinforcement learning to the optimization control of air compressor cluster systems is expected to achieve a more efficient and intelligent control strategy, thereby reducing energy consumption and improving system performance, and has great application potential.

[0035] The present invention discloses an air compressor cluster optimization control method based on reinforcement learning, comprising the following steps: receiving target loads in different time periods in a control cycle, and selecting multiple working state sets corresponding to the target loads; wherein the multiple working state sets are working state sets under multiple different working conditions, and the working state sets include the operating state of each air compressor in the air compressor cluster system; selecting the working state sets for each time period in the control cycle based on a reinforcement learning method; and operating the air compressor cluster system based on the working state sets.

[0036] The present invention constructs a set of working states under different working conditions and selects the working state sets for each time period through a reinforcement learning method, thereby obtaining a control method with the lowest operating energy consumption of an air compressor cluster system, thereby reducing the operating energy consumption of the air compressor cluster system.

[0037] like Figure 1The figure shows the process of the optimization control method of the air compressor cluster system based on reinforcement learning. Specifically, before selecting multiple working state sets corresponding to the target load, it also includes: dividing the load interval of the air compressor cluster system into multiple sub-intervals according to a preset step size; calculating the working state set of the air compressor cluster system corresponding to each sub-interval under different working conditions; wherein the different working conditions are that the predetermined air compressor in the air compressor cluster system is in the on state or not in the off state.

[0038] In other words, the cluster system of air compressor units is modeled, and the control variables and parameters of each type of air compressor are determined according to the type of air compressor used, and the relationship between the energy consumption, gas output and control variables of each air compressor is fitted. Among them, the control variable of the fixed-frequency screw air compressor is the working state (shutdown, loading, unloading), the control variable of the variable-frequency screw air compressor is the frequency, and the control variable of the centrifugal air compressor is the IGV (inlet regulating valve) opening and BOV (i.e., vent regulating valve) opening.

[0039] Then, according to the load range of the air compressor cluster system, the reinforcement learning algorithm is used to calculate the optimal solution when the equipment is set to be turned on / off within the load range. During the calculation process, the full load range can be divided into multiple different load point values. In other words, the optimal solution corresponding to the load of the air compressor cluster system is calculated according to a certain step size.

[0040] In addition, since the start-stop process of the air compressor will also generate start-stop costs, the optimal solution of the air compressor cluster system is calculated under different working conditions of each air compressor. For example, if the air compressor cluster system has two fixed-frequency screw air compressors, the calculated optimal solutions include 11, 12, 22, 02, and 00, where 11 represents the working condition of both fixed-frequency screw air compressors being unloaded, 12 represents 1# (i.e., the first fixed-frequency screw air compressor) being unloaded and 2# (i.e., the second fixed-frequency screw air compressor) being loaded, 22 represents the working condition of both fixed-frequency screw air compressors being loaded, 02 represents 1# being shut down and 2# being loaded, and 00 represents the working condition of both fixed-frequency screw air compressors being shut down.

[0041] Finally, based on the solution set and gas demand obtained in the previous step, reinforcement learning is used to optimize the optimal operating parameters of each air compressor at different times within the control cycle.

[0042] The present invention uses a reinforcement learning algorithm to solve the optimal solution of the operating parameters of each air compressor per hour under the gas load within the control cycle. Reinforcement learning is a field in machine learning that emphasizes how to act based on the environment to maximize expected benefits. The environment is usually abstracted as a Markov decision process (MDP). MDP is a four-tuple, including a state set S, an action set A, and a reward function R. The agent selects an action based on its current state, transfers to the next state after executing the action, and obtains the current state and the reward related to the selection of the action. The goal is to learn a strategy by interacting with the environment to maximize the cumulative reward.

[0043] When using reinforcement learning to solve the problem of air compressor cluster system control, the environment refers to the time-coupled, dynamically changing gas load, and the optimization goal refers to the system operating cost. The solution and objective function obtained in each round of exploration are quantitatively evaluated through the utility function, and the results of the quantitative evaluation are reflected in each decision step in generating this set of solutions. The decision function can give a decision choice that is more conducive to the final objective function given the state of the solution and the optional decision variables. The complete solution obtained at the end of this round of exploration is evaluated by the utility function, and the decision function is updated. At this point, a round of solution exploration and optimization is completed. With multiple rounds of solution iterations, the exploration solution given by the algorithm will converge to the possible optimal solution with a high probability. Figure 2 Shown is a diagram of the reinforcement learning principle.

[0044] like Figure 3 The figure shows the framework diagram of the reinforcement learning method solution process. Among them, the state value table is functionally equivalent to the decision function, and the update of the state value table is completed using the above-mentioned Q-Learning algorithm and Bellman equation. Monte Carlo simulation includes multi-step action selection decisions. One round of Monte Carlo simulation gives a complete exploration solution, and the calculation of utility value is realized through the utility function we designed. After multiple rounds of Monte Carlo simulation, the exploration solution of this algorithm is likely to converge to the optimal solution. The optimal solution result in the exploration process is the optimal solution given by the algorithm.

[0045] In one embodiment, the air compressor cluster system has a total of 7 air compressors, 1# screw air compressor, 2# screw air compressor, 3# screw air compressor, 4# screw air compressor, 5# screw air compressor, 6# variable frequency screw air compressor, and 7# centrifugal air compressor.

[0046] Among them, the gas output of 1#, 2#, 3#, 4# and 5# screw air compressors when loaded is 23.5m 3 / min、69.2m 3 / min、69.2m 3 / min、70.6m 3 / min、70.6m 3 / min, the energy consumption during loading is 132kW, 350kW, 350kW, 400kW, 400kW, the gas production during unloading is 0, and the energy consumption during unloading is 58.8kW, 140kW, 140kW, 160kW, 160kW. The minimum and maximum frequencies of the 6# variable frequency screw air compressor are 20HZ and 50HZ, respectively, and the corresponding gas production is 12m 3 / min、40m 3 / min, the corresponding energy consumption is 57kW and 180kW respectively. The minimum and maximum opening of IGV of 7# centrifugal air compressor are 25% and 100% respectively, and the corresponding gas production is 80m 3 / min、100m 3 / min, the corresponding energy consumption is 363kW and 435kW respectively, the minimum and maximum opening of BOV are 0% and 100% respectively, and the corresponding air discharge volume is 0 and 50m 3 / min, when BOV is opened, it is actually the deflation process which does not generate energy consumption.

[0047] A model was established based on the data of the above air compressors. The polynomial fitting method was used to fit the relationship between the frequency and gas production of the 6# variable frequency screw air compressor, the frequency and energy consumption of the 6# variable frequency screw air compressor, the IGV opening and gas production of the 7# centrifugal air compressor, the IGV opening and energy consumption of the 7# centrifugal air compressor, and the BOV opening and air discharge volume of the 7# centrifugal air compressor. The relationship was used to calculate the gas production and energy consumption corresponding to different solution sets.

[0048] 1) When setting the optimal solution for when the device is turned on / off within the load range:

[0049] Action and action space definition:

[0050] For a fixed-frequency screw air compressor, the action space of its operating state is [0,1,2], where 0 represents shutdown, 1 represents unloading, and 2 represents loading.

[0051] For variable frequency screw air compressors, the frequency action space is [0,20,21,...,49,50], where 0 represents shutdown.

[0052] For a centrifugal air compressor, the action space of its IGV opening is [0, 0.25, 0.26, ..., 0.99, 1.00], and the action space of BOV opening is [0, 0.01, 0.02, ..., 0.99, 1.00], where 0 represents shutdown.

[0053] In the present invention, calculating the working state set of the air compressor cluster system corresponding to each sub-interval under different working conditions includes: calculating using a reinforcement learning method; wherein the total energy consumption and penalty term of the air compressor cluster system are used as the efficiency function, and the penalty term is calculated based on the difference between the gas output and the load of the air compressor cluster system.

[0054] Action a={a1,a2,...,a8}, where a1,a2,...,a8 respectively represent the operating status of 1# screw air compressor, the operating status of 2# screw air compressor, the operating status of 3# screw air compressor, the operating status of 4# screw air compressor, the operating status of 5# screw air compressor, the frequency of 6# variable frequency screw air compressor, the IGV opening of 7# centrifugal air compressor, and the BOV opening of 7# centrifugal air compressor.

[0055] The utility function is defined as: (total energy consumption of 7 air compressors + penalty term)*(-1). The penalty term is proportional to the difference between gas production and load, and is calculated as: Among them, targe_q is the load, and q is the gas production calculated after the actual selected action.

[0056] In this air compressor cluster system, the maximum load is the sum of the maximum gas output of the seven air compressors. If the step length is set to 10, the load traversal range is 0 to 440 m 3 / min, and four situations where 6# variable frequency screw air compressor and 7# centrifugal air compressor are turned on and off are selected as different working conditions.

[0057] Working condition 1 is that the 6# variable frequency screw air compressor is not turned on. At this time, the action space of the 6# variable frequency screw air compressor becomes [0], and the action space of other air compressors remains unchanged.

[0058] Working condition 2 is when the 6# variable frequency screw air compressor is turned on. At this time, the action space of the 6# variable frequency screw air compressor becomes [20,21,...,49,50], and the action space of other air compressors remains unchanged.

[0059] Working condition 3 is that the 7# centrifugal air compressor is not turned on. At this time, the action space of the IGV and BOV of the 7# centrifugal air compressor becomes [0], and the action space of other air compressors remains unchanged.

[0060] Working condition 4 is that the 7# centrifugal air compressor is turned on. At this time, the action spaces of the IGV and BOV of the 7# centrifugal air compressor are changed to [0.25, 0.26, ..., 0.99, 1.00] and [0.01, 0.02, ..., 0.99, 1.00] respectively, and the action spaces of other air compressors remain unchanged.

[0061] Reinforcement learning is used to calculate the optimal solution under each load. The optimal solution includes the optimal solutions under different working conditions. When there are four working conditions, four optimal solutions are obtained.

[0062] Among them, the maximum number of iterations k = 90000, epoch is the current number of iterations, Each time an action is selected, a random number in the range of [0,1) is generated. When the random number is less than β, an action is randomly generated in the action space. When the random number is greater than or equal to β, the action corresponding to the maximum value of the utility function in all current Qtable tables is selected. The optimal solutions and corresponding energy consumption values ​​of the four working conditions under each load are calculated in sequence. For example, when the load is 100, the optimal solutions and energy consumption of the four working conditions are:

[0063] [0,0,0,0,0,0,100,0], energy consumption = 435; [0,0,0,0,0,29,32,2], energy consumption = 456.9648;

[0064] [0,0,2,0,0,42,0,0], energy consumption = 490.3351; [0,0,0,0,0,0,100,0], energy consumption = 435.

[0065] 2) Obtain the optimal solution for the gas load in 24 time periods throughout the day.

[0066] The working state set of each time period in the control cycle is selected based on the reinforcement learning method, including: taking the working state set under multiple different working conditions corresponding to each time period as the action space, taking the working state set of the air compressor cluster system in different time periods as the state space, and taking the inverse of the sum of the total electricity cost and the start-stop cost of the air compressor cluster system in the control cycle as the utility function value to select the working state set of each time period in the control cycle.

[0067] Action space definition: The optimal solutions of the four working conditions can be found under the corresponding load at each moment, so the action space at each time is [1, 2, 3, 4], where [1, 2, 3, 4] represents the solutions of working conditions 1, 2, 3, and 4 respectively.

[0068] State space definition: (time, solution), where time is the 24 time periods in a day, 0, 1, 2, ..., 23, and the solution is the solution of the working condition corresponding to the load selection.

[0069] The utility function is defined as: (total electricity cost for 24 hours in a day + start-stop cost)*(-1), where the sum of electricity costs is the sum of electricity costs at each hour, and the electricity cost at each hour is equal to energy consumption multiplied by the electricity price. The start-stop cost is the total start-stop cost of 7 air compressors in a day. Among them, the cost of starting and stopping a fixed-frequency air compressor once is 129.5, the cost of starting and stopping a variable-frequency air compressor once is 200.2, and the cost of starting and stopping a centrifugal air compressor once is 300. The start-stop cost is a preset parameter for each air compressor, which can be obtained based on the purchase cost and service life of the air compressor.

[0070] Assume that the load in a day is [56, 63, 72, 61, 59, 48, 46, 42, 120, 117, 126, 115, 112, 107, 110, 109, 105, 102, 82, 87, 85, 77, 72, 56], and the electricity price at different times is [0.23656, 0.23656, 0.23656, 0.23656, 0.23656, 0.236 ,0.23656,0.23656,0.75127,0.75127,0.75127,1.15997,1.15997,1.15997,0.75127,0.75127,0.75127,1.15997,1.15997,1.15997,1.15997,1.15997,0.75127,0.75127].

[0071] When obtaining the above load, since the sub-interval of this embodiment is established with a step size of 10, there are no non-integer loads such as loads 56 and 63 in the database. Therefore, it is necessary to first determine the sub-interval to which the target load belongs, and select multiple working state sets corresponding to the sub-interval. Specifically, in the present invention, the load can be rounded up to ten for selection, so that part of the pipeline loss can be considered. After the optimization is completed, the PID optimization method is used for fine-tuning, which can greatly shorten the calculation time and improve the extreme efficiency.

[0072] More specifically, the maximum number of iterations k is designed to be 100000, and epoch is the current number of iterations.

[0073]

[0074] Each time an action is selected, a random number in the range [0,1) is generated. When the random number is less than β, an action is randomly generated in the action space. When the random number is greater than or equal to β, the action corresponding to the maximum value of the utility function in all current Qtable tables is selected.

[0075] For example, in the first iteration, the load at time 0 is 56, and the action selected is 2. The solution at time 0 is recorded as (2, [2, 0, 0, 0, 0, 47, 0, 0], 315.0498). The solutions at 0, 1, 2, ..., 23 are recorded in sequence. This is the solution for the first iteration. The corresponding electricity fee and start-stop cost are calculated based on this solution. The optimal strategy for 24 moments in a day is obtained by continuous iteration as follows:

[0076] Running status of 1# screw air compressor: [0,0,0,0,2,2,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];

[0077] Running status of 2# screw air compressor: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];

[0078] Running status of 3# screw air compressor: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];

[0079] The operating status of 4# screw air compressor: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];

[0080] Running status of 5# screw air compressor: [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0];

[0081] Frequency of 6# variable frequency screw air compressor:

[0082] [0,0,0,0,47,38,38,38,35,35,46,35,32,21,21,21,21,21,0,0,0,0,0,0];

[0083] IGV opening of 7# centrifugal air compressor:

[0084] [26,27,25,27,0,0,0,0,85,85,84,85,85,91,91,91,91,91,63,63,63,25,25,26];

[0085] BOV opening: [53,32,0,32,0,0,0,0,0,0,4,0,0,0,0,0,0,0,0,0,0,0,0,0,53].

[0086] In summary, the air compressor cluster optimization control method based on reinforcement learning of the present invention is a data-driven decision-making problem solving method, which utilizes the advantages of reinforcement learning without specific environmental model parameters and self-supervised machine learning methods that acquire knowledge through interaction with the environment. In the iterative solution process, the solution space of the operating status of each device in the air compressor cluster system can be continuously explored through the objective function and constraint conditions, and the relatively optimal operating status solution set can be found. Adaptive adjustments are made according to environmental changes (including gas demand, time-of-use electricity prices, etc.), thereby improving the flexibility and adaptability of the operation of the air compressor cluster system. The rate and time of the solution can also be well controlled by adjusting the parameters, providing a reliable and real-time optimization control strategy, improving the energy efficiency and control accuracy of the air compressor cluster system, and fully tapping the energy-saving potential of the air compressor cluster system to reduce energy consumption.

[0087] The present invention discloses an air compressor cluster optimization control device based on reinforcement learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the method is implemented when the processor executes the computer program.

[0088] The present invention also discloses an embodiment that provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0089] The present invention also provides a computer program product. When the computer program product runs on a data storage device, the data storage device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0090] If the integrated unit module is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying the computer program code to a storage device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electrical carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a disk or an optical disk.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0093] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0094] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Claims

1. An air compressor cluster optimization control method based on reinforcement learning, characterized in that: The following steps are involved: Receive target loads at different time periods in a control cycle, and select multiple working state sets corresponding to the target loads; wherein the multiple working state sets are working state sets under multiple different working conditions, and the working state sets include the operating state of each air compressor in the air compressor cluster system; Selecting a set of working states for each period in the control cycle based on a reinforcement learning method; The air compressor cluster system is operated based on the working state set.

2. The air compressor cluster optimization control method based on reinforcement learning according to claim 1, characterized in that: Before selecting a plurality of working state sets corresponding to the target load, the method further includes: Dividing the load interval of the air compressor cluster system into a plurality of sub-intervals according to a preset step length; The working state set of the air compressor cluster system corresponding to each of the sub-intervals under different working conditions is calculated; wherein the different working conditions are that a predetermined air compressor in the air compressor cluster system is in an on state or an off state.

3. The air compressor cluster optimization control method based on reinforcement learning as claimed in claim 2, characterized in that: Selecting a plurality of working state sets corresponding to the target load includes: The subinterval to which the target load belongs is determined, and a plurality of working state sets corresponding to the subinterval are selected.

4. The air compressor cluster optimization control method based on reinforcement learning as described in claim 2 or 3, characterized in that: Selecting the working state set for each period in the control cycle based on the reinforcement learning method includes: The working state sets under multiple different working conditions corresponding to each time period are taken as the action space, the working state sets of the air compressor cluster system in different time periods are taken as the state space, and the negative number of the sum of the total electricity cost and the start-stop cost of the air compressor cluster system in the control period is taken as the utility function value to select the working state sets of each time period in the control period.

5. The air compressor cluster optimization control method based on reinforcement learning as claimed in claim 4, characterized in that: Selecting the working state set for each period in the control cycle based on the reinforcement learning method also includes: Generate a random number in the range [0,1) each time you select an action; When the random number is less than the threshold, an action is randomly selected in the action space.

6. The air compressor cluster optimization control method based on reinforcement learning as claimed in claim 5, characterized in that: When the random number is greater than or equal to the threshold, the action corresponding to the maximum value of the utility function is selected.

7. The air compressor cluster optimization control method based on reinforcement learning as described in claim 5 or 6, characterized in that: The threshold calculation method is: Among them, β is the threshold, k is the maximum number of iterations, and epoch is the current number of iterations.

8. The air compressor cluster optimization control method based on reinforcement learning as claimed in claim 2, characterized in that: Calculating the working state set of the air compressor cluster system corresponding to each of the sub-intervals under different working conditions includes: The calculation is performed using a reinforcement learning method; wherein the total energy consumption of the air compressor cluster system and a penalty term are used as the efficiency function, and the penalty term is calculated based on the difference between the gas output and the load of the air compressor cluster system.

9. The air compressor cluster optimization control method based on reinforcement learning as claimed in claim 8, characterized in that: The penalty term is calculated as follows: Among them, punish represents the penalty term, targe_q represents the load, and q represents the gas production.

10. An air compressor cluster optimization control device based on reinforcement learning, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

Images