Smart park resource scheduling method, device and equipment and readable storage medium

Abstract

The invention discloses a smart park resource scheduling method, device and equipment and a readable storage medium, and relates to the technical field of resource scheduling, and the method comprises the following steps: firstly constructing a preset DQN-PPO hybrid decision network; then obtaining a plurality of historical state space vectors, and training a preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a near-end strategy optimization subnet; then obtaining a current state space vector, inputting the current state space vector into a preset DQN-PPO hybrid decision network, and outputting a current optimal discrete action and a current optimal continuous action; and finally, scheduling resources according to the output current optimal discrete action and the current optimal continuous action. The optimal continuous action and the optimal discrete action are respectively output through the DQN-PPO hybrid decision network, so that the low precision or fuzzy result of the output result of the existing scheme is avoided, and the park resource scheduling is more reasonable.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of resource scheduling, and in particular to a smart park resource scheduling method, device and equipment and a readable storage medium. BACKGROUND

[0002] In recent years, deep reinforcement learning technology has shown application potential in the field of energy scheduling. As an important part of urban energy systems, the efficiency of energy scheduling in smart parks directly affects the economy, stability and sustainability of the park. Traditional energy scheduling methods mainly rely on manual experience to develop fixed rules or optimize based on mathematical programming models. These methods have obvious limitations when facing dynamic changes in park energy systems and are difficult to adapt to complex scenarios in the park.

[0003] The existing park resource scheduling algorithm has the problems of low accuracy or unclear range, and resource waste or resource shortage may occur during park resource scheduling. SUMMARY

[0004] The embodiments of the present application provide a smart park resource scheduling method, device, equipment and readable storage medium to solve the technical problems of low accuracy or fuzzy results of the output of the existing traditional single algorithm in related technologies.

[0005] In a first aspect, a smart park resource scheduling method is provided, comprising the following steps: constructing a preset DQN-PPO hybrid decision network; obtaining a plurality of historical state space vectors, and training the preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; obtaining a current state space vector, inputting the current state space vector into the preset DQN-PPO hybrid decision network, and outputting a current optimal discrete action and a current optimal continuous action; scheduling resources according to the output current optimal discrete action and the output current optimal continuous action.

[0006] In some embodiments, the construction of the preset DQN-PPO hybrid decision network comprises: constructing a deep Q network subnet based on a value-based optimal discrete action processing mechanism; constructing a proximal policy optimization subnet based on a policy-based optimal continuous action processing mechanism; combining the deep Q network subnet and the proximal policy optimization subnet to obtain the DQN-PPO hybrid decision network.

[0007] In some embodiments, the construction of the deep Q network subnet based on the value-based optimal discrete action processing mechanism comprises: action value function of the deep Q network subnet is constructed; The action value function of the deep Q network subnet is maximized.

[0008] In some embodiments, the proximal policy optimization subnet of the mechanism for constructing the optimal continuous action based on the policy includes: A random policy function of the proximal policy optimization subnet is constructed; The random policy function of the proximal policy optimization subnet is maximized by policy gradient ascent to maximize the cumulative reward.

[0009] In some embodiments, the plurality of historical state space vectors are obtained, and the preset DQN-PPO hybrid decision network is trained according to the plurality of historical state space vectors, including: Step 201: Obtain a first historical state space vector; Step 202: input the first historical state space vector into the preset DQN-PPO hybrid decision network, and output a first set of optimal discrete actions and optimal continuous actions; Step 203: Put the first set of optimal discrete actions and optimal continuous actions and the first historical state space vector into the experience replay pool; Step 204: Obtain an action sample from the experience replay pool, and update the network parameters of the optimal action value function of the deep Q network subnet and the optimal value subnet function and the optimal policy subnet function of the proximal policy optimization subnet in the DQN-PPO hybrid decision network; Step 205: Obtain a next historical state space vector, and repeat the above steps 201 to 204 until the training is completed.

[0010] In some embodiments, the action sample is obtained from the experience replay pool, and the network parameters of the deep Q network subnet and the proximal policy optimization subnet in the DQN-PPO hybrid decision network are updated, including: The network parameters of the deep Q network subnet are updated as: ; The network parameters of the proximal policy optimization subnet are updated as:

[0011]

[0012] wherein, is a learning rate; is the gradient of the loss function of the deep Q network subnet with respect to the parameter ; is the gradient of the total loss function of the proximal policy optimization subnet with respect to the parameter a gradient of the value loss function with respect to the parameter a gradient of the value loss function with respect to the parameter a gradient of the value loss function with respect to the parameter

[0013] In some embodiments, the scheduling of the resource according to the current optimal discrete action and the current optimal continuous action comprises: The current optimal discrete action at least comprises: a current energy storage charge and discharge state, and a current air conditioner power gear position. The current optimal continuous action at least comprises: a current energy storage charging ratio, a current photovoltaic charging ratio, and a current wind power charging ratio.

[0014] In a second aspect, a smart park resource scheduling apparatus is provided, comprising: A construction unit configured to construct a preset DQN-PPO hybrid decision network. An acquisition unit configured to acquire a plurality of historical state space vectors, and train the preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet. An output unit configured to acquire a current state space vector, input the current state space vector into the preset DQN-PPO hybrid decision network, and output a current optimal discrete action and a current optimal continuous action. A scheduling unit configured to schedule a resource according to the output current optimal discrete action and the output current optimal continuous action.

[0015] In a third aspect, a computer device is provided, comprising a memory and a processor, the memory stores at least one instruction, the at least one instruction is loaded and executed by the processor to implement the aforementioned smart park resource scheduling method.

[0016] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium stores computer instructions, when the computer instructions are executed by a computer, the computer instructions cause the computer to execute the aforementioned smart park resource scheduling method.

[0017] The technical scheme provided by the present application brings the following beneficial effects: The embodiment of the present application provides a smart park resource scheduling method, device and equipment and readable storage medium, first, a preset DQN-PPO hybrid decision network is constructed; then a plurality of historical state space vectors are acquired, and the preset DQN-PPO hybrid decision network is trained according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; then a current state space vector is acquired, the current state space vector is input into the preset DQN-PPO hybrid decision network, and a current optimal discrete action and a current optimal continuous action are output; and finally, resources are scheduled according to the output current optimal discrete action and the current optimal continuous action. The DQN-PPO hybrid decision network is used to replace the original single algorithm to perform the method of park resource scheduling, the precision and accuracy of the output action are improved, the output fuzzy action is avoided, and the optimization of the park resource scheduling is realized. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 A flowchart of a smart park resource scheduling method provided by the embodiment of the present application is shown in the figure. Figure 2 A DQN-PPO hybrid decision network training flowchart provided by the embodiment of the present application is shown in the figure. Figure 3 A structural schematic diagram of a smart park resource scheduling device provided by the embodiment of the present application is shown in the figure. Figure 4 A structural schematic diagram of a computer device provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] The embodiment of the present application provides a smart park resource scheduling method, which can solve the technical problems of low precision or fuzzy output of single algorithm in existing smart park resource use.

[0022] Referring to Figure 1 As shown in the embodiments of the present application, a smart park resource scheduling method is provided, comprising the following steps: Step S1: constructing a preset DQN-PPO hybrid decision network; Step S2: obtaining a plurality of historical state space vectors, and training the preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; Step S3: obtaining a current state space vector, inputting the current state space vector into the preset DQN-PPO hybrid decision network, and outputting a current optimal discrete action and a current optimal continuous action; Step S4: scheduling resources according to the output current optimal discrete action and the output current optimal continuous action.

[0023] In the process of scheduling the smart park, the preset DQN-PPO hybrid decision network is first constructed; then a plurality of historical state space vectors are obtained, and the preset DQN-PPO hybrid decision network is trained according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; then a current state space vector is obtained, and the current state space vector is input into the preset DQN-PPO hybrid decision network, and a current optimal discrete action and a current optimal continuous action are output; finally, resources are scheduled according to the output current optimal discrete action and the output current optimal continuous action. By using the historical state vectors to train the DQN-PPO hybrid decision network, the number of samples required for training is greatly reduced. By using the DQN-PPO hybrid decision network to replace the original single algorithm for park resource scheduling, the accuracy and precision of park resource scheduling are improved, and the occurrence of resource waste or resource shortage in park resource scheduling is avoided.

[0024] In an optional embodiment, the deep Q network subnet of the action value function based optimal discrete action processing mechanism comprises: An action value function of the deep Q network subnet is constructed ; ; Wherein the network parameters of the deep Q network subnet are ; The action value function of the deep Q network subnet is maximized to output the optimal discrete action ; The training target of the deep Q network subnet is to minimize the mean square error of and , wherein the loss function is: ; The current network parameters θ are updated by minimizing the loss function through stochastic gradient descent (SGD): Network parameters of the deep Q-network subnet The update is: ; is the environment state of the state space vector at time t; is the optimal discrete action at time t; represents the selection of the discrete action that can maximize the value of is the experience replay pool; is the learning rate; is the gradient of the loss function with respect to ; is the optimal discrete action value function of the current deep Q-network subnet, and is the optimal discrete action value function of the target deep Q-network subnet.

[0025] The deep Q-network subnet is specifically used to process and output discrete actions, avoiding the loss of precision caused by mapping discrete actions to continuous actions, using a shared experience pool to reduce the sample size required for training, and shortening the training period.

[0026] In an optional embodiment, the proximal policy optimization subnet for constructing a policy-based optimal continuous action processing mechanism comprises: Policy probability distribution function for constructing a proximal policy optimization subnet ; The random policy function of the proximal policy optimization subnet is processed by maximizing the cumulative reward through policy gradient ascent, and the optimal continuous action is output. The proximal policy optimization subnet approximates the random policy through a deep neural network and outputs the optimal continuous action, and is composed of a preset policy subnet and a preset value subnet, wherein the parameters of the policy subnet are and the parameters of the value subnet are ; The training target of the proximal policy optimization subnet is to maximize the expected cumulative reward , that is: ; A batch of continuous action trajectory samples are randomly sampled from the experience replay pool , and the current state value and the next state value are obtained through the value subnet.

[0027] The temporal difference method is used to calculate the immediate value of continuous actions, and if is a non-terminal state (not to the end of the scheduling period), then:

[0028] if is a terminal state (end of the scheduling period), then:

[0029] wherein is the discount factor.

[0030] Set the advantage function , , the advantage function is set as: ; Through iteration, the policy subnetwork and the value subnetwork parameter are updated. First, the current policy distribution is calculated according to the current state:

[0031] Define the preset policy subnetwork update rate : ; The clipping objective function of the proximal policy optimization subnetwork is defined as: ; An entropy regularization term is introduced , and the final objective function of the proximal policy optimization subnetwork is: ; The parameters are updated by gradient ascent: :

[0032] Define the fitting objective of the value subnetwork as , and the learning state corresponds to the real value:

[0033] The mean square error (MSE) is used to measure the deviation between the output of the value subnetwork and the target value, and the loss function is: ; The parameters are updated by minimizing the value loss by gradient descent: : .

[0034] ​By constructing a near-end strategy optimization subnet, continuous actions can be processed accurately and flexibly, solving the problems of poor accuracy and weak adaptability of traditional algorithms in park resource scheduling, and improving the economy and stability of park scheduling.

[0035] In an optional embodiment, the step of acquiring multiple historical state space vectors and training a preset DQN-PPO hybrid decision network based on the multiple historical state space vectors includes: Step 201: Obtain the first historical state space vector; Step 202: Input the first historical state space vector into the preset DQN-PPO hybrid decision network, and output the first set of optimal discrete actions. and optimal continuous action ; Step 203: Assign the first set of optimal discrete actions and optimal continuous action The first historical state space vector is placed into the experience replay pool D. middle; Step 204: Obtain action samples from the experience replay pool using DQN sampling. Training, PPO sampling Train and update the network parameters of the optimal action value function of the deep Q network subnet, the optimal value subnet function of the proximal policy optimization subnet, and the optimal policy subnet function in the DQN-PPO hybrid decision network; Step 205: Obtain the next historical state space vector, and repeat steps 201 to 204 above until training ends.

[0036] By using historical state space vectors for direct training, training costs are reduced, eliminating the need for extensive trial and error. The optimal continuous and discrete action samples are then put back into the experience replay pool, facilitating further optimization and training of the model, thereby reducing costs and improving accuracy.

[0037] See Figure 3 As shown in the figure, this embodiment of the invention also provides a smart park resource scheduling device, including: a construction unit, an acquisition unit, an output unit, and a scheduling unit.

[0038] The building unit is used to construct a preset DQN-PPO hybrid decision network; The acquisition unit is used to acquire multiple historical state space vectors and train a preset DQN-PPO hybrid decision network based on the multiple historical state space vectors; wherein, the preset DQN-PPO hybrid decision network includes a deep Q network subnetwork and a near-end policy optimization subnetwork; The output unit is configured to acquire a current state space vector, input the current state space vector into a preset DQN-PPO hybrid decision network, and output a current optimal discrete action and a current optimal continuous action. The scheduling unit is configured to schedule resources according to the output current optimal discrete action and the output current optimal continuous action.

[0039] The embodiment of the present application also provides a computer device, which comprises a memory, a processor and a network interface connected through a system bus, at least one instruction is stored in the memory, the at least one instruction is loaded and executed by the processor to realize all steps or part of steps of the smart park resource scheduling method.

[0040] The network interface is configured to perform network communication, such as sending the allocated task. Figure 4 The structure shown in the figure is only a block diagram of part of the structure related to the present application scheme, and does not constitute a limitation on the computer device to which the present application scheme is applied, and the specific computer device can comprise more or less components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0041] The processor can be a CPU, and can also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), ready programmable gate arrays (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic discrete hardware components, etc. The general-purpose processor can be a microprocessor, or the processor can also be any conventional processor, etc. The processor is the control center of the computer device, and connects all parts of the computer device through various interfaces and lines.

[0042] The memory can be used to store computer programs and / or modules, and the processor realizes various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and calling data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application program required by a function (such as a video playing function, an image playing function, etc.), and the like; and the data storage area can store data created according to the use of the mobile phone (such as video data, image data, etc.), and the like. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0043] In one embodiment of the application, the processor is configured to run a computer program stored in the memory to implement the following steps: Referring to Figure 1 The embodiment of the application provides a smart park resource scheduling method, which comprises the following steps: Step S1: constructing a preset DQN-PPO hybrid decision network; Step S2: obtaining a plurality of historical state space vectors, and training the preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; Step S3: obtaining a current state space vector, inputting the current state space vector into the preset DQN-PPO hybrid decision network, and outputting a current optimal discrete action and a current optimal continuous action; Step S4: scheduling resources according to the output current optimal discrete action and the output current optimal continuous action.

[0044] In an optional embodiment, the deep Q network subnet of the value-based optimal discrete action processing mechanism comprises: An action value function of the deep Q network subnet is constructed ; ; The network parameters of the deep Q network subnet are ; The action value function of the deep Q network subnet is maximized to output an optimal discrete action ; The training target of the deep Q network subnet is to minimize and the mean square error of ; The current network parameters θ are updated by minimizing the loss function through stochastic gradient descent (SGD): Network parameters of the deep Q-network subnet are updated as: ; is the environment state of the state space vector at time t; is the optimal discrete action at time t; represents the discrete action that maximizes the value of at time t, is the experience replay pool; is the learning rate; is the gradient of the loss function with respect to ; is the optimal discrete action value function of the current deep Q-network subnet, is the optimal discrete action value function of the target deep Q-network subnet.

[0045] In an optional embodiment, the proximal policy optimization subnet for constructing the policy-based optimal continuous action processing mechanism comprises: A policy probability distribution function for constructing the proximal policy optimization subnet ; A stochastic policy function of the proximal policy optimization subnet is maximized by policy gradient ascent to process the cumulative reward, and outputs the optimal continuous action The proximal policy optimization subnet approximates the stochastic policy by a deep neural network, outputs the optimal continuous action, and is composed of a preset policy subnet and a preset value subnet, wherein the parameters of the policy subnet are and the parameters of the value subnet are ; The training target of the proximal policy optimization subnet is to maximize the expected cumulative reward , that is: ; A batch of continuous action trajectory samples are randomly sampled from the experience replay pool , and the current state value and the next state value are obtained by the value subnet.

[0046] The temporal difference method is used to calculate the immediate value of the continuous action, and if is a non-terminal state (not to the end of the scheduling period), then:

[0047] If is the termination state (end of the scheduling period), then:

[0048] where is the discount factor.

[0049] Set the advantage function , , the advantage function is set to: ; Update the policy subnetwork and value subnetwork parameters through iteration. First, calculate the current policy distribution according to the current state:

[0050] Define the preset policy subnetwork update rate : ; The clipping objective function of the proximal policy optimization subnetwork is defined as: ; Introduce an entropy regularization term , and the final objective function of the proximal policy optimization subnetwork is: ; Update the parameters by gradient ascent: Define the fitting objective of the value subnetwork as

[0051] , and the learning state corresponds to the true value:

[0052] The mean square error (MSE) is used to measure the deviation between the output of the value subnetwork and the target value, and the loss function is: ; Minimize the value loss by gradient descent to update the parameters : .

[0053] In an optional embodiment, the obtaining a plurality of historical state space vectors, and training a preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors, comprises: Step 201: obtaining a first historical state space vector; ​​Step 202: Input the first historical state space vector into the preset DQN-PPO hybrid decision network, and output the first set of optimal discrete actions. and optimal continuous action ; Step 203: Select the first set of optimal discrete actions and optimal continuous action The first historical state space vector is placed into the experience replay pool D. middle; Step 204: Obtain action samples from the experience replay pool using DQN sampling. Training, PPO sampling Train and update the network parameters of the optimal action value function of the deep Q network subnet, the optimal value subnet function of the proximal policy optimization subnet, and the optimal policy subnet function in the DQN-PPO hybrid decision network; Step 205: Obtain the next historical state space vector, and repeat steps 201 to 204 above until training ends.

[0054] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the steps of the aforementioned smart park resource scheduling method.

[0055] The embodiments of the present invention can implement all or part of the aforementioned processes, or they can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various methods described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0056] Those skilled in the art will appreciate that embodiments of the present application can be devised for a variety of applications. It is therefore intended that the present application cover all such modifications and variations of the example embodiments described herein including combinations of features of the described embodiments. It is intended that this application embrace all alternatives fairly within the scope of the description.

[0057] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. It is also possible in the present application that steps can be executed in different sequence, where this is explicitly indicated or otherwise is apparent from this description.

[0058] The sequence number of the above-mentioned embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments.

[0059] The present application is described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 The means for carrying out the functions specified in a flowchart or multiple flows and / or blocks Figure 1 The means for carrying out the functions specified in a flowchart or multiple flows and / or blocks

[0060] The above description is intended to enable the person skilled in the art to understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for resource scheduling in a smart park, characterized in that, The method comprises the following steps: constructing a preset DQN-PPO hybrid decision network; obtaining a plurality of historical state space vectors, and training the preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; obtaining a current state space vector, inputting the current state space vector into the preset DQN-PPO hybrid decision network, and outputting a current optimal discrete action and a current optimal continuous action; scheduling resources according to the output current optimal discrete action and the output current optimal continuous action. 2.The smart park resource scheduling method of claim 1, wherein, The method of constructing the preset DQN-PPO hybrid decision network comprises: constructing a deep Q network subnet based on a value-based optimal discrete action processing mechanism; constructing a proximal policy optimization subnet based on a policy-based optimal continuous action processing mechanism; combining the deep Q network subnet and the proximal policy optimization subnet to obtain the DQN-PPO hybrid decision network. 3.The smart park resource scheduling method of claim 2, wherein, The method of constructing the deep Q network subnet based on the value-based optimal discrete action processing mechanism comprises: constructing an action value function of the deep Q network subnet; maximizing the action value function of the deep Q network subnet. 4.The method of Claim 3, wherein, The method of constructing the proximal policy optimization subnet based on the policy-based optimal continuous action processing mechanism comprises: constructing a random policy function of the proximal policy optimization subnet; maximizing the cumulative reward of the random policy function of the proximal policy optimization subnet through policy gradient ascent. 5.The wisdom zone resource scheduling optimization method based on deep reinforcement learning according to claim 4, characterized in that, The method of obtaining a plurality of historical state space vectors and training the preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors comprises: step 201: obtaining a first historical state space vector; step 202: inputting the first historical state space vector into the preset DQN-PPO hybrid decision network to output a first set of optimal discrete actions and optimal continuous actions; step 203: placing the first set of optimal discrete actions and optimal continuous actions and the first historical state space vector into an experience replay pool; step 204: obtaining an action sample from the experience replay pool, and updating network parameters of an optimal action value function of the deep Q network subnet and optimal value subnet functions and optimal policy subnet functions of the proximal policy optimization subnet in the DQN-PPO hybrid decision network; step 205: obtaining a next historical state space vector, and repeating the above steps 201 to 204 until the training is completed. 6.The method of Claim 5, wherein, The method of obtaining an action sample from the experience replay pool and updating network parameters of the deep Q network subnet and the proximal policy optimization subnet in the DQN-PPO hybrid decision network comprises: network parameters of the optimal action-value function of the deep Q-network subnetwork updated to read: ； Network parameters of optimal value subnetwork functions and optimal policy subnetwork functions of a proximal policy optimization subnetwork Updated to read: wherein, is the learning rate; is the gradient of the deep Q-network subnet loss function with respect to the parameters ; is the gradient of the total loss function of the on-policy optimization subnet with respect to the parameters ; is the gradient of the value loss function with respect to the parameters .

7. The smart park resource scheduling method according to claim 2, characterized in that: the current optimal discrete action at least comprises a current energy storage charging and discharging state and a current air conditioner power gear position; the current optimal continuous action at least comprises a current energy storage charging ratio, a current photovoltaic charging ratio, and a current wind power charging ratio.

8. A smart park resource scheduling apparatus, characterized in that, The method comprises: a construction unit configured to construct a preset DQN-PPO hybrid decision network; The acquisition unit is configured to acquire a plurality of historical state space vectors, and train a preset DQN-PPO hybrid decision network according to the plurality of historical state space vectors; wherein the preset DQN-PPO hybrid decision network comprises a deep Q network subnet and a proximal policy optimization subnet; The output unit is configured to acquire a current state space vector, input the current state space vector into the preset DQN-PPO hybrid decision network, and output a current optimal discrete action and a current optimal continuous action; The scheduling unit is configured to schedule resources according to the output current optimal discrete action and the output current optimal continuous action.

9. A computer device, comprising: The memory and the processor, at least one instruction is stored in the memory, the at least one instruction is loaded and executed by the processor, to realize the intelligent park resource scheduling method in any one of claims 1 to 7. The computer readable storage medium stores computer instructions, when the computer instructions are executed by the computer, the computer executes the intelligent park resource scheduling method in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: ​

Images