Port equipment cluster multi-energy control method based on quantum deep reinforcement learning

Abstract

The invention relates to a port equipment cluster multi-energy control method based on quantum deep reinforcement learning, and the method comprises the steps: obtaining the energy related data and equipment state data of a port, and carrying out the data preprocessing; converting the preprocessed energy related data and equipment state data into quantum state data according to a quantum coding rule; according to the quantum state data, generating probability distribution for selecting an energy combination by using a quantum strategy network, and selecting an optimal energy combination according to a greedy strategy; an optimal energy combination is optimized by utilizing a reward function, value network updating and quantum back propagation optimization; and controlling energy equipment of the port according to the optimized optimal energy combination. Compared with the prior art, the method can comprehensively consider the cost, the energy utilization rate, carbon emission and other factors according to the actual operation condition of the port, accurately selects the most suitable energy, effectively improves the energy utilization efficiency, and reduces the energy waste.

Description

Technical Field

[0001] The present invention relates to, and in particular to, a multi-energy control method for a port equipment cluster based on quantum deep reinforcement learning. Background Art

[0002] Against the backdrop of rising environmental awareness and energy restructuring, ports, as logistics hubs and major energy consumers, are facing profound changes in their energy management and utilization. Currently, the energy supply for port equipment clusters primarily includes electricity, thermal energy, hydrogen energy, and renewable energy sources such as wind and solar energy.

[0003] With technological advancements, ports are no longer solely reliant on traditional energy sources like utility power and diesel. Renewable energy is now accounting for a growing share of their energy systems. For example, some ports have installed wind turbines and solar photovoltaic panels to power lighting and small equipment. Furthermore, some ports have begun developing energy management systems that use sensors and monitoring equipment to monitor energy production, transmission, and consumption in real time. These systems collect energy data and provide information support for port energy management. Furthermore, in response to environmental protection requirements, port companies are implementing measures to reduce energy consumption and carbon emissions. For example, these measures include optimizing port operational processes and adopting energy-saving equipment.

[0004] The shortcomings of existing technologies include low energy utilization efficiency, unreasonable energy matching leading to energy waste, and low equipment energy conversion efficiency.

[0005] Chinese patent application CN117787589A discloses a method for rapid multi-energy coordinated scheduling based on a quantum neural network. This method constructs a quantum neural network-based power generation prediction model, trains a QNN model using a large number of training samples containing historical renewable energy output level measurement data, load levels, generator output levels, and unit start-up and shutdown status data. A mapping model is established with real-time wind, solar, and load measurements as input and unit output and unit combination as output. A closed-loop hot start framework is constructed for the QNN and multi-energy system optimization model. However, this patent only considers operating costs in its objective function and does not explicitly mention the comprehensive calculation of energy costs or carbon emissions. It primarily focuses on energy generation prediction and scheduling optimization, but does not deeply integrate factors related to equipment operating status. Summary of the Invention

[0006] The purpose of this invention is to overcome the defects of the above-mentioned existing technologies and provide a multi-energy control method for port equipment clusters based on quantum deep reinforcement learning. It can comprehensively consider factors such as cost, energy utilization, carbon emissions, etc. according to the actual operation of the port, accurately select the most suitable energy, effectively improve energy utilization efficiency, and reduce energy waste.

[0007] The purpose of the present invention can be achieved by the following technical solutions:

[0008] A multi-energy control method for port equipment clusters based on quantum deep reinforcement learning, the method comprising:

[0009] Obtain energy-related data and equipment status data of the port and perform data preprocessing;

[0010] Converting the pre-processed energy-related data and device status data into quantum state data according to quantum coding rules;

[0011] Based on quantum state data, a quantum strategy network is used to generate a probability distribution for selecting energy combinations and the optimal energy combination is selected according to a greedy strategy.

[0012] Optimize the optimal energy mix using reward functions, value network updates, and quantum backpropagation tuning;

[0013] Control the port's energy equipment based on the optimized optimal energy combination.

[0014] Furthermore, the energy-related data includes carbon emissions, costs and utilization rates of electrical energy, thermal energy and renewable energy.

[0015] Furthermore, the cost of the electric energy C g The calculation expression is:

[0016] C g =P g ×E g +α×(1-S g )×E g +β×M g ,

[0017] Among them, P g is the real-time electricity price of the city electricity, E g is the mains electricity usage, S g is the grid stability parameter, α is the coefficient related to the loss caused by grid instability, M g is the maintenance cost associated with the utility access equipment, and β is the maintenance cost coefficient;

[0018] The carbon emissions of the electricity CE g The calculation expression is:

[0019] CE g =CF g ×E g ,

[0020] Among them, CF g is the carbon emission factor of local utility power generation, E g is the mains electricity usage;

[0021] The cost of the thermal energy C d The calculation expression is:

[0022] C d =P d ×E d +γ×I d +δ×T d +∈×M d ,

[0023] Among them, P d is the diesel price, E d is the diesel usage, γ is the inventory cost coefficient, I d is the diesel inventory balance, δ is the transportation cost coefficient, T d is the transportation distance of diesel from the storage location to the equipment in use, ∈ is the diesel engine maintenance cost coefficient, M d is the diesel engine maintenance cost;

[0024] The carbon emissions of the heat energy CE d The calculation expression is:

[0025] CE d =ρ d ×LHV d ×EF d ×E d

[0026] Among them, ρ d is the density of diesel, LHV d is the lower calorific value of diesel, EF d is the carbon emission factor of diesel, E d is the diesel usage;

[0027] The cost of renewable energy C r The calculation expression is:

[0028]

[0029] Where θ is the depreciation coefficient of renewable energy equipment, C r-t is the total investment cost of renewable energy power generation equipment, E r-i is the electricity generated by renewable energy in the i-th time period, E r is the current amount of renewable energy power generation, μ is the maintenance cost coefficient of renewable energy equipment, M r is the maintenance cost of renewable energy equipment;

[0030] Carbon emissions of renewable energy CE r Considered as 0, that is, CE r =0.

[0031] Furthermore, the process of selecting the optimal energy combination includes:

[0032] Based on the port's energy-related data and equipment status data, a quantum strategy network is used to generate a probability distribution for selecting energy combinations;

[0033] The comprehensive evaluation value of each energy source is calculated using the comprehensive evaluation value formula, which is:

[0034]

[0035] Among them, w1, w2, and w3 are the weight coefficients of cost, utilization rate, and carbon emission, i.e., the probability distribution of selecting energy combination. is the ratio of the current real-time cost of energy to the reference cost, is the ratio of current energy utilization to reference utilization, is the ratio of carbon emissions from current energy to reference carbon emissions;

[0036] The energy combination with the largest comprehensive evaluation value is selected as the current optimal energy combination.

[0037] Furthermore, the data preprocessing includes:

[0038] using a cleaning algorithm to remove noise outliers from the energy-related data and the equipment status data;

[0039] The energy-related data and the equipment status data are normalized.

[0040] Furthermore, the reward function is expressed as:

[0041]

[0042] in, Is the best energy source currently best the cost, is the best energy utilization, is the carbon footprint of the best energy source, is the optimal energy usage, Q t is the target workload of port operations, Q a is the actual amount of work completed, C ref is the reference cost, U ref is the reference utilization, CE ref is the reference carbon emissions, w4, w5, w6, w7 are weight coefficients,

[0043] w4+w5+w6+w7=1,

[0044] 0≤w4,w5,w6,w7≤1.

[0045] Furthermore, the expression for updating the value network is:

[0046]

[0047] Among them, S t is the quantum state at time t, a t is the energy source chosen at time t, γ is the discount factor;

[0048] The quantum states include quantum states related to energy supply and consumption, quantum states related to equipment operation status, and quantum states related to environment and strategy, wherein:

[0049] The quantum states related to energy supply and consumption include the renewable energy-dominated supply state, the energy supply and demand balance wave state, and the energy supply tension state;

[0050] The quantum states related to the equipment operation status include the equipment efficient and stable operation state, key equipment failure state, and equipment maintenance and upgrade state;

[0051] The quantum states related to the environment and strategy include the strict implementation state of environmental protection strategy, the incentive state of energy subsidy strategy, and the emergency state for special weather or sudden events.

[0052] Furthermore, the expression for quantum back propagation tuning is:

[0053]

[0054] Among them, D(θ t ) is the gradient memory term based on time decay in the port multi-energy scenario,

[0055]

[0056] Among them, λ is the decay rate and K is the length of the historical window, that is, recent data has a higher weight on parameter updates.

[0057] Furthermore, according to the optimized optimal energy combination, a control signal generated by the mapping relationship is used to control the energy equipment of the port. The generation formula of the control signal is:

[0058]

[0059] Among them, E n represents the real-time usage of the nth type of energy, ω n represents the weight coefficient of energy n, θ is the equipment operating status parameter, τ is the environmental parameter, and α and β are adjustment coefficients.

[0060] Furthermore, after controlling the energy equipment of the port, the port data is also monitored in real time, and the comprehensive evaluation results are calculated using the comprehensive evaluation index formula, and the comprehensive evaluation results are fed back to adjust the weight parameters of the reward function, the parameters of the value network, or the parameters of the quantum strategy network, wherein,

[0061] The comprehensive evaluation index formula is:

[0062]

[0063] Among them, E is the actual energy efficiency, E t is the target energy efficiency, D is the actual equipment failure rate, and D b is the benchmark failure rate, i.e. the historical average, C is the actual carbon emissions, and C t is the carbon emission constraint, ω E 、ω D 、ω C is the dynamic weight coefficient.

[0064] Compared with the prior art, the present invention has the following beneficial effects:

[0065] 1. This invention proposes a comprehensive control method for different energy devices. This method can accurately select the most appropriate energy usage ratio based on the actual port operation conditions, taking into account factors such as cost, energy utilization, and carbon emissions, effectively improving energy utilization efficiency and reducing energy waste. Through quantum coding, this invention maps multi-source heterogeneous data into high-dimensional quantum states and uses quantum superposition properties to analyze the probability distribution of energy combinations in parallel, solving the "curse of dimensionality" problem of traditional methods in high-dimensional state spaces. This invention uses a comprehensive evaluation value formula and a multi-parameter reward function to uniformly quantify objectives such as cost, utilization, carbon emissions, and workload deviation. Weight coefficients are used to dynamically adjust the optimization priority in different scenarios, and the value network is updated to further introduce long-term benefit evaluation to avoid local optimality. Based on quantum backpropagation tuning and comprehensive evaluation indicator feedback, this invention can respond in real time to sudden conditions such as port operation fluctuations and equipment failures.

[0066] 2. This invention can accurately calculate the costs of different energy sources and fully consider cost factors in the energy equipment control process, selecting lower-cost energy sources, thereby reducing the port's energy procurement and usage costs. At the same time, by optimizing the operation and management of energy equipment, it reduces equipment maintenance costs and losses, further reducing operating costs.

[0067] 3. This invention prioritizes carbon emissions, incorporating a carbon emission penalty term into the reward function to force the model to favor low-carbon energy. It also defines a "renewable energy-dominated state" in quantum state partitioning to maximize clean energy use. Low-carbon energy sources are prioritized during energy selection and energy equipment control, effectively reducing carbon emissions at the port, helping it achieve its low-carbon operation goals and aligning with current environmental protection development trends.

[0068] 4. This invention uses quantum deep reinforcement learning technology. The overall algorithm can continuously learn and adapt to changes in the port energy environment and increased management needs. Through real-time monitoring and feedback mechanisms, it dynamically optimizes energy equipment control to ensure continuous improvement in energy equipment control performance.

[0069] 5. The quantum deep recurrent reinforcement learning algorithm proposed in this paper continuously monitors and provides feedback, effectively solving the problem of insufficient decision-making performance of quantum reinforcement learning in existing technologies under partially observable environments. It improves the accuracy and reliability of energy equipment control and provides a more scientific decision-making basis for port energy management.

[0070] 6. This invention uses quantum parallel computing to quickly process high-dimensional data and explore the global optimal solution, avoiding falling into local optimality like traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 Flow chart of the method of the present invention;

[0072] Figure 2 This is a system structure diagram of the present invention. DETAILED DESCRIPTION

[0073] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0074] Example 1

[0075] This embodiment discloses a multi-energy control method for port equipment cluster based on quantum deep reinforcement learning. The method is as follows: Figure 1 As shown, the method steps include:

[0076] S1, obtains the port’s energy-related data and equipment status data, and performs data preprocessing;

[0077] S2, converting the pre-processed energy-related data and equipment status data into quantum state data according to quantum coding rules;

[0078] S3, based on the quantum state data, uses the quantum strategy network to generate the probability distribution of energy combination selection and selects the optimal energy combination according to the greedy strategy;

[0079] S4, optimizes the optimal energy combination using reward function, value network update and quantum backpropagation tuning;

[0080] S5, controls the energy equipment of the port according to the optimized optimal energy combination;

[0081] S6 monitors port data in real time, calculates comprehensive evaluation results using the comprehensive evaluation index formula, and feeds back the comprehensive evaluation results to adjust the weight of the reward function or the parameters of the quantum strategy network.

[0082] In step S1, the energy-related data includes carbon emissions, costs, and utilization rates of electrical energy, thermal energy, and renewable energy.

[0083] The cost of electricity C g The calculation expression is:

[0084] C g =P g ×E g +α×(1-S g )×E g +β×M g ,

[0085] Among them, P g is the real-time electricity price of the city electricity, E g is the mains electricity usage, S g is the grid stability parameter, α is the coefficient related to the loss caused by grid instability, M g is the maintenance cost associated with the utility access equipment, and β is the maintenance cost coefficient;

[0086] Carbon emissions from electricity CE g The calculation expression is:

[0087] CE g =CF g ×E g ,

[0088] Among them, CF g is the carbon emission factor of local utility power generation, E g is the mains electricity usage;

[0089] The cost of heat energy C d The calculation expression is:

[0090] C d =P d ×E d +γ×I d+δ×T d +∈×M d ,

[0091] Among them, P d is the diesel price, E d is the diesel usage, γ is the inventory cost coefficient, I d is the diesel inventory balance, δ is the transportation cost coefficient, T d is the transportation distance of diesel from the storage location to the equipment in use, ∈ is the diesel engine maintenance cost coefficient, M d is the diesel engine maintenance cost;

[0092] Carbon emissions from thermal energy CE d The calculation expression is:

[0093] CE d =ρ d ×LHV d ×EF d ×E d

[0094] Among them, ρ d is the density of diesel, LHV d is the lower calorific value of diesel, EF d is the carbon emission factor of diesel, E d is the diesel usage;

[0095] The cost of renewable energy C r The calculation expression is:

[0096]

[0097] Where θ is the depreciation coefficient of renewable energy equipment, C r-t is the total investment cost of renewable energy power generation equipment, E r-i is the electricity generated by renewable energy in the i-th time period, E r is the current amount of renewable energy power generation, μ is the maintenance cost coefficient of renewable energy equipment, M r is the maintenance cost of renewable energy equipment;

[0098] Carbon emissions from renewable energy CE r Considered as 0, that is, CE r =0.

[0099] Data preprocessing includes:

[0100] Use cleaning algorithms to remove noise outliers from energy-related data and equipment status data;

[0101] Normalize energy-related data and equipment status data.

[0102] In step S3, the process of selecting the optimal energy combination includes:

[0103] Based on the port's energy-related data and equipment status data, the quantum strategy network is used to generate a probability distribution for selecting energy combinations. The probability distribution of the selected energy combination is the weight coefficients w1, w2, and w3 corresponding to cost, utilization rate, and carbon emissions, where w1+w2+w3=1, and w1, w2, w3≤1. The weight coefficients are used to balance the importance of cost, utilization rate, and carbon emissions. The weights are determined based on the actual needs and goals of port operations. For example, if cost control is more important, w1 can be appropriately increased; if the focus is on efficient energy utilization, w2 can be increased; if low carbon emissions are the goal, w3 can be increased.

[0104] The comprehensive evaluation value formula is used to calculate the comprehensive evaluation value of each energy source. The comprehensive evaluation value formula is:

[0105]

[0106] in, is the ratio of the current real-time cost of energy to the reference cost, is the ratio of current energy utilization to reference utilization, is the ratio of carbon emissions from current energy to reference carbon emissions;

[0107] The energy with the largest comprehensive evaluation value is selected as the current best energy source, that is:

[0108]

[0109] In step S4, the reward function is expressed as:

[0110]

[0111] in, Is the best energy source currently best the cost, is the best energy utilization, is the carbon footprint of the best energy source, is the optimal energy usage, Q t is the target workload of port operations, Q a is the actual amount of work completed, C ref is the reference cost, U ref is the reference utilization, CE ref is the reference carbon emissions, w4, w5, w6, w7 are weight coefficients,

[0112] w4+w5+w6+w7=1,

[0113] 0≤w4,w5,w6,w7≤1.

[0114] Data such as weight coefficients, reference utilization rates, and reference carbon emissions are historical averages or preset target values, determined based on the long-term data and goals of port operations.

[0115] In step S4, the value network is updated in the form of the quantum Bellman equation. The expression for the value network update is:

[0116]

[0117] Among them, S t is the quantum state at time t, a t is the energy source chosen at time t, γ is the discount factor;

[0118] Quantum states include quantum states related to energy supply and consumption, quantum states related to equipment operation status, and quantum states related to environment and strategy.

[0119] The quantum states related to energy supply and consumption include the renewable energy-dominated supply state, the energy supply and demand balance wave state, and the energy supply tension state;

[0120] Among them, S re1 Indicates the dominant supply state of renewable energy, and sets the solar power generation power to P solar , when P solar ≥0.8P solar-rated (P solar-rated is the rated power of solar panels); wind power generation is P wind ,0.6P wind-rated ≤P wind ≤0.8P wind-rated (P wind-rated is the rated power of the wind turbine); the mains electricity demand is E grid-demand , 0.3E total-demand ≤E grid-demand ≤0.4E total-demand (E total-demand is the total energy demand of the port); the renewable energy storage capacity is S re-storage , S re-storage ≥0.6S re-storage-max (S re-storage-max is the maximum storage capacity of renewable energy), it is in this quantum state. In this state, according to the definition of the reward function R, due to the abundant supply of renewable energy, its cost C is relatively low (C is mainly fixed costs such as equipment depreciation), the utilization rate U is high, and the carbon emissions C is e Almost zero, after substitution, the reward function R will increase significantly. According to the value network update formula, the Q value continues to increase, so in energy equipment control, renewable energy will be more preferred to power port equipment.

[0121] Among them, S fluct Indicates the dynamics of energy supply and demand balance. When the electricity demand E elec-demand The change in a short time ΔE elec-demand Meet 0.2E elec-demand-before ≤ΔE elec-demand ≤0.3E elec-demand-before

[0122] (E elec-demand-before is the electricity demand before the change); the change in solar power generation power ΔP solar Satisfy -0.5P solar-before ≤ΔP solar ≤-0.3P solar-before (P solar-before is the solar power generation power before the change); the wind power generation power fluctuation range is ±0.2P wind-before (P wind-before is the wind power generation power before the change); the number of diesel generator starts N diesel-start When it reaches 3-5 times within this period, it enters this quantum state. In this state, energy device control must comprehensively weigh the real-time costs, supply stability and reward function calculation results of multiple energy sources;

[0123] Among them, S shortage Indicates a state of energy supply tension. When the mains power supply power P grid-supply ≤0.5P grid-supply-normal (P grid-supply-normal is the normal mains power supply power); renewable energy generation cannot meet the demand, that is, E re-generation ≤0.3E total-demand (E re-generation is the amount of electricity generated by renewable energy); diesel stock S diesel-stock Lower than safety stock S diesel-safety-stock 50% of S diesel-stock ≤0.5S diesel-safety-stock When in this quantum state, in S shortage Under this state, energy equipment control gives priority to ensuring the operation of key equipment. Assuming that the energy demand of key equipment is E critical-demand , allocate energy according to equipment priority and energy emergency distribution degree, giving priority to meeting E critical-demand supply energy to the equipment.

[0124] The quantum states related to the equipment operation status include the equipment efficient and stable operation state, key equipment failure state, and equipment maintenance and upgrade state;

[0125] Among them, S stable Indicates that the equipment is in efficient and stable operation. When the port equipment failure rate F rate ≤5%, comprehensive energy utilization rate of equipment U totalWhen ≥85%, it is in this state;

[0126] Among them, S fault Indicates the failure state of key equipment. Assume that a large transformer failure causes power supply interruption in a certain area. The total power of the equipment in the area is P fault-area , accounting for 10%-20% of the total power of the port equipment, that is, 0.1P total-equipment ≤P fault-area ≤0.2P total-equipment (P total-equipment is the total equipment power of the port), and affects the collaborative operation of other equipment. The degree of impact coefficient of collaborative operation is I co-work , 0.3≤I co-work ≤0.5, at this time, the energy equipment control quickly adjusts the energy distribution strategy and starts the backup power generation equipment to temporarily power the key equipment;

[0127] Among them, S maintain Indicates the status of equipment maintenance and upgrades. For example, some old diesel engines undergo upgrades and renovations. During the renovation period, their energy consumption is unstable and their efficiency is reduced by 20%-30%. That is, the engine efficiency during the renovation period is reduced. Energy equipment control adjusts energy supply according to the equipment maintenance plan and energy consumption changes. For equipment under maintenance, the energy supply ratio is reduced. For other normally operating equipment, energy distribution is optimized to make up for the energy utilization gap caused by equipment maintenance.

[0128] The quantum states related to environment and strategy include the strict implementation state of environmental protection strategy, the incentive state of energy subsidy strategy, and the emergency state of special weather or sudden events;

[0129] Among them, S env-strict Indicates the strict implementation of the environmental protection strategy. When the environmental protection strategy requires the port's overall carbon emissions to be reduced by more than 30% within a specific time period T, that is, (C e-current (t) is the carbon emission in the current time period t, C e-before (t) is the carbon emissions in the same period before the implementation of the strategy, it is in this state;

[0130] Among them, S subsidy Indicates the incentive state of the energy subsidy strategy. When there is a subsidy strategy for hydrogen energy, the subsidy amount reaches 30%-50% of the cost of hydrogen energy use, and it is in this state;

[0131] Among them, S emergencyIndicates an emergency state due to special weather or sudden events. For example, during a typhoon, excessive winds may cause wind turbines to stop operating, or water may accumulate in parts of the port, affecting the normal use of equipment. In this state, energy equipment control prioritizes personnel safety and the protection of key equipment, suspending operations in dangerous areas. In terms of energy distribution, priority is given to powering drainage equipment and emergency lighting equipment. At this time, energy costs and carbon emissions are relatively secondary factors, and ensuring port safety and basic functional operations becomes the primary goal. The reward function focuses on evaluating the role of energy in the operation and safety of key equipment.

[0132] In step S4, quantum back propagation tuning uses the collected state transition information of the port multi-energy system to adjust the parameters in the quantum strategy network and the value network using the quantum back propagation algorithm. It uses the dynamic double damping factor to update the entire network based on complex parameters to achieve precise adjustment of the parameters, thereby optimizing the selection of the optimal energy combination.

[0133] The expression for quantum backpropagation tuning is:

[0134]

[0135] Among them, D(θ t ) is the gradient memory term based on time decay in the port multi-energy scenario,

[0136]

[0137] Among them, λ is the decay rate and K is the length of the historical window, that is, recent data has a higher weight on parameter updates.

[0138] The optimized optimal energy combination uses the control signal generated by the mapping relationship to control the energy equipment in the port. The generation formula of the control signal is:

[0139]

[0140] Among them, E n represents the real-time usage of the nth type of energy, ω n represents the weight coefficient of energy n, θ is the equipment operating status parameter, τ is the environmental parameter, and α and β are adjustment coefficients.

[0141] Different control signal generation methods and communication protocols are also used in the control of different energy equipment.

[0142] In step S7, after controlling the energy equipment of the port, the port data is also monitored in real time, and the comprehensive evaluation results are calculated using the comprehensive evaluation index formula, and the comprehensive evaluation results are fed back to adjust the weight of the reward function or the parameters of the quantum strategy network, wherein,

[0143] The comprehensive evaluation index formula is:

[0144]

[0145] Among them, E is the actual energy efficiency, E t is the target energy efficiency, D is the actual equipment failure rate, and D b is the benchmark failure rate, i.e. the historical average, C is the actual carbon emissions, and C t is the carbon emission constraint, ω E 、ω D 、ω C is the dynamic weight coefficient.

[0146] Example 2

[0147] This embodiment, based on the above-mentioned embodiment 1, discloses an example of the actual use of a multi-energy control method for a port equipment cluster based on quantum deep reinforcement learning.

[0148] Consider a port with an annual throughput of 5 million TEUs and an average daily electricity demand of 50 MWh. Its energy mix includes utility power, diesel generators, a solar photovoltaic system (10 MW installed capacity), hydrogen fuel cells (2,000 kg hydrogen storage capacity), and wind power (5 MW installed capacity). The port's goal is to reduce energy costs, minimize carbon emissions, and increase the utilization of renewable energy.

[0149] 1. Data collection and monitoring implementation: Install sensors at key locations such as the port's substations, boiler rooms, hydrogen stations, wind turbines, solar panels, and various energy equipment, including power sensors, temperature sensors, pressure sensors, flow sensors, etc. The sensors collect data at set time intervals and transmit the data to the data processing center through wired or wireless communication technology. The data processing center stores and conducts preliminary analysis of the collected data in real time to ensure the integrity and accuracy of the data. Assume the status of each sensor deployed in a port: the light sensor can collect real-time light intensity (average 300W / m 2 ); the wind speed sensor collects a real-time wind speed of 8 m / s; the grid monitors the real-time electricity price of 0.12 $ / kWh, and the grid stability parameter α = 0.8; the current oil storage capacity is 5000 L, and the diesel price is 1.2 $ / L; the hydrogen storage capacity is 1500 kg, and the hydrogen cost is 8 $ / kg; and the port's current electricity demand is 45 MWh.

[0150] 2. Data Preprocessing: After receiving the raw data, the data processing center applies statistical cleaning algorithms, such as the 3σ criterion, to remove noise outliers that significantly deviate from the normal range. Normalization algorithms, such as the minimum-maximum normalization method, are used to map data of different dimensions to the [0, 1] interval and convert them into quantum state data according to quantum coding rules.

[0151] 3. Select the optimal energy mix: Determine the weighting factors for cost, energy utilization, and carbon emissions based on the current stage of port operations and key objectives. For example, during the key phase of energy conservation and emission reduction, the carbon emissions weighting factor can be appropriately increased. The above comprehensive energy assessment formula can be used to select the most appropriate energy source for the current situation.

[0152] According to the port's low-carbon goals, weights are set.

[0153] ω1=0.4(cost) ω2=0.3(utilization) ω3=0.3(carbon emissions),

[0154] Table 1 Calculation table of various energy parameters

[0155] Energy type <![CDATA[Cost (C j )]]> <![CDATA[Utilization rate (U j )]]> <![CDATA[Carbon emissions (E j )]]> solar energy 0.08$ / kWh 0.75 <![CDATA[0.0kg CO2 / kWh]]> wind energy 0.10$ / kWh 0.65 <![CDATA[0.0kg CO2 / kWh]]> Mains electricity 0.12$ / kWh 0.85 <![CDATA[0.7kg CO2 / kWh]]> Hydrogen Energy 0.15$ / kWh 0.80 <![CDATA[0.2kg CO2 / kWh]]> diesel fuel 0.18$ / kWh 0.70 <![CDATA[2.5kg CO2 / kWh]]>

[0156] Calculation of comprehensive evaluation value (taking solar energy as an example):

[0157]

[0158] Similarly, calculate the Z value of other energy sources and choose the energy combination with the highest Z value: solar energy (0.507) + wind energy (0.423) + hydrogen energy (0.455)

[0159] Reward function calculation:

[0160]

[0161] Quantum strategy network optimization: Update the parameter θ through the quantum back propagation algorithm, the formula is:

[0162]

[0163] 4. Decision Execution and Equipment Control Implementation: After outputting the optimal energy combination, the intelligent control signal generation model uses pre-trained complex mapping relationships to generate control signals applicable to various energy devices. Through a specific communication protocol, the control signals are transmitted to the port's energy equipment control system. The equipment collaborative control model monitors the operating status of the equipment and energy flow changes in real time. Based on the collaborative relationship between equipment and the dynamic balance requirements of energy flow, it dynamically adjusts the control signals to ensure efficient and stable operation of energy equipment. The control signal u is defined as:

[0164]

[0165] Among them, E n represents the real-time usage of the nth type of energy, ω n represents the weight coefficient of energy n, θ is the equipment operating status parameter, τ is the environmental parameter, and α and β are adjustment coefficients.

[0166] According to this formula, the energy allocation ratio of a certain port at this time is 50% solar energy, 30% wind energy, and 20% hydrogen energy;

[0167] The output power of the solar photovoltaic system is: 10MW×50%=5MW;

[0168] The output power of the wind turbine is: 5MW×30%=1.5MW;

[0169] The output power of the hydrogen fuel cell is: 45MWh×20%=9MWh.

[0170] 5. Evaluation and Feedback Implementation: Regularly calculate evaluation indicators such as energy efficiency improvement rate, cost reduction rate, and carbon emission reduction rate, and compare and analyze these indicators with preset target values. Based on the evaluation results, adjust relevant parameters through the feedback mechanism. For example, if energy efficiency does not meet the expected target, appropriately adjust the weight coefficient related to energy utilization in the reward function, prompting subsequent energy equipment control to pay more attention to improving energy efficiency, forming a closed-loop adaptive optimization system;

[0171] Real-time detection results of a port:

[0172] Actual energy cost: 0.08 × 5 + 0.10 × 1.5 + 0.15 × 9 = 1.9 $ / h;

[0173] Carbon emissions: 0.0×5+0.0×1.5+0.2×9=1.8kg CO2 / h;

[0174] Feedback optimization: Adjust the weights ω1 = 0.35, ω2 = 0.35, and ω3 = 0.3 based on actual data to increase utilization priority.

[0175] In summary, the implementation effect of the method according to Example 1 at a certain port is as follows:

[0176] Cost reduction: Compared with the pure mains power solution (0.12×45=5.4$ / h), the cost is reduced by 64.8%;

[0177] Carbon emissions reduction: Compared with the diesel solution (2.5×45=112.5kg CO2 / h), carbon emissions are reduced by 98.4%.

[0178] Example 3

[0179] This embodiment, based on the above embodiment 1, discloses a multi-energy control system for port equipment clusters based on quantum deep reinforcement learning, such as Figure 2 As shown, this system includes:

[0180] Data acquisition module: acquires energy-related data and equipment status data of the port;

[0181] Data preprocessing module: preprocesses energy-related data and equipment status data, and converts the preprocessed energy-related data and equipment status data into quantum state data according to quantum coding rules;

[0182] Quantum decision module: Based on quantum state data, the quantum strategy network is used to generate the probability distribution of energy combination selection and select the optimal energy combination according to the greedy strategy. The reward function, value network update and quantum backpropagation tuning are used to optimize the optimal energy combination.

[0183] Execution feedback module: Based on the optimized optimal energy combination, control the port's energy equipment, complete the control of multiple energy equipment, monitor port data in real time, calculate the comprehensive evaluation results using the comprehensive evaluation index formula, and feedback the comprehensive evaluation results to adjust the weight of the reward function or the parameters of the quantum strategy network.

[0184] The specific details of the above modules can be understood by referring to the relevant descriptions and effects in Example 1.

[0185] Example 4

[0186] Based on Example 1, this embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned multi-energy control method for port equipment cluster based on quantum deep reinforcement learning.

[0187] At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for its operations. The processor reads the corresponding computer program from the non-volatile storage into the internal memory and then runs it to implement the aforementioned multi-energy control method for port equipment clusters based on quantum deep reinforcement learning. Of course, in addition to software implementation, the present invention does not exclude other implementation methods, such as logic devices or a combination of software and hardware. In other words, the execution of the following processing flow is not limited to individual logic units and can also be hardware or logic devices.

[0188] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0189] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0190] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A multi-energy control method for port equipment cluster based on quantum deep reinforcement learning, characterized in that: The method comprises: Obtain energy-related data and equipment status data of the port and perform data preprocessing; Converting the pre-processed energy-related data and device status data into quantum state data according to quantum coding rules; Based on quantum state data, a quantum strategy network is used to generate a probability distribution for selecting energy combinations and the optimal energy combination is selected according to a greedy strategy. Optimize the optimal energy mix using reward functions, value network updates, and quantum backpropagation tuning; Control the port's energy equipment based on the optimized optimal energy combination.

2. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: The energy-related data includes carbon emissions, costs and utilization rates of electricity, thermal energy and renewable energy.

3. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 2 is characterized in that: The cost of the electrical energy C g The calculation expression is: C g =P g ×E g +α×(1-S g )×E g +β×M g , Among them, P g is the real-time electricity price of the city electricity, E g is the mains electricity usage, S g is the grid stability parameter, α is the coefficient related to the loss caused by grid instability, M g is the maintenance cost associated with the utility access equipment, and β is the maintenance cost coefficient; The carbon emissions of the electricity CE g The calculation expression is: WHAT g =CF g ×E g , Among them, CF g is the carbon emission factor of local utility power generation, E g is the mains electricity usage; The cost of the thermal energy C d The calculation expression is: C d =P d ×E d +γ×I d +δ×T d +∈×M d , Among them, P d is the diesel price, E d is the diesel usage, γ is the inventory cost coefficient, I d is the diesel inventory balance, δ is the transportation cost coefficient, T d is the transportation distance of diesel from the storage location to the equipment in use, ∈ is the diesel engine maintenance cost coefficient, M d is the diesel engine maintenance cost; The carbon emissions of the heat energy CE d The calculation expression is: WHAT d =ρ d ×LHV d ×EF d ×E d Among them, ρ d is the density of diesel, LHV d is the lower calorific value of diesel, EF d is the carbon emission factor of diesel, E d is the diesel usage; The cost of renewable energy C r The calculation expression is: Where θ is the depreciation coefficient of renewable energy equipment, C r-t is the total investment cost of renewable energy power generation equipment, E r-i is the electricity generated by renewable energy in the i-th time period, E r is the current amount of renewable energy power generation, μ is the maintenance cost coefficient of renewable energy equipment, M r is the maintenance cost of renewable energy equipment; Carbon emissions of renewable energy CE r Considered as 0, ie CE r =0.

4. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 3 is characterized in that: The process of selecting the optimal energy mix includes: Based on the port's energy-related data and equipment status data, a quantum strategy network is used to generate a probability distribution for selecting energy combinations; The comprehensive evaluation value of each energy source is calculated using the comprehensive evaluation value formula, which is: Among them, w1, w2, and w3 are the weight coefficients of cost, utilization rate, and carbon emission, i.e., the probability distribution of selecting energy combination. is the ratio of the current real-time cost of energy to the reference cost, is the ratio of current energy utilization to reference utilization, is the ratio of carbon emissions from current energy to reference carbon emissions; The energy combination with the largest comprehensive evaluation value is selected as the current optimal energy combination.

5. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: The data preprocessing includes: using a cleaning algorithm to remove noise outliers from the energy-related data and the equipment status data; The energy-related data and the equipment status data are normalized.

6. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: The expression of the reward function is: in, Is the best energy source currently best the cost, is the best energy utilization, is the carbon footprint of the best energy source, is the optimal energy usage, Q t is the target workload of port operations, Q a is the actual amount of work completed, C ref is the reference cost, U ref is the reference utilization, CE ref is the reference carbon emissions, w4, w5, w6, w7 are weight coefficients, w4+w5+w6+w7=1, 0≤w4,w5,w6,w7≤1.

7. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: The expression for the value network update is: Among them, S t is the quantum state at time t, a t is the energy source chosen at time t, γ is the discount factor; The quantum states include quantum states related to energy supply and consumption, quantum states related to equipment operation status, and quantum states related to environment and strategy, wherein: The quantum states related to energy supply and consumption include the renewable energy-dominated supply state, the energy supply and demand balance wave state, and the energy supply tension state; The quantum states related to the equipment operation status include the equipment efficient and stable operation state, key equipment failure state, and equipment maintenance and upgrade state; The quantum states related to the environment and strategy include the strict implementation state of environmental protection strategy, the incentive state of energy subsidy strategy, and the emergency state of special weather or sudden events.

8. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: The expression of quantum back propagation tuning is: Where D(θ t ) is the gradient memory term based on time decay in the port multi-energy scenario, Among them, λ is the decay rate and K is the length of the historical window, that is, recent data has a higher weight on parameter updates.

9. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: According to the optimized optimal energy combination, the control signal generated by the mapping relationship is used to control the energy devices of the port. The generation formula of the control signal is: Among them, E n represents the real-time usage of the nth type of energy, ω n represents the weight coefficient of energy n, θ is the equipment operating status parameter, τ is the environmental parameter, and α and β are adjustment coefficients.

10. The multi-energy control method for port equipment cluster based on quantum deep reinforcement learning according to claim 1 is characterized in that: After controlling the energy equipment of the port, the port data is also monitored in real time, the comprehensive evaluation results are calculated using the comprehensive evaluation index formula, and the comprehensive evaluation results are fed back to adjust the weight parameters of the reward function, the parameters of the value network or the parameters of the quantum strategy network, wherein, The comprehensive evaluation index formula is: Among them, E is the actual energy efficiency, E t is the target energy efficiency, D is the actual equipment failure rate, and D b is the benchmark failure rate, i.e. the historical average, C is the actual carbon emissions, and C t is the carbon emission constraint, ω E 、ω D 、ω C is the dynamic weight coefficient.

Images