Air source heat pump current distribution method and system and medium

Abstract

The invention relates to an air source heat pump current distribution method and system and a medium. The method comprises the following steps: constructing a state vector based on a heat pump operation state parameter and an environment parameter; the execution network based on the self-adaptive dynamic programming algorithm generates a control action vector for the state vector; the evaluation network based on an adaptive dynamic programming algorithm generates a long-term cost estimation value corresponding to each state-action combination by using the state vector and the control action vector; performing minimization calculation on the long-term cost estimation value, and selecting an action with the minimum long-term cost estimation value as a final action instruction; and performing current distribution based on the final action instruction, and iteratively updating the state vector, executing the network weight and evaluating the network weight. Intelligent and accurate current distribution of the heat pump system can be realized, the equipment operation stability and the energy utilization efficiency are effectively improved, and the system energy consumption and the component loss are reduced.

Description

Technical Field

[0001] This invention relates to the field of power system technology, and specifically to an air source heat pump current distribution method, system, and medium. Background Technology

[0002] Air source heat pumps, as an energy-saving device, have been widely used in heating. Existing air source heat pump control mostly adopts rule-based control strategies. The core bottleneck is that fixed rules cannot adapt to changing systems, especially when coordinating the energy consumption distribution of multiple actuators such as compressors, fans, water pumps, and auxiliary electric heaters (PTC). It is impossible to achieve global dynamic optimization, resulting in problems such as compressor priority but rigid adjustment, or fixed rules for multi-component linkage and sluggish response. Summary of the Invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, the present invention proposes an air source heat pump current distribution method, system and medium.

[0004] This invention provides a current allocation method for an air source heat pump, comprising: constructing a state vector based on heat pump operating state parameters and environmental parameters, wherein the heat pump operating state parameters include total current, operating current, operating voltage, water temperature, heat exchanger inlet and outlet temperatures, and equipment operating parameters, and the environmental parameters include indoor and outdoor temperatures and time parameters; generating a control action vector from the state vector using an execution network based on an adaptive dynamic programming algorithm; generating a long-term cost estimate for each state-action combination using an evaluation network based on an adaptive dynamic programming algorithm; minimizing the long-term cost estimate and selecting the action with the smallest long-term cost estimate as the final action instruction; allocating current based on the final action instruction and iteratively updating the state vector, execution network weights, and evaluation network weights.

[0005] In one optional implementation, the execution network based on the adaptive dynamic programming algorithm generates a control action vector from the state vector, specifically including: weighted summation of the components of the state vector with the connection weights from the input layer to the hidden layer of the execution network to obtain the hidden layer node input of the execution network; applying Sigmoid activation processing to the hidden layer node input of the execution network to obtain the hidden layer node output of the execution network; and weighted summation of the hidden layer node output of the execution network with the connection weights from the hidden layer to the output layer of the execution network to obtain the output node output of the execution network. The output nodes of all nodes constitute the control action vector, which corresponds to the current distribution scheme of the compressor, PTC heater, water pump, and fan.

[0006] In one optional implementation, the evaluation network based on the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination by combining the state vector and the control action vector. Specifically, this includes: weighting the components of the state vector with the connection weights from the input layer to the hidden layer of the evaluation network to obtain the input of the hidden layer node; performing Sigmoid activation on the input of the hidden layer node to obtain the output of the hidden layer node; and weighting the output of the hidden layer node with the connection weights from the hidden layer to the output layer to obtain the output of the output layer node. The output of the output layer node for all nodes represents the long-term cost estimate for each state-action combination.

[0007] In one optional implementation, the long-term cost estimate is minimized, and the action with the smallest long-term cost estimate is selected as the final action instruction, specifically including: Random perturbation is added to the control action vector to generate multiple candidate actions; based on the principle of minimizing long-term cost, the action with the smallest corresponding long-term cost estimate is selected as the final action instruction.

[0008] In one optional implementation, current allocation is performed based on the final action command, and the state vector, execution network weights, and evaluation network weights are iteratively updated, specifically including: Based on the final action command controller, the control action vector is parsed into the drive signal of each actuator and issued; based on the drive signal, the actuator drives the corresponding component to run and iteratively updates the state vector; online learning is performed based on the immediate effect after the action is executed, and the execution network weight and evaluation network weight are iteratively updated.

[0009] In one optional implementation, online learning is performed based on the immediate effect after the action is executed, and the execution network weights and evaluation network weights are iteratively updated. Specifically, this includes: calculating the utility function at time k based on the immediate effect after the action is executed; calculating the evaluation network error based on the utility function at time k; updating the evaluation network weights with the goal of minimizing the squared error of the evaluation network error; and updating the execution network weights with the goal of minimizing the long-term cost estimate of the execution network.

[0010] In one optional implementation, updating the evaluation network weights with the objective of minimizing the squared error of the evaluation network error specifically includes: calculating the weight matrix from the hidden layer to the output layer of the evaluation network with the objective of minimizing the squared error of the evaluation network error. : ; ; Calculate the weight matrix from the input layer to the hidden layer of the evaluation network. : ; ;in, To evaluate the network learning rate; To evaluate the network's error signal at time k; for In the The amount of weight update per step; This is the transpose of the output vector of the hidden layer node in the evaluation network at time k. This is the weight matrix from the hidden layer to the output layer of the network at time k. This is the updated weight matrix from the hidden layer to the output layer for evaluating the network at time k+1. for In the The amount of weight update per step; This is the transpose of the network input vector at time k; This is a nonlinear correction term based on the hidden layer output; To evaluate the weight matrix from the input layer to the hidden layer of the network at time k+1 after the update.

[0011] The execution network weights are updated with the goal of minimizing the long-term cost estimate of the execution network. Specifically, this includes calculating the weight matrix from the hidden layer to the output layer of the execution network, with the goal of minimizing the long-term cost estimate of the execution network. : ; Calculate the weight matrix from the input layer to the hidden layer of the execution network. : ; ;in, , to implement the network learning rate; To execute the weight matrix from the hidden layer to the output layer of the network The weight update amount at step k; The output vector of the hidden layer node is transposed at time k. This is the output vector of the hidden layer node in the network at time k. This is the transpose of the auxiliary weight matrix at time k; This is the updated weight matrix from the hidden layer to the output layer of the network at time k+1. To execute the weight matrix from the input layer to the hidden layer of the network The weight matrix at step k; and These are the transpose of the auxiliary weight matrix for evaluating the network and the transpose of the weight matrix from the hidden layer to the output layer of the execution network, respectively. This is the weight matrix from the input layer to the hidden layer of the network after the update at time k+1.

[0012] On the other hand, the present invention also proposes an air source heat pump current distribution system, comprising: The sensing module includes a current sensor, a voltage sensor, and a temperature sensor, used to collect system operating status and environmental parameters in real time. The heat pump operating status parameters include total current, operating current, operating voltage, water temperature, heat exchanger inlet and outlet temperatures, and equipment operating parameters. The environmental parameters include indoor and outdoor temperatures and time parameters. The decision module includes a processor running an adaptive dynamic programming algorithm, which constructs a state vector based on the heat pump operating status parameters and environmental parameters, and generates a control action vector based on the execution network of the adaptive dynamic programming algorithm. The evaluation network of the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination based on the state vector and the control action vector, performs a minimization calculation on the long-term cost estimate, selects the action with the smallest long-term cost estimate as the final action instruction, and iteratively updates the state vector, execution network weights, and evaluation network weights. The execution module includes a frequency converter and a solid-state relay to execute the final action instruction issued by the decision module, causing the execution module to drive the corresponding components to operate.

[0013] The present invention also proposes a storage medium storing a computer program, which, when executed by a processor, implements any of the air source heat pump current distribution methods described above.

[0014] The beneficial technical effects of this invention are as follows: Traditional control strategies based on fixed thresholds cannot adapt to the nonlinear and multivariate coupling characteristics of heat pump operation, and are prone to deviating from the high-efficiency point under varying operating conditions. This invention constructs a state vector by integrating the multi-dimensional operating state of the heat pump with environmental parameters, and relies on the execution network and evaluation network of an adaptive dynamic programming algorithm to generate control action vectors and accurately evaluate long-term costs. The system iterates online with the goal of minimizing long-term costs, enabling intelligent and precise allocation of the heat pump system current, thereby systematically improving the overall energy efficiency under all operating conditions, realizing dynamic global optimization instead of static rules, and tapping the system's energy efficiency potential. Unlike the independent control of each component, this invention intelligently coordinates the operation of the compressor, PTC heater, water pump, and fan through a unified long-term cost optimization objective. The algorithm autonomously learns to prioritize driving high-efficiency components (such as variable frequency compressors), activate auxiliary heat sources as needed, and dynamically match fan speed and water pump flow rate, avoiding common internal friction phenomena such as over-response, thereby directly reducing the total power consumption of the system while meeting the requirements, achieving precise coordination of multiple components, and reducing ineffective energy consumption. The weights of the system's execution network and evaluation network can be updated online based on actual operating feedback. The control strategy always maintains accurate tracking of the system's optimal operating point, overcoming the drawback of traditional fixed strategies whose effectiveness declines with increasing operating time. It achieves long-term stability and optimization of energy utilization efficiency, and while possessing continuous self-learning capabilities, it also ensures the long-term efficient and stable operation of the system. Attached Figure Description

[0015] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is an overall architecture diagram of an air source heat pump current distribution system provided in an embodiment of the present invention; Figure 2 A flowchart of an air source heat pump current distribution method provided in an embodiment of the present invention; Figure 3 A flowchart of closed-loop control and learning of the ADP algorithm for the current distribution method of an air source heat pump provided in an embodiment of the present invention; Figure 4 A flowchart illustrating a scenario example of the air source heat pump current distribution method provided in an embodiment of the present invention.

[0016] Explanation of reference numerals in the attached diagram: 10, Perception module; 20, Decision module; 30, Execution module. Detailed Implementation

[0017] With the development of artificial intelligence technology, the Adaptive Dynamic Programming (ADP) algorithm has shown great potential in the field of optimization control. It can continuously optimize control strategies through online learning and is applicable to nonlinear and time-varying systems. The ADP algorithm defines state vectors directly related to heat pump performance, outputting abstract control strategies that are precisely parsed into current distribution commands for each actuator, achieving precise regulation of heating and energy storage to achieve energy conservation. Currently, there are no reports on applying the ADP algorithm to the current distribution control of air source heat pumps.

[0018] The following is in conjunction with the appendix Figures 1 to 4 The present application will be further described in detail with reference to the embodiments. It is understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] Please refer to Figure 1This invention proposes an air source heat pump current distribution system, comprising: a sensing module 10, including a current sensor, a voltage sensor, and a temperature sensor, for real-time acquisition of system operating status and environmental parameters. The heat pump operating status parameters include total current, operating current, operating voltage, water temperature, heat exchanger inlet and outlet temperatures, and equipment operating parameters. The environmental parameters include indoor and outdoor temperatures and time parameters. A decision module 20 includes a processor running an adaptive dynamic programming algorithm, which constructs a state vector based on the heat pump operating status parameters and environmental parameters, and generates a control action vector based on the execution network of the adaptive dynamic programming algorithm. An evaluation network based on the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination, minimizes the long-term cost estimate, selects the action with the smallest long-term cost estimate as the final action instruction, and iteratively updates the state vector, execution network weights, and evaluation network weights. An execution module 30 includes a frequency converter and a solid-state relay to execute the final action instruction issued by the decision module 20, thereby driving the corresponding components to operate.

[0021] In this embodiment, the air source heat pump current distribution system includes a system body, a sensing module 10, a decision module 20, and an execution module 30. The system body includes a compressor, a PTC heater, a water pump, and a fan. The four components work together to achieve intelligent and efficient distribution of the air source heat pump current, thereby improving the stability and energy efficiency of the heat pump operation.

[0022] The main body of the system is the functional executor of the air source heat pump, comprising four key components: a compressor, a PTC heater, a water pump, and a fan. The compressor, as the core power unit of the heat pump system, is responsible for driving the refrigerant circulation to achieve heat transport and exchange. The PTC heater, as an auxiliary heating device, starts in low-temperature environments to supplement heating power. The water pump drives water circulation, achieving efficient transfer of heat between the refrigerant in the condenser (heat exchanger) and the circulating water in the water tank. The fan improves the heat transfer coefficient between the outdoor evaporator and the air, ensuring a stable output of heating or cooling performance. All components operate according to a preset collaborative logic, jointly completing the core functions of the heat pump system, including heating, cooling, and temperature control.

[0023] The sensing module 10 integrates current sensors, voltage sensors, and temperature sensors, along with a signal preprocessing unit and a data acquisition terminal. Current sensors are deployed on the system's main power supply circuit and the branch power supply circuits for the compressor, PTC heater, water pump, and fan, collecting real-time data on the system's total input current and the operating current of each component, accurately capturing dynamic changes in the current of each branch. Voltage sensors monitor the system's input operating voltage and the terminal voltage of each component, providing timely feedback on voltage fluctuations. Temperature sensors are deployed at multiple points to monitor the water temperature in the water tank, the inlet and outlet temperatures of the heat exchanger, the indoor ambient temperature, and the outdoor ambient temperature, comprehensively understanding the system's heat exchange status and environmental temperature conditions. Simultaneously, the sensing module 10 connects to a time parameter acquisition unit to record time information such as system operating time and seasonal cycles. The collected heat pump operating status parameters (total current, operating current of each component, operating voltage, water temperature, inlet and outlet temperatures of the heat exchanger, equipment start-up and shutdown status, etc.) and environmental parameters (indoor and outdoor temperatures, time parameters) are filtered, amplified, and converted from analog to digital by the signal preprocessing unit, forming a standardized and highly reliable dataset that is uploaded to the decision module 20 to provide comprehensive and accurate data support for subsequent control decisions.

[0024] The core of the decision module 20 is a high-performance processor equipped with an adaptive dynamic programming algorithm. This processor integrates an execution network, an evaluation network, and an iterative optimization unit. First, based on the operating status parameters and environmental parameters uploaded by the sensing module 10, the processor constructs a state vector containing multi-dimensional information such as current, voltage, temperature, and time, comprehensively representing the real-time operating status and environmental conditions of the heat pump system. The execution network takes this state vector as input and, combined with the operating characteristics of the compressor, PTC heater, water pump, and fan, and current distribution constraints, generates multiple sets of differentiated control action vectors. Each set of control action vectors corresponds to a current distribution scheme and operating frequency adjustment strategy for each component. The evaluation network matches the state vector with each set of control action vectors and calculates the long-term cost estimate for each state-action combination by constructing a long-term cost function covering multi-dimensional indicators such as system energy consumption, equipment operating losses, temperature regulation deviation, and component lifespan loss. Subsequently, the iterative optimization unit minimizes the long-term cost estimates for each combination and selects the action combination with the smallest long-term cost estimate as the final action instruction. Meanwhile, the decision module 20 continuously updates the state vector, execution network weights, and evaluation network weights based on the latest operational status feedback data, thereby continuously optimizing the algorithm model and ensuring that the control decisions always adapt to the dynamic changes in the system's operational status and environment.

[0025] The execution module 30 mainly includes a frequency converter, a solid-state relay, and a drive circuit. The frequency converter is connected to variable frequency loads such as compressors, fans, and water pumps, and precisely controls the operating speed and current of the loads by adjusting the output frequency. The solid-state relay is connected to the power supply circuit of fixed frequency loads such as PTC heaters, realizing the on / off control and current distribution adjustment of the fixed frequency loads. After receiving the final action command from the decision module 20, the execution module 30, according to the command signal, controls the frequency converter to output the corresponding frequency and the solid-state relay to perform on / off actions, thereby driving the compressor, PTC heater, water pump, and fan to operate according to the optimized current distribution scheme, ensuring that all components are always in a highly efficient and coordinated working state.

[0026] This solution addresses the operational characteristics of different functional components such as compressors, PTC heaters, water pumps, and fans. By combining real-time collected status and environmental parameters, it generates an optimal current distribution scheme through an adaptive dynamic programming algorithm. This allows for precise control of the operating current and frequency of each component, avoiding current distribution imbalances, ensuring efficient and coordinated operation of all components, and improving the overall heating / cooling efficiency of the heat pump system.

[0027] Decision module 20, with the core objective of minimizing long-term costs, incorporates indicators such as system energy consumption and equipment losses into the cost function for optimization. It can rationally allocate current to various components while meeting user temperature requirements, reducing ineffective energy consumption and excessive power output. Especially in low-temperature environments, by precisely controlling the start-up timing and operating current of the PTC heater, it avoids prolonged high-power operation, significantly reducing overall system energy consumption and improving the economic efficiency of equipment operation.

[0028] The execution module 30, through precise control of the frequency converter and solid-state relays, can avoid damage to core components caused by current surges and voltage fluctuations, ensuring stable operation of each component within the optimal current range. Simultaneously, the iterative optimization characteristics of the adaptive dynamic programming algorithm can promptly correct operational deviations caused by environmental changes or equipment aging, reducing component failure rates, decreasing equipment maintenance costs, and effectively extending the overall service life of the heat pump system.

[0029] The system achieves intelligent control of the heat pump system through the automated collaborative operation of the sensing, decision-making, and execution modules 30, eliminating the need for manual real-time adjustments to the current distribution scheme. Simultaneously, the system can automatically adjust its operating strategy based on time parameters and changes in ambient temperature, adapting to the usage needs of different seasons and time periods, significantly reducing manual intervention costs and enhancing the convenience and intelligence of system management.

[0030] like Figure 2 and Figure 3 As shown, this embodiment of the invention provides a current distribution method for an air source heat pump, comprising the following steps: Step S101: Construct a state vector based on the heat pump operating status parameters and environmental parameters. The heat pump operating status parameters include total current, operating current, operating voltage, water temperature, heat exchanger inlet and outlet temperatures, and equipment operating parameters. The environmental parameters include indoor and outdoor temperatures and time parameters.

[0031] Step S103: The execution network based on the adaptive dynamic programming algorithm generates control action vectors from the state vectors; Step S105: The evaluation network based on the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination by combining the state vector and the control action vector. Step S107: Minimize the long-term cost estimate and select the action with the smallest long-term cost estimate as the final action instruction; Step S109: Distribute current based on the final action command, and iteratively update the state vector, execute network weights, and evaluate network weights.

[0032] This air-source heat pump current allocation method constructs a state vector by integrating the multi-dimensional operating status of the heat pump and environmental parameters. Relying on the execution network and evaluation network of the adaptive dynamic programming algorithm, it generates control action vectors and accurately assesses long-term costs. With the goal of minimizing costs, it outputs the optimal current allocation command and completes model iterative optimization. This method enables intelligent and precise allocation of current in the heat pump system, effectively improving equipment operation stability and energy utilization efficiency, reducing system energy consumption and component wear, and extending equipment lifespan.

[0033] Further, step S103, the execution network based on the adaptive dynamic programming algorithm generates control action vectors from the state vectors, specifically including the following steps: Step S1031: The input of the hidden layer node of the execution network is obtained by weighted summation of the components of the state vector and the connection weights from the input layer to the hidden layer of the execution network.

[0034] Step S1033: Perform Sigmoid activation processing on the input of the hidden layer node of the execution network to obtain the output of the hidden layer node of the execution network.

[0035] Step S1035: The output of the execution network output layer node is obtained by weighted summation of the output of the execution network hidden layer node and the connection weight from the execution network hidden layer to the output layer.

[0036] Among them, the output of the execution network output layer nodes of all nodes constitutes the control action vector, and the control action vector corresponds to the current distribution scheme of the compressor, PTC heater, water pump and fan.

[0037] In this embodiment, after the sensor packages and uploads the collected data (such as total household current, actual water temperature in the water tank, ambient temperature, time, etc.) to the ADP controller at fixed intervals (e.g., 10 seconds / time), the controller performs normalization and noise reduction preprocessing on the multi-dimensional data, and the ADP controller constructs a state vector. , where k represents the current time.

[0038] The state vector The input is fed into the execution network, and the forward computation process of the execution network is as follows: ; ; ; In the above formula, n is the dimension of the state vector; q is the number of nodes executing the hidden layer of the network; and m is the number of output layer nodes (corresponding to the four types of equipment: compressor, PTC heater, water pump, and fan). To execute the input of the j-th node in the hidden layer of the network; To implement the connection weights from the network input layer to the hidden layer; To execute the output of the j-th node in the network hidden layer, These are the connection weights from the hidden layer to the output layer. The network ultimately outputs a control action vector. This vector represents the current distribution scheme for the compressor, PTC heater, water pump, and fan.

[0039] The inputs to each node in the hidden layer are processed by a Sigmoid nonlinear activation, mapping the input values ​​to the (0,1) interval, thus obtaining the output of the j-th node in the hidden layer of the execution network. The output values ​​of all nodes in the output layer are integrated into a control action vector u(k), which directly serves as the current allocation scheme for the four types of devices, guiding the power regulation and operating status control of the devices.

[0040] By using the nonlinear mapping of the Sigmoid activation function, the hidden layer input can be nonlinearly transformed, solving the problem that linear models cannot fit the operating laws of complex systems. This enables the generated control action vector to accurately match the dynamic operating characteristics of the heat pump water heating system and reduces current distribution deviation.

[0041] The sensor collects data at fixed intervals and constructs a state vector. Combined with the forward fast calculation process of the execution network, it can realize real-time perception of the system's operating status and instant generation of control commands. At the same time, the characteristics of the adaptive dynamic programming algorithm enable the execution network to dynamically adjust the connection weights as the system parameters change, adapting to the equipment control requirements under different ambient temperatures and water demand.

[0042] The control action vector precisely distributes current to core equipment such as compressors and PTC heaters, which can avoid inefficient operation of the equipment, ensure that the water tank heating temperature meets the heating demand, and reduce the inefficient consumption of the total household current, thus achieving energy-saving operation of the system.

[0043] By using the state vector of the heat pump water heating system and the control action vector generated by the execution network as dual inputs, a long-term cost estimate is generated through a three-step forward calculation process. This enables a quantitative assessment of the long-term benefits of the "state-action" combination, providing a basis for weight optimization of the execution network and iterative control strategies.

[0044] Further, in step S105, the evaluation network based on the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination from the state vector and the control action vector, specifically including the following steps: Step S1051: The input of the hidden layer node of the evaluation network is obtained by weighted summation of the components of the state vector and the connection weights from the input layer to the hidden layer of the evaluation network.

[0045] Step S1053: Perform Sigmoid activation processing on the input of the hidden layer node of the evaluation network to obtain the output of the hidden layer node of the evaluation network.

[0046] Step S1055: The output of the evaluation network output layer node is obtained by weighted summation of the output of the hidden layer node of the evaluation network and the connection weight from the hidden layer to the output layer of the evaluation network.

[0047] In this system, the output layer nodes of the evaluation network for all nodes output the long-term cost estimate corresponding to each state-action combination.

[0048] The state vector With control motion vector The inputs are fed into the evaluation network, and the forward computation process of the evaluation network is as follows: ; ; ; In the above formula, n is the dimension of the joint input vector; p is the number of nodes in the evaluation network hidden layer; To evaluate the input of the j-th node in the hidden layer of the network; To evaluate the connection weights from the network input layer to the hidden layer; To evaluate the output of the j-th node in the hidden layer of the network; To evaluate the connection weights from the hidden layers to the output layer of the network, we evaluate the network output scalar. That is, the long-term cost estimate, used to quantify the current state. Execute control actions The expected total long-term cost.

[0049] The system real-time state vector x(k) constructed by the ADP controller is dimensionally matched and data fused with the control action vector u(k) output by the execution network to form the joint input vector of the evaluation network. This vector contains both system operating state information and device control strategy information, fully reflecting the "state-action" combination characteristics.

[0050] This scalar is used to quantify the expected long-term total cost of operating a heat pump water heating system under the current combination of "state x(k)-action u(k)" (such as the comprehensive cost in terms of energy consumption, equipment loss, heating efficiency, etc.).

[0051] The evaluation network overcomes the limitations of traditional instant cost evaluation by quantifying the long-term cost of "state-action" combinations. This allows the ADP algorithm to iteratively optimize the connection weights of the execution network from the perspective of long-term system operation, avoiding problems such as excessive long-term energy consumption and aggravated equipment wear caused by pursuing short-term heating efficiency.

[0052] By using the Sigmoid activation function to perform a nonlinear transformation on the hidden layer input, the complex nonlinear mapping relationship of "state-action-cost" in the heat pump water heating system can be accurately fitted, solving the problem that linear models cannot capture the dynamic characteristics of the system, and making the long-term cost estimate more in line with the actual operating cost pattern.

[0053] The long-term cost estimate output by the evaluation network provides a clear optimization objective for the weight update of the execution network, enabling the ADP algorithm to achieve closed-loop iteration of the control strategy based on cost feedback. At the same time, the integrated design of dual input vectors allows the evaluation network to directly evaluate specific "state-action" combinations, reducing invalid computation and improving the algorithm's iteration efficiency.

[0054] By quantitatively assessing long-term costs, control action vectors that are "low in energy consumption, low in equipment wear and tear, and excellent in heating effect" can be selected. This not only reduces the long-term operating costs of heat pump water heating systems, but also avoids problems such as frequent start-ups and shutdowns of equipment caused by unreasonable current distribution, thus extending the service life of core equipment such as compressors and PTC heaters.

[0055] Further, step S107 involves minimizing the long-term cost estimate and selecting the action with the smallest long-term cost estimate as the final action instruction. This includes the following steps: Step S1071: Add random perturbation to the control action vector to generate multiple candidate actions; Step S1073: Based on the principle of minimizing long-term costs, select the action with the smallest corresponding long-term cost estimate as the final action instruction.

[0056] The ADP control core is based on the principle of minimizing long-term costs when proposing control actions. Make decisions. Typically, Actions evaluated as superior by the network have been directly adopted as the final action instructions. In the early stages of system learning, to encourage exploration, [further steps will be taken]. Based on this, multiple candidate actions are generated by adding random perturbations, and the corresponding actions are selected. The smallest action is taken as the final action instruction. The controller will... The system analyzes and sends out drive signals (such as target frequencies) to each actuator (compressor inverter, PTC solid-state relay, water pump, and fan inverter). These actuators drive their corresponding components, thereby changing the heat pump's heating power and total power consumption, which in turn affects the total household current and water tank temperature, updating the system status accordingly. .

[0057] In the early stages of system learning, random perturbations are added to generate candidate actions based on the suggested control action u(k), which breaks the limitation of relying solely on existing experience in the early stages and allows for the exploration of more potential efficient control strategies. At the same time, actions are selected based on the principle of minimizing long-term costs, ensuring that the exploration process always revolves around the long-term benefits of the system and achieving a balance between "exploring new strategies" and "utilizing excellent strategies".

[0058] It is suggested that the control action u(k) has passed the long-term cost evaluation of the evaluation network and become the basis for the better action. On this basis, perturbation screening can further lock the solution with the minimum long-term cost from the candidate actions, so that the final action command issued is more in line with the operating requirements of the heat pump system and reduces the generation of ineffective control decisions.

[0059] The final control action u(k) is parsed into drive signals for each actuator and sent out. This allows for direct adjustment of the operating status of core components such as the compressor and PTC heater, rapid changes in the heat pump's heating power and total power consumption, and timely adjustment of the household's total current and water tank temperature. This enables efficient updates of the system status from x(k) to x(k+1), adapting to dynamic changes in heating demand and the environment.

[0060] The initial exploration of perturbations accumulated more "state-action-cost" sample data for the ADP algorithm. This data fed back into the weight optimization of the evaluation network and the execution network, enabling the algorithm to continuously learn and iterate during system operation. As the running time increases, the adaptability and optimization effect of the control strategy will continue to improve.

[0061] Further, step S109 involves allocating current based on the final action command, and iteratively updating the state vector, executing network weights, and evaluating network weights. This specifically includes the following steps: Step S1091: Based on the final motion instruction, the controller parses the control motion vector into the drive signal of each actuator and sends it out.

[0062] Step S1093: Based on the driving signal, the actuator drives the corresponding component to run and iteratively updates the state vector.

[0063] Step S1095: Based on the immediate effect after the action is executed, perform online learning and iteratively update the execution network weights and evaluation network weights.

[0064] After the ADP controller sends the control action vector u(k) corresponding to the final action command, it maps the values ​​of each dimension in the vector to the drive signals of the compressor inverter, PTC solid-state relay, water pump inverter, and fan inverter according to the preset parsing rules. These signals include the target operating frequency of the compressor, the on / off duty cycle of the PTC heater, and the target speed of the water pump and fan. The controller then sends these drive signals to the corresponding actuators in real time through an industrial communication bus (such as Modbus or CAN bus).

[0065] After receiving the drive signal, each actuator drives the core components such as the compressor, PTC heater, water pump and fan to operate according to the instructions, directly changing the heating power and total power consumption of the heat pump system. The change in system power consumption further affects the distribution of the total current in the household. At the same time, the adjustment of the heat pump heating power will change the rate of temperature rise of the water in the water tank, ultimately causing the heat pump system to iteratively update from the current operating state x(k) to a new operating state x(k+1). The sensor synchronously collects the status data of x(k+1) and uploads it to the ADP controller.

[0066] The controller quickly parses the control action vector into the drive signal of the actuator and sends it out, enabling components such as compressors and PTC heaters to respond to commands in real time and adjust their operating status, and quickly complete the iterative update of the system status; at the same time, the online iteration of network weights enables the control strategy to keep up with the changes in the system operating status and adapt to the dynamic fluctuations of external conditions such as ambient temperature and water demand.

[0067] The ADP controller extracts real-time effect data after the action is executed, including the change in the state vector from x(k) to x(k+1), the deviation between actual and expected energy consumption, and the actual temperature rise of the water in the tank. Using this real-time effect data as training samples, the connection weights of the execution network and the evaluation network are learned and iteratively updated online using optimization algorithms such as gradient descent, so that the model parameters of the two networks are more in line with the dynamic characteristics of the actual operation of the system.

[0068] Through a closed-loop process of "action execution - state update - weight iteration", the actual effect of equipment operation is directly fed back to the parameter optimization of the execution network and the evaluation network, allowing the algorithm to continuously adjust the model based on the real operating data of the system, breaking through the limitations of offline training and realizing online self-optimization of the control strategy.

[0069] Online learning based on the immediate effects after action execution provides real and real-time training samples for network weight updates, avoiding the deviation between theoretical models and actual systems; continuous sample input and weight iteration can continuously correct the network's fitting error, improve the accuracy of the execution network in generating control actions and the evaluation network in assessing long-term costs, and enhance the algorithm's generalization ability to different operating scenarios.

[0070] The weighted iterative execution network can generate better control action vectors, making the actuator drive the core components more accurately and avoiding problems such as low equipment energy efficiency caused by control command deviations. At the same time, the smooth iterative update of the system state reduces sudden changes in water temperature and current, reduces the probability of frequent equipment start-ups and shutdowns, and extends the service life of key components such as compressors and water pumps.

[0071] More specifically, step S1095 involves online learning and iteratively updating the execution network weights and evaluation network weights based on the immediate effects after the action is executed. This includes the following steps: Step S10951: Calculate the utility function at time k based on the immediate effect after the action is executed; Step S10953: Calculate the evaluation network error based on the utility function at time k; Step S10955: Update the evaluation network weights with the goal of minimizing the squared error of the evaluation network error; Step S10957: Update the execution network weights with the goal of minimizing the long-term cost estimate of the execution network.

[0072] Calculate the utility function at time k based on the immediate effect after the action is performed. Its formula is: ; in, and These are positive definite weight matrices, used to balance state deviations (such as water temperature errors) and control energy consumption. As a priority guarantee penalty term, when the priority guarantee penalty term in the state vector is equal to 1 (indicating that there is a high-priority load demand and the total current is tight), this term will incur a significant cost, thereby forcing the strategy to reduce the PTC current in subsequent optimizations.

[0073] The network error is evaluated by calculating the utility function U(k) combined with the discount factor. By combining immediate utility, future cost estimation, and discount factor, this approach quantifies the immediate cost of the current "state-action" while also taking into account the long-term cost impact of subsequent states. This allows error calculation to fully reflect the prediction bias of the evaluation network and provides an accurate basis for weight updates.

[0074] The utility function incorporates positive definite weight matrices Q and R, which can respectively adjust the weighting of state deviations (such as water temperature errors) and control energy consumption in the cost assessment. This allows for flexible adjustment of the trade-off strategy based on the actual operating needs of the heat pump system (such as prioritizing water temperature during peak water usage periods and energy saving during off-peak periods); priority guarantee penalty items are also included. The design allows for forced reduction of PTC current when total current is tight, prioritizing the needs of high-priority loads and adapting to complex household power usage scenarios.

[0075] Evaluate network error The calculation formula is: ; in, It is a discount factor and 0 < ≤1, used to balance the importance of immediate and future costs; To evaluate the network based on the new state The calculated cost estimate.

[0076] The evaluation network, with its update objective of "minimizing the squared error," can quickly correct the prediction bias of the evaluation network, making the long-term cost estimate more in line with reality. The execution network, with its update objective of "minimizing the long-term cost estimate," directly anchors the long-term benefit optimization control strategy of the system, so that the weight updates of both networks have a clear optimization direction, avoiding the inefficiency caused by aimless iteration.

[0077] when When the value is 1, a significant penalty is triggered, forcing the strategy to reduce the PTC current. This ensures that core power needs are prioritized when the total household current is tight and there are high-priority load demands, preventing other high-priority devices from malfunctioning due to excessive current consumption by the heat pump system. At the same time, by adjusting the weights of state deviation and energy consumption, the drastic fluctuations in water temperature and current are reduced, improving the operational stability of the heat pump system.

[0078] The design of the steps from calculating the utility function and evaluating network error to updating weights for each objective forms a closed-loop online learning process of "effect feedback - error calculation - weight optimization". It can quickly complete the iteration of network parameters based on the real-time effect of action execution, and can continuously optimize without offline training, thus improving the learning and adaptation efficiency of the ADP algorithm in real systems.

[0079] To minimize the squared error To achieve this, update the evaluation network weights.

[0080] Step S10955, with the objective of minimizing the squared error of the evaluation network, updates the evaluation network weights, specifically including the following steps: Step S109551: With the objective of minimizing the squared error of the evaluation network, calculate the weight matrix from the hidden layer to the output layer of the evaluation network. : ; ; Step S109553: Calculate the weight matrix from the input layer to the hidden layer of the evaluation network. : ; ; in, To evaluate the network learning rate, the step size used to control the weight updates is... The larger the weight, the faster the update speed; conversely, the smaller the weight, the slower the update. To evaluate the error signal at time k, reflecting the degree of deviation between the long-term cost estimate output by the evaluation network and the actual cost; This is the transpose of the output vector of the hidden layer node in the evaluation network at time k. for The weight update amount at step k; To evaluate the weight matrix from the hidden layer to the output layer at time k+1 after the update, the weights are derived from the original weights. Plus update volume get; for The weight update amount at step k; This is the transpose of the network input vector (state and action combination vector) at time k. The nonlinear correction term based on the hidden layer output is obtained by subtracting the element-wise square of the hidden layer output vector from 1, and is used to optimize the nonlinear fitting effect of weight updates. To evaluate the weight matrix from the input layer to the hidden layer at time k+1 after the update, the original weights are used... Plus update volume get.

[0081] Learning rate Adjusting the weight update step size allows for flexible adjustment of the update rate based on the operating stage of the heat pump system (such as rapid learning in the early stage and fine-tuning in the later stage), avoiding the problems of weight oscillation caused by excessively large step sizes and slow learning caused by excessively small step sizes. At the same time, by combining the evaluation network error and the hidden layer output to calculate the update amount, the weight adjustment can directly target the network prediction deviation and improve the accuracy of the update.

[0082] The introduction of a nonlinear correction term for the Hadamard product into the weight update formula from the input layer to the hidden layer can adapt to the complex nonlinear mapping relationship of "state-action-cost" in the heat pump system. This makes the evaluation network's estimation of long-term costs more consistent with the actual operating law of the system and reduces the linear model. The resulting fitting error.

[0083] For each hidden layer to the output layer Input layer to hidden layer Design independent update formulas and steps to achieve hierarchical optimization of weights at different levels of the evaluation network, avoid optimization interference between levels caused by overall updates, and allow each layer weight to specifically correct its own fitting bias, thereby improving the overall learning efficiency of the evaluation network.

[0084] To minimize the squared error "The weight update rule designed for the target is an optimization strategy belonging to the gradient descent class, which can make the evaluation network error continuously decrease along the gradient direction and ensure that the error gradually converges to a minimum value; at the same time, the hierarchical update design further reduces the fluctuations in the weight iteration process, making the long-term cost estimate of the evaluation network tend to be stable and reliable."

[0085] Furthermore, step S10957, with the objective of minimizing the long-term cost estimate of the execution network, updates the execution network weights, specifically including the following steps: Step S109571: With the objective of minimizing the long-term cost estimate of the execution network, calculate the weight matrix from the hidden layer to the output layer of the execution network. : ; ; Step S109573: Calculate the weight matrix from the input layer to the hidden layer of the execution network. : ; ; in, The learning rate is used to control the step size for updating network weights. To execute the weight matrix from the hidden layer to the output layer of the network The weight update amount at step k reflects the magnitude and direction of the weight adjustment for that layer. The output vector of the hidden layer node is transposed at time k. The output vector of the hidden layer node of the evaluation network at time k is used as the parameter input across the network to participate in the network weight update. This is the transpose of the auxiliary weight matrix at time k (which, in context, is the auxiliary parameter matrix preset by the algorithm). This is the updated weight matrix for executing the network's hidden layer to output layer at time k+1, derived from the original weights. Overlapping update volume get; To execute the weight matrix from the input layer to the hidden layer of the network The weight update amount at step k; and These are the transpose of the auxiliary weight matrix of the evaluation network and the transpose of the weight matrix from the hidden layer to the output layer of the execution network, respectively, used to construct the weight update correlation logic across networks; This is the output vector of the hidden layer node at time k. To update the weight matrix from the input layer to the hidden layer at time k+1, the weights are changed from the original weights. Plus update volume get.

[0086] The evaluation network is introduced into the network weight update formula. , By establishing parameter correlation and collaborative optimization mechanism between the two networks, the weight adjustment of the execution network can be based on the error and fitting effect of the evaluation network, making the overall optimization direction of the ADP algorithm more unified and avoiding policy decoupling caused by independent updates of the two networks.

[0087] Perform the operation on the hidden layer to the output layer of the network respectively. Input layer to hidden layer Design differentiated update formulas, combined with the execution network itself. State vector By evaluating the cross-layer parameters of the network, the weight updates at different layers can be specifically adapted to their respective fitting tasks; at the same time, the learning rate... Adjusting the update step size allows for flexible adjustment of the learning rate based on the system's operational phase, preventing weight oscillations or excessively slow convergence.

[0088] With the core objective of "minimizing long-term cost estimation", the execution network weight update rule is designed. Combined with the error feedback of the evaluation network, the parameters are iterated so that the weight adjustment of the execution network always revolves around the long-term benefits of the system. This can continuously optimize the generation effect of control action vectors and gradually reduce the long-term operating costs of the heat pump system (such as energy consumption and equipment loss).

[0089] The layered and network-specific weight update steps form a closed-loop online learning process of "evaluating network error feedback → executing network weight iteration". Parameter optimization can be completed based on real-time system data without offline training, improving the actual adaptation efficiency and iteration speed of the ADP algorithm in heat pump systems.

[0090] Steps S101 to S109 are repeated to form a continuous "perception-decision-execution-learning" closed loop. Through iterative processing of massive amounts of online data, the evaluation network gradually learns to accurately predict the long-term cost of any "state-action" pair, while the execution network learns to generate current allocation actions that minimize the predicted cost. Specifically, the utility function includes a priority guarantee penalty term. When the priority flag in the system state In this case, the penalty term will dominate the optimization direction. The algorithm will thus learn autonomously and eventually solidify into a safeguard strategy: when the total household current is tight and high-priority loads need electricity, it will automatically generate actions to prioritize reducing the current of the PTC heater, thereby ensuring that the highest priority goal of household power safety is met, while continuously optimizing system energy efficiency and heating performance under other operating conditions.

[0091] like Figure 4 The image shows an example of an application scenario.

[0092] 1. During system startup (rapid heating scenario), to quickly raise the water temperature to the set temperature, the core optimization objective of the ADP algorithm is to minimize the water temperature deviation to reach the set temperature as quickly as possible while satisfying grid safety constraints. This multi-objective optimization problem is achieved through the following mechanism: the water temperature deviation in the state vector is relatively large at this time, driving the algorithm to prioritize heating performance. Simultaneously, the total household current in the state vector, along with the penalty term for total power consumption in the utility function, constitutes a constraint, ensuring that the optimization process takes grid safety into account. In this scenario, the immediate cost calculated by the evaluation network is dominated by the water temperature deviation term, but the total power consumption term still maintains an effective weight. Under the principle of minimizing long-term costs, the execution network, through its online learned strategies, outputs a current allocation action that achieves an optimal balance among rapid heating performance, system startup efficiency, and instantaneous grid impact. This manifests as follows: the variable frequency compressor obtains higher current and frequency within the grid's allowable range, serving as the primary heat source; the PTC auxiliary heater obtains some current to compensate for the insufficient performance of the heat pump in the initial startup phase, but its current value is strictly constrained by the algorithm to prevent it from superimposing with the compressor current and causing instantaneous grid overload; the water pump and fan obtain the necessary current to operate at higher speeds to ensure effective heat exchange, but their power consumption is also included in the overall optimization framework.

[0093] 2. In outdoor low-temperature (high-load scenario), the ambient temperature collected by the sensor drops significantly. This information is encoded into the state vector. Simultaneously, the water temperature in the tank may deviate from the set value due to insufficient heating capacity, increasing the water temperature deviation term in the state vector. The utility function built into the evaluation network assigns a high weight to the water temperature deviation term. When the water temperature cannot be maintained due to efficiency decay, the immediate cost increases significantly. At this point, driven by the principle of minimizing long-term costs, the execution network autonomously adjusts the structure of its output actions through its online learned strategy: the PTC heater becomes the main force, receiving most of the current and heating at full capacity; the variable frequency compressor receives some current to maintain medium-frequency operation; the fan receives less current to maintain partial fan speed operation and reduce heat exchange intensity; and the water pump receives the necessary current, operating at high capacity to ensure stable water flow.

[0094] 3. User-set temperature changes or water usage surges (dynamic load response scenario): These surges or temperature changes immediately cause a sharp increase in the deviation between the tank water temperature and the target value. This sudden change is rapidly encoded into the state vector. In the utility function of the evaluation network, the water temperature deviation term has a high weight, thus the immediate cost spikes instantly. Faced with this rapidly increasing cost, the execution network must urgently generate an action that quickly suppresses the water temperature deviation while minimizing long-term costs. The algorithm responds to the current state using its online learned strategy: the variable frequency compressor, as the most energy-efficient main heating component, increases its current to most effectively boost heating power. The execution network autonomously calculates and outputs a significantly increased compressor current command. The PTC, as an auxiliary heat source, also has its startup or current boosting decisions made autonomously by the algorithm. If the PTC was previously off, it receives some current and starts. If the total heating demand exceeds the compressor's individual boosting capacity, the execution network automatically allocates some current to the PTC for rapid power replenishment. To match the increased heat load and ensure timely heat exchange, the execution network slightly increases the current of the water pumps and fans, but their priority and increase are lower than those of direct heating components to optimize overall energy efficiency. When the system begins to respond and the rate of water temperature change changes, the state vector is updated accordingly. The ADP algorithm re-evaluates the cost and optimizes actions in each control cycle, thus forming a dynamic and smooth load tracking capability. 4. Nighttime Heat Storage: To operate at the lowest cost while maximizing overall system thermal energy reserves, the time information in the state vector allows the algorithm to identify the current nighttime heat storage period. Simultaneously, the core optimization objective of minimum-cost heat storage in this scenario is embedded into the utility function through algorithm design: during nighttime, the weight of the water temperature deviation term in this function decreases (allowing for slow heat storage), and the penalty coefficient for total energy consumption cost also decreases accordingly (lower off-peak electricity prices), but the weight of the energy efficiency term relatively increases. In this scenario, the immediate cost calculated by the evaluation network is primarily driven by energy consumption and energy efficiency. Driven by the principle of minimizing long-term costs, the execution network explores and learns the optimal actions: the variable frequency compressor allocates most of the current; the PTC does not allocate current and is turned off; the water pump allocates a small amount of current and operates at low speed, just meeting the heat exchange requirements; the fan allocates a small amount of current and operates at low speed.

[0095] 5. Partial Load (Suitable Ambient Temperature, Low Demand): Under this condition, the core optimization goal of the system is to achieve maximum operating energy efficiency while precisely maintaining the set temperature. The strategy formation logic of the ADP algorithm in this scenario is as follows: The state vector accurately reflects the characteristics of this scenario: the ambient temperature is within a suitable range, the water tank temperature deviates very little from the target value, and the total current demand is low. This provides objective conditions for the system to pursue ultimate energy efficiency. In this scenario, in the utility function built into the evaluation network, the water temperature deviation term has a very small value due to high control precision, and the optimization dominance shifts to the energy efficiency optimization term. The cost function designed by the algorithm rewards high-efficiency operating states, that is, for actions that produce the same amount of heat but consume less total power, the long-term cost estimate is lower. The optimal action calculated and output by the execution network to minimize the cost is manifested as: allocating most of the current to the most energy-efficient variable frequency compressor, allowing it to operate in the high-efficiency range; allocating zero current to the PTC, shutting it down; and allocating only a small amount of current necessary to meet the minimum heat exchange requirements to the water pump and fan, allowing them to operate at low speed.

[0096] The present invention also proposes an electronic device, comprising: at least one processor; a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform an air source heat pump current distribution method.

[0097] The present invention also proposes a storage medium storing a computer program, which, when executed by a processor, implements an air source heat pump current distribution method.

[0098] In the above embodiments, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0099] Optionally, the processor executes the method steps described in the above embodiments based on the program code stored in the storage medium.

[0100] Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0101] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be implemented using general-purpose computing devices, which can be centralized on a single computing device or distributed on a network of multiple computing devices.

[0102] Alternatively, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computing device. Furthermore, in some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this disclosure is not limited to any particular combination of hardware and software.

[0103] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the inventive concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for current distribution in an air source heat pump, characterized in that, include: A state vector is constructed based on heat pump operating status parameters and environmental parameters. The heat pump operating status parameters include total current, operating current, operating voltage, water temperature, heat exchanger inlet and outlet temperatures, and equipment operating parameters. The environmental parameters include indoor and outdoor temperatures and time parameters. An execution network based on an adaptive dynamic programming algorithm generates control action vectors from the state vectors. The evaluation network based on the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination from the state vector and the control action vector. The long-term cost estimate is minimized, and the action with the smallest long-term cost estimate is selected as the final action instruction. Current allocation is performed based on the final action command, and the state vector, execution network weights, and evaluation network weights are iteratively updated.

2. The air source heat pump current distribution method according to claim 1, characterized in that, An execution network based on an adaptive dynamic programming algorithm generates control action vectors from the state vectors, specifically including: The input of the hidden layer node of the execution network is obtained by weighted summation of the components of the state vector and the connection weights from the input layer to the hidden layer of the execution network; The input of the execution network hidden layer node is processed by Sigmoid activation to obtain the execution network hidden layer node. Output of hidden layer nodes in the line network; The output of the execution network output layer node is obtained by weighted summation of the output of the execution network hidden layer node and the connection weight from the execution network hidden layer to the output layer. The outputs of all nodes in the execution network output layer constitute the control action vector, which corresponds to the current distribution scheme of the compressor, PTC heater, water pump and fan.

3. The air source heat pump current distribution method according to claim 1, characterized in that, The evaluation network based on the adaptive dynamic programming algorithm generates a long-term cost estimate for each state-action combination from the state vector and the control action vector, specifically including: The input of the hidden layer node of the evaluation network is obtained by weighted summation of the components of the state vector and the connection weights from the input layer to the hidden layer of the evaluation network. The input of the hidden layer node of the evaluation network is processed by Sigmoid activation to obtain the output of the hidden layer node of the evaluation network. The output of the evaluation network output layer node is obtained by weighted summing the output of the evaluation network hidden layer node and the connection weight from the evaluation network hidden layer to the output layer. In this system, the output layer nodes of the evaluation network for all nodes output the long-term cost estimate corresponding to each state-action combination.

4. The air source heat pump current distribution method according to any one of claims 1 to 3, characterized in that, The long-term cost estimate is minimized, and the action with the smallest long-term cost estimate is selected as the final action instruction. Specifically, this includes: Random perturbations are added to the control action vector to generate multiple candidate actions; Based on the principle of minimizing long-term costs, the action with the smallest estimated long-term cost is selected as the final action instruction.

5. The air source heat pump current distribution method according to any one of claims 1 to 3, characterized in that, Based on the final action command, current is allocated, and the state vector, execution network weights, and evaluation network weights are iteratively updated, specifically including: Based on the final motion command controller, the control motion vector is parsed into the drive signal of each actuator and then sent out. Based on the driving signal, the actuator drives the corresponding component to run and iteratively updates the state vector; The system learns online and iteratively updates the execution network weights and evaluation network weights based on the immediate effects of the actions performed.

6. The air source heat pump current distribution method according to claim 5, characterized in that, Based on the immediate effects after action execution, online learning is performed and the execution network weights and evaluation network weights are iteratively updated. Specifically, this includes: Calculate the utility function at time k based on the immediate effect after the action is performed; The network error is evaluated based on the utility function at time k. The evaluation network weights are updated with the objective of minimizing the squared error of the evaluation network. The execution network weights are updated with the goal of minimizing the long-term cost estimate of the execution network.

7. The air source heat pump current distribution method according to claim 6, characterized in that, The evaluation network weights are updated with the objective of minimizing the squared error of the evaluation network error, specifically including: With the objective of minimizing the squared error of the evaluation network, the weight matrix from the hidden layer to the output layer of the evaluation network is calculated. : ； ； Calculate the weight matrix from the input layer to the hidden layer of the evaluation network. : ； ； in, To evaluate the network learning rate; To evaluate the network's error signal at time k; for In the The amount of weight update per step; This is the transpose of the output vector of the hidden layer node in the evaluation network at time k. This is the weight matrix from the hidden layer to the output layer of the network at time k. This is the updated weight matrix from the hidden layer to the output layer for evaluating the network at time k+1. for In the The amount of weight update per step; This is the transpose of the network input vector at time k; This is a nonlinear correction term based on the hidden layer output; This is the weight matrix from the input layer to the hidden layer of the network at the updated time k+1.

8. The air source heat pump current allocation method according to claim 6, characterized in that, with the goal of minimizing the long-term cost estimate of the execution network, the execution network weights are updated, specifically including: With the objective of minimizing the squared error of the evaluated network, the weight matrix from the hidden layer to the output layer of the execution network is calculated. : ; ; Calculate the weight matrix from the input layer to the hidden layer of the execution network. : ； ； in, To implement the network learning rate; To execute the weight matrix from the hidden layer to the output layer of the network The weight update amount at step k; The output vector of the hidden layer node is transposed at time k. This is the output vector of the hidden layer node in the network at time k. This is the transpose of the auxiliary weight matrix at time k; This is the updated weight matrix from the hidden layer to the output layer of the network at time k+1. To execute the weight matrix from the input layer to the hidden layer of the network The weight matrix at step k; and These are the transpose of the auxiliary weight matrix for evaluating the network and the transpose of the weight matrix from the hidden layer to the output layer of the execution network, respectively. This is the weight matrix from the input layer to the hidden layer of the network after the update at time k+1.

9. An air source heat pump current distribution system, characterized in that, include: The sensing module includes a current sensor, a voltage sensor, and a temperature sensor, used to collect system operating status and environmental parameters in real time. The heat pump operating status parameters include total current, operating current, operating voltage, water temperature, heat exchanger inlet and outlet temperatures, and equipment operating parameters. The environmental parameters include indoor and outdoor temperatures and time parameters. The decision module includes a processor that runs an adaptive dynamic programming algorithm, constructs a state vector based on heat pump operating state parameters and environmental parameters, generates a control action vector from the state vector using an execution network based on the adaptive dynamic programming algorithm, and generates a long-term cost estimate for each state-action combination using an evaluation network based on the adaptive dynamic programming algorithm, minimizes the long-term cost estimate, selects the action with the smallest long-term cost estimate as the final action instruction, and iteratively updates the state vector, execution network weights, and evaluation network weights. The execution module, including a frequency converter and a solid-state relay, executes the final action command issued by the decision module, thereby driving the corresponding component to operate.

10. A storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the air source heat pump current distribution method according to any one of claims 1 to 8.

Images