Building energy consumption multi-region collaborative control method, device and equipment and storage medium

Abstract

The application provides a building energy consumption multi-region collaborative control method, device, equipment and storage medium, which can be applied to the fields of artificial intelligence and building energy consumption control technology. The method comprises the following steps: processing temperature evaluation information and energy consumption evaluation information of a sample space region in a sample building to obtain an initial evaluation result of a sample environment temperature within a preset time period, the temperature evaluation information being determined by sample temperature information of the sample space region, and the energy consumption evaluation information being determined by sample power information, sample energy consumption information and cost information of the sample space region; obtaining a control intelligent agent based on sample information, the initial evaluation result, a loss function of a respective strategy model of the sample space region and a sample associated region, and a fusion strategy; obtaining action information of a valve opening degree of a control device based on state information and associated region information of a space region in the building, performing a control operation on the valve opening degree based on the action information, and obtaining energy consumption control information of the space region.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and building energy consumption control technology, specifically to a method, apparatus, equipment, and storage medium for multi-zone collaborative control of building energy consumption. Background Technology

[0002] In building energy consumption control technologies, the main approach is to use models for predictive control, such as Model Predictive Control (MPC), or to combine Centralized Reinforcement Learning (CRL) with a shared strategy to control ventilation equipment in all areas of a building, thereby achieving global control of building energy consumption.

[0003] However, due to differences in orientation, building envelope, indoor heat sources (personnel, equipment), usage functions, and schedules, different areas within a building exhibit significant differences in thermodynamic characteristics. Shared strategies struggle to perceive and adapt to these thermal differences, resulting in poor overall energy efficiency. Relying on a single strategy model necessitates changes to the input and output dimensions of the entire strategy model when the building's spatial layout changes, requiring retraining and increasing system consumption. Furthermore, centralized reinforcement learning fails to consider the interrelationships between areas within the building, making it difficult to generate effective control strategies and thus unable to meet the energy control needs of practical applications. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method, apparatus, equipment, storage medium and program product for improving multi-zone coordinated control of building energy consumption.

[0005] According to a first aspect of the present invention, a multi-regional collaborative control method for building energy consumption is provided, comprising: processing temperature assessment information and energy consumption assessment information of sample spatial regions in a sample building to obtain an initial assessment result of the sample ambient temperature of the sample spatial regions within a preset time period, wherein the temperature assessment information is determined by the sample temperature information of the sample spatial regions, and the energy consumption assessment information is determined by the sample power information, sample energy consumption information, and cost information of the sample spatial regions; obtaining a control agent based on the sample information, the initial assessment result, the loss function of the strategy model of each of the sample spatial regions and the associated sample regions, and a fusion strategy; using the control agent based on the state information of the spatial regions in the building and the associated region information to obtain action information for controlling the valve opening of control devices in the spatial regions, and performing control operations on the valve opening based on the action information to obtain energy consumption control information of the spatial regions.

[0006] The second aspect of the present application provides a building energy consumption multi-region collaborative control device, comprising: an information processing module, configured to process temperature evaluation information and energy consumption evaluation information of a sample space region in a sample building to obtain an initial evaluation result of a sample environment temperature of the sample space region in a preset time period, wherein the temperature evaluation information is determined by sample temperature information of the sample space region, and the energy consumption evaluation information is determined by sample power information, sample energy consumption information and cost information of the sample space region; an agent obtaining module, configured to obtain a control agent based on sample information, the initial evaluation result, a loss function of a respective policy model of the sample space region and a sample associated region, and a fusion policy; and a control module, configured to obtain action information for controlling a valve opening degree of a control device in the space region based on state information and associated region information of the space region in the building by using the control agent, and perform a control operation on the valve opening degree based on the action information to obtain energy consumption control information of the space region.

[0007] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method.

[0008] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0009] The fifth aspect of the present application further provides a computer program product comprising a computer program or instructions, wherein the computer program or instructions are executed by a processor to implement the steps of the method.

[0010] According to the building energy consumption multi-region collaborative control method, device, equipment, storage medium and program product provided by the present application, since the loss function of the sample associated region policy model is fused in the training phase of the control agent, the joint evaluation mechanism of the associated region shared information and the local region is introduced, the thermal correlation between different regions in the building is considered, the control agent satisfies more collaborative and flexible decision output, and different space regions have respective policy models to avoid retraining of the model as a whole under the changed state, thereby reducing the system resource consumption. In the application phase, by fusing the associated region information, the control agent can perceive the multi-dimensional information of a larger range of the space region, so that the heat transfer and energy consumption influence between different regions in the building can be considered when making decisions, the global energy consumption degradation caused by ignoring the thermal differences between different regions is avoided, the valve opening degree of the control device is adjusted by fusing the associated region information, the energy consumption cost of the whole building is significantly reduced under the premise of meeting the environmental temperature comfort. BRIEF DESCRIPTION OF DRAWINGS

[0011] The above and other objects, features and advantages of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0012] Figure 1 An application scenario diagram of the building energy consumption multi-region collaborative control method, device, equipment, storage medium and program product according to the embodiments of the present application is shown;

[0013] Figure 2 A flowchart of the building energy consumption multi-region collaborative control method according to the embodiments of the present application is shown;

[0014] Figure 3 An example schematic diagram of the building energy consumption multi-region collaborative control process according to the embodiments of the present application is shown;

[0015] Figure 4 A structural block diagram of the building energy consumption multi-region collaborative control device according to the embodiments of the present application is shown;

[0016] Figure 5 A block diagram of an electronic device suitable for implementing the building energy consumption multi-region collaborative control method according to the embodiments of the present application is shown. DETAILED DESCRIPTION

[0017] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. It is to be understood, however, that the description is merely exemplary of the present application, and is not intended to limit the scope of the present application. In the following detailed description of the embodiments of the present application, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present application.

[0018] The terms used herein are merely used to describe specific embodiments, and are not intended to limit the present application. The terms "include", "comprise" and the like used herein indicate the presence of the described features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0019] All terms used herein, including technical and scientific terms, have the meanings commonly understood by one of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.

[0020] In the case of using expressions such as "at least one of A, B, and C", it generally should be interpreted to include any of A, B, or C individually, as well as a combination of A, B, and C, unless the context clearly indicates otherwise (e.g., "at least one of A, B, and C" would include A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together).

[0021] For building energy consumption control, in the related art, mainly model is used for prediction control, or global control of building energy consumption is realized by combining centralized reinforcement learning and shared strategy. However, due to different orientations, envelope structures, indoor heat sources (personnel, equipment), use functions and time tables, the thermodynamic characteristics of different regions in the building are significantly different, and the shared strategy is difficult to perceive and adapt to these thermal differences, resulting in poor overall energy consumption effect; relying on the same strategy model, when the building space layout changes, the input and output dimensions of the whole strategy model need to be changed, and need to be retrained, increasing system consumption; centralized reinforcement learning lacks consideration of the relevance of internal regions of the building, and it is difficult to generate effective control strategies, and cannot meet the needs of energy consumption control in actual applications.

[0022] Therefore, the present application provides a building energy consumption multi-region collaborative control method, device, equipment, storage medium and program product, the method comprising: processing temperature evaluation information and energy consumption evaluation information of a sample space region in a sample building to obtain an initial evaluation result of a sample environment temperature of the sample space region in a preset time period, wherein the temperature evaluation information is determined by sample temperature information of the sample space region, and the energy consumption evaluation information is determined by sample power information, sample energy consumption information and cost information of the sample space region; based on the sample information, the initial evaluation result, the loss function and the fusion strategy of the respective strategy models of the sample space region and the sample associated region, a control agent is obtained; using the control agent based on the state information and the associated region information of the space region in the building, action information for controlling the valve opening of the control device in the space region is obtained, and a control operation is performed on the valve opening based on the action information to obtain energy consumption control information of the space region.

[0023] According to the embodiment of the present application, since the training phase of the control agent fuses the loss function of the sample correlation region strategy model, the joint evaluation mechanism of the correlation region shared information and the local region is introduced, the thermal correlation between different regions in the building is considered, so that the control agent satisfies more collaborative and flexible decision output, and different spatial regions have their own strategy model to avoid retraining of the whole model under the changed state, reducing the system resource consumption. In the application stage, by fusing the correlation region information, the control agent can perceive the multi-dimensional information of a larger range of spatial regions, so that the heat transfer and energy consumption influence between different regions in the building can be considered when making decisions, avoiding the global energy consumption degradation caused by ignoring the thermal differences between different regions, realizing the fusion of correlation region information to adjust the valve opening of the control equipment, and significantly reducing the overall energy consumption cost of the building under the premise of meeting the environmental temperature comfort.

[0024] Figure 1 An application scenario diagram of the building energy consumption multi-region collaborative control method, device, equipment, storage medium and program product according to the embodiment of the present application is shown.

[0025] As shown in Figure 1 , the application scenario 100 according to the embodiment can include a terminal device 101, a data acquisition device 102, a server 103 and a network 104. The network 104 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 104 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0026] The terminal device 101 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0027] The data acquisition device 102 can be used to acquire multi-dimensional information in the building and the building space in real time, and the device types include but are not limited to temperature acquisition devices, wind speed acquisition devices, energy consumption acquisition devices, power acquisition devices, load acquisition devices and operation information of different air conditioning devices.

[0028] The user can use the terminal device 101 to interact with the server 103 through the network 104 to receive or send messages, etc. For example, the received energy consumption result information drives the device to execute the corresponding instructions.

[0029] The server 103 can be a server or an intelligent agent for controlling building energy consumption, such as processing real-time data from the data acquisition device 102, making decisions according to a preset strategy (such as an energy saving strategy) or a strategy model; or issuing adjustment instructions to the terminal device 101.

[0030] It should be noted that the building energy consumption multi-region collaborative control method provided by the embodiment of the present application can be generally executed by the server 103. Correspondingly, the building energy consumption multi-region collaborative control device provided by the embodiment of the present application can be generally arranged in the server 103. The building energy consumption multi-region collaborative control method provided by the embodiment of the present application can also be executed by a server or a server cluster different from the server 103 and capable of communicating with the terminal device 101 and / or the server 103. Correspondingly, the building energy consumption multi-region collaborative control device provided by the embodiment of the present application can also be arranged in a server or a server cluster different from the server 103 and capable of communicating with the terminal device 101 and / or the server 103.

[0031] It should be understood that Figure 1 The number of terminal devices, data acquisition devices, networks and servers in the system is only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

[0032] Figure 2 A flowchart of the building energy consumption multi-region collaborative control method according to the embodiment of the present application is shown.

[0033] As Figure 2 shown, the building energy consumption multi-region collaborative control method of this embodiment includes operations S210-S230.

[0034] In operation S210, the temperature evaluation information and the energy consumption evaluation information of the sample space region in the sample building are processed to obtain the initial evaluation result of the sample environment temperature of the sample space region in the preset time period, wherein the temperature evaluation information is determined by the sample temperature information of the sample space region, and the energy consumption evaluation information is determined by the sample power information, the sample energy consumption information and the cost information of the sample space region.

[0035] In operation S220, based on the sample information, the initial evaluation result, the loss function of the respective policy model of the sample space region and the sample associated region, and the fusion policy, the control agent is obtained.

[0036] In the embodiments of the present application, the sample building can be a commercial building in a historical time range, serving as a sample in the training phase. The sample space region can be a different functional space or room in the sample building. After the current sample action is performed, a plurality of temperature information at a later time can be predicted, and the plurality of temperature information can be evaluated to obtain temperature evaluation information. A plurality of energy consumption related information obtained after the current sample action is performed can be fused to obtain energy consumption evaluation information. The building energy consumption information evaluated by the current action information. The initial evaluation result can be the result after fusing the temperature evaluation information and the energy consumption evaluation information. The sample temperature information can include the latter sample temperature information and the reference temperature information corresponding to the latter sample temperature information. The reference temperature information can be the expected temperature information determined according to actual needs or tests.

[0037] In the embodiments of the present application, the sample power information can be the power information of the air conditioning equipment or related equipment in the sample space region, and the sample energy consumption information can be the power consumption information of the air conditioning equipment or related equipment in the sample space region. The cost information can be the cost information of the energy consumed by the sample building in the historical time period.

[0038] In the embodiments of the present application, the sample information can include sample state information and sample action information. The initial evaluation result can be a reward result obtained after the sample space region performs a current control action at a current time. The sample associated region can be a region associated with the sample space region with an association degree greater than a preset condition. The association degree can be determined based on a plurality of association information. The sample space region and the sample associated region have respective policy models, and different policy models have corresponding loss functions in the training process. The fusion policy can be a policy for fusing a plurality of output results of different policy models.

[0039] For example, after the current sample action is performed, a plurality of temperature information and a plurality of energy consumption related information at a later time can be predicted, and weights corresponding to the plurality of temperature information and the plurality of energy consumption related information can be determined according to actual conditions or importance degrees, so as to weight the plurality of temperature information and the plurality of energy consumption related information respectively by using the weights, to obtain a reward result of performing the current sample action as the initial evaluation result.

[0040] For example, after obtaining the initial evaluation result, the associated region of the sample space region can be determined according to the association degree, and the policy model of the sample space region and the policy model of the sample associated region are obtained. Based on the sample state information, the sample action information, the initial evaluation result, and the loss function of each different policy model obtained after fusing the associated region information, the initial control agent is trained to obtain the control agent.

[0041] In operation S230, the control agent is used to obtain action information for controlling the valve opening degree of the control device in the space region based on the state information and the associated region information of the space region in the building, and perform a control operation on the valve opening degree based on the action information to obtain energy consumption control information of the space region.

[0042] In embodiments of the present application, the space region can be a specific functional space or room in a building in an actual application scenario. The state information can be a variety of information collected in real time by different data collection devices for the space region, including temperature information (such as indoor temperature and outdoor temperature), air speed information of the air conditioning device, energy consumption information, occupancy information (such as personnel occupancy information and device occupancy information), heat load information, and power information of the air conditioning device.

[0043] In embodiments of the present application, the associated region information can include associated state information and associated action information of the associated region. The control device can be a device for controlling the ambient temperature of the space region. The valve opening degree can be the opening degree of the valve of the air conditioning device, which can be represented by an opening level. The energy consumption control information can be the overall energy consumption control result and the corresponding temperature control result of the space region in a preset time period.

[0044] For example, after the control agent is trained, the control agent can be used to process the acquired temperature information, air speed information, energy consumption information, occupancy information, heat load information, power information, associated state information, and associated action information to obtain current action information, and then use the current action information to perform a control operation on the valve opening degree to obtain the overall energy consumption control result and the corresponding temperature control result of the space region in a preset time period.

[0045] According to embodiments of the present application, since the training phase of the control agent fuses the loss function of the sample associated region strategy model, introduces a joint evaluation mechanism of the associated region shared information and the local region, considers the thermal correlation between different regions in the building, the control agent satisfies more collaborative and flexible decision output, and different space regions have their own strategy model to avoid retraining of the model as a whole under changing conditions, thereby reducing system resource consumption. In the application stage, by fusing the associated region information, the control agent can perceive more multi-dimensional information in a larger range of the space region, so that the heat transfer and energy consumption influence between different regions in the building can be considered when making decisions, avoiding ignoring the thermal differences between different regions to cause global energy consumption degradation, realizing the fusion of the associated region information to adjust the valve opening degree of the control device, and significantly reducing the overall energy consumption cost of the building under the premise of meeting the comfort of the ambient temperature.

[0046] According to an embodiment of the present application, based on state information of a space region in a building and associated region information, action information for controlling a valve opening degree of a control device in the space region is obtained, including: fusing the state information with associated state information in the associated region information to obtain fused state information; and mapping the fused state information to the action information based on a mapping strategy, wherein the mapping strategy includes any one of a deterministic strategy and a random strategy.

[0047] In an embodiment of the present application, the fused state information can be state information obtained after the state information and the associated state information are combined, fused or spliced. The specific content of the associated state information is the same as or similar to that of the state information, which is not described in detail here. The mapping strategy can be a plurality of strategies for mapping the state information (or the associated state information) to the action information (or the associated action information). The deterministic strategy can be a strategy for one-to-one mapping different state information to corresponding action information based on a specific mapping relationship; and the random strategy can be a strategy for mapping the state information to the action information in real time and dynamically according to a preset probability.

[0048] It can be understood that the fused state information allows the control agent to make a decision based not only on its own (space region) information, but also on relevant information of the associated region, thereby making a more global and collaborative decision.

[0049] For example, the state information and the associated state information can be spliced in the feature dimension of the state vector to obtain spliced features as the fused state information.

[0050] For example, using an attention mechanism, the control agent can take the state information of the space region A as a query, and the associated state information of the associated region B as a key and a value, so as to calculate the correlation degree (attention weight) between A and B, and then use the weight to perform weighted summation on the information of B, and then fuse it with the state of A itself to obtain the fused state information.

[0051] According to an embodiment of the present application, by dynamically measuring the importance of different neighbor information, for example, the space region A can pay more attention to the state of the associated region B with the strongest thermal coupling in the upwind direction, and through the attention weight, it can be analyzed that the control agent refers to the associated regions of different correlation degrees when making a decision.

[0052] In an embodiment of the present application, after obtaining the fused state information, it can be mapped to specific action information through a policy network for action control.

[0053] For example, for the deterministic strategy, the strategy network can be a deterministic function, which can output a determined and optimal action information by inputting the fusion state information.

[0054] For the random strategy, the strategy network can output a probability distribution of the action, and the control agent can sample the action according to the probability distribution.

[0055] In the related art, each area controller makes a decision in isolation, which can easily lead to local optimization and global conflict. For example, while space area A is cooling, adjacent area B is heating, the energy is offset, and great waste is caused.

[0056] For the state information, taking the space area i of the building as an example (i=1,...,M), the state information can be determined by the following formula (1):

[0057] (1);

[0058] wherein, can be the expected temperature information of the space area i, can be the indoor temperature of the space area i, can be the wind speed information of the air conditioning equipment of the space area i, can be the energy consumption information of the space area i, can be the heat load information of the space area i, can be the power information of the air conditioning equipment of the space area i. can be the outdoor temperature information. t is the time t.

[0059] It can be understood that the content and determination method of the associated action information of the associated area are the same as those of the above formula (1), and details are not described here. After obtaining the state information and the associated state information, the two kinds of state information can be fused to obtain the fusion state information.

[0060] According to the embodiment of the present application, by fusing the associated state information of the associated area, the decision of the control agent has a system view, which can actively avoid the strategy conflict with the neighbors, coordinate the equipment operation mode, reduce the energy consumption, and reduce the energy consumption from the whole building level.

[0061] According to an embodiment of the present application, the building energy consumption multi-region collaborative control method further comprises: determining a correlation region from the plurality of adjacent regions when a correlation degree between the space region and the plurality of adjacent regions meets a preset condition, wherein the correlation degree can be determined based on distance information, device information, and envelope information between the space region and the plurality of adjacent regions; and obtaining correlation state information and correlation action information of the correlation region as correlation region information.

[0062] In an embodiment of the present application, the correlation region can be determined from the plurality of regions adjacent to the space region based on the correlation degree. The distance information can be a planar distance or a spatial distance between the space region and the adjacent region. The device information can be a relationship between an air conditioning device of the space region and an air conditioning device of the adjacent region, for example, whether the air conditioning systems or air conditioning devices between the two regions are independent of each other. The envelope information can include thermal performance information, material information, and thickness information of the building envelope between the space region and the adjacent region.

[0063] For example, a distance threshold corresponding to the distance information, a device correlation condition corresponding to the device information, and a structure condition corresponding to the envelope information can be determined respectively, so as to determine which regions in the plurality of adjacent regions meet at least one of the distance threshold, the device correlation condition, and the structure condition, and determine the adjacent region as the correlation region when at least one of them is met.

[0064] For example, geometric information and attribute information of the space region and all adjacent regions can be extracted from a building information model or a plan, including but not limited to geometric boundary coordinates of each region; position, type (such as load-bearing wall, partition wall), thickness of the wall, and the space separated by the wall; room function, ventilation system partition identification, etc.

[0065] For the correlation region of the space region determined based on the correlation degree, taking the distance threshold as an example, the Euclidean distance between the respective centroids of the space region and the adjacent region can be calculated, or the minimum distance between the boundaries of the two regions can be calculated, for example, the distance threshold is 2 meters, and the adjacent region can be determined as the correlation region when the spatial distance or the horizontal distance is less than 2 meters.

[0066] For example, wall data is analyzed to determine whether two regions share a complete wall, a wall with a door or window, or a completely independent partition. The structure condition can be "sharing a wall with a door", and the adjacent region is determined as the correlation region. Alternatively, mechanical, electrical, and piping information in the building information model is checked to determine whether two regions belong to the same ventilation system partition, for example, the device condition meets "sharing a set of ventilation systems", and the two regions are determined as the correlation region.

[0067] According to the embodiment of the present application, the fuzzy space perception is converted into accurate and calculable data through the multiple conditions and threshold judgments in the correlation degree, the large-scale and multi-condition correlation analysis can be realized, the correlation area is calculated for all space areas of the whole building at the same time, and the processing efficiency and accuracy are improved.

[0068] According to the embodiment of the present application, the action information includes multiple opening degree operation information of the valve; the control operation is performed on the valve opening degree based on the action information, and the energy consumption control information of the space area is obtained, including: determining the target operation information indicated by the action information and the speed information of the control device, wherein the target operation information is associated with the speed information; performing the control operation on the valve opening degree based on the target operation information and the speed information to obtain the energy consumption control information, and the energy consumption control information includes the energy consumption result and the environmental temperature information corresponding to the energy consumption result.

[0069] In the embodiment of the present application, the multiple opening degrees correspond to multiple levels of the opening degree of the valve of the control device, for example, the percentage of the opening of the valve, 0% represents complete closing, and 100% represents complete opening. The action information can be the information of multiple control actions pre-stored in the system database. The speed information can be the speed level or speed value that the control device can perform. The target operation information can be the current control action determined from the multiple action information according to the speed level or speed value of the control device at the current moment, to control the valve opening degree. The energy consumption result can indicate the energy consumption information corresponding to the action information at the current moment, which can be determined in combination with the energy price. The environmental temperature can be the indoor environmental temperature of the space area obtained after the intelligent agent performs the control action. The energy consumption control information can also include the air flow speed and the user's body feeling state.

[0070] For example, each space area (i=1,...,M) can use different fan coil controllers to calculate the air outlet speed level of the air conditioning device, which can be denoted as , that is, the control intelligent agent of the space area i=1,...,M can select the action information (ai(t)) according to different strategies (pi(t)) at the moment t∈{0,1,2,3}. The air outlet speed level and the action information can be obtained through the following formulas (2)-(3):

[0071] (2);

[0072] (3);

[0073] , wherein can be the state information of the space area i at the moment t.

[0074] ​Each spatial region controls the decision output of the agent, which can convert its "thinking" result (select an abstract action) into specific control instructions for physical control devices, which can be fan coil units (FCUs). The action information set can be mapped to multiple operating levels of the fan (such as off, low speed, medium speed, and high speed), realizing a control closed loop from the information space to the physical space.

[0075] According to an embodiment of the present application, the sample temperature information includes post-temperature information and reference temperature information corresponding to the post-temperature information; the building energy consumption multi-region collaborative control method further includes: obtaining temperature information after a control operation on the valve opening based on the sample action information at the current time as the post-temperature information; and obtaining temperature evaluation information based on the post-temperature information and the reference temperature information.

[0076] In an embodiment of the present application, the post-temperature information can be temperature information of the sample space region at the next time after the current action is executed, which can be denoted as , which can represent a result generated after the initial control agent executes an action (such as adjusting the FCU) at time t, and is used to evaluate the effectiveness of the current control action.

[0077] In an embodiment of the present application, the reference temperature information can be expected temperature information of the sample space region at the next time after the current action is executed, which can be denoted as , which can be a fixed value, or can dynamically change according to a schedule, a personnel density, and the like. The temperature evaluation information can be used to evaluate the deviation between the temperature information of the sample space region at the next time after the current action is executed and the reference temperature information.

[0078] For example, the square of the difference between the post-temperature information and the reference temperature information can be calculated, and the square term can realize that whether the temperature is too high or too low, it will be punished, and a greater penalty will be given to a greater deviation, thereby encouraging the control agent to stably maintain the environmental temperature near the set value.

[0079] For example, if the difference between the indoor environmental temperature and the reference temperature information is large, the corresponding penalty is also large (i.e., the reward is a large negative number), indicating that the currently executed control action is unreasonable. If the difference between the indoor environmental temperature and the reference temperature information is small, the value corresponding to the temperature evaluation information is small, and the penalty is also small, indicating that the executed control action is reasonable.

[0080] According to an embodiment of the present application, the building energy consumption multi-region collaborative control method further includes: obtaining energy consumption evaluation information of the sample action information at the current time based on the combined information obtained based on the combined sample power information and the sample energy consumption information and the cost information.

[0081] In the embodiments of the present application, the sample energy consumption information can include energy consumption information (which can be denoted as ) of a sample space area fan coil and energy price information (which can be denoted as ), such as time-of-use electricity price and real-time electricity price. The energy consumption evaluation information can be a result obtained by combining, splicing or adding the sample power information, the energy consumption information and the energy price information.

[0082] For example, the sample power information and the energy consumption information can be combined to obtain combined information, and the combined information can be multiplied with the energy price information to obtain an energy consumption evaluation result of performing the sample action at the current time.

[0083] It can be understood that if the control action causes the fan coil and other air conditioning equipment to consume a large amount of electricity, and the high electricity price or low priority period in the time-of-use electricity price corresponding to the electricity, the value corresponding to the energy consumption evaluation result is relatively large, the penalty is relatively large, and it is indicated that the current control action is unreasonable. If the energy consumption is very low, or the energy is consumed in the low electricity price period, the value corresponding to the energy consumption evaluation result is relatively small, the penalty is relatively small, and it is indicated that the current control action is relatively reasonable.

[0084] According to the embodiments of the present application, the temperature evaluation information and the energy consumption evaluation information of the sample space area in the sample building are processed to obtain an initial evaluation result of the sample environment temperature of the sample space area in a preset time period, including: respectively weighting the temperature evaluation information by using a temperature weight and weighting the energy consumption evaluation information by using an energy consumption weight to obtain a plurality of evaluation information corresponding to a plurality of time in the preset time period; and combining the plurality of evaluation information to obtain the initial evaluation result.

[0085] In the embodiments of the present application, the temperature weight and the energy consumption weight can be dynamically generated weight values according to actual conditions or test results, which can be respectively denoted as λ1 and λ2, and can be used to balance the importance between the two optimization objectives of temperature comfort and energy consumption cost. The plurality of evaluation information can be evaluation results corresponding to a plurality of time periods respectively.

[0086] For example, after obtaining the temperature evaluation information and the energy consumption evaluation information, the temperature evaluation information can be weighted by using the dynamically generated temperature weight to obtain a first weighted result, and the energy consumption evaluation information can be weighted by using the energy consumption weight to obtain a second weighted result. It can be understood that the first weighted result and the second weighted result can each include a plurality of sub-results corresponding to different time. Thus, the first weighted result and the second weighted result can be combined, spliced or added to obtain the initial evaluation result.

[0087] After obtaining the temperature evaluation information and the energy consumption evaluation information, the temperature evaluation information and the energy consumption evaluation information can be combined to obtain the initial evaluation result.

[0088] For example, the initial control agent can monitor the sample state information at the next moment after performing the current control operation to evaluate the effectiveness of the control operation on the control device in the sample space region i', determine evaluation information at a moment (e.g., moment t) in a preset period The calculation method of the t-moment evaluation information is shown in the following formula (4):

[0089] (4);

[0090] wherein, may represent the energy consumption information of the fan coil in the sample space region i', may be the refrigeration or heating power of the air conditioning system in the sample space region i'. may be the temperature evaluation information, may be the energy consumption evaluation information. may represent the energy price information corresponding to the sample building, for example, time-of-use electricity price or real-time electricity price. λ1 and λ2 may be the temperature weight and the energy consumption weight, respectively.

[0091] It can be understood that the initial evaluation result may represent the immediate reward that the agent (e.g., the FCU controller) located in the sample space region i' can obtain at the moment t, and the value is negative, which can indicate that it is a penalty item (Penalty). The goal of the control agent can be to minimize the absolute value of this negative value (i.e., the minimum penalty and the maximum reward) by adjusting the strategy.

[0092] After obtaining a plurality of evaluation information corresponding to a plurality of moments in a preset period, the plurality of evaluation information can be summed to obtain the initial evaluation result of the sample space region i'. The initial evaluation result can be calculated by the following formula (5):

[0093] (5);

[0094] wherein, N is the total number of moments in the preset period.

[0095] According to the embodiments of the present application, the balance between the environmental temperature and the energy saving efficiency is realized through the respective weights corresponding to the temperature and the energy consumption, and the control agent trained through the iterative optimization of the initial control agent can select the action sequence with the minimum value of the initial evaluation result (total penalty), thereby improving the intelligent control efficiency and precision of the building energy consumption.

[0096] According to an embodiment of the present application, the loss function comprises a first loss function of the sample space region policy model and a second loss function of the sample associated region policy model; the building energy consumption multi-region collaborative control method further comprises: obtaining the first loss function based on the reward information of the sample action information, the current value information, the subsequent value information, the weight of the current value information and the weight of the subsequent value information; obtaining the second loss function based on the reward information of the sample associated action information in the sample associated region, the current associated value information, the subsequent associated value information, the weight of the current associated value information and the weight of the subsequent associated value information, wherein the current associated value information and the subsequent associated value information fuse the sample state information and the sample action information of the sample space region.

[0097] In an embodiment of the present application, the policy model of the sample space region can be denoted as a first policy model, and the policy model of the sample associated region can be denoted as a second policy model. The reward information can be an immediate reward result corresponding to a control action performed by an initial agent in a current state. The current value information can be a predicted value of the total income obtained by the first policy model under the current sample state information. The subsequent value information can be a maximum value in the income prediction of all possible actions under the subsequent sample state information at the next moment, which can represent an estimated value of the best future income obtained in the subsequent state. The weight of the current value information and the weight of the subsequent value information respectively indicate the degree of attention of the initial control agent to future rewards, that is, the greater the weight of the subsequent value information, the higher the degree of attention of the initial control agent to the state information at the future moment.

[0098] The current associated value information can be a predicted value of the income of the current associated state-action pair by the second policy model of the associated agent j in the sample associated region, and the input information includes the current state information, the current action information of itself, and the current state information and the current action information of the initial agent i initial . Similarly, the subsequent associated value information also fuses the subsequent state information and the subsequent action information of the initial agent i initial , which will not be described in detail here.

[0099] It should be noted that, in addition to the current associated value information and the subsequent associated value information described above, the determination method and corresponding content of the weight of the reward information, the current associated value information and the subsequent associated value information of the sample associated action information in the sample associated region are similar to those of the corresponding information of the sample space region, which will not be described in detail here.

[0100] For the sample space region i', a neural network can be used to infinitely approach , and the network parameter can be represented by θ. The training loss function of the neural network which is calculated as shown in the following equation (6):

[0101] (6);

[0102] wherein, may represent the first policy model of the sample space region i', γmay be a discount factor, i.e., the weight of the current value information and the subsequent value information, D k+1 may represent the trajectory termination. kmay represent the size of the mini-batch of samples randomly drawn from the experience replay buffer. may represent the initial agent i initial obtained instant reward result. may represent the maximum value among the predicted values of the benefits of all possible action information under the next time state information. may represent the predicted value of the total benefit that the first policy model at the current time can obtain by taking an action under the current state information.

[0103] In order to make the control agent fully explore the environment in the early stage, and gradually tend to use the optimal strategy in the later stage, and finally converge to a control strategy that is both globally optimal and stable. The control agent can balance exploration and exploitation by selecting actions through random numbers of the greedy strategy in the decision-making stage.

[0104] For example, when the random number ϵ≥δ, the control agent can select the action that is currently considered the best, i.e., the action with the maximum benefit value, so that the control agent uses the learned knowledge to obtain high benefits.

[0105] When the random number ϵ<δ, the control agent can randomly select an action, so that the control agent tries actions that are currently not good but may be good in the future, avoiding falling into a local optimal solution, as shown in the following equation (7):

[0106] (7);

[0107] wherein, may represent the action finally selected and executed by the initial control agent at time step t. may represent the policy of the initial agent i initial may be a function that decides what action to take according to the current state information. ϵmay be a random number uniformly distributed in the interval [0, 1], which can be resampled at each decision-making step. δmay be a threshold value of the exploration rate, which is a hyperparameter (for example, set to 0.1), which determines the probability of the agent exploring, A random number between 0 and 1 can be generated. The current state information can be characterized, and the action that maximizes the reward value can be selected, and y can represent the parameters of the first policy model.

[0108] For the associated agent j of the sample space region i', the loss function (second loss function) The determination process considers the experience data (sample state information and sample action information) of the sample space region i', and the experience data of the sample space region i' is used for learning, so as to realize knowledge transfer and collaboration, as shown in the following formula (8):

[0109] (8);

[0110] Among them, The maximum operation can be represented, which means that the second policy model of the associated agent j takes the maximum value in the reward value prediction of all possible actions initial of the initial agent i under the state information of the initial agent i , which can represent the best future reward value estimated by The second policy model can represent the predicted value of the reward of the current state-action pair.

[0111] According to the embodiments of the present application, the random number selection of the greedy strategy can make the initial control agent fully explore the environment in the initial stage, and gradually tend to use the optimal strategy in the later stage, and finally converge to a stable control strategy that is both globally optimal and stable. At the same time, based on the interaction experience of the initial agent, the parameters of the policy network of the associated agent are updated, so that a good strategy can be learned faster, and the training efficiency of the control agent is improved.

[0112] According to the embodiments of the present application, based on the loss function and the fusion strategy of the respective policy models of the sample information, the initial evaluation result, the sample space region and the sample associated region, the control agent is obtained, including: obtaining the first intermediate policy model and the second intermediate policy model under the condition that the first loss function and the second loss function meet the preset loss condition; based on the fusion strategy, the output results of the first intermediate policy model and the second intermediate policy model are fused to obtain a fusion result; under the condition that the fusion result meets the preset fusion condition, the control agent is obtained.

[0113] In the embodiments of the present application, the preset loss condition can be used to judge whether the predicted value of the strategy model meets the condition of the prediction accuracy requirement. The first intermediate strategy model can be a strategy model in the training stage corresponding to the sample space region, and the second intermediate strategy model can be a strategy model in the training stage corresponding to the sample associated region. The fusion result can be a joint revenue value obtained after the output results of different strategy models are fused, which can be denoted as joint Q value.

[0114] Each sample space region i' can retain its own action value function But when the action is selected, the joint Q value of the sample associated region will be integrated, and the determination method of the joint Q value can be shown in the following formula (9):

[0115] (9);

[0116] Among them, Can represent the joint Q value, Can represent the coordination weight between the sample space region i' and its associated region j. Can represent the action value function of the sample space region i' itself. Can represent the action value function of the associated region j.

[0117] The joint Q value can be the fusion of the control agent's own information and the associated agent's associated region information, so that the decision of each agent is no longer isolated, but considers the synergistic effect of the whole system, which helps to avoid local selfish decision and pursue a better global solution.

[0118] (10);

[0119] Among them, Can represent the action finally selected by the sample space region i' at time t. a i Can represent the action information set. Can represent the joint action value function calculated according to the above formula (9) at time t. Argmax can be a mathematical operator used to find the independent variable (i.e. action information) that makes the function value behind it reach the maximum value.

[0120] According to an embodiment of the present application, by reserving the control advantage of the independent space area of the building, the heat interaction across the area is realized through the cooperation of the associated area. By fusing the Q value of the local and the associated area for decision-making, and sharing the experience data of the associated area, the fusion of the experience data of the neighborhood can enhance the experience storage of the system, allowing the agent of each area to learn not only from its own experience, but also from the experience of its adjacent area, realizing more coordinated and adaptive HVAC control, not only maintaining the adaptability of the specific area, but also reducing the energy fluctuation, solving the key gap existing in the previous time scale focusing work. Especially suitable for commercial buildings with strong space heat correlation.

[0121] Figure 3 An example schematic diagram of a building energy consumption multi-area collaborative control process according to an embodiment of the present application is shown.

[0122] As shown in (a) of FIG. 1, Figure 3 In the training phase of the initial control agent, the temperature evaluation information 301 and the energy consumption evaluation information 302 of the sample space area can be processed to obtain the initial evaluation result 303 of the sample environment temperature of the sample space area within a preset time period; and based on the sample information 304, the initial evaluation result 303, the loss function 305 of the respective policy model of the sample space area and the sample associated area, and the fusion policy 306, the control agent 307 is obtained. The loss function 305 can include a first loss function 3051 and a second loss function 3052.

[0123] As shown in (b) of FIG. 1, Figure 3 After obtaining the control agent 307, in the application stage, the control agent 307 can be used to fuse the state information 310 with the associated state information 320 in the associated area information to obtain the fused state information 330; and based on the mapping policy 340, the fused state information is mapped to the action information 350, the mapping policy 340 includes any one of the deterministic policy 341 and the random policy 342; and then based on the action information 350, the control operation is performed on the valve opening 360 to obtain the energy consumption control information 370 of the space area.

[0124] Based on the above building energy consumption multi-area collaborative control method, the present application further provides a building energy consumption multi-area collaborative control device. The device will be described in detail below. Figure 4

[0125] Figure 4 A structural block diagram of a building energy consumption multi-area collaborative control device according to an embodiment of the present application is shown.

[0126] As shown in (a) of FIG. 1, Figure 4 ​As shown, the building energy consumption multi-region collaborative control device 400 of the embodiment includes an information processing module 410, an agent obtaining module 420, and a control module 430.

[0127] The information processing module 410 is configured to process temperature evaluation information and energy consumption evaluation information of a sample space region in a sample building to obtain an initial evaluation result of a sample environment temperature of the sample space region within a preset time period, wherein the temperature evaluation information is determined by sample temperature information of the sample space region, and the energy consumption evaluation information is determined by sample power information, sample energy consumption information, and cost information of the sample space region. In an embodiment, the information processing module 410 can be configured to perform the operation S210 described above, and details are not repeated here.

[0128] The agent obtaining module 420 is configured to obtain a control agent based on sample information, the initial evaluation result, a loss function of a respective policy model of the sample space region and a sample associated region, and a fusion policy. In an embodiment, the agent obtaining module 420 can be configured to perform the operation S220 described above, and details are not repeated here.

[0129] The control module 430 is configured to obtain action information for controlling a valve opening degree of a control device in a space region based on state information and associated region information of the space region in the building by using the control agent, and perform a control operation on the valve opening degree based on the action information to obtain energy consumption control information of the space region. In an embodiment, the control module 430 can be configured to perform the operation S230 described above, and details are not repeated here.

[0130] According to the embodiment of the present application, the information processing module 410, the agent obtaining module 420, and the control module 430 in the building energy consumption multi-region collaborative control device 400 are used. Since the training phase of the control agent fuses the loss function of the sample associated region policy model, introduces the joint evaluation mechanism of the associated region shared information and the local region, considers the thermal correlation between different regions in the building, makes the control agent satisfy more collaborative and flexible decision output, and different space regions have respective policy models to avoid retraining of the model as a whole under varying conditions, the system resource consumption is reduced. In the application phase, by fusing the associated region information, the control agent can perceive more large-range multi-dimensional information of the space region, so that the heat transfer and energy consumption influence between different regions in the building can be considered when making decisions, the global energy consumption is prevented from being deteriorated due to ignoring the thermal differences between different regions, the associated region information is fused to adjust the valve opening degree of the control device, the energy consumption cost of the whole building is significantly reduced under the premise of meeting the environmental temperature comfort.

[0131] According to an embodiment of the present application, the control module 430 comprises a fusion submodule and a mapping submodule. The fusion submodule is configured to fuse the state information with the associated state information in the associated region information to obtain fused state information. The mapping submodule is configured to map the fused state information to action information based on a mapping strategy, wherein the mapping strategy comprises any one of a deterministic strategy and a random strategy.

[0132] According to an embodiment of the present application, the building energy consumption multi-region collaborative control device further comprises a region determination module and an action information acquisition module. The region determination module is configured to determine an associated region from a plurality of adjacent regions in a case where an association degree between a space region and the plurality of adjacent regions satisfies a preset condition, wherein the association degree can be determined based on distance information, equipment information and envelope information between the space region and the plurality of adjacent regions. The action information acquisition module is configured to acquire associated state information and associated action information of the associated region as the associated region information.

[0133] According to an embodiment of the present application, the action information comprises a plurality of opening degree operation information for executing the valve. The control module 430 further comprises a speed information determination submodule and an execution submodule. The speed information determination submodule is configured to determine target operation information indicated by the action information and speed information of the control equipment, wherein the target operation information is associated with the speed information. The execution submodule is configured to perform a control operation on the valve opening based on the target operation information and the speed information to obtain energy consumption control information, wherein the energy consumption control information comprises an energy consumption result and environmental temperature information corresponding to the energy consumption result.

[0134] According to an embodiment of the present application, the sample temperature information comprises a subsequent temperature information and reference temperature information corresponding to the subsequent temperature information. The building energy consumption multi-region collaborative control device further comprises a temperature information acquisition module and an evaluation information determination module. The temperature information acquisition module is configured to acquire temperature information after performing a control operation on the valve opening based on the sample action information at the current time as the subsequent temperature information. The evaluation information determination module is configured to obtain temperature evaluation information based on the subsequent temperature information and the reference temperature information.

[0135] According to an embodiment of the present application, the building energy consumption multi-region collaborative control device further comprises a combination module configured to obtain energy consumption evaluation information of the sample action information at the current time based on combination information obtained based on the combination sample power information and the sample energy consumption information and cost information.

[0136] According to an embodiment of the present application, the information processing module 410 comprises a weighting submodule and an information combination submodule. The weighting submodule is configured to weight the temperature evaluation information using a temperature weight and weight the energy consumption evaluation information using an energy consumption weight, respectively, to obtain a plurality of evaluation information corresponding to a plurality of time points in a preset time period. The information combination submodule is configured to combine the plurality of evaluation information to obtain an initial evaluation result.

[0137] According to an embodiment of the present application, the loss function comprises a first loss function of the sample space region strategy model and a second loss function of the sample correlation region strategy model. The building energy consumption multi-region collaborative control device further comprises a first loss function determination module and a second loss function determination module. The first loss function determination module is configured to obtain the first loss function based on the reward information, the current value information, the subsequent value information, the weight of the current value information, and the weight of the subsequent value information of the sample action information. The second loss function determination module is configured to obtain the second loss function based on the reward information, the current correlation value information, the subsequent correlation value information, the weight of the current correlation value information, and the weight of the subsequent correlation value information of the sample correlation action information in the sample correlation region, wherein the current correlation value information and the subsequent correlation value information fuse the sample state information and the sample action information of the sample space region.

[0138] According to an embodiment of the present application, the agent obtaining module 420 comprises a model determination submodule, a result fusion submodule, and an agent obtaining submodule. The model determination submodule is configured to obtain a first intermediate strategy model and a second intermediate strategy model when the first loss function and the second loss function satisfy a preset loss condition. The result fusion submodule is configured to fuse the output results of the first intermediate strategy model and the second intermediate strategy model based on a fusion strategy to obtain a fusion result. The agent obtaining submodule is configured to obtain a control agent when the fusion result satisfies a preset fusion condition.

[0139] According to an embodiment of the present application, any of the information processing module 410, the agent obtaining module 420 and the control module 430 can be combined in one module, or any of them can be split into multiple modules. Alternatively, at least part of the function of one or more of these modules can be combined with at least part of the function of the other modules, and implemented in one module. According to an embodiment of the present application, at least one of the information processing module 410, the agent obtaining module 420 and the control module 430 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on board, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging a circuit, etc. or implemented by hardware or firmware, or implemented in any one of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the information processing module 410, the agent obtaining module 420 and the control module 430 can be at least partially implemented as a computer program module which, when executed, can perform the corresponding function.

[0140] Figure 5 A block diagram of an electronic device suitable for implementing the method of multi-zone collaborative control of building energy consumption according to an embodiment of the present application is shown.

[0141] As shown in Figure 5 , the electronic device 500 according to an embodiment of the present application comprises a processor 501 which can perform various appropriate actions and processes according to the program stored in a read-only memory (ROM) 502 or the program loaded from the storage portion 508 into a random access memory (RAM) 503. The processor 501 can comprise, for example, a general purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special purpose microprocessor (such as an application specific integrated circuit (ASIC)), etc. The processor 501 can also comprise an on-board memory for cache use. The processor 501 can comprise a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0142] In the RAM 503, various programs and data required for the operation of the electronic device 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other via the bus 504. The processor 501 performs various operations of the method flow according to the embodiments of the present application by executing the programs in the ROM 502 and / or the RAM 503. It should be noted that the programs can also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 can also perform various operations of the method flow according to the embodiments of the present application by executing the programs stored in the one or more memories.

[0143] According to the embodiments of the present application, the electronic device 500 can further include an input / output (I / O) interface 505, which is also connected to the bus 504. The electronic device 500 can further include one or more of the following components connected to the input / output (I / O) interface 505: an input part 506 including a keyboard, a mouse, etc.; an output part 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 508 including a hard disk, etc.; and a communication part 509 including a network interface card such as a LAN card, a modem, etc. The communication part 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the input / output (I / O) interface 505 as necessary. A removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read out therefrom is installed in the storage part 508 as necessary.

[0144] The present application also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present application is implemented.

[0145] According to an embodiment of the present application, the computer readable storage medium can be a non-transitory computer readable storage medium, for example, can include but not limited to: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, the computer readable storage medium can include one or more memories of the ROM 502 and / or the RAM 503 described above and / or in addition to the ROM 502 and the RAM 503.

[0146] Embodiments of the present application also include a computer program product, which includes a computer program containing program codes for executing the method shown in the flow chart. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the building energy consumption multi-region collaborative control method provided by the embodiments of the present application.

[0147] The above functions defined in the system / device of the embodiments of the present application are performed when the computer program is executed by the processor 501. According to an embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0148] In one embodiment, the computer program can rely on tangible storage media such as optical storage media, magnetic storage media, etc. In another embodiment, the computer program can also be transmitted, distributed, downloaded and installed in the form of signals on a network medium, and downloaded and installed through the communication part 509, and / or installed from the detachable medium 511. The program codes contained in the computer program can be transmitted by any suitable network medium, including but not limited to wireless, wired, etc., or any suitable combination of the foregoing.

[0149] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the detachable medium 511. When the computer program is executed by the processor 501, the above functions defined in the system of the embodiments of the present application are performed. According to an embodiment of the present application, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0150] According to embodiments of the present application, program code for implementing the computer programs provided by embodiments of the present application can be written in any combination of one or more programming languages, and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The programming language can include, but is not limited to, Java, C++, python, "C" language, or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the remote computing device, or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0151] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0152] Those skilled in the art will appreciate that the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways, even if such combinations or integrations are not expressly noted in the present application. In particular, the features recited in the various embodiments of the present application can be combined and / or integrated in a variety of ways without departing from the spirit and scope of the present application. All such combinations and / or integrations are within the scope of the present application.

[0153] The embodiments of the present application described above are merely intended to illustrate the present application. These embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present application. Although the above describes each embodiment separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Numerous alternatives and modifications of embodiments of the present application can be made by those skilled in the art without departing from the scope of the present application, and all such alternatives and modifications are to be included within the scope of the present application.

Claims

1. A building energy consumption multi-region collaborative control method, characterized in that, The method comprises: processing temperature evaluation information and energy consumption evaluation information of a sample space region in a sample building to obtain an initial evaluation result of a sample environment temperature of the sample space region in a preset time period, wherein the temperature evaluation information is determined by sample temperature information of the sample space region, and the energy consumption evaluation information is determined by sample power information, sample energy consumption information and cost information of the sample space region; based on sample state information and sample action information in sample information, the initial evaluation result, loss functions of respective policy models of the sample space region and a sample associated region, and a fusion policy, obtaining a control agent, comprising: in a case where a first loss function of a policy model of the sample space region and a second loss function of a policy model of the sample associated region satisfy a preset loss condition, obtaining a first intermediate policy model and a second intermediate policy model; based on the fusion policy, fusing output results of the first intermediate policy model and the second intermediate policy model to obtain a fusion result; in a case where the fusion result satisfies a preset fusion condition, obtaining the control agent; based on state information and associated region information of a space region in a building, obtaining action information for controlling a valve opening degree of a control device in the space region, and performing a control operation on the valve opening degree based on the action information to obtain energy consumption control information of the space region.

2. The method of claim 1, wherein, based on state information and associated region information of a space region in a building, obtaining action information for controlling a valve opening degree of a control device in the space region, comprising: fusing the state information with associated state information in the associated region information to obtain fused state information; based on a mapping policy, mapping the fused state information to the action information, wherein the mapping policy comprises any one of a deterministic policy and a random policy.

3. The method according to claim 1 or 2, characterized in that, The method further comprises: in a case where an association degree between the space region and a plurality of adjacent regions satisfies a preset condition, determining the associated region from the plurality of adjacent regions, wherein the association degree is determined based on distance information, device information and envelope structure information between the space region and the plurality of adjacent regions; obtaining associated state information and associated action information of the associated region as the associated region information.

4. The method of claim 1, wherein, The action information comprises a plurality of opening operation information for performing on a valve; based on the action information, performing a control operation on the valve opening degree to obtain energy consumption control information of the space region, comprising: determining target operation information indicated by the action information and speed information of the control device, wherein the target operation information is associated with the speed information; based on the target operation information and the speed information, performing a control operation on the valve opening degree to obtain the energy consumption control information, the energy consumption control information comprising energy consumption results and environment temperature information corresponding to the energy consumption results.

5. The method of claim 1, wherein, The sample temperature information comprises subsequent temperature information and reference temperature information corresponding to the subsequent temperature information; The method further comprises: Obtain temperature information after a control operation on the valve opening based on sample action information at the current moment as the after temperature information; Obtain the temperature evaluation information based on the after temperature information and the reference temperature information.

6. The method of claim 1, wherein, The method further comprises: Obtain energy consumption evaluation information of the sample action information at the current moment based on combined information obtained by combining the sample power information and the sample energy consumption information and the cost information.

7. The method according to claim 5 or 6, characterized in that, Process the temperature evaluation information and the energy consumption evaluation information of the sample space region in the sample building to obtain an initial evaluation result of the sample environment temperature of the sample space region within a preset time period, comprising: Respectively weight the temperature evaluation information using a temperature weight and weight the energy consumption evaluation information using an energy consumption weight to obtain a plurality of evaluation information corresponding to a plurality of moments within the preset time period; Combine the plurality of evaluation information to obtain the initial evaluation result.

8. The method of claim 5, wherein, The method further comprises: Obtain the first loss function based on reward information of the sample action information, current value information, after value information, weights of the current value information and the after value information, wherein the current value information is a predicted value of total income obtained by the first policy model under the current sample state information, and the after value information is a maximum value in income prediction of all possible actions under the after sample state information at the next moment; Obtain the second loss function based on reward information of sample associated action information in the sample associated region, current associated value information, after associated value information, and weights of the current associated value information and the after associated value information, wherein the current associated value information and the after associated value information fuse the sample state information and the sample action information of the sample space region, wherein the current associated value information is a predicted value of income of the current associated state-action pair by the second policy model of the associated agent of the sample associated region, and the after associated value information fuses the after state information and the after action information of the initial agent.

9. A building energy consumption multi-zone collaborative control device, characterized in that, The device comprises: An information processing module for processing the temperature evaluation information and the energy consumption evaluation information of the sample space region in the sample building to obtain an initial evaluation result of the sample environment temperature of the sample space region within a preset time period, wherein the temperature evaluation information is determined by sample temperature information of the sample space region, and the energy consumption evaluation information is determined by sample power information, sample energy consumption information, and cost information of the sample space region; An agent obtaining module for obtaining a control agent based on sample state information and sample action information in sample information, the initial evaluation result, loss functions of respective policy models of the sample space region and the sample associated region, and fusion strategies, comprising: A model determining submodule for obtaining a first intermediate policy model and a second intermediate policy model when the first loss function of the policy model of the sample space region and the second loss function of the policy model of the sample associated region meet a preset loss condition; The result fusion submodule is configured to fuse outputs of the first intermediate strategy model and the second intermediate strategy model based on the fusion strategy to obtain a fusion result. The agent obtaining submodule is configured to obtain the control agent when the fusion result meets a preset fusion condition. The control module is configured to obtain action information for controlling a valve opening degree of a control device in a space region in a building based on state information and associated region information of the space region by using the control agent, and perform a control operation on the valve opening degree based on the action information to obtain energy consumption control information of the space region.

10. An electronic device comprising: one or more processors; memory for storing one or more computer programs, characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed by the processor to implement the steps of the method according to any one of claims 1-8.

Images