Heat source switching method and device based on machine learning, equipment and medium
By applying machine learning-based heat source switching methods in the energy management system, dynamically adjusting the heat source distribution strategy, the challenge of heat source scheduling in complex heat source systems is solved, and efficient and flexible heat source management and energy utilization efficiency are achieved.
Patent Information
- Application Number
- CN202411567540.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2025-03-04
AI Technical Summary
In modern energy management systems, especially in complex and diversified heat source systems, it is challenging to achieve efficient and flexible heat source scheduling to accurately match user needs while reducing operating costs and carbon emissions.
Using a heat source switching method based on machine learning, a heating strategy classification model and reinforcement learning model are established by obtaining heating data of multiple heat sources in the heating system, and the heat source allocation strategy is dynamically adjusted to meet user needs and respond to environmental changes.
It realizes more intelligent heat source scheduling, improves the flexibility and response speed of the system, and can achieve maximum energy returns and emission reduction effects while meeting user needs.
Smart Images
Figure CN119558571A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of heat source switching technology, and in particular to a heat source switching method, device, equipment and medium based on machine learning. Background Art
[0002] In the modern energy management system, especially in the construction of complex and diversified heat source systems, achieving efficient and flexible heat source scheduling to accurately match user needs while ensuring the minimization of operating costs and significant reduction of carbon emissions constitutes a major challenge in the current energy management field.
[0003] The unpredictability of environmental conditions and the diversity of user needs further increase the difficulty of formulating heat source scheduling strategies. For example, the frequent occurrence of extreme weather events may lead to unstable heat source supply, while changes in user preferences require heat source switching strategies to have higher flexibility and responsiveness. These factors make it difficult for traditional heat source switching mechanisms, which mostly rely on static rules and manual experience, to achieve the optimal goal of energy conservation and emission reduction while ensuring user satisfaction. In view of this, it is particularly important to explore a new method that can comprehensively consider multiple factors and dynamically adjust the heat source allocation strategy. Summary of the invention
[0004] In view of the above problems, embodiments of the present invention are proposed to provide a heat source switching method, device, equipment and medium based on machine learning that overcome the above problems or at least partially solve the above problems.
[0005] In order to solve the above problems, an embodiment of the present invention discloses a heat source switching method based on machine learning, the method comprising acquiring heating data of multiple heat sources of a heating system;
[0006] Obtaining a heating strategy classification model according to heating data of multiple heat sources of the heating system;
[0007] Determining a first heating strategy for multiple heat sources of the heating system according to the heating strategy classification model, and controlling the multiple heat sources of the heating system according to the first heating strategy;
[0008] Obtaining a heating strategy reinforcement learning model according to heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy;
[0009] According to the heating strategy reinforcement learning model, a second heating strategy for the multiple heat sources of the heating system is determined, and the multiple heat sources of the heating system are controlled according to the second heating strategy.
[0010] Optionally, the method further comprises:
[0011] Determining whether the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets a preset model optimization condition;
[0012] The step of obtaining a heating strategy reinforcement learning model based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy includes:
[0013] When the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions, a heating strategy reinforcement learning model is obtained based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy.
[0014] Optionally, obtaining the heating strategy reinforcement learning model according to the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy includes:
[0015] preprocessing heating data of the multiple heat sources obtained after the first heating strategy controls the multiple heat sources of the heating system;
[0016] Determine a training data set and a test data set based on heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the preprocessed first heating strategy;
[0017] Performing multi-objective learning training on the reinforcement learning model according to the training data set;
[0018] Performing a performance evaluation on the reinforcement learning model after multi-objective learning training according to the test data set;
[0019] When the evaluation result meets the preset evaluation index, the heating strategy reinforcement learning model is output.
[0020] Optionally, the performing multi-objective learning training on the reinforcement learning model includes:
[0021] At least one of the operating cost, carbon emissions, and user satisfaction is used as an immediate reward for the multi-objective learning training; the operating cost is used to motivate the reinforcement learning model to select a strategy with lower operating cost; the carbon emissions are used to motivate the reinforcement learning model to select a strategy with lower carbon emissions; and the user satisfaction is used to motivate the reinforcement learning model to select a strategy that meets user needs to a greater extent.
[0022] Optionally, obtaining a heating strategy classification model according to the heating data of the multiple heat sources of the heating system includes:
[0023] When the heating data meets the preset initial data condition, a heating strategy classification model is obtained according to the heating data of the multiple heat sources of the heating system;
[0024] The method further comprises:
[0025] When the heating data does not meet the preset initial data conditions, obtaining the regional initial heating strategy classification model parameters from the server;
[0026] Determining a regional initial heating strategy classification model according to the regional initial heating strategy classification model parameters;
[0027] According to the regional initial heating strategy classification model, a third heating strategy for the multiple heat sources of the heating system is determined, and the multiple heat sources of the heating system are controlled according to the third heating strategy.
[0028] Optionally, the heating data of the multiple heat sources of the heating system include user data, and the method further includes:
[0029] According to the user data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the third heating strategy, the regional initial heating strategy classification model is fine-tuned for user personalization to obtain the heating strategy classification model.
[0030] Optionally, the method further comprises:
[0031] The heating data of the multiple heat sources of the heating system are transmitted to the server, so that the server obtains the classification model parameters of the initial heating strategy for the region according to the heating data of the multiple heat sources of the heating system transmitted by multiple users in the same region.
[0032] On the other hand, an embodiment of the present invention discloses a heat source switching device based on machine learning, the device comprising:
[0033] A heating data acquisition module is used to acquire heating data of multiple heat sources of a heating system;
[0034] A classification model acquisition module, used to obtain a heating strategy classification model according to the heating data of multiple heat sources of the heating system;
[0035] a classification strategy execution module, configured to determine a first heating strategy for a plurality of heat sources of the heating system according to the heating strategy classification model, and control the plurality of heat sources of the heating system according to the first heating strategy;
[0036] a learning model acquisition module, configured to obtain a heating strategy reinforcement learning model according to heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy;
[0037] A learning strategy execution module is used to determine a second heating strategy for multiple heat sources of the heating system according to the heating strategy reinforcement learning model, and control the multiple heat sources of the heating system according to the second heating strategy.
[0038] Optionally, the device further comprises:
[0039] a model optimization judgment submodule, used to judge whether the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions;
[0040] The learning model acquisition module includes:
[0041] A learning model construction submodule is used to obtain a heating strategy reinforcement learning model based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy, when the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions.
[0042] Optionally, the learning model construction submodule includes:
[0043] a data preprocessing unit, configured to preprocess heating data of the multiple heat sources obtained after the first heating strategy controls the multiple heat sources of the heating system;
[0044] a data set determining unit, configured to determine a training data set and a test data set based on heating data of the plurality of heat sources obtained after the first heating strategy is preprocessed to control the plurality of heat sources of the heating system;
[0045] A multi-objective learning unit, used to perform multi-objective learning training on the reinforcement learning model according to the training data set;
[0046] A performance evaluation unit, used to perform performance evaluation on the reinforcement learning model after multi-objective learning training according to the test data set;
[0047] The model output unit is used to output the heating strategy reinforcement learning model when the evaluation result meets the preset evaluation index.
[0048] Optionally, the multi-objective learning unit includes:
[0049] A multi-objective confirmation subunit is used to use at least one of the operating cost, carbon emissions, and user satisfaction as an immediate reward for the multi-objective learning training; the operating cost is used to motivate the reinforcement learning model to select a strategy with lower operating cost; the carbon emissions are used to motivate the reinforcement learning model to select a strategy with lower carbon emissions; and the user satisfaction is used to motivate the reinforcement learning model to select a strategy that meets user needs to a greater extent.
[0050] Optionally, the classification model acquisition module includes:
[0051] A classification model acquisition submodule, for obtaining a heating strategy classification model according to the heating data of multiple heat sources of the heating system when the heating data meets a preset initial data condition;
[0052] The device also includes:
[0053] An initial classification parameter acquisition submodule, used for acquiring regional initial heating strategy classification model parameters from a server when the heating data does not meet the preset initial data conditions;
[0054] The initial component model determination submodule is used to determine a third heating strategy for multiple heat sources of the heating system according to the regional initial heating strategy classification model, and control the multiple heat sources of the heating system according to the third heating strategy.
[0055] Optionally, the heating data of the multiple heat sources of the heating system include user data, and the device further includes:
[0056] The initial component model determination unit is used to perform user-personalized fine-tuning on the regional initial heating strategy classification model based on user data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the third heating strategy to obtain the heating strategy classification model.
[0057] Optionally, the device further comprises:
[0058] The data transmission submodule is used to transmit the heating data of multiple heat sources of the heating system to the server, so that the server obtains the initial heating strategy classification model parameters of the region based on the heating data of multiple heat sources of the heating system transmitted by multiple users in the same region.
[0059] Correspondingly, an embodiment of the present invention discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various steps of the above-mentioned embodiment of the heat source switching method based on machine learning are implemented.
[0060] Correspondingly, an embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various steps of the above-mentioned embodiment of the heat source switching method based on machine learning are implemented.
[0061] The embodiments of the present invention include the following advantages: by establishing a classification model based on the heating data of multiple heat sources in the heating system, it is possible to intelligently identify and classify different heating demand scenarios based on the heating data of multiple heat sources, which helps to initially realize user needs and respond to environmental changes. Furthermore, through the implementation of the classification model heating strategy and data feedback, a reinforcement learning model is constructed in combination with other heating data so that the reinforcement learning model can further optimize the strategy on the basis of the classification strategy, and can more deeply meet user needs and respond to environmental changes more sensitively while maximizing energy benefits. The embodiments of the present invention abandon the static rules and manual experience that traditional heat source switching technology relies on, and use machine learning to formulate and optimize heat source scheduling strategies, thereby improving the intelligence level of the system and achieving both energy conservation and emission reduction and user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0062] Figure 1 is a flowchart of a heat source switching method embodiment based on machine learning of the present invention;
[0063] Figure 2 is a flowchart of another embodiment of a heat source switching method based on machine learning of the present invention;
[0064] Figure 3 It is a structural block diagram of an embodiment of a heat source switching device based on machine learning of the present invention. DETAILED DESCRIPTION
[0065] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0066] Multiple heat sources refer to the existence of multiple heat sources in a heating system, which together provide heat to heat users. These heat sources may include power plants that produce electricity and heat, thermal power plants that improve energy efficiency through cogeneration, boiler rooms that provide heat by burning primary energy such as coal, natural gas, heavy oil, and solar thermal sources, geothermal heat sources, and other types of heat sources.
[0067] One of the core concepts of an embodiment of the present invention is that when the amount of data is insufficient to support the training of a reinforcement learning model, a classification model that relies less on the amount of data is used to formulate a preliminary switching strategy. After the classification model has been implemented for a period of time, when the amount of data is sufficient to train the reinforcement learning model, the reinforcement learning model is further trained and used to implement a more intelligent heat source switching strategy formulation.
[0068] Reference Figure 1 , shows a flow chart of the steps of an embodiment of a heat source switching method based on machine learning of the present invention, which may specifically include the following steps:
[0069] Step 101, obtaining heating data of multiple heat sources of a heating system;
[0070] Before building the model, it is necessary to collect heating data from multiple heat sources in the heating system;
[0071] In one example, the heating data may be collected through sensor collection, data collection based on the Internet of Things, and manual data collection such as user input, etc., which is not limited in this embodiment of the present invention.
[0072] The collected heating data may include: historical operation data such as the output power, operation time, energy efficiency, fuel consumption, etc. of each heat source; environmental data such as outdoor temperature, humidity, weather forecast, etc.; demand data such as the user's heat demand, usage time period, etc. Cost data such as fuel prices, electricity prices, maintenance costs, etc. Switching strategies such as the heat source switching strategies adopted in different states in history and their effects. The collected data is preprocessed to make the data format more consistent with the format of model processing. After removing missing values and outliers from the data, historical data can be generated, such as daily heat demand, operation costs, outdoor temperature, etc.
[0073] Step 102: Obtain a heating strategy classification model based on the heating data of multiple heat sources of the heating system.
[0074] In machine learning, a classification model is a supervised learning method that uses a training data set to learn the mapping relationship between input features and output categories, so that it can make classification predictions for new data.
[0075] In one example, the heating strategy classification model can be a classification model such as logistic regression, decision tree, random forest, gradient boosting tree, support vector machine (SVM) and naive Bayes, which is not limited in the embodiment of the present invention.
[0076] In one embodiment, step 102 includes the following sub-steps:
[0077] Step 103, determining a first heating strategy for multiple heat sources of the heating system according to the heating strategy classification model, and controlling the multiple heat sources of the heating system according to the first heating strategy;
[0078] Step 104, obtaining a heating strategy reinforcement learning model according to heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy;
[0079] When the amount of data is sufficient to support the training of the reinforcement learning model, the heating system will train and use the reinforcement learning model to further formulate heat source switching strategies.
[0080] Reinforcement Learning (RL) is a machine learning algorithm that is mainly based on the core concept of Markov Decision Process (MDP), but is not limited to the MDP framework. Reinforcement learning allows the agent to continuously try and learn in the environment, with the goal of maximizing the cumulative reward, so that it has the ability to learn and adapt autonomously, and can make optimized decisions in a dynamic environment.
[0081] Reinforcement learning models are generally composed of the following components: Agent, Environment, State, Action, Reward, and Policy. The agent will select actions based on the current state, the environment will feedback new states and rewards based on the actions, and the agent will adjust its strategy based on these feedbacks to optimize the long-term cumulative rewards.
[0082] Step 105: determining a second heating strategy for the multiple heat sources of the heating system according to the heating strategy reinforcement learning model, and controlling the multiple heat sources of the heating system according to the second heating strategy.
[0083] By establishing a classification model based on the heating data of multiple heat sources in the heating system, it is possible to intelligently identify and classify different heating demand scenarios based on the heating data of multiple heat sources, which helps to initially meet user needs and respond to environmental changes. Furthermore, through the implementation of the classification model heating strategy and data feedback, a reinforcement learning model is constructed in combination with other heating data so that the reinforcement learning model can further optimize the strategy on the basis of the classification strategy, and can meet user needs to a deeper degree and respond to environmental changes more sensitively while achieving maximum energy benefits. The embodiment of the present invention abandons the static rules and manual experience that traditional heat source switching technology relies on, and adopts machine learning to formulate and optimize heat source scheduling strategies, thereby improving the intelligence level of the system and achieving both energy conservation and emission reduction and user experience.
[0084] Reference Figure 2 , shows a flow chart of another heat source switching method based on machine learning in Embodiment 2 of the present invention, which may specifically include the following steps:
[0085] Step 201, obtaining heating data of multiple heat sources of a heating system;
[0086] Before building the model, it is necessary to collect heating data from multiple heat sources in the heating system;
[0087] In one example, the heating data may be collected through sensor collection, data collection based on the Internet of Things, and manual data collection such as user input, etc., which is not limited in this embodiment of the present invention.
[0088] The collected heating data may include: historical operation data such as the output power, operation time, energy efficiency, fuel consumption, etc. of each heat source; environmental data such as outdoor temperature, humidity, weather forecast, etc.; demand data such as the user's heat demand, usage time period, etc. Cost data such as fuel prices, electricity prices, maintenance costs, etc. Switching strategies such as the heat source switching strategies adopted in different states in history and their effects. The collected data is preprocessed to make the data format more consistent with the format of model processing. After removing missing values and outliers from the data, historical data can be generated, such as daily heat demand, operation costs, outdoor temperature, etc.
[0089] Step 202, obtaining a heating strategy classification model according to the heating data of multiple heat sources of the heating system;
[0090] In machine learning, a classification model is a supervised learning method that uses a training data set to learn the mapping relationship between input features and output categories, so that it can make classification predictions for new data.
[0091] In one example, the heating strategy classification model can be a classification model such as logistic regression, decision tree, random forest, gradient boosting tree, support vector machine (SVM) and naive Bayes, which is not limited in the embodiment of the present invention.
[0092] In one embodiment, step 202 includes the following sub-steps:
[0093] Sub-step S11, when the heating data meets the preset initial data condition, obtaining a heating strategy classification model according to the heating data of multiple heat sources of the heating system;
[0094] When the amount of heating data reaches the preset initial data condition, the collected heating data is used to build a classification model.
[0095] The method may further comprise the following sub-steps:
[0096] Sub-step S21, when the heating data does not meet the preset initial data conditions, obtaining regional initial heating strategy classification model parameters from the server;
[0097] When the amount of heating data does not reach the preset initial data condition, so that the classification model cannot be constructed based on the collected data, the heating system obtains from the server the parameters of the regional initial heating strategy classification model constructed by the server using the user's location habits and environmental data collected by the server, so that the heating system can construct the regional initial heating strategy classification model based on the parameters.
[0098] Sub-step S22, determining a regional initial heating strategy classification model according to the regional initial heating strategy classification model parameters;
[0099] Sub-step S23, determining a third heating strategy for the multiple heat sources of the heating system according to the regional initial heating strategy classification model, and controlling the multiple heat sources of the heating system according to the third heating strategy.
[0100] The embodiments of the present invention can provide users with an initial classification model to achieve a preliminary intelligent classification effect when the amount of data is insufficient, such as when a user applies a heating system equipped with the method for the first time, thereby improving the environmental adaptability of the heating system and meeting the needs of users.
[0101] In one embodiment, the heating data of the plurality of heat sources of the heating system include user data, and the method further includes the following sub-steps:
[0102] Sub-step S31, based on the user data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the third heating strategy, the user-personalized fine-tuning of the regional initial heating strategy classification model is performed to obtain the heating strategy classification model.
[0103] After using the regional initial heating strategy classification model to control the heat source switching strategy for a period of time, the relevant parameters of the regional initial heating strategy classification model are adjusted according to the real-time feedback of the user, so that the model can better fit the needs of the user and meet the personalized needs of the user.
[0104] In one embodiment, the method further comprises the following sub-steps:
[0105] Sub-step S41, transmitting the heating data of the multiple heat sources of the heating system to the server, so that the server obtains the classification model parameters of the initial heating strategy of the region according to the heating data of the multiple heat sources of the heating system transmitted by multiple users in the same region.
[0106] Transmitting the heating data of multiple users to the server allows the server to preliminarily formulate a regional initial heating strategy classification model parameter based on the heating data of multiple users in the area for the user's first use, and fit the environmental factors of the heating system.
[0107] Step 203, determining a first heating strategy for the multiple heat sources of the heating system according to the heating strategy classification model, and controlling the multiple heat sources of the heating system according to the first heating strategy;
[0108] Step 204, determining whether the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization condition;
[0109] Step 205, when the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions, a heating strategy reinforcement learning model is obtained according to the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy.
[0110] When the amount of data meets the preset model optimization conditions, that is, it is sufficient to support the training of the reinforcement learning model, the heating system will train and use the reinforcement learning model to further formulate heat source switching strategies.
[0111] Reinforcement Learning (RL) is a machine learning algorithm that is mainly based on the core concept of Markov Decision Process (MDP), but is not limited to the MDP framework. Reinforcement learning allows the agent to continuously try and learn in the environment, with the goal of maximizing the cumulative reward, so that it has the ability to learn and adapt autonomously, and can make optimized decisions in a dynamic environment.
[0112] Reinforcement learning models are generally composed of the following components: Agent, Environment, State, Action, Reward, and Policy. The agent will select actions based on the current state, the environment will feedback new states and rewards based on the actions, and the agent will adjust its strategy based on these feedbacks to optimize the long-term cumulative rewards.
[0113] In one embodiment, step 205 includes the following sub-steps:
[0114] Sub-step S51, pre-processing the heating data of the multiple heat sources obtained after the first heating strategy controls the multiple heat sources of the heating system;
[0115] In order to make the heating data meet the requirements of the reinforcement learning model, the heating system will pre-clean the collected heating data.
[0116] In one example, pre-cleaning may be to fill in missing data using interpolation, normalize data, reduce data dimension, enhance data using sampling, etc. The method for pre-cleaning data may also be any other pre-cleaning method, which is not limited in the embodiment of the present invention.
[0117] Sub-step S52, determining a training data set and a test data set based on heating data of the multiple heat sources obtained after the first heating strategy is pre-processed to control the multiple heat sources of the heating system;
[0118] When training a reinforcement learning model, it is generally necessary to divide the data into a training set and a test set, use the training set to train the model and use the test set to evaluate the accuracy of the model.
[0119] Sub-step S53, performing multi-objective learning training on the reinforcement learning model according to the training data set;
[0120] Multi-objective learning is a method commonly used in the field of machine learning, especially in reinforcement learning and multi-task learning. It refers to a learning method that pursues multiple learning objectives simultaneously under limited time and resources.
[0121] In one example, the multi-objective learning method may be: a weighted sum method, a Pareto frontier method, a multi-objective evolutionary algorithm, etc. The embodiment of the present invention does not limit this.
[0122] Sub-step S54, performing performance evaluation on the reinforcement learning model after multi-objective learning training according to the test data set;
[0123] Sub-step S55, when the evaluation result meets the preset evaluation index, output the heating strategy reinforcement learning model.
[0124] The embodiment of the present invention constructs a reinforcement learning model applied to the heating system by means of multi-objective reinforcement learning, which can meet multiple goals involved in the heating system, such as improving heating efficiency, reducing energy consumption, reducing emissions, and improving user comfort.
[0125] In one embodiment, sub-step S53 includes the following sub-steps:
[0126] Sub-step S531, at least one of the operating cost, carbon emissions, and user satisfaction is used as an immediate reward for the multi-objective learning training; the operating cost is used to motivate the reinforcement learning model to select a strategy with lower operating cost; the carbon emissions are used to motivate the reinforcement learning model to select a strategy with lower carbon emissions; the user satisfaction is used to motivate the reinforcement learning model to select a strategy that meets user needs to a higher degree. The reward target can also be other indicators such as business indicators such as heating efficiency, heating quality, and energy diversity, which are not limited in the embodiments of the present invention.
[0127] By taking negative operating costs, negative carbon emissions, and positive user satisfaction as rewards for multi-objective learning, the system can be motivated to choose a switching strategy with lower operating costs, lower carbon emissions, and better satisfaction with user needs;
[0128] In one example, the user's emphasis on reward goals can be adjusted by setting different weights. For example, some users may care more about the heating effect and relatively less about the operating cost. When training the model, the weights can be adjusted to obtain a reinforcement learning model that meets the user's personalized needs.
[0129] Step 206: Determine a second heating strategy for the multiple heat sources of the heating system according to the heating strategy reinforcement learning model, and control the multiple heat sources of the heating system according to the second heating strategy.
[0130] By establishing a classification model based on the heating data of multiple heat sources in the heating system, it is possible to intelligently identify and classify different heating demand scenarios based on the heating data of multiple heat sources, which helps to initially meet user needs and respond to environmental changes. Furthermore, through the implementation of the classification model heating strategy and data feedback, a reinforcement learning model is constructed in combination with other heating data so that the reinforcement learning model can further optimize the strategy on the basis of the classification strategy, and can meet user needs to a deeper degree and respond to environmental changes more sensitively while achieving maximum energy benefits. The embodiment of the present invention abandons the static rules and manual experience that traditional heat source switching technology relies on, and adopts machine learning to formulate and optimize heat source scheduling strategies, thereby improving the intelligence level of the system and achieving both energy conservation and emission reduction and user experience.
[0131] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0132] Reference Figure 3 , shows a structural block diagram of an embodiment of a heat source switching device based on machine learning of the present invention, which may specifically include the following modules:
[0133] A heating data acquisition module 301 is used to acquire heating data of multiple heat sources of a heating system;
[0134] Before building the model, it is necessary to collect heating data from multiple heat sources in the heating system;
[0135] In one example, the heating data may be collected through sensor collection, data collection based on the Internet of Things, and manual data collection such as user input, etc., which is not limited in this embodiment of the present invention.
[0136] The collected heating data may include: historical operation data such as the output power, operation time, energy efficiency, fuel consumption, etc. of each heat source; environmental data such as outdoor temperature, humidity, weather forecast, etc.; demand data such as the user's heat demand, usage time period, etc. Cost data such as fuel prices, electricity prices, maintenance costs, etc. Switching strategies such as the heat source switching strategies adopted in different states in history and their effects. The collected data is preprocessed to make the data format more consistent with the format of model processing. After removing missing values and outliers from the data, historical data can be generated, such as daily heat demand, operation costs, outdoor temperature, etc.
[0137] A classification model acquisition module 302 is used to obtain a heating strategy classification model according to the heating data of multiple heat sources of the heating system;
[0138] In machine learning, a classification model is a supervised learning method that uses a training data set to learn the mapping relationship between input features and output categories, so that it can make classification predictions for new data.
[0139] In one example, the heating strategy classification model can be a classification model such as logistic regression, decision tree, random forest, gradient boosting tree, support vector machine (SVM) and naive Bayes, which is not limited in the embodiment of the present invention.
[0140] A classification strategy execution module 303, configured to determine a first heating strategy for the multiple heat sources of the heating system according to the heating strategy classification model, and control the multiple heat sources of the heating system according to the first heating strategy;
[0141] A learning model acquisition module 304, configured to obtain a heating strategy reinforcement learning model according to heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy;
[0142] When the amount of data is sufficient to support the training of the reinforcement learning model, the heating system will train and use the reinforcement learning model to further formulate heat source switching strategies.
[0143] Reinforcement Learning (RL) is a machine learning algorithm that is mainly based on the core concept of Markov Decision Process (MDP), but is not limited to the MDP framework. Reinforcement learning allows the agent to continuously try and learn in the environment, with the goal of maximizing the cumulative reward, so that it has the ability to learn and adapt autonomously, and can make optimized decisions in a dynamic environment.
[0144] Reinforcement learning models are generally composed of the following components: Agent, Environment, State, Action, Reward, and Policy. The agent will select actions based on the current state, the environment will feedback new states and rewards based on the actions, and the agent will adjust its strategy based on these feedbacks to optimize the long-term cumulative rewards.
[0145] The learning strategy execution module 305 is used to determine a second heating strategy for the multiple heat sources of the heating system according to the heating strategy reinforcement learning model, and control the multiple heat sources of the heating system according to the second heating strategy.
[0146] In one embodiment, the device further comprises:
[0147] a model optimization judgment submodule, used to judge whether the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions;
[0148] The learning model acquisition module includes:
[0149] A learning model construction submodule is used to obtain a heating strategy reinforcement learning model based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy, when the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions.
[0150] When the amount of data meets the preset model optimization conditions, that is, it is sufficient to support the training of the reinforcement learning model, the heating system will train and use the reinforcement learning model to further formulate heat source switching strategies.
[0151] In one embodiment, the learning model construction submodule includes:
[0152] a data preprocessing unit, configured to preprocess heating data of the multiple heat sources obtained after the first heating strategy controls the multiple heat sources of the heating system;
[0153] a data set determining unit, configured to determine a training data set and a test data set based on heating data of the plurality of heat sources obtained after the first heating strategy is preprocessed to control the plurality of heat sources of the heating system;
[0154] A multi-objective learning unit, used to perform multi-objective learning training on the reinforcement learning model according to the training data set;
[0155] A performance evaluation unit, used to perform performance evaluation on the reinforcement learning model after multi-objective learning training according to the test data set;
[0156] The model output unit is used to output the heating strategy reinforcement learning model when the evaluation result meets the preset evaluation index.
[0157] The embodiment of the present invention constructs a reinforcement learning model applied to the heating system by means of multi-objective reinforcement learning, which can meet multiple goals involved in the heating system, such as improving heating efficiency, reducing energy consumption, reducing emissions, and improving user comfort.
[0158] In one embodiment, the multi-objective learning unit includes:
[0159] A multi-objective confirmation subunit is used to use at least one of the operating cost, carbon emissions, and user satisfaction as an immediate reward for the multi-objective learning training; the operating cost is used to motivate the reinforcement learning model to select a strategy with lower operating cost; the carbon emissions are used to motivate the reinforcement learning model to select a strategy with lower carbon emissions; and the user satisfaction is used to motivate the reinforcement learning model to select a strategy that meets user needs to a greater extent.
[0160] By taking negative operating costs, negative carbon emissions, and positive user satisfaction as rewards for multi-objective learning, the system can be motivated to choose a switching strategy with lower operating costs, lower carbon emissions, and better satisfaction with user needs;
[0161] In one embodiment, the classification model acquisition module includes:
[0162] A classification model acquisition submodule, for obtaining a heating strategy classification model according to the heating data of multiple heat sources of the heating system when the heating data meets a preset initial data condition;
[0163] The device also includes:
[0164] An initial classification parameter acquisition submodule, used for acquiring regional initial heating strategy classification model parameters from a server when the heating data does not meet the preset initial data conditions;
[0165] The initial component model determination submodule is used to determine a third heating strategy for multiple heat sources of the heating system according to the regional initial heating strategy classification model, and control the multiple heat sources of the heating system according to the third heating strategy.
[0166] The embodiments of the present invention can provide users with an initial classification model to achieve a preliminary intelligent classification effect when the amount of data is insufficient, such as when a user applies a heating system equipped with the method for the first time, thereby improving the environmental adaptability of the heating system and meeting the needs of users.
[0167] In one embodiment, the heating data of the plurality of heat sources of the heating system include user data, and the device further includes:
[0168] The initial component model determination unit is used to perform user-personalized fine-tuning on the regional initial heating strategy classification model based on user data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the third heating strategy to obtain the heating strategy classification model.
[0169] After using the regional initial heating strategy classification model to control the heat source switching strategy for a period of time, the relevant parameters of the regional initial heating strategy classification model are adjusted according to the real-time feedback of the user, so that the model can better fit the needs of the user and meet the personalized needs of the user.
[0170] In one embodiment, the device further comprises:
[0171] The data transmission submodule is used to transmit the heating data of multiple heat sources of the heating system to the server, so that the server obtains the initial heating strategy classification model parameters of the region based on the heating data of multiple heat sources of the heating system transmitted by multiple users in the same region.
[0172] By establishing a classification model based on the heating data of multiple heat sources in the heating system, it is possible to intelligently identify and classify different heating demand scenarios based on the heating data of multiple heat sources, which helps to initially meet user needs and respond to environmental changes. Furthermore, through the implementation of the classification model heating strategy and data feedback, a reinforcement learning model is constructed in combination with other heating data so that the reinforcement learning model can further optimize the strategy on the basis of the classification strategy, and can meet user needs to a deeper degree and respond to environmental changes more sensitively while achieving maximum energy benefits. The embodiment of the present invention abandons the static rules and manual experience that traditional heat source switching technology relies on, and adopts machine learning to formulate and optimize heat source scheduling strategies, thereby improving the intelligence level of the system and achieving both energy conservation and emission reduction and user experience.
[0173] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0174] An embodiment of the present invention further provides an electronic device, including:
[0175] It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned heat source switching method embodiment based on machine learning are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0176] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned embodiment of a heat source switching method based on machine learning are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0177] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0178] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
[0179] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0180] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0181] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0182] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.
[0183] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or terminal device including the elements.
[0184] The above is a detailed introduction to the heat source switching method, device, equipment and medium based on machine learning provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A heat source switching method based on machine learning, characterized in that: The method comprises: Obtain heating data of multiple heat sources of a heating system; Obtaining a heating strategy classification model according to heating data of multiple heat sources of the heating system; Determining a first heating strategy for multiple heat sources of the heating system according to the heating strategy classification model, and controlling the multiple heat sources of the heating system according to the first heating strategy; Obtaining a heating strategy reinforcement learning model according to heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy; According to the heating strategy reinforcement learning model, a second heating strategy for the multiple heat sources of the heating system is determined, and the multiple heat sources of the heating system are controlled according to the second heating strategy.
2. The method according to claim 1, characterized in that: The method further comprises: Determining whether the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets a preset model optimization condition; The step of obtaining a heating strategy reinforcement learning model based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy includes: When the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy meets the preset model optimization conditions, a heating strategy reinforcement learning model is obtained based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy.
3. The method according to claim 2, characterized in that The step of obtaining a heating strategy reinforcement learning model based on the heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy includes: preprocessing heating data of the multiple heat sources obtained after the first heating strategy controls the multiple heat sources of the heating system; Determine a training data set and a test data set based on heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the preprocessed first heating strategy; Performing multi-objective learning training on the reinforcement learning model according to the training data set; Performing a performance evaluation on the reinforcement learning model after multi-objective learning training according to the test data set; When the evaluation result meets the preset evaluation index, the heating strategy reinforcement learning model is output.
4. The method according to claim 3, characterized in that The multi-objective learning training of the reinforcement learning model includes: At least one of the operating cost, carbon emissions, and user satisfaction is used as an immediate reward for the multi-objective learning training; the operating cost is used to motivate the reinforcement learning model to select a strategy with lower operating cost; the carbon emissions are used to motivate the reinforcement learning model to select a strategy with lower carbon emissions; and the user satisfaction is used to motivate the reinforcement learning model to select a strategy that meets user needs to a greater extent.
5. The method according to claim 1 or 3, characterized in that: The step of obtaining a heating strategy classification model based on the heating data of the multiple heat sources of the heating system comprises: When the heating data meets the preset initial data condition, a heating strategy classification model is obtained according to the heating data of the multiple heat sources of the heating system; The method further comprises: When the heating data does not meet the preset initial data conditions, obtaining the regional initial heating strategy classification model parameters from the server; Determining a regional initial heating strategy classification model according to the regional initial heating strategy classification model parameters; According to the regional initial heating strategy classification model, a third heating strategy for the multiple heat sources of the heating system is determined, and the multiple heat sources of the heating system are controlled according to the third heating strategy.
6. The method according to claim 5, characterized in that The heating data of the plurality of heat sources of the heating system include user data, and the method further includes: According to the user data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the third heating strategy, the regional initial heating strategy classification model is fine-tuned for user personalization to obtain the heating strategy classification model.
7. The method according to claim 5, characterized in that The method further comprises: The heating data of the multiple heat sources of the heating system are transmitted to the server, so that the server obtains the classification model parameters of the initial heating strategy for the region according to the heating data of the multiple heat sources of the heating system transmitted by multiple users in the same region.
8. A heat source switching device based on machine learning, characterized in that: The device comprises: A heating data acquisition module, used to acquire heating data of multiple heat sources of a heating system; A classification model acquisition module, used to obtain a heating strategy classification model according to the heating data of multiple heat sources of the heating system; a classification strategy execution module, configured to determine a first heating strategy for a plurality of heat sources of the heating system according to the heating strategy classification model, and control the plurality of heat sources of the heating system according to the first heating strategy; a learning model acquisition module, configured to obtain a heating strategy reinforcement learning model according to heating data of the multiple heat sources obtained after controlling the multiple heat sources of the heating system according to the first heating strategy; A learning strategy execution module is used to determine a second heating strategy for multiple heat sources of the heating system according to the heating strategy reinforcement learning model, and control the multiple heat sources of the heating system according to the second heating strategy.
9. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of a heat source switching method based on machine learning as described in any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a heat source switching method based on machine learning as described in any one of claims 1 to 7 are implemented.