Water demand prediction method and device based on machine learning and big data coupling, and medium
By combining machine learning and big data in water demand prediction, a composite water demand prediction model is solved, and the problem of difficult to take into account both the prediction accuracy and convergence speed in the existing technology is achieved, and the water demand prediction effect with high precision and rapid convergence is achieved.
Patent Information
- Application Number
- CN202411543482.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-31
- Publication Date
- 2025-02-14
AI Technical Summary
Existing water volume prediction methods are difficult to take into account both prediction accuracy and convergence speed. Traditional methods have uncertainties when determining influencing factors, while new methods such as neural networks and machine learning are prone to fall into local optimal solutions or overfitting and underfitting.
Using the water demand prediction method based on the coupling of machine learning and big data, we will build a basic prediction model that considers holidays and extreme weather, and use big data to combine algorithms to build a composite water prediction model to improve prediction accuracy and rapid convergence. .
It achieves a high prediction accuracy of water demand, and at the same time, the model can converge quickly, has good generalization ability, and can effectively capture the trend, seasonality and randomness of the water time series.
Smart Images
Figure CN119443391A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of water demand prediction, and in particular to a water demand prediction method, device and medium based on machine learning and big data coupling. Background Art
[0002] The water supply system is related to the lives of urban residents. The water volume of the urban water supply network provides services for the scheduling decision-making of the water supply system. It is the basis and prerequisite for ensuring the balance of supply and demand, safe operation of the network, energy-saving and optimized scheduling, etc. Scientific and accurate short-term water volume forecasting is an important basis for achieving efficient operation and intelligent scheduling of the water supply system.
[0003] At present, the main methods commonly used in water demand forecasting are divided into traditional forecasting methods (regression analysis, ARIMA, etc.) and new forecasting methods (neural networks, machine learning, combined algorithms, etc.). When using regression analysis for forecasting modeling, it is necessary to determine the accurate factors affecting water demand in order to establish a regression equation. Different influencing factors will have a great impact on the regression results. The ARIMA model has a high degree of fit for linear time series, while urban water demand will change with the influence of time and weather conditions, and has the characteristics of complexity, nonlinearity, and time-varying. The neural network model has a strong nonlinear mapping ability. When using it for forecasting, the optimal network structure is difficult to determine and is prone to local optimal solutions. The main difficulty of machine learning algorithms in the actual modeling process is that it is difficult to balance the training error of the training set and the generalization error of the test set, and it is easy to overfit or underfit in the forecasting process. The main advantages of machine learning models are that the model is simpler, the modeling and forecasting speed is faster, and multi-step forecasting can be achieved. In addition, when the data volume of the research object is large and the completeness is good, the prediction accuracy of the model is high, which is suitable for extrapolation forecasting.
[0004] In summary, there is currently a lack of a water demand prediction method that can take into account both prediction accuracy and convergence speed. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a water demand prediction method, equipment and medium based on the coupling of machine learning and big data. Based on the principle of machine learning, a basic prediction model considering factors such as holidays and extreme weather is established, and machine learning is introduced on the basis of big data to perform algorithm combination domestication, calibrate and predict the model, so that it can achieve high prediction accuracy and the model can converge quickly.
[0006] The purpose of the present invention can be achieved by the following technical solutions:
[0007] According to a first aspect of the present invention, a method for water demand prediction based on coupling of machine learning and big data is provided, comprising the following steps:
[0008] Obtain historical data on factors affecting water demand and water demand;
[0009] Perform data preprocessing on historical data of water demand;
[0010] Using big data analysis methods, we analyze the water use characteristics of historical water demand data based on historical data of factors affecting water demand, and obtain water use characteristic indicators;
[0011] Construct multiple single-algorithm water volume prediction models based on different machine learning algorithms, use historical data to train each single-algorithm water volume prediction model, evaluate the trained models, and select alternative models from the single-algorithm water volume prediction models based on the evaluation results;
[0012] A composite water volume prediction model is constructed based on the alternative models and the prediction coefficients corresponding to each alternative model, and the composite water volume prediction model is trained using historical data. During the training process, the parameters of each alternative model in the composite water volume prediction model are fixed, and only the prediction coefficients are trained; wherein both the single algorithm water volume prediction model and the composite water volume prediction model use factors affecting water demand, water use characteristic indicators and water demand at historical moments as model inputs, and use water demand at the prediction moment as model output;
[0013] Obtain the data of factors affecting water demand at the time to be predicted and the historical data of water demand for a preset time length before the time to be predicted, and use the trained composite water demand prediction model to predict water demand.
[0014] The data of factors affecting water demand include meteorological data, date data and social factor data. The meteorological data includes time information and the average temperature, maximum temperature, minimum temperature, average relative humidity, precipitation, and weather conditions corresponding to the time information; the date data includes date, week attribute and holiday attribute, wherein the week attribute indicates which day of the week the current date belongs to, and the holiday attribute indicates whether the current date is a holiday; the social factor data includes the main water use properties in different regions and the situation of users with large water demand.
[0015] The historical data of water demand includes the identification of the water plant or the independently measured water use area, date, time, total water volume and water volume at each moment.
[0016] The data preprocessing is specifically as follows: in the water demand data sequence, filtering and cleaning are performed according to data inspection rules to remove abnormal point data; quality inspection and scanning of the water demand data are performed, and based on the rule that the water demand at each moment of the day is a one-dimensional random non-stationary process and obeys the normal distribution, irregular data are corrected.
[0017] The specific method for obtaining the water use characteristic index is as follows: data processing is performed according to the date data of the acquisition period, and the date type of the current date is distinguished according to the weekly attribute and the holiday attribute, and the date type is one of a working day, a weekend or a holiday. Based on the date type, a holiday correction coefficient is determined according to the big data analysis method; based on the temperature data in the meteorological data, the temperature data is classified according to the preset temperature range, the water demand data in different temperature ranges are distinguished, and the correction coefficients for different temperature ranges are determined according to the big data analysis method.
[0018] The single algorithm water volume prediction model is expressed as:
[0019] Q di = k1 k2 F(T i ,D i ,W i ,H i ,q1,…,q n )
[0020] Among them, Q di is the predicted water demand at prediction time point i; T i is the temperature at the predicted time point i; D i is the humidity at predicted time point i; H i is the weather condition at the predicted time point i; W i is the date type of the predicted time point i; q1,…,q n is the n historical water demand data before the prediction time point i; k1 is the holiday correction coefficient; k2 is the correction coefficient for different temperature ranges; F represents the prediction model.
[0021] The composite water volume prediction model is expressed as:
[0022]
[0023] Among them, Q d is the predicted water demand of the composite water quantity prediction model; α j is the prediction coefficient of the jth alternative model, N is the number of alternative models; is the predicted water demand of the jth alternative model at the prediction time point i.
[0024] The single algorithm water quantity prediction model and the composite water quantity prediction model use mean absolute percentage error to evaluate the error between actual water demand and predicted water demand during the evaluation process.
[0025] According to a second aspect of the present invention, there is provided an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and the method described above is implemented when the processor executes the program.
[0026] According to a third aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the program implements the method described above when executed by a processor.
[0027] Compared with the prior art, the present invention has the following beneficial effects:
[0028] 1. The present invention introduces machine learning coupled big data model and applies it to water volume forecasting, which can well capture the trend, seasonality and randomness of the time series of minute-level and daily-level water volume, and has good compatibility with the influence of exogenous variables.
[0029] 2. The present invention utilizes machine learning to combine algorithms, adds a variety of excellent and mature algorithms, uses machine learning to train historical data, mixes multiple algorithms to establish a composite prediction model, and continuously uses measured data to compare and automatically adjust with predicted data. The prediction accuracy is higher than that of a single algorithm.
[0030] 3. The present invention has good generalization ability, and when training the composite prediction model, the trained single algorithm model parameters are fixed, and only the prediction coefficients are trained, and the performance indicators in terms of convergence speed and convergence accuracy are better. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 is a flow chart of the method of the present invention;
[0032] Figure 2 It is a schematic diagram of the training process of the composite water volume prediction model of the present invention;
[0033] Figure 3 It is a schematic diagram of the prediction process of the present invention;
[0034] Figure 4 A diagram showing the prediction result of water demand at a certain moment in an embodiment;
[0035] Figure 5 This is a diagram of daily water demand prediction results in an embodiment. DETAILED DESCRIPTION
[0036] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0037] This embodiment provides a water demand prediction method based on machine learning and big data coupling. Figure 1 As shown, the following steps are included:
[0038] S1, obtain historical data on factors affecting water demand and water demand.
[0039] Factors that usually affect water demand can be divided into three categories: climate conditions (including temperature, precipitation, etc.), date types (including holidays, seasons, etc.), and social factors (including population, policies, etc.).
[0040] In this embodiment, the data of factors affecting water demand include meteorological data, date data and social factor data. The meteorological data includes time information and the average temperature, maximum temperature, minimum temperature, average relative humidity, precipitation, and weather conditions corresponding to the time information; the date data includes date, week attribute and holiday attribute, wherein the week attribute indicates which day of the week the current date belongs to, and the holiday attribute indicates whether the current date is a holiday; the social factor data includes the main water use properties in different regions and the situation of users with large water demand.
[0041] The historical data of water demand includes the identification of the water plant or the water use area of independent metering, date, time, total water volume and water volume at each moment. The length of the time interval will affect the accuracy of water volume forecasting. According to the collection conditions and calculation granularity of the monitoring equipment, the water volume collection interval that can be selected is 1 day, 1 hour, 15 minutes, etc.
[0042] S2, preprocessing the historical data of water demand.
[0043] In the water demand sequence, due to the influence of random factors such as data acquisition equipment or communication process, abnormal points that are contrary to the actual situation are generated. These abnormal points are mixed into the normal sequence, which will increase the overall noise of the sequence and affect the statistical analysis conclusions of the data. Therefore, before using these data, the abnormal point data must be corrected or eliminated to remove the influence of "pseudo data".
[0044] This embodiment performs the following data cleaning process to ensure the authenticity, completeness and accuracy of the data: in the water demand data sequence, filtering and cleaning are performed according to the data inspection rules to remove abnormal point data and prevent the entry of irregular data; the water demand data is scanned for quality inspection, and irregular data is corrected based on the rule that the water demand at each moment of the day is a one-dimensional random non-stationary process and obeys the normal distribution.
[0045] S3, using big data analysis methods, analyzes water use characteristics of historical water demand data based on historical data of factors affecting water demand, and obtains water use characteristic indicators.
[0046] Data processing is performed based on the date data of the acquisition period, and the date type of the current date is distinguished according to the weekly attribute and the holiday attribute. The date type is one of a working day, a weekend or a holiday. Based on the date type, the holiday correction coefficient k1 is determined according to the big data analysis method; based on the temperature data in the meteorological data, classification is performed according to the preset temperature range, the water demand data in different temperature ranges are distinguished, and the correction coefficient k2 for different temperature ranges is determined according to the big data analysis method.
[0047] S4, construct multiple single-algorithm water volume prediction models based on different machine learning algorithms, use historical data to train each single-algorithm water volume prediction model, evaluate the trained model, and select alternative models from the single-algorithm water volume prediction model according to the evaluation results.
[0048] Historical data is the original sample data of the prediction algorithm. The amount of sample data and the degree of consistency with the prediction time data directly affect the accuracy of the final prediction. The present invention will add historical water demand data of n time periods during the prediction process to make the prediction results more accurate and close to the real data.
[0049] The single algorithm water volume prediction model is established according to the following principles:
[0050] 1) Rationality of the model: The prediction model is a simulation of the development laws of actual things, and the prediction results calculated by the model should be consistent with the development trend of things.
[0051] 2) Predictive ability: Determine whether the model explains the development of things during the forecast period. Whether the model prediction results are within the allowable error range.
[0052] 3) Stability: whether a model accurately reflects the development and changes of the object over a long period of time. Whether it still has strong predictive ability when external conditions change.
[0053] 4) Simplicity: Simple models should be preferred among models with the same predictive ability.
[0054] The constructed single algorithm water volume prediction model is expressed as:
[0055] Q di = k1 k2 F(T i ,D i ,W i ,H i ,q1,…,q n )
[0056] Among them, Q di is the predicted water demand at the predicted time point i (time / day); T i is the temperature at the predicted time point i; D iis the humidity at predicted time point i; H i is the weather condition at the predicted time point i; W i is the date type of the predicted time point i; q1,…,q n is the n historical water demand data before the prediction time point i; k1 is the holiday correction coefficient; k2 is the correction coefficient for different temperature ranges; F represents the prediction model.
[0057] S5. A composite water volume prediction model is constructed based on alternative models and the prediction coefficients corresponding to each alternative model, and the composite water volume prediction model is trained using historical data. During the training process, the parameters of each alternative model in the composite water volume prediction model are fixed, and only the prediction coefficients are trained. The single algorithm water volume prediction model and the composite water volume prediction model both use factors affecting water demand, water use characteristic indicators and water demand at historical moments as model inputs, and use water demand at the prediction moment as model output.
[0058] The composite water volume prediction model is expressed as:
[0059]
[0060] Among them, Q d is the predicted water demand of the composite water quantity prediction model; α j is the prediction coefficient of the jth alternative model, N is the number of alternative models; is the predicted water demand of the jth alternative model at the prediction time point i.
[0061] like Figure 2 As shown in the figure, four single-algorithm water volume prediction models are used as alternative models to construct a composite water volume prediction model, and the prediction coefficients are iteratively optimized until the minimum accuracy is met and the model training is completed.
[0062] In this embodiment, the single-algorithm water volume prediction model and the composite water volume prediction model use mean absolute percentage error to evaluate the error between the actual water demand and the predicted water demand during the evaluation process.
[0063] S6, obtaining data on factors influencing water demand at the time to be predicted and historical data on water demand for a preset time period before the time to be predicted, and using the trained composite water demand prediction model to perform water demand prediction.
[0064] like Figure 3 The figure shows the prediction process of the present invention, which obtains the prediction date, prediction time point, prediction plant information and weather condition data, and inputs them into the trained model to obtain the daily water demand prediction results and the hourly water demand prediction results.
[0065] In order to verify the performance of the present invention, this embodiment conducted experiments on a large-scale water supply enterprise within a one-year time range, including 11 groups of water demand and water demand at each time, including the head office and 10 regions, and analyzed and compared the machine learning and big data coupling algorithm with seasonal exponential smoothing, adaptive filtering, AR, moving average, and ARMA algorithms. The experiments were trained and tested in accordance with the experimental regulations of the corresponding data sets. From Table 1, Figure 4 , Figure 5 It can be seen from the comparison that the daily average absolute percentage error of the prediction result of the present invention is within 3%, and the average absolute percentage error of the moment prediction is within 10%. When predicting the moment and daily water demand, a better average absolute percentage error (MAPE) can be obtained than the traditional method. This fully proves that the algorithm based on machine learning and big data coupling of the present invention can more accurately predict daily water demand and moment water demand.
[0066] Table 1 Experimental data of the dataset
[0067]
[0068] The electronic device of the present invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The CPU, ROM and RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.
[0069] Multiple components in the device are connected to the I / O interface, including: input units, such as keyboards, mice, etc.; output units, such as various types of displays, speakers, etc.; storage units, such as disks, optical disks, etc.; and communication units, such as network cards, modems, wireless communication transceivers, etc. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunication networks.
[0070] The processing unit performs the various methods and processes described above, such as methods S1 to S6. For example, in some embodiments, methods S1 to S6 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via a ROM and / or a communication unit. When the computer program is loaded into the RAM and executed by the CPU, one or more steps of methods S1 to S6 described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute methods S1 to S6 in any other appropriate manner (e.g., by means of firmware).
[0071] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
[0072] The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.
[0073] In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0074] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. A water demand prediction method based on machine learning and big data coupling, characterized in that: The following steps are involved: Obtain historical data on factors affecting water demand and water demand; Perform data preprocessing on historical data of water demand; Using big data analysis methods, we analyze the water use characteristics of historical water demand data based on historical data of factors affecting water demand, and obtain water use characteristic indicators; Construct multiple single-algorithm water volume prediction models based on different machine learning algorithms, use historical data to train each single-algorithm water volume prediction model, evaluate the trained models, and select alternative models from the single-algorithm water volume prediction models based on the evaluation results; A composite water volume prediction model is constructed based on the alternative models and the prediction coefficients corresponding to each alternative model, and the composite water volume prediction model is trained using historical data. During the training process, the parameters of each alternative model in the composite water volume prediction model are fixed, and only the prediction coefficients are trained; wherein both the single algorithm water volume prediction model and the composite water volume prediction model use factors affecting water demand, water use characteristic indicators and water demand at historical moments as model inputs, and use water demand at the prediction moment as model output; Obtain the data of factors affecting water demand at the time to be predicted and the historical data of water demand for a preset time length before the time to be predicted, and use the trained composite water demand prediction model to predict water demand.
2. According to claim 1, a water demand prediction method based on machine learning and big data coupling is characterized in that: The data of factors affecting water demand include meteorological data, date data and social factor data. The meteorological data includes time information and the average temperature, maximum temperature, minimum temperature, average relative humidity, precipitation, and weather conditions corresponding to the time information; the date data includes date, week attribute and holiday attribute, wherein the week attribute indicates which day of the week the current date belongs to, and the holiday attribute indicates whether the current date is a holiday; the social factor data includes the main water use properties in different regions and the situation of users with large water demand.
3. The water demand prediction method based on machine learning and big data coupling according to claim 1 is characterized in that: The historical data of water demand includes the identification of the water plant or the independently measured water use area, date, time, total water volume and water volume at each moment.
4. The water demand prediction method based on machine learning and big data coupling according to claim 1 is characterized in that: The data preprocessing is specifically as follows: in the water demand data sequence, filtering and cleaning are performed according to data inspection rules to remove abnormal point data; quality inspection and scanning of the water demand data are performed, and based on the rule that the water demand at each moment of the day is a one-dimensional random non-stationary process and obeys the normal distribution, irregular data are corrected.
5. The water demand prediction method based on machine learning and big data coupling according to claim 2 is characterized in that: The specific method for obtaining the water use characteristic index is as follows: data processing is performed according to the date data of the acquisition period, and the date type of the current date is distinguished according to the weekly attribute and the holiday attribute, and the date type is one of a working day, a weekend or a holiday. Based on the date type, a holiday correction coefficient is determined according to the big data analysis method; based on the temperature data in the meteorological data, the temperature data is classified according to the preset temperature range, the water demand data in different temperature ranges are distinguished, and the correction coefficients for different temperature ranges are determined according to the big data analysis method.
6. The water demand prediction method based on machine learning and big data coupling according to claim 5 is characterized in that: The single algorithm water volume prediction model is expressed as: Q di =k1 k2 F(T i ,D i ,W i ,H i ,q1,…,q n ) Among them, Q di is the predicted water demand at prediction time point i; T i is the temperature at the predicted time point i; D i is the humidity at predicted time point i; H i is the weather condition at the predicted time point i; W i is the date type of the predicted time point i; q1,…,q n is the n historical water demand data before the prediction time point i; k1 is the holiday correction coefficient; k2 is the correction coefficient for different temperature ranges; F represents the prediction model.
7. The water demand prediction method based on machine learning and big data coupling according to claim 6 is characterized in that: The composite water volume prediction model is expressed as: Among them, Q d is the predicted water demand of the composite water quantity prediction model; α j is the prediction coefficient of the jth alternative model, N is the number of alternative models; is the predicted water demand of the jth alternative model at the prediction time point i.
8. The water demand prediction method based on machine learning and big data coupling according to claim 1 is characterized in that: The single algorithm water quantity prediction model and the composite water quantity prediction model use mean absolute percentage error to evaluate the error between actual water demand and predicted water demand during the evaluation process.
9. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.