A method for detecting anomaly in gas consumption data based on big data
By analyzing the habitual regularity and change characteristics of gas consumption data based on big data, combined with the SARIMA model, the problem of inaccurate detection of gas consumption data in the prior art is solved, and the accuracy of detection and identification of hardware facilities are improved. ability.
Patent Information
- Application Number
- CN202411170741.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-26
AI Technical Summary
The prior art has inaccuracy in the detection of abnormal gas usage data, especially because abnormal data caused by hardware facilities failures have not exceeded the rated range and are not detected.
Using a big data-based method, by obtaining the gas usage value for each sampling period, analyzing the habitual regularity, gas usage variation and regular trend variation, calculating abnormality indicators and adjustment degrees, combining with the SARIMA model for abnormality detection, reducing the adjustment of heating Effect on test results.
It improves the accuracy of abnormal detection of gas consumption data, can better reflect abnormal data of hardware facilities, and reduces misjudgment.
Smart Images

Figure CN118690311B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of gas data anomaly detection, and in particular to a gas usage data anomaly detection method based on big data. Background Art
[0002] Gas usage data usually refers to data that records and quantifies gas consumption within a specific time period. Through real-time monitoring and analysis of gas usage, gas companies can reasonably arrange gas supply plans to ensure the stability and safety of gas supply. Gas usage data is an important basis for gas companies to manage supply and demand balance. During the use of gas, there may be problems such as hardware failure or damage that may cause abnormal gas usage data, and it is necessary to detect abnormal gas usage data.
[0003] In the process of detecting abnormal data, the existing technology is usually based on the rated working range of the physical equipment. If the gas usage data exceeds the rated range, it will be marked as abnormal. Due to the failure of some hardware facilities, the abnormal gas usage data does not exceed the rated range, resulting in inaccurate abnormal detection of gas usage data. Summary of the invention
[0004] In order to solve the technical problem of inaccurate detection of abnormal gas usage data in the prior art, the purpose of the present invention is to provide a method for detecting abnormal gas usage data based on big data. The technical solution adopted is as follows:
[0005] A method for detecting anomaly in gas usage data based on big data, the method comprising:
[0006] Obtain the gas consumption values of all sampling intervals in each sampling period;
[0007] Obtain each adjacent period of the sampling period in a preset time sequence range; obtain the habitual regularity of the sampling period according to the similarity between the sampling period and the gas usage values of all sampling intervals corresponding to each adjacent period;
[0008] According to the difference between the local distribution characteristics of the gas usage values in the preset time series range of the sampling period, the gas usage change degree of the sampling period is obtained; according to the difference between the habitual regularity of the sampling period in the preset time series range, the regular trend change degree of the sampling period is obtained; according to the gas usage change degree and the regular trend change degree, the gas adjustment degree of the sampling period is obtained;
[0009] According to the gas adjustment degree and the habitual regularity, an abnormal index of a sampling period is obtained; according to the abnormal index of all sampling periods and the abnormal situation of the gas usage value, an abnormal gas usage value is obtained.
[0010] Furthermore, the method for obtaining the habit regularity includes:
[0011] According to the time sequence of each sampling interval under the sampling period, obtain the time sequence number of each sampling interval;
[0012] According to the difference between the gas usage value of the sampling period and the adjacent period of the surrounding sampling period at the same time sequence number, the custom difference index between the sampling period and the adjacent period is obtained;
[0013] The habit difference index of the sampling period and all adjacent periods is comprehensively considered to obtain the habit regularity of the sampling period; the habit difference index and the habit regularity are negatively correlated.
[0014] Furthermore, the method for obtaining the habit difference index of the comprehensive sampling period and all adjacent periods to obtain the habit regularity of the sampling period includes:
[0015] The mean of the habit difference index between the sampling period and all adjacent periods is calculated, and the mean is negatively correlated to obtain the habit regularity of the sampling period.
[0016] Furthermore, the method for obtaining the gas consumption variation degree includes:
[0017] The gas usage values of all sampling intervals in adjacent cycles are integrated to obtain the periodic gas value of the adjacent cycles;
[0018] In the preset time range of the sampling period, according to the overall distribution of the periodic gas values of all adjacent periods on the time front side of the sampling period, the overall gas value on the time front side of the sampling period is obtained;
[0019] In the preset time range of the sampling period, according to the overall distribution of the periodic gas values of all adjacent periods at the time rear side of the sampling period, the overall gas value at the time rear side of the sampling period is obtained;
[0020] The degree of change of the gas usage in the sampling period is obtained according to the difference between the overall gas value at the rear side of the timing sequence and the overall gas value at the front side of the timing sequence in the sampling period.
[0021] Furthermore, the method for obtaining the regular trend change degree includes:
[0022] Obtaining the regular mutation degree of the sampling period according to the prominence of the habitual regularity of the sampling period in the preset time series range;
[0023] According to the difference between the local trend characteristics of the habitual regularity of the sampling period in the preset time series range, the regular trend change index of the sampling period is obtained;
[0024] The regular trend change degree is obtained according to the regular mutation degree and the regular trend change index; the regular mutation degree and the regular trend change index are both positively correlated with the regular trend change degree.
[0025] Furthermore, the method for obtaining the regular mutation degree includes:
[0026] Calculate the average of the habitual regularity corresponding to all adjacent periods in the preset time sequence range of the sampling period to obtain the surrounding regularity index of the sampling period;
[0027] The absolute value of the difference between the habitual regularity of the sampling period and the surrounding regularity index is calculated to obtain the regular mutation degree of the sampling period.
[0028] Furthermore, the method for obtaining the regular trend change indicator includes:
[0029] In the preset time range of the sampling period, according to the deviation between the habitual regularity of all adjacent periods on the time front side of the sampling period, the downward trend degree of the time front side of the sampling period is obtained;
[0030] In the preset time range of the sampling period, according to the deviation between the habitual regularity of all adjacent periods on the time rear side of the sampling period, the upward trend degree of the time rear side of the sampling period is obtained;
[0031] According to the downward trend degree of the front side of the time series and the upward trend degree of the back side of the time series, a regular trend change index of the sampling period is obtained; the downward trend degree of the front side of the time series and the upward trend degree of the back side of the time series are both positively correlated with the regular trend change index.
[0032] Furthermore, the method for obtaining the time series front side downward trend degree includes:
[0033] Calculate the difference between the habit regularity of each adjacent period and the previous adjacent period to obtain the change parameters of the adjacent periods;
[0034] In all adjacent periods on the time front side of the sampling period, all adjacent periods with negative change parameters are counted to obtain the downward trend degree of the time front side of the sampling period.
[0035] Furthermore, the method for obtaining the gas adjustment degree includes:
[0036] The product of the gas usage variation degree and the regular trend variation degree is calculated, and the product is normalized to obtain the gas adjustment degree of the sampling period.
[0037] Furthermore, the method for obtaining the abnormal gas usage value includes:
[0038] Using the SARIMA model, anomaly detection is performed on the gas usage values of all sampling intervals in all sampling periods to obtain anomaly scores of the gas usage values;
[0039] Calculate the product of the abnormal score of the gas usage value and the abnormal index of the sampling period to which it belongs, and obtain the abnormal degree;
[0040] The gas usage value whose abnormality is greater than the preset abnormality threshold is marked as an abnormal gas usage value.
[0041] The present invention has the following beneficial effects:
[0042] The main purpose of the present invention is to detect abnormalities in gas consumption values. First, considering that the gas consumption values of the sampling intervals included in the sampling period of a unit restaurant are similar to those of the surrounding sampling periods, the habitual regularity of the sampling period is obtained by analyzing the similarity between the gas consumption values of the sampling period and all the sampling intervals corresponding to the adjacent periods. The greater the habitual regularity, the more regular the gas consumption values in the sampling period are, and the less likely the gas consumption values contained in the sampling period are abnormal data. In order to analyze the possibility of adjusting the heating in the sampling period, considering that the unit restaurant has a relatively fixed meal supply in the sampling period during the non-heating period and the gas consumption of the unit restaurant is mainly used for meal preparation, that is, the gas consumption of each sampling period of the unit restaurant in the non-heating period is relatively close, considering that the gas consumption of the unit restaurant in the heating period is for meal preparation and heating, the meal supply of the unit restaurant in the sampling period is relatively fixed and the gas consumption for heating is relatively fixed, that is, the gas consumption of each sampling period of the unit restaurant in the heating period is relatively close and the gas consumption is relatively large, the gas adjustment degree of the sampling period is obtained; the greater the gas adjustment degree, the more likely the heating is adjusted in the sampling period. In order to analyze the abnormal gas usage values caused by hardware facilities such as equipment, pipelines, and valves, it is necessary to reduce the impact of adjusting the heating on the abnormal analysis of the gas usage values when conducting abnormal analysis, so that the abnormal gas usage values can better reflect the abnormal data of the hardware facilities and improve the accuracy of abnormal detection of gas usage values. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0044] Figure 1 A flowchart of a method for detecting anomalies in gas usage data based on big data provided by an embodiment of the present invention;
[0045] Figure 2A flow chart of a method for obtaining habit regularity provided by an embodiment of the present invention;
[0046] Figure 3 A flow chart of a method for obtaining a gas usage variation degree provided by an embodiment of the present invention;
[0047] Figure 4 A flow chart of a method for obtaining a regular trend change degree provided by an embodiment of the present invention;
[0048] Figure 5 A structural diagram of a gas usage data anomaly detection system based on big data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0049] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, features and effects of a method for detecting abnormal gas usage data based on big data proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0050] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0051] The following is a detailed description of a method for detecting anomalies in gas usage data based on big data provided by the present invention in conjunction with the accompanying drawings.
[0052] The present invention provides a method for detecting abnormal gas consumption data based on big data. Figure 1 , which shows a flow chart of a method for detecting abnormal gas usage data based on big data provided by an embodiment of the present invention, the method comprising the following steps:
[0053] Step S1: Obtain the gas consumption values of all sampling intervals in each sampling period.
[0054] In order to detect anomalies in gas usage data, it is first necessary to obtain the gas usage values of all sampling intervals in each sampling period.
[0055] In the gas place to be detected, an intelligent metering device is installed to detect the gas consumption of the gas place to be detected, and the gas consumption is uploaded to the gas detection system, and the gas consumption values of all sampling intervals in each sampling period in the total historical sampling interval of the gas place to be detected are obtained from the gas detection system. In one embodiment of the present invention, 365 days are used as the total historical sampling interval, and the beginning to the end of a day is used as a sampling period; one hour is used as the sampling interval, for example, in the sampling period, all sampling intervals are 0:00-1:00, 1:00-2:00, 2:00-3:00...21:00-22:00, 22:00-23:00, 23:00-24:00.
[0056] It should be noted that, in order to facilitate calculation, all indicator data involved in the calculation in the embodiment of the present invention are preprocessed to eliminate the dimension effect. The specific means of eliminating the dimension effect are technical means well known to those skilled in the art and are not limited here.
[0057] The present invention takes a unit restaurant in which the food supply quantity and meal preparation time period are relatively fixed in each sampling cycle as an example. The unit restaurant will start the gas stove in the meal preparation time period to frequently consume gas, and the gas consumption in the non-meal preparation time period is relatively less. The gas usage habits of the unit restaurant in the sampling cycle are regular; considering that the main function of the unit restaurant is to provide meals and the food supply quantity in each sampling cycle is relatively fixed, the gas usage values of the sampling intervals included in the sampling cycle of the unit restaurant and the surrounding sampling cycles are similar.
[0058] Step S2: Obtain each adjacent period of the sampling period within a preset time sequence range; obtain the habitual regularity of the sampling period based on the similarity between the sampling period and the gas usage values of all sampling intervals corresponding to each adjacent period.
[0059] The main purpose of the present invention is to detect anomalies in gas usage values. First, considering that the sampling period of a unit restaurant is similar to the gas usage values in the sampling intervals contained in the surrounding sampling periods, the habitual regularity of the sampling period is obtained by analyzing the similarity between the gas usage values of the sampling period and all the sampling intervals corresponding to the adjacent periods. The greater the habitual regularity, the more regular the gas usage values in the sampling period are, and the less likely the gas usage values contained in the sampling period are to be abnormal data.
[0060] In order to analyze the characteristics of the sampling periods around the sampling period, all adjacent periods of the sampling period in a preset time sequence range are determined based on the time sequence method. Preferably, in one embodiment of the present invention, the method for obtaining the adjacent periods includes:
[0061] All sampling periods are sorted in chronological order, and any sampling period is used as the period to be constructed; a preset number of sampling periods before the period to be constructed, the period to be constructed, and a preset number of sampling periods after the period to be constructed are used as the preset timing range of the period to be constructed, and all sampling periods except the period to be constructed in the preset timing range are used as the adjacent periods of the period to be constructed, that is, the adjacent periods of the sampling period are obtained. In one embodiment of the present invention, the preset number is 7, and the preset number is mainly used to determine the number of adjacent periods of the sampling period. If the number of adjacent periods required is small, the preset number can be appropriately reduced, and the implementer can set it according to the implementation scenario.
[0062] See also Figure 2 , which shows a flow chart of a method for obtaining a habit regularity in one embodiment of the present invention. Preferably, in one embodiment of the present invention, the method for obtaining a habit regularity includes:
[0063] Step S201: according to the time sequence of each sampling interval in the sampling period, obtain the time sequence number of each sampling interval.
[0064] In order to subsequently analyze the similarities between the gas usage values corresponding to the sampling intervals at the same time sequence position in the sampling period, the time sequence numbers of all the sampling intervals in the sampling period are obtained.
[0065] In one embodiment of the present invention, all sampling intervals in the sampling period are sorted according to time sequence, and all sampling intervals in the sampling period are 0:00-1:00, 1:00-2:00, 2:00-3:00...21:00-22:00, 22:00-23:00, 23:00-24:00 in sequence; the time sequence numbers of all sampling intervals in the sampling period are: 1, 2, 3...22, 23, 24 in sequence.
[0066] Step S202: according to the difference between the gas usage values of the sampling period and the adjacent period at the same time sequence number, obtain the habitual difference index between the sampling period and the adjacent period.
[0067] Considering that the gas consumption values of the sampling period and the sampling interval included in the adjacent period of the unit restaurant are similar, the habit difference index of the sampling period and the adjacent period is obtained. The larger the habit difference index, the greater the difference in gas consumption values of the sampling period and the sampling interval included in the adjacent period.
[0068] In one embodiment of the present invention, a method for obtaining a habitual difference index includes: calculating the average of the absolute values of the differences between the gas usage values of a sampling period and adjacent periods at all time sequences, obtaining the habitual difference index between the sampling period and adjacent periods, and reflecting the difference situation through the absolute value of the difference.
[0069] Step S203: The habit difference index of the sampling period and all adjacent periods is integrated to obtain the habit regularity of the sampling period; the habit difference index and the habit regularity are negatively correlated.
[0070] The habit difference index of the comprehensive sampling period and all adjacent periods is used to obtain the habit regularity of the sampling period; the larger the habit regularity, the smaller the difference in gas consumption value between the sampling period and the sampling intervals contained in all adjacent periods, and the more regular the habit of the sampling period is.
[0071] In one embodiment of the present invention, a method for obtaining a habit regularity includes: calculating the mean of the habit difference index between a sampling period and all adjacent periods, performing negative correlation mapping on the mean, and obtaining the habit regularity of the sampling period. In one embodiment of the present invention, a method for negative correlation mapping is: using the opposite of the mean square error as a power of an exponential function with a natural constant as the base to obtain a negative correlation mapping result. It should be noted that in other embodiments of the present invention, other negative correlation mapping methods such as opposite numbers and reciprocals may also be selected, which are all technical means well known to those skilled in the art and will not be elaborated or limited here.
[0072] In a specific method of an embodiment of the present invention, the expression of habit regularity is:
[0073] ; The sampling period The regularity of habits; For the sampling period In The gas consumption value corresponding to the time sequence number; For the sampling period No. In the adjacent cycles, The gas consumption value corresponding to the time sequence number; is the total number of time series; The sampling period The total number of all adjacent cycles of ; The sampling period With habit difference index of adjacent periods; For The exponential function of base .
[0074] The present invention takes an office restaurant that uses gas to provide heating and turns the heating on and off according to weather and seasonal characteristics as an example. For example, the office restaurant will use heating when the weather is cold in winter and turn off the heating after the weather turns warmer. Turning the heating on and off will cause a significant change in the gas usage value. In order to avoid interference with abnormal detection of the gas usage value, it is necessary to analyze the possibility of adjusting the heating during the sampling period.
[0075] Step S3: According to the difference between the local distribution characteristics of the gas usage values in the preset time range of the sampling period, obtain the gas usage variation of the sampling period; according to the difference between the habitual regularity of the sampling period in the preset time range, obtain the regular trend variation of the sampling period; according to the gas usage variation and the regular trend variation, obtain the gas adjustment degree of the sampling period.
[0076] In order to analyze the possibility of adjusting the heating during the sampling period, considering that the meal supply of the unit restaurant during the non-heating period is relatively fixed and the gas consumption of the unit restaurant is mainly used for meal preparation, that is, the gas consumption of the unit restaurant in each sampling period during the non-heating period is relatively close, considering that the gas consumption of the unit restaurant in the heating period is for meal preparation and heating, the meal supply of the unit restaurant during the sampling period is relatively fixed and the gas consumption for heating is relatively fixed, that is, the gas consumption of the unit restaurant in each sampling period during the heating period is relatively close and the gas consumption is relatively large, the gas adjustment degree of the sampling period is obtained; the larger the gas adjustment degree, the more likely it is that the heating will be adjusted during the sampling period.
[0077] In order to analyze the possibility of adjusting the heating during the sampling period, considering the large difference in gas consumption between the heating period and the non-heating period, please refer to Figure 3 , which shows a flow chart of a method for obtaining a gas usage variation degree in one embodiment of the present invention. The method for obtaining a gas usage variation degree includes:
[0078] Step S301: The gas usage values of all sampling intervals in adjacent cycles are integrated to obtain the periodic gas values of adjacent cycles.
[0079] The cycle gas value reflects the gas consumption of adjacent cycles. The larger the cycle gas value, the greater the gas consumption of adjacent cycles.
[0080] In one embodiment of the present invention, the accumulated value of the gas usage values of all sampling intervals in adjacent cycles is calculated as the periodic gas value of the adjacent cycles.
[0081] Step S302: in a preset time range of the sampling period, according to the overall distribution of the periodic gas values of all adjacent periods at the time front side of the sampling period, the overall gas value at the time front side of the sampling period is obtained.
[0082] The overall gas value at the front side of the time series reflects the gas consumption in the period before the sampling period. The larger the overall gas value at the front side of the time series, the greater the gas consumption in the period before the sampling period.
[0083] In one embodiment of the present invention, within a preset time range of a sampling period, the average of the periodic gas values of all adjacent periods on the time front side of the sampling period is calculated to obtain the overall gas value on the time front side of the sampling period.
[0084] Step S303: in a preset time range of the sampling period, according to the overall distribution of the periodic gas values of all adjacent periods at the time rear side of the sampling period, the overall gas value at the time rear side of the sampling period is obtained.
[0085] The overall gas value on the back side of the time series reflects the gas consumption in a period of time after the sampling period. The larger the overall gas value on the back side of the time series, the greater the gas consumption in a period of time after the sampling period.
[0086] In one embodiment of the present invention, within a preset time range of a sampling period, the mean of the periodic gas values of all adjacent periods on the time rear side of the sampling period is calculated to obtain the overall gas value on the time rear side of the sampling period; the overall distribution is reflected by the mean.
[0087] Step S304: Obtain the degree of change of gas usage in the sampling period according to the difference between the overall gas value at the rear side of the sampling period and the overall gas value at the front side of the sampling period.
[0088] Taking into account the large difference in gas consumption between the heating period and the non-heating period, the greater the change in gas consumption, the greater the difference in gas consumption before and after the sampling period, and the more likely it is that the heating will be adjusted during the sampling period.
[0089] In one embodiment of the present invention, the absolute value of the difference between the overall gas value at the rear side of the sampling period and the overall gas value at the front side of the sampling period is calculated to obtain the degree of change in gas consumption; the difference is reflected by the absolute value of the difference.
[0090] At this point, the degree of change in gas consumption is obtained.
[0091] In order to analyze the possibility of adjusting the heating during the sampling period, considering that the gas consumption in the heating period and the non-heating period is quite different, the gas consumption in each sampling period in the heating period is relatively close, and the gas consumption in each sampling period in the non-heating period is relatively close, it can be known based on step S2 that the habitual regularity of the sampling period is obtained based on the difference in gas consumption between the sampling period and all adjacent periods. It can be known that when the heating is adjusted during the sampling period, the habitual regularity of the period before the sampling period shows a downward trend until it drops to the habitual regularity of the sampling period, and the habitual regularity of the period after the sampling period shows an upward trend. Please refer to Figure 4 , which shows a flow chart of a method for obtaining a regular trend change degree in one embodiment of the present invention, the method for obtaining a regular trend change degree includes:
[0092] Step S311: Obtain the regular mutation degree of the sampling period according to the prominence of the habitual regularity of the sampling period in a preset time series range.
[0093] The greater the regularity mutation, the greater the difference between the habitual regularity of the sampling period and the habitual regularity of the adjacent period, and the more likely the sampling period is to adjust the heating.
[0094] In one embodiment of the present invention, a method for obtaining a regular mutation degree includes: calculating the mean of the habitual regularities corresponding to all adjacent periods in a preset time series range of a sampling period to obtain a surrounding regularity index of the sampling period; calculating the absolute value of the difference between the habitual regularity of the sampling period and the surrounding regularity index to obtain a regular mutation degree of the sampling period. In other embodiments of the present invention, a method for obtaining a regular mutation degree includes: calculating the absolute value of the difference between the habitual regularity of the sampling period and the habitual regularity corresponding to each adjacent period in a preset time series range of the sampling period, calculating the mean of all the absolute values of the difference, and obtaining a regular mutation degree of the sampling period.
[0095] Step S312: Obtain a regular trend change index of the sampling period according to the difference between the local trend characteristics of the habitual regularity of the sampling period in a preset time series range.
[0096] The larger the regularity trend change index is, the more the habit regularity in the period before the sampling period shows a downward trend and the more the corresponding habit regularity in the period after the sampling period shows an upward trend, and the more likely the heating is to be adjusted in the sampling period.
[0097] In one embodiment of the present invention, a method for obtaining a regular trend change indicator includes:
[0098] In the preset time series range of the sampling period, the downward trend degree of the timing front side of the sampling period is obtained according to the deviation between the habitual regularity of all adjacent periods on the timing front side of the sampling period; in the preset time series range of the sampling period, the upward trend degree of the timing back side of the sampling period is obtained according to the deviation between the habitual regularity of all adjacent periods on the timing back side of the sampling period; according to the downward trend degree of the timing front side and the upward trend degree of the timing back side, the regular trend change index of the sampling period is obtained; the downward trend degree of the timing front side and the upward trend degree of the timing back side are both positively correlated with the regular trend change index.
[0099] For the above steps, specifically, in the preset time range of the sampling period, calculate the difference in the habitual regularity between each adjacent period and its previous adjacent period, obtain the change parameter of the adjacent period, and reflect the deviation through the difference; in all adjacent periods on the timing front side of the sampling period, count all adjacent periods with negative change parameters to obtain the downward trend degree of the timing front side of the sampling period. In all adjacent periods on the timing rear side of the sampling period, count all adjacent periods with positive change parameters to obtain the upward trend of the timing rear side of the sampling period. Calculate the sum of the downward trend degree of the timing front side and the upward trend degree of the timing rear side to obtain the regular trend change index of the sampling period.
[0100] Step S313: Obtain the regular trend change degree according to the regular mutation degree and the regular trend change index; the regular mutation degree and the regular trend change index are both positively correlated with the regular trend change degree.
[0101] The regular mutation degree and regular trend change indicators are combined to obtain the regular trend change degree; the greater the regular trend change degree, the more likely it is that the heating will be adjusted during the sampling period.
[0102] In one embodiment of the present invention, the sum of the regular mutation degree and the regular trend change index is calculated to obtain the regular trend change degree.
[0103] At this point, the degree of change of the regular trend is obtained.
[0104] In order to more accurately analyze the possibility of adjusting heating during the sampling period, the impact of adjusting heating on heating consumption before and after the sampling period, as well as the impact of adjusting heating on the habitual regularity before and after the sampling period, are combined. The gas adjustment degree of the sampling period is obtained according to the change degree of gas consumption and the change degree of regular trend. The greater the gas adjustment degree, the greater the possibility of adjusting heating during the sampling period.
[0105] In one embodiment of the present invention, a method for obtaining a gas adjustment degree includes:
[0106] The product of the gas consumption change degree and the regular trend change degree is calculated, and the product is normalized to obtain the gas adjustment degree of the sampling period. In one embodiment of the present invention, the normalization method is: normalization is performed using a linear normalization method, and the value range is limited to between 0 and 1.
[0107] At this point, the gas adjustment degree is obtained.
[0108] Step S4: Obtain the abnormal index of the sampling period according to the gas adjustment degree and the habitual regularity; obtain the abnormal gas usage value according to the abnormal index of all sampling periods and the abnormal situation of the gas usage value.
[0109] In order to analyze the abnormal gas usage values caused by hardware facilities such as equipment, pipelines, and valves, it is necessary to reduce the impact of adjusting the heating on the abnormal analysis of the gas usage values when conducting abnormal analysis, so that the abnormal gas usage values can better reflect the abnormal data of the hardware facilities and improve the accuracy of abnormal detection of gas usage values.
[0110] Taking into account that abnormal hardware facilities may lead to irregular gas usage during the sampling period, that is, the regularity of the sampling period is low, the impact of reducing regularity due to heating adjustment is reduced, and the abnormal index of the sampling period is obtained; the larger the abnormal index, the less likely it is that the heating will be adjusted during the sampling period, and the more likely that the irregularity of the sampling period is caused by hardware facilities.
[0111] Preferably, in one embodiment of the present invention, the method for obtaining abnormal indicators includes:
[0112] Calculate the difference between 1 and the habitual regularity, calculate the sum of the gas adjustment degree and the denominator adjustment factor, calculate the ratio of the difference to the sum, normalize the comparison value, and obtain the abnormal index of the sampling period. In one embodiment of the present invention, the denominator adjustment factor is used to ensure that the denominator is not 0. The denominator adjustment factor is a number greater than 0. This embodiment sets it to 0.001, and the implementer can set it according to the implementation scenario. In one embodiment of the present invention, the normalization method is: use the standard normalization method for normalization, and limit the value range to between 0 and 1.
[0113] Through the SARIMA model, the gas usage value anomaly score is used to analyze the gas usage value caused by the abnormal hardware facilities. On this basis, the product of the anomaly score of the gas usage value and the anomaly index of the sampling period to which it belongs is calculated to obtain the degree of anomaly. The greater the degree of anomaly, the more likely it is that the gas usage value anomaly is caused by the abnormal hardware facilities.
[0114] Preferably, in one embodiment of the present invention, the method for obtaining the abnormal gas usage value includes:
[0115] The SARIMA model (Seasonal Autoregressive Integrated Moving Average) is used to detect anomalies in the gas usage values of all sampling intervals in all sampling periods to obtain anomaly scores of the gas usage values. It should be noted that the SARIMA model is a technical means well known to those skilled in the art. Here, only a simple description is given of the method of using the SARIMA model to detect anomalies in the gas usage values of all sampling intervals in all sampling periods to obtain anomaly scores of the gas usage values:
[0116] The SARIMA model is used to perform anomaly detection on the gas usage values of all sampling intervals in all sampling periods to obtain the standardized residual of each gas usage value; the absolute value of the standardized residual is used as the anomaly score of the gas usage value.
[0117] The product of the abnormal score of the gas usage value and the abnormal index of the sampling period to which it belongs is calculated to obtain the degree of abnormality; the gas usage value with an abnormal degree greater than the preset abnormal threshold is marked as an abnormal gas usage value. In one embodiment of the present invention, the preset abnormal threshold is 0.7, and the preset abnormal threshold is used to filter out abnormal gas usage values. When more abnormal gas usage values are required, the preset abnormal threshold can be appropriately reduced, and the implementer can set it according to the implementation scenario.
[0118] Specifically, all abnormal gas usage values are sent to the staff using the terminal. The staff analyzes the performance type of the abnormal gas usage values based on the abnormal characteristics of the data and determines the cause of this type of abnormality. Based on different causes, the hardware facilities such as equipment, pipelines, valves, or the system's own data are repaired and optimized to reduce safety hazards and improve user safety.
[0119] In summary, the embodiment of the present invention provides a method for detecting anomalies in gas usage data based on big data, firstly obtaining the habitual regularity of the sampling period; obtaining the gas usage variation of the sampling period according to the difference between the local distribution characteristics of the gas usage value in the preset time series range of the sampling period; obtaining the regular trend variation of the sampling period according to the difference in the habitual regularity of the sampling period in the preset time series range; obtaining the gas adjustment of the sampling period according to the gas usage variation and the regular trend variation; obtaining the abnormal index of the sampling period according to the gas adjustment and habitual regularity; obtaining the abnormal gas usage value according to the abnormal index of all sampling periods and the abnormal situation of the gas usage value. In the embodiment of the present invention, by reducing the influence of adjusting the heating on the abnormal analysis of the gas usage value, the abnormal gas usage value can better reflect the abnormal data of the hardware facilities, and improve the accuracy of abnormal detection of the gas usage value.
[0120] The present invention also proposes a gas consumption data anomaly detection system based on big data, please refer to Figure 5 , which shows a structure diagram of a gas usage data anomaly detection system based on big data provided by an embodiment of the present invention, the system includes: a data acquisition module 101, a habit law analysis module 102, a gas adjustment analysis module 103 and an anomaly detection module 104.
[0121] The data acquisition module 101 is used to obtain the gas usage values of all sampling intervals in each sampling period.
[0122] The habit regularity analysis module 102 is used to obtain each adjacent period of the sampling period in a preset time sequence range; according to the similarity between the sampling period and the gas usage values of all sampling intervals corresponding to each adjacent period, the habit regularity of the sampling period is obtained.
[0123] The gas adjustment analysis module 103 is used to obtain the degree of change of gas usage in a sampling period according to the difference between the local distribution characteristics of the gas usage values in a preset time range of the sampling period; to obtain the degree of change of the regular trend of the sampling period according to the difference in the habitual regularity of the sampling period in a preset time range; and to obtain the degree of gas adjustment of the sampling period according to the degree of change of gas usage and the degree of change of the regular trend.
[0124] The anomaly detection module 104 is used to obtain an abnormal index of a sampling period according to the gas adjustment degree and the habitual regularity; and obtain an abnormal gas usage value according to the abnormal index of all sampling periods and the abnormal situation of the gas usage value.
[0125] It should be noted that: the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the gas consumption data anomaly detection system based on big data and the gas consumption data anomaly detection method based on big data provided in the above embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0126] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0127] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.
Claims
1. A method for detecting anomaly in gas usage data based on big data, characterized in that: The method comprises: Obtain the gas consumption values of all sampling intervals in each sampling period; Obtain each adjacent period of the sampling period within a preset time sequence range; obtain the habitual regularity of the sampling period according to the similarity between the sampling period and the gas usage values of all sampling intervals corresponding to each of the adjacent periods; According to the difference between the local distribution characteristics of the gas usage values in the preset time series range of the sampling period, the gas usage change degree of the sampling period is obtained; according to the difference between the habitual regularity of the sampling period in the preset time series range, the regular trend change degree of the sampling period is obtained; according to the gas usage change degree and the regular trend change degree, the gas adjustment degree of the sampling period is obtained; According to the gas adjustment degree and the habitual regularity, an abnormal index of a sampling period is obtained; according to the abnormal index of all sampling periods and the abnormal situation of the gas usage value, an abnormal gas usage value is obtained; The method for obtaining the habit regularity includes: According to the time sequence of each sampling interval under the sampling period, obtain the time sequence number of each sampling interval; According to the difference between the gas usage value of the sampling period and the adjacent period at the same time sequence number, obtaining the custom difference index between the sampling period and the adjacent period; The habit difference index of the sampling period and all adjacent periods is integrated to obtain the habit regularity of the sampling period; the habit difference index and the habit regularity are negatively correlated; The method for obtaining the gas consumption variation degree includes: The gas usage values of all sampling intervals in adjacent cycles are integrated to obtain the periodic gas value of the adjacent cycles; In the preset time range of the sampling period, according to the overall distribution of the periodic gas values of all adjacent periods on the time front side of the sampling period, the overall gas value on the time front side of the sampling period is obtained; In the preset time range of the sampling period, according to the overall distribution of the periodic gas values of all adjacent periods at the time rear side of the sampling period, the overall gas value at the time rear side of the sampling period is obtained; According to the difference between the overall gas value at the rear side of the time sequence of the sampling period and the overall gas value at the front side of the time sequence, the gas usage change degree of the sampling period is obtained; The method for obtaining the regular trend change degree includes: Obtaining the regular mutation degree of the sampling period according to the prominence of the habitual regularity of the sampling period in the preset time series range; According to the difference between the local trend characteristics of the habitual regularity of the sampling period in the preset time series range, the regular trend change index of the sampling period is obtained; According to the regular mutation degree and the regular trend change index, the regular trend change degree is obtained; the regular mutation degree and the regular trend change index are both positively correlated with the regular trend change degree; The method for obtaining the regular mutation degree includes: Calculate the average of the habitual regularity corresponding to all adjacent periods in the preset time sequence range of the sampling period to obtain the surrounding regularity index of the sampling period; Calculate the absolute value of the difference between the habitual regularity of the sampling period and the surrounding regularity index to obtain the regularity mutation degree of the sampling period; The method for obtaining the regular trend change indicator includes: In the preset time range of the sampling period, according to the deviation between the habitual regularity of all adjacent periods on the time front side of the sampling period, the downward trend degree of the time front side of the sampling period is obtained; In the preset time range of the sampling period, according to the deviation between the habitual regularity of all adjacent periods on the time rear side of the sampling period, the upward trend degree of the time rear side of the sampling period is obtained; According to the downward trend degree of the front side of the time series and the upward trend degree of the back side of the time series, a regular trend change index of the sampling period is obtained; the downward trend degree of the front side of the time series and the upward trend degree of the back side of the time series are both positively correlated with the regular trend change index; The method for obtaining the regular trend change degree according to the regular mutation degree and the regular trend change index comprises: Calculate the sum of the regular mutation degree and the regular trend change index to obtain the regular trend change degree; The method for obtaining the gas adjustment degree includes: The product of the gas usage variation degree and the regular trend variation degree is calculated, and the product is normalized to obtain the gas adjustment degree of the sampling period.
2. According to the method for detecting abnormality of gas consumption data based on big data according to claim 1, it is characterized in that: The method for obtaining the habit difference index of the comprehensive sampling period and all adjacent periods to obtain the habit regularity of the sampling period includes: The mean of the habit difference index between the sampling period and all adjacent periods is calculated, and the mean is negatively correlated to obtain the habit regularity of the sampling period.
3. According to the method for detecting anomaly of gas consumption data based on big data in claim 1, it is characterized in that: The method for obtaining the downward trend degree of the front side of the time series includes: Calculate the difference between the habit regularity of each adjacent period and the previous adjacent period to obtain the change parameters of the adjacent periods; In all adjacent periods on the time front side of the sampling period, all adjacent periods with negative change parameters are counted to obtain the downward trend degree of the time front side of the sampling period.
4. According to the method for detecting anomaly of gas consumption data based on big data according to claim 1, it is characterized in that: The method for obtaining the abnormal gas usage value includes: Using the SARIMA model, anomaly detection is performed on the gas usage values of all sampling intervals in all sampling periods to obtain anomaly scores of the gas usage values; Calculate the product of the abnormal score of the gas usage value and the abnormal index of the sampling period to which it belongs, and obtain the abnormal degree; The gas usage value whose abnormality is greater than the preset abnormality threshold is marked as an abnormal gas usage value.
Citation Information
Patent Citations
Abnormity processing method based on gas flow detection deep learning
CN116257811A
Abnormality detection method and device for gas data
CN116485041A
Animal husbandry water source pollution monitoring and early warning method and system based on big data analysis
CN117786584A
Method for monitoring operation state of numerical control machine tool
CN117798744A
Remote monitoring system
US10607475B1
