A time-of-use line loss anomaly diagnosis method based on Flink and regression model

Abstract

The application discloses a kind of based on Flink and regression model's time-sharing line loss abnormal diagnosis method, including steps: by offline historical line loss data establishment and train regression model;Massive measurement data real-time concurrent access is realized based on DataHub component, the time-sharing line loss of line is calculated after received measurement data is calculated by Flink real-time processing, and abnormal diagnosis of massive time-sharing line loss data is completed in combination with the regression model, the time-sharing line loss data is marked and divided into normal data or abnormal data;Real-time high-speed storage is carried out to the time-sharing line loss data marked and divided into normal data or abnormal data after diagnosis using cache database Redis, and normal time-sharing line loss data is fed back to the regression model, and the parameters of regression model are adjusted in real time.The application improves the timeliness of abnormal diagnosis of power consumption behavior by time-sharing line loss on one hand, and improves the accuracy of abnormal diagnosis of power consumption behavior by online updating regression model on the other hand.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power grid fault detection, in particular to a time-of-use line loss anomaly diagnosis method based on Flink and a regression model. BACKGROUND

[0002] With the continuous development of smart grid informatization, various control systems and information systems generate massive measurement data. As the data basis for the digital transformation of power grids, power grid measurement data supports real-time interaction of terminals in various fields of new power systems, and promotes the positive development of power grids. The digital transformation of power grids promotes the line loss monitoring of transmission lines. At present, based on the line variable relationship and the frozen meter reading of electric energy meters, the time efficiency of line loss calculation has been improved from daily to hourly. Monitoring the time-of-use line loss state of transmission lines helps to identify abnormal behaviors such as electricity theft and illegal electricity use, and effectively ensures the safety and stability of power grid production.

[0003] At present, the diagnosis of abnormal electricity use behavior through time-of-use line loss is limited by the large amount of data, resulting in low timeliness and low diagnosis accuracy. SUMMARY

[0004] The present application provides a time-of-use line loss anomaly diagnosis method based on Flink and a regression model to solve the technical problem of low timeliness and low diagnosis accuracy in diagnosing abnormal electricity use behavior through time-of-use line loss.

[0005] The technical solution adopted by the present application is as follows:

[0006] A time-of-use line loss anomaly diagnosis method based on Flink and a regression model, comprising the steps of:

[0007] establishing and training a regression model with line loss data and time information as independent variables through offline historical line loss data;

[0008] Based on the DataHub component, massive measurement data is realized in real time and concurrently accessed. After receiving the measurement data through real-time processing of Flink, the time-of-use line loss of the line is calculated, and the regression model is combined to complete the anomaly diagnosis of massive time-of-use line loss data. The time-of-use line loss data is marked and divided into normal data or abnormal data;

[0009] The time-of-use line loss data marked and divided into normal data or abnormal data after diagnosis is stored in real time and at high speed by using the cache database Redis, and the normal time-of-use line loss data is fed back to the regression model to adjust the parameters of the regression model in real time.

[0010] Further, before the regression model with line loss data and time information as independent variables is established and trained through offline historical line loss data, the method further comprises the steps of:

[0011] Let D = {D1, D2, …, D } represent a set of historical line loss data of a line, and the set of historical line loss data D

[0012]

[0013] A new set of historical line loss data is obtained, that is,

[0014] D = {D1, D2, …, D N};

[0015] Setting a set of time information related to historical line loss data includes:

[0016] Let W = {W1, W2, …, W N} represent a set of historical line loss data corresponding to the day of the week, where W n ∈ {1, 2, 3, 4, 5, 6, 7} represents the line loss data D n is collected on the day of the week W n ;

[0017] Let Y = {Y1, Y2, …, Y N} represent a set of historical line loss data corresponding to special dates, where Y n ∈ {1, 2, …, K}, K can be expanded according to demand, and the meaning of different values of Y n is defined according to demand;

[0018] Let H = {H1, H2, …, H N} represent the historical line loss data corresponding to the hour, where H n ∈ {0, 1, …, 23} represents the time of collecting historical line loss data D n is the H n th hour of the day;

[0019] According to the historical line loss data and the corresponding time information, a set of training data T is generated: Let the set T = {T1, T2, …, T M} represent the training data set, T M is composed of q historical line loss data and the time information corresponding to T M ,

[0020] M = N-q, T M = {D M , D M+1 , …, D M+q-1 , W q+M , Y q+M , H q+M}, the corresponding label of T M is Dq+M , and the corresponding label set T = {D q+1 , D q+2 , …, D q+M}.

[0021] Further, the regression model with line loss data and time information as independent variables is established and trained by offline historical line loss data, and specifically includes the following steps:

[0022] Based on historical data, q regression models with line loss data and related time information as independent variables are established:

[0023]

[0024] wherein n = q + 1, q + 2, …, N, is the predicted value of the model output result D n , c is a constant term, is a model parameter, the element in indicates the influence degree of the corresponding parameter on the predicted value , e n is the residual error of the model prediction and obeys normal distribution;

[0025] The regression model is trained using the generated training data set until a set of parameters is found to minimize the residual error of the training data set.

[0026] Further, the residual error is equivalent to the loss loss, i.e.:

[0027]

[0028] or,

[0029] The loss loss adopts square loss or mean square error loss, and the regression model training method at least includes gradient descent method and quasi-Newton method.

[0030] Further, the DataHub component is used to realize real-time and concurrent access of massive measurement data, and the line loss by time is calculated after the received measurement data is processed in real time by Flink. The regression model is combined to complete the abnormal diagnosis of massive line loss data by time, and the line loss data by time is marked and divided into normal data or abnormal data, and specifically includes the following steps:

[0031] The Flink is used to access measurement data in real time from the DataHub, and then the line loss data by time of each line is calculated;

[0032] The line loss data by time D z of a certain line is preprocessed, and D zThe time-sharing line loss data and time information of D z-q ,D z-q-1 ,…,D z-1 ,W z ,Y z ,H z} are input features, and the input features are taken as input of the regression model corresponding to the line to obtain a predicted value Let represent a prediction error, and take the prediction error of the first K line loss data of D z , denoted as a set E = {e z-K ,e z-K+1 ,…,e z-1}, the mean and variance of the set E are μ and σ 2 , the data D z is diagnosed according to the following formula, and the time-sharing line loss data is marked as normal data or abnormal data:

[0033]

[0034] Where λ is a real number.

[0035] Further, when the time-sharing line loss data is diagnosed as abnormal data, the method further comprises the steps of:

[0036] The time-sharing line loss data diagnosed as abnormal data is further judged as mild abnormality, moderate abnormality and extreme abnormality according to the prediction error according to the following formula:

[0037]

[0038] In order to make the above formula not conflict, λ is less than 2.

[0039] Further, the time-sharing line loss data marked as normal data or abnormal data after diagnosis is stored in real time at high speed by using a cache database Redis, and the normal time-sharing line loss data is fed back to the regression model, and the parameters of the regression model are adjusted in real time, and the method comprises the steps of:

[0040] If the time-sharing line loss data is marked as normal data, the data is directly written into the cache database Redis;

[0041] If the time-sharing line loss data is marked as abnormal data, the abnormal situation is fed back to the application side, and alarm information is displayed, and then the line loss data is reviewed, wherein the processing methods of different degrees of abnormality are different, and corresponding processing measures are formulated according to business logic;

[0042] If the line loss data is indeed abnormal after review, mark the line loss data as corresponding abnormal data and write it into the cache database redis; if the line loss data is not abnormal, mark the line loss data as normal data and write it into the cache database redis, and simultaneously, adaptively adjust λ under the condition of meeting the preset interval constraint, i.e. λ = λ + γ, where γ is a set constant;

[0043] When all the line loss data marked as normal data accumulates to a set number, it is fed back to the regression model for continuous training of the regression model and real-time adjustment of the parameters of the regression model.

[0044] Another aspect of the present application also provides a time-of-use line loss anomaly diagnosis device based on Flink and a regression model, comprising:

[0045] A regression model construction module is configured to establish and train a regression model with line loss data and time information as independent variables through offline historical line loss data;

[0046] An anomaly diagnosis module is configured to realize real-time and concurrent access of massive measurement data based on a DataHub component, calculate the time-of-use line loss of a line after real-time processing of the received measurement data by Flink, and complete anomaly diagnosis of massive time-of-use line loss data in combination with the regression model, and mark and distinguish the time-of-use line loss data as normal data or abnormal data;

[0047] A diagnosis result storage and feedback module is configured to store the time-of-use line loss data marked and distinguished as normal data or abnormal data in real time at high speed by using a cache database Redis, and feed back the normal time-of-use line loss data to the regression model for real-time adjustment of the parameters of the regression model.

[0048] Another aspect of the present application also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the time-of-use line loss anomaly diagnosis method based on Flink and a regression model when executing the program.

[0049] Another aspect of the present application also provides a storage medium comprising a stored program, wherein the program controls the device where the storage medium is located to execute the steps of the time-of-use line loss anomaly diagnosis method based on Flink and a regression model when the program is executed.

[0050] Compared with the prior art, the present application has the following beneficial effects:

[0051] The application provides a time-sharing line loss abnormality diagnosis method based on Flink and a regression model.

[0052] On one hand, the application realizes real-time and concurrent access of massive measurement data and time-sharing line loss calculation by using Flink and a DataHub component, and improves the timeliness of abnormality diagnosis of power consumption behavior by time-sharing line loss; on the other hand, the application constantly feeds back normal line loss data stored at high speed in real time to the corresponding regression model while using offline historical line loss data to establish and train a regression model with line loss data and time information as independent variables to complete abnormality diagnosis of massive time-sharing line loss data, so that the regression model parameters are updated online, obviously, with the increase of data volume, the adaptability of the regression model will gradually increase over time, thereby the accuracy of abnormality diagnosis of power consumption behavior is improved adaptively.

[0053] In addition to the purposes, features and advantages described above, the application has other purposes, features and advantages. The application will be further described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0054] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. In the drawings:

[0055] Figure 1 FIG. 1 is a flowchart of a time-sharing line loss abnormality diagnosis method based on Flink and a regression model according to a preferred embodiment of the application.

[0056] Figure 2 FIG. 2 is a flowchart of a time-sharing line loss abnormality diagnosis method based on Flink and a regression model according to another preferred embodiment of the application.

[0057] Figure 3 FIG. 3 is a flowchart of abnormality diagnosis of time-sharing line loss data of a line according to a preferred embodiment of the application.

[0058] Figure 4Fig. 1 is a schematic diagram of a Flink and regression model-based sub-hourly line loss anomaly diagnosis device module according to a preferred embodiment of the present application.

[0059] Figure 5 Fig. 2 is a schematic block diagram of an electronic device according to a preferred embodiment of the present application.

[0060] Figure 6 Fig. 3 is an internal structure diagram of a computer device according to a preferred embodiment of the present application. DETAILED DESCRIPTION

[0061] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0062] Reference Figure 1 and Figure 2 The preferred embodiments of the present application provide a Flink and regression model-based sub-hourly line loss anomaly diagnosis method, comprising the steps of:

[0063] S1, establishing and training a regression model with line loss data and time information as independent variables through offline historical line loss data;

[0064] S2, realizing real-time and concurrent access of massive measurement data based on a DataHub component, calculating the sub-hourly line loss of a line after real-time processing of the received measurement data by Flink, and completing anomaly diagnosis of massive sub-hourly line loss data in combination with the regression model, marking and dividing the sub-hourly line loss data into normal data or abnormal data;

[0065] S3, storing the sub-hourly line loss data marked and divided into normal data or abnormal data after diagnosis in real time and at high speed by using a cache database Redis, and feeding back the normal sub-hourly line loss data to the regression model to adjust the parameters of the regression model in real time.

[0066] The embodiments provide a Flink and regression model-based sub-hourly line loss anomaly diagnosis method, which establishes and trains a regression model with line loss data and time information as independent variables through offline historical line loss data; realizes real-time and concurrent access of massive measurement data based on a DataHub component, calculates the sub-hourly line loss of a line after real-time processing of the received measurement data by Flink, and completes anomaly diagnosis of massive sub-hourly line loss data in combination with the regression model, marking and dividing the sub-hourly line loss data into normal data or abnormal data; stores the sub-hourly line loss data marked and divided into normal data or abnormal data after diagnosis in real time and at high speed by using a cache database Redis, and feeds back the normal sub-hourly line loss data to the regression model to adjust the parameters of the regression model in real time.

[0067] This embodiment utilizes Flink and DataHub components to achieve real-time concurrent access to massive measurement data and time-sharing line loss calculation. Flink, with its excellent stream processing, distributed, and high-performance characteristics, has become a mainstream technology for efficient computation and deep data mining of massive amounts of data, effectively meeting the real-time requirements of business operations and improving the timeliness of diagnosing abnormal electricity consumption behavior through time-sharing line loss. Furthermore, while using offline historical line loss data to establish and train a regression model with line loss data and time information as independent variables to complete the anomaly diagnosis of massive time-sharing line loss data, this embodiment also continuously feeds back real-time, high-speed stored normal line loss data to the corresponding regression model, enabling online updates of the regression model parameters. Clearly, as the amount of data increases, the adaptability of the regression model will gradually increase over time, thereby adaptively improving the accuracy of abnormal electricity consumption behavior diagnosis.

[0068] Specifically, before establishing and training a regression model with line loss data and time information as independent variables using offline historical line loss data, the method further includes the following steps:

[0069] S101, Order This represents a set of historical line loss data for a single line (assuming that all historical line loss values ​​have been adjusted, i.e., data that is unusable or incalculable due to missing data, jumps, outliers, etc.). In other words, assuming the set... There are no abnormal data values ​​in the dataset; all values ​​are genuine and without anomalies (based on historical line loss data). Difference processing yields set D, which enhances the stationarity of the data.

[0070]

[0071] We obtain a new set of historical line loss data, namely:

[0072] D = {D1,D2,…,D} N};

[0073] S102. Set the time information set related to historical line loss data, including:

[0074] Let W = {W1, W2, ..., W} N} represents the set of days of the week corresponding to the historical line loss data, where W n ∈{1,2,3,4,5,6,7} represents the line loss data D. n It's weekday W n The collected data;

[0075] Let Y = {Y1, Y2, ..., Y} N} indicates that the historical line loss data corresponds to a set of specific dates, where Y n∈{1,2,…,K}, where the value of K can be expanded according to requirements, and the corresponding Y n The meaning of different values ​​is defined according to the requirements, for example, Y n =1 indicates that line loss data D is collected. n The time is a regular date, Y n =2 indicates that line loss data D is collected. n The time is the Spring Festival;

[0076] Let H = {H1, H2, ..., H} N} represents the hourly time corresponding to the historical line loss data, where H n ∈{0,1,…,23} represents the collection of historical line loss data D n The time is the Hth day of that day. n Hours;

[0077] S103. Generate a training data set T based on historical line loss data and corresponding time information: Let set T = {T1, T2, ..., T...} M} represents the training data set, T M Composed of q historical line loss data and T M The corresponding time information constitutes M = Nq.

[0078] T M ={D M D M+1 ,…,D M+q-1 W q+M ,Y q+M H q+M}, T M The corresponding tag is D q+M Therefore, the corresponding label set T = {D} q+1 D q+2 ,…,D q+M}

[0079] Specifically, the step of establishing and training a regression model using offline historical line loss data as independent variables includes the following steps:

[0080] S11. Establish a regression model based on historical data, using q line loss data points and 3 relevant time information points as independent variables:

[0081]

[0082] Where n = q+1, q+2, ..., N, Output the model result D n The predicted value, where c is a constant term. For model parameters, The elements in the table represent the corresponding parameters for the predicted value. the degree of influence of e n is the residual of the model prediction and obeys a normal distribution;

[0083] S12, train the regression model using the generated training data set until a set of parameters minimizes the residual of the training data set.

[0084] wherein the residual is equivalent to the loss loss, that is:

[0085]

[0086] or,

[0087] The loss loss adopts a square loss or a mean square error loss, etc. Correspondingly, the regression model training method includes a gradient descent method, a quasi-Newton method, etc. Those skilled in the art can make a corresponding selection according to the needs.

[0088] The reason why the embodiment considers the week, hour and special date attributes in the input features is that: first, the power consumption data may be related to whether it is the weekend, for example, if there is no work on the weekend, the power consumption of a certain line will decrease; second, the power consumption data may be time-division, for example, in the early morning, during working hours, etc., the power consumption of some lines will decrease, while during off-work hours, the power consumption will relatively increase; third, the power consumption data may be related to special holidays, for example, during the holiday, the power consumption of some lines will also change accordingly. In summary, considering time information can make the basis of the prediction more sufficient, thereby improving the accuracy of the prediction result and reducing the misdiagnosis rate of the anomaly diagnosis.

[0089] Specifically, the massive measurement data real-time concurrent access is realized based on the DataHub component, the time-division line loss of the line is calculated after the received measurement data is processed in real time by Flink, and the anomaly diagnosis of massive time-division line loss data is completed in combination with the regression model, the time-division line loss data is marked and divided into normal data or abnormal data, and specifically includes the following steps:

[0090] S21, measurement data is accessed in real time from DataHub by Flink, and then the time-division line loss data of each line is quickly calculated by using the high-performance computing capability of Flink;

[0091] S22, the time-division line loss data D z of a certain line is preprocessed, and the time-division line loss data D z of the line and the time information are composed into input features {D z-q ,D z-q-1 ,…,D z-1 ,W z ,Y z ,H z} as the input of the regression model corresponding to the line to obtain a predicted value Let denote the prediction error, and take D z the prediction error of the first K line loss data, denoted as a set E = {e z-K , e z-K+1 , …, e z-1}, the mean and variance of the set E are μ and σ 2 , according to the following formula to diagnose the data D z , and the time-sharing line loss data is marked as normal data or abnormal data:

[0092]

[0093] Where λ is a real number.

[0094] Preferably, as Figure 3 shown, when the time-sharing line loss data is diagnosed as abnormal data, further comprising the steps of:

[0095] S23, according to the following formula, the time-sharing line loss data diagnosed as abnormal data is further judged as mild abnormality, moderate abnormality and extreme abnormality according to the prediction error:

[0096]

[0097] Where, in order to make the above formula not conflict, λ is less than 2, and a feasible value range of λ is [1.5, 2), of course, λ can also be adjusted according to the actual situation.

[0098] Specifically, in the diagnosis of the embodiment, first, it is judged whether the line loss data is normal, if abnormal, secondary judgment is carried out; if normal, the diagnosis is ended, and the line loss data is marked as normal. For secondary judgment, first, it is judged whether the line loss data is extreme abnormality, if not extreme abnormality, tertiary judgment is carried out; if it is extreme abnormality, the diagnosis is ended, and the line loss data is marked as extreme abnormality. For tertiary judgment, first, it is judged whether the line loss data is moderate abnormality, if it is moderate abnormality, the diagnosis is ended, and the line loss data is marked as moderate abnormality; otherwise, the line loss data is marked as mild abnormality.

[0099] Specifically, as Figure 3 shown, the time-sharing line loss data marked and distinguished as normal data or abnormal data after diagnosis is stored in real time at high speed by using the cache database Redis, and the normal time-sharing line loss data is fed back to the regression model, and the parameters of the regression model are adjusted in real time, which specifically includes the steps of:

[0100] S31, if the time-sharing line loss data is marked and distinguished as normal data, the data is directly written into the cache database Redis;

[0101] S32. If the time-sharing line loss data is identified and marked as abnormal data, the abnormal situation is fed back to the application side, an alarm message is displayed, and then the line loss data is reviewed. The methods for handling different degrees of abnormality are different, and corresponding handling measures are formulated according to the business logic.

[0102] S33. If, after review, the line loss data is indeed abnormal, then the line loss data is marked as the corresponding abnormal data and written to the cache database Redis; if the line loss data is not abnormal, then the line loss data is marked as normal data and written to the cache database Redis. Obviously, the model diagnosis is wrong at this time. One possible reason is that λ is too small, causing the model to misjudge normal data as abnormal data. At this time, under the preset interval constraint, λ is adaptively adjusted, that is, λ = λ + γ, where γ is a sufficiently small set constant.

[0103] S34. When all the line loss data marked as normal data accumulates to a set number, it is fed back to the regression model to continue training the regression model and adjust the parameters of the regression model in real time.

[0104] In this embodiment, when the amount of line loss data marked as normal data accumulates to a certain level, the normal data is fed back to the regression model for continued model training. Clearly, feeding back data that the model predicted incorrectly to continue training corrects the model's parameters, thereby enhancing the model's generalization ability in real time.

[0105] like Figure 4 As shown, another embodiment of this application also provides a time-sharing line loss anomaly diagnosis device based on Flink and regression models, including:

[0106] The regression model building module is used to build and train a regression model with line loss data and time information as independent variables using offline historical line loss data.

[0107] The anomaly diagnosis module is used to realize the real-time concurrent access of massive measurement data based on the DataHub component. After processing the received measurement data in real time through Flink, it calculates the time-sharing line loss of the line and completes the anomaly diagnosis of massive time-sharing line loss data by combining the regression model, and marks the time-sharing line loss data as normal data or abnormal data.

[0108] The diagnostic result storage and feedback module is used to store the time-sharing line loss data, which is marked as normal or abnormal after diagnosis, in real time and at high speed using the cache database Redis, and to feed back the normal time-sharing line loss data to the regression model to adjust the parameters of the regression model in real time.

[0109] like Figure 5As shown, the preferred embodiments of the present application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the Flink and regression model based time-of-use line loss anomaly diagnosis method in the above embodiments when executing the program.

[0110] As shown, the preferred embodiments of the present application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the Flink and regression model based time-of-use line loss anomaly diagnosis method in the above embodiments when executing the program. Figure 6 As shown, the preferred embodiments of the present application also provide a computer device, which can be a terminal or a living body detection server, and its internal structure diagram can be as shown in Figure 6 The computer device includes a processor, a memory, and a network interface connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the computer device is used to communicate with other computer devices outside through network connection. The computer program is executed by the processor to implement the steps of the Flink and regression model based time-of-use line loss anomaly diagnosis method.

[0111] Those skilled in the art can understand that Figure 6 The structure shown in the above embodiments is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0112] The preferred embodiments of the present application also provide a storage medium, which includes a stored program, wherein the program controls the device where the storage medium is located to execute the steps of the Flink and regression model based time-of-use line loss anomaly diagnosis method in the above embodiments when running.

[0113] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a group of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0114] If the functions described in the method of the embodiments are realized in the form of software function units and sold or used as independent products, they can be stored in one or more computer device readable storage media. Based on such understanding, the part of the prior art or the part of the technical solution of the embodiments of the present application that makes a contribution to the prior art can be embodied in the form of a software product stored in a storage medium, including a number of instructions for causing a computer device (which can be a personal computer, a server, a mobile computing device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0115] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0116] The present application is described with reference to flowcharts and / or block diagrams according to the methods, devices (systems), and computer program products of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in one or more flows and / or blocks.

[0117] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1the function specified in the one or more blocks.

[0118] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flows Figure 1 the flow or flows and / or blocks Figure 1 the function specified in the one or more blocks.

[0119] Although preferred embodiments of the application have been described, those skilled in the art will be able to make additional modifications and variations to the described embodiments without departing from the spirit and scope of the application. Accordingly, the appended claims are intended to encompass all such modifications and variations as falling within the scope of the application.

[0120] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A Flink and regression model-based time-of-use line loss anomaly diagnosis method, characterized in that, The method comprises the steps of: establishing and training a regression model with line loss data and time information as independent variables through offline historical line loss data; based on the DataHub component, realizing real-time and concurrent access of massive measurement data, calculating the time-sharing line loss of the line after Flink real-time processing of the received measurement data, and combining the regression model to complete the abnormal diagnosis of massive time-sharing line loss data, and marking and dividing the time-sharing line loss data into normal data or abnormal data; using the cache database Redis to store the time-sharing line loss data marked and divided into normal data or abnormal data in real time at high speed, and feeding back the normal time-sharing line loss data to the regression model to adjust the parameters of the regression model in real time, specifically comprising the steps of: if the time-sharing line loss data is marked and divided as normal data, the data is directly written into the cache database Redis; if the time-sharing line loss data is marked and divided as abnormal data, the abnormal situation is fed back to the application side to display alarm information, and then the line loss data is reviewed, wherein the processing methods for different degrees of abnormality are different, and corresponding processing measures are formulated according to business logic; if the line loss data is indeed abnormal after review, the line loss data is marked as corresponding abnormal data and written into the cache database Redis; if the line loss data is not abnormal, the line loss data is marked as normal data and written into the cache database Redis, and λ is adaptively adjusted under the condition of meeting the preset interval constraint, i.e. λ = λ + γ, where γ is a constant; when all the line loss data marked as normal data accumulates to a set amount, it is fed back to the regression model for continuous training of the regression model and real-time adjustment of the parameters of the regression model.

2. The Flink and regression model-based time-of-use line loss anomaly diagnosis method according to claim 1, characterized in that, Before the regression model with line loss data and time information as independent variables is established and trained through offline historical line loss data, the method further comprises the steps of: Let denote a set of historical line loss data for a line, and let D denote the set of differences obtained by differencing the set of historical line loss data obtaining a new historical line loss data set, i.e. D = {D1, D2,..., D N}; setting a time information set related to the historical line loss data comprises: Let W = {W1, W2, …, W N} represent the set of days of the week corresponding to the time at which the historical line loss data was collected, where W n e {1, 2, 3, 4, 5, 6, 7} represents the line loss data D n collected on day W n . Let Y = {Y1, Y2, …, Y N} represent the set of special dates corresponding to the time of the historical line loss data, where Y n ∈ {1, 2, …, K}, the value of K can be expanded according to requirements, and the meanings of different values of Y n are defined according to requirements; Let H = {H1, H2,..., H N} denote the hour time corresponding to the historical line loss data, where H n ∈ {0, 1,..., 23} indicates that the time of collecting the historical line loss data D n is the H n th hour of the day; According to historical line loss data and corresponding time information, a training data set T is generated: let the set T = {T1, T2, …, Tn} represent the training data set, T M n represents the number of historical line loss data. M The training data set T is composed of q historical line loss data and the corresponding time information T M ​ M = N - q, T M = {D M ,D M+1 ,…,D M+q-1 ,W q+M ,Y q+M ,H q+M}, T M The corresponding label is D q+M , and thus the corresponding label set T = {D q+1 ,D q+2 ,…,D q+M}.

3. The Flink and regression model-based time-of-use line loss anomaly diagnosis method according to claim 2, characterized in that, the regression model with line loss data and related time information as independent variables is established based on historical data, specifically comprising the steps of: the residual error is equivalent to the loss loss, i.e. where n = q + 1, q + 2,..., N, is the prediction value of the model output result D n , c is a constant term, is a model parameter, the element in the matrix represents the degree of influence of the corresponding parameter on the prediction value , e n is the residual of the model prediction and is subject to a normal distribution; training the regression model with the generated training data set until a set of parameters is found minimizing the residuals of the training data set.

4. The Flink and regression model-based time-of-use line loss anomaly diagnosis method according to claim 3, characterized in that, or, the loss loss adopts a square loss or a mean square error loss, and the regression model training method at least comprises a gradient descent method and a quasi-Newton method. based on the DataHub component, realizing real-time and concurrent access of massive measurement data, calculating the time-sharing line loss of the line after Flink real-time processing of the received measurement data, and combining the regression model to complete the abnormal diagnosis of massive time-sharing line loss data, and marking and dividing the time-sharing line loss data into normal data or abnormal data, specifically comprising the steps of:

5. The Flink and regression model-based time-of-use line loss anomaly diagnosis method according to claim 3, characterized in that, using Flink to access measurement data from DataHub in real time, and then calculating the time-sharing line loss data of each line; wherein λ is a real number. the time-of-use line loss data D of a current line z Preprocessing D z , the time-of-use line loss data and time information of D z-q , D z-q-1 , …, D z-1 , W z , Y z , H z to form an input feature {D z , D z-K , …, D z-K+1}, and taking the input feature as the input of a regression model corresponding to the line to obtain a predicted value Let represent the prediction error, take the prediction errors of the first K line loss data of D z , and represent them as a set E = {e z-K , e z-K+1 , …, e z-1}, the mean and variance of the set E are μ and σ 2 , perform anomaly diagnosis on the data D z according to the following formula, and mark the time-of-use line loss data as normal data or abnormal data: after the time-sharing line loss data is diagnosed as abnormal data, the method further comprises the steps of:

6. The Flink and regression model-based time-of-use line loss anomaly diagnosis method according to claim 5, characterized in that, ​ The time-sharing line loss data diagnosed as abnormal data is further judged as mild abnormality, moderate abnormality and extreme abnormality according to the prediction error according to the following formula: Wherein, in order to make the formula not conflict, λ is less than 2.

7. A time-of-use line loss anomaly diagnosis device based on Flink and a regression model, characterized in that, Comprise: The regression model construction module is used to establish and train a regression model with line loss data and time information as independent variables through offline historical line loss data; The abnormality diagnosis module is used to realize real-time and concurrent access of massive measurement data based on a DataHub component, calculate the time-sharing line loss of the line after receiving the measurement data through Flink real-time processing, and complete the abnormality diagnosis of massive time-sharing line loss data in combination with the regression model, and mark and divide the time-sharing line loss data as normal data or abnormal data; The diagnosis result storage and feedback module is used to store the time-sharing line loss data marked and divided as normal data or abnormal data in real time and high speed by using a cache database Redis, and feed back the normal time-sharing line loss data to the regression model to adjust the parameters of the regression model in real time, and is specifically used for: If the time-sharing line loss data is marked and divided as normal data, the data is directly written into the cache database Redis; If the time-sharing line loss data is marked and divided as abnormal data, the abnormality is fed back to the application side to display alarm information, and then the line loss data is reviewed, wherein the processing methods of different degrees of abnormality are different, and corresponding processing measures are formulated according to business logic; If the line loss data is indeed abnormal after review, the line loss data is marked as corresponding abnormal data and written into the cache database Redis; if the line loss data is not abnormal, the line loss data is marked as normal data and written into the cache database Redis, and λ is adaptively adjusted under the condition of meeting the preset interval constraint, i.e. λ = λ + γ, γ is a set constant; When all the line loss data marked as normal data accumulates to a set number, it is fed back to the regression model for continuous training of the regression model and real-time adjustment of the parameters of the regression model.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the time-sharing line loss abnormality diagnosis method based on Flink and regression model according to any one of claims 1-6 when executing the program.

9. A storage medium comprising a stored program, which when executed controls the device where the storage medium is located to perform the steps of the time-sharing line loss abnormality diagnosis method based on Flink and regression model according to any one of claims 1-6.

Images