Line loss data synchronization method and device, electronic equipment and storage medium

Abstract

The invention discloses a line loss data synchronization method and device, electronic equipment and a storage medium, and belongs to the technical field of data processing. The method comprises the following steps: extracting historical table code curve data from a Hive database; according to the historical table code curve data, the real-time table code curve data and the archive information of the users, the equipment and the courts, determining the electric quantity value used by each user, and obtaining the line loss rate of the preset power supply area or the courts; according to the line loss rate and a preset line loss range, determining a line loss type corresponding to the preset power supply area or the transformer area; and under the condition that the line loss type is abnormal line loss, synchronizing the table code curve data corresponding to each user in the preset power supply area or the area to a data medium station through a bitsail platform. When the line loss type is the abnormal line loss, the table code curve data corresponding to each user is synchronized to the data medium station through the bitsail platform, the abnormal line loss data can be synchronized to the data medium station in time, synchronization between different data sources is achieved, and the line loss abnormal data synchronization speed is increased.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular relates to a line loss data synchronization method, device, electronic device and storage medium. Background Art

[0002] Abnormal line loss data is generated during power transmission. In the prior art, due to the inconsistency of data formats among business subsystems, data conversion operations are required for cross-system line loss data synchronization, which reduces the efficiency of synchronization between line loss data and other business data. The real-time requirements for line loss data synchronization are often not effectively met. Summary of the invention

[0003] The present application proposes a line loss data synchronization method, device, electronic device and storage medium to solve the problems existing in the related art in the refrigerator food recognition scenario, such as high collection cost, food close to the camera may be blocked from view and cannot be collected, and inaccurate recognition results.

[0004] In a first aspect, the present application provides a line loss data synchronization method, the method comprising:

[0005] Extract historical table code curve data from Hive database;

[0006] Determine the power consumption value of each user in the preset power supply area or area according to the historical meter curve data and the real-time meter curve data, as well as the archive information of the user, the equipment and the area;

[0007] Performing regionalized aggregation analysis on the power value to obtain a line loss rate in a preset power supply area or substation;

[0008] Determine the line loss type corresponding to the preset power supply area or substation according to the line loss rate and preset line loss range of the preset power supply area or substation;

[0009] When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the table code curve data corresponding to each user in the preset power supply area or substation is synchronized to the data center through the bitsail platform.

[0010] The line loss data synchronization method provided in the embodiment of the present application extracts historical table code curve data from the Hive database, determines the electricity value used by each user in the preset power supply area or substation according to the historical table code curve data and the real-time table code curve data, as well as the archival information of users, equipment and substations, performs regionalized aggregation analysis on the electricity value, and obtains the line loss rate of the preset power supply area or substation. When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the table code curve data corresponding to each user in the preset power supply area or substation is synchronized to the data middle station through the bitsail platform, and the abnormal line loss data in the Hive database can be synchronized to the data middle station in time, and can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time performance of the synchronization of abnormal line loss data, so that the line loss anomaly can be analyzed in time on the data middle station, thereby improving the stability of power grid operation.

[0011] According to one embodiment of the present application, the method further includes:

[0012] Acquire the file information, and store the file information in a relational database;

[0013] The real-time table code curve data is obtained, and the real-time table code curve data is synchronized to a Hive database.

[0014] In the above technical solution, the archive information is stored in a relational database, and the real-time table code curve data is obtained and synchronized to the Hive database. The synchronization of the real-time table code curve data can be realized, and it can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time performance of data synchronization and improving the operating efficiency and service quality of the power system.

[0015] According to an embodiment of the present application, synchronizing the real-time table code curve data to a Hive database includes:

[0016] The real-time table code curve data is stored in the Kafka message queue in JSON format;

[0017] Extracting the real-time code curve data in JSON format from the Kafka message queue through the bitsail platform, parsing the real-time code curve data in JSON format, and obtaining structured data corresponding to the real-time code curve data;

[0018] Matching the structured data with the archive information to determine the user and station area corresponding to the real-time table code curve data;

[0019] The real-time table code curve data and the users and stations corresponding to the real-time table code curve data are synchronized to the Hive database.

[0020] In the above technical solution, the bitsail platform extracts the real-time table code data from the Kafka message queue for analysis, obtains the structured data corresponding to the real-time table code data, matches the structured data with the archival information, determines the users and substations corresponding to the real-time table code curve data, and synchronizes the real-time table code curve data and the users and substations corresponding to the real-time table code curve data to the Hive database, thereby improving the speed and accuracy of the synchronization of the real-time table code curve data.

[0021] According to one embodiment of the present application, the method further includes:

[0022] Capturing archive change data from the relational database using Flink’s change data capture function;

[0023] Write the archive change data into the Hive archive table through Flink's Hive connector;

[0024] The archive change data is synchronized from the Hive archive table to the Hive database through the bitsail platform.

[0025] In the above technical solution, the archive change data is captured from the relational database through Flink's change data capture function, and the bitsail platform synchronizes the archive change data from the Hive archive table to the Hive database, thereby realizing synchronization between different data sources. It is suitable for data integration of multiple data sources and can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time performance of data synchronization.

[0026] According to an embodiment of the present application, performing regionalized aggregate analysis on the power value to obtain a line loss rate of a preset power supply area or substation includes:

[0027] According to the power consumption value of each user in the preset power supply area or substation area, the total power consumption value of all users in the preset power supply area or substation area is calculated;

[0028] Obtaining the difference between the power value used by the total meter of a preset power supply area or area and the total power value used by all users in the preset power supply area or area;

[0029] The line loss rate of the preset power supply area or substation is determined according to the difference and the power value used by the total meter of the preset power supply area or substation.

[0030] In the above technical solution, the difference is obtained by the power value used by the total meter of the preset power supply area or substation and the total power value used by all users in the preset power supply area or substation. According to the difference and the power value used by the total meter of the preset power supply area or substation, the line loss rate of the preset power supply area or substation is obtained. The loss occurring during the power transmission process can be quickly identified, the line loss abnormality can be discovered in time and corresponding measures can be taken, thereby improving the level of line loss management.

[0031] According to an embodiment of the present application, determining the line loss type corresponding to the preset power supply area or substation according to the line loss rate and the preset line loss range of the preset power supply area or substation includes:

[0032] When the line loss rate of the preset power supply area or substation is greater than the maximum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is high line loss;

[0033] When the line loss rate of the preset power supply area or substation is less than the minimum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is low line loss;

[0034] The abnormal line loss includes the high line loss and the low line loss.

[0035] In the above technical scheme, by comparing the line loss rate of the preset power supply area or substation with the preset line loss range, the line loss type corresponding to the preset power supply area or substation can be determined, and the preset power supply area or substation corresponding to the abnormal line loss can be located, which facilitates the subsequent synchronization of abnormal line loss data.

[0036] According to one embodiment of the present application, the real-time meter curve data includes current, voltage, meter reading, timestamp and user ID, and the real-time meter curve data is obtained through 96-point synchronous sampling and phase-by-phase statistics.

[0037] In the above technical solution, real-time meter code curve data is obtained through 96-point synchronous sampling and phase statistics, and current, voltage, and meter readings at multiple time points can be collected, which improves the real-time and accuracy of data acquisition and is conducive to timely detection of abnormal data.

[0038] In a second aspect, the present application provides a line loss data synchronization device, the device comprising:

[0039] Table code data acquisition unit, used to extract historical table code curve data from the Hive database;

[0040] The power determination unit is used to determine the power value used by each user in the preset power supply area or substation according to the historical meter code curve data and the real-time meter code curve data, as well as the archive information of the user, the equipment and the substation;

[0041] An aggregate analysis unit, used to perform regional aggregate analysis on the power value to obtain a line loss rate in a preset power supply area or substation;

[0042] A line loss type determination unit, configured to determine a line loss type corresponding to the preset power supply area or substation according to a line loss rate and a preset line loss range of the preset power supply area or substation;

[0043] The abnormal line loss synchronization unit is used to synchronize the table code curve data corresponding to each user in the preset power supply area or substation to the data center through the bitsail platform when the line loss type corresponding to the preset power supply area or substation is abnormal line loss.

[0044] In the above technical scheme, historical table code curve data is extracted from the Hive database, and the electricity value used by each user in the preset power supply area or substation is determined according to the historical table code curve data and the real-time table code curve data, as well as the archival information of users, equipment and substations. The electricity value is regionalized and aggregated for analysis to obtain the line loss rate of the preset power supply area or substation. When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the table code curve data corresponding to each user in the preset power supply area or substation is synchronized to the data middle platform through the bitsail platform, so that the abnormal line loss data in the Hive database can be synchronized to the data middle platform in time, and can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time synchronization of abnormal line loss data, so that the line loss anomaly can be analyzed in time at the data middle platform, thereby improving the stability of power grid operation.

[0045] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the line loss data synchronization method as described in the first aspect above is implemented.

[0046] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the line loss data synchronization method as described in the first aspect above is implemented.

[0047] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the line loss data synchronization method as described in the first aspect.

[0048] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the line loss data synchronization method as described in the first aspect above.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 is a flow chart of a line loss data synchronization method provided by some embodiments of the present application;

[0052] Figure 2 It is a structural diagram of realizing data synchronization using the bitsail platform provided in some embodiments of the present application;

[0053] Figure 3 is a schematic diagram of the structure of a line loss data synchronization device provided in some embodiments of the present application;

[0054] Figure 4 It is a schematic diagram of the structure of an electronic device provided in some embodiments of the present application.

[0055] Description of reference numerals:

[0056] 30: Line loss data synchronization device; 301: Meter code data acquisition unit; 302: Electricity quantity determination unit;

[0057] 303: aggregation analysis unit; 304: line loss type determination unit; 305: abnormal line loss synchronization unit;

[0058] 400: electronic device; 401: processor; 402: memory. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0060] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0061] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0062] With the continuous advancement of the intelligent and digital process of the power system, power companies have put forward higher requirements for real-time monitoring and accurate analysis of power grid operation data in the process of achieving efficient operation and improving service quality. Line loss management is a key link in measuring power transmission efficiency and ensuring economic benefits. The accuracy and timeliness of its data directly affect the decision-making and operation and maintenance strategies of power companies.

[0063] There is a close connection between electricity quantity and line loss. Electricity quantity refers to the total amount of electric energy consumed in the power system, while line loss refers to the electric energy loss caused by various factors (such as resistance, inductance, capacitance, etc.) during power transmission. In the power industry, electricity quantity is usually measured by electric energy meters, while line loss is calculated on the basis of electricity quantity measurement by counting the difference between the power supply and the power sales. Line loss is usually measured by line loss rate, that is, the ratio of line loss load to power supply load. Line loss rate is a comprehensive reflection of the management level and economic benefits of power grid companies. Scientific and reasonable line loss management methods are crucial to the stable development of power transmission and distribution and power consumption projects. They can effectively reduce the loss of electric energy in the distribution network and achieve the goal of energy-saving development of the power grid.

[0064] However, in the actual collection, transmission and processing of line loss data, it is often affected by factors such as equipment failure, environmental interference, and calculation errors, resulting in abnormal situations. The existing business systems have the following two defects in synchronizing line loss data:

[0065] Inconsistency in data formats and standards: In the power industry, there is a widespread problem of inconsistency in data formats and standards between various business subsystems, and this phenomenon is particularly prominent when processing line loss data. As a key indicator for measuring power transmission efficiency, the accuracy and integrity of line loss data directly affect the operating efficiency and decision-making quality of power companies. Due to the lack of unified data sources, the integration of these related data faces huge challenges. Different business systems may use their own internal data sources, encoding rules, and even data exchange protocols, resulting in cumbersome and time-consuming data conversion operations when interacting across systems, greatly reducing the efficiency of synchronizing line loss data with other business data. The real-time requirements of line loss data are often not effectively met, which poses an obstacle to activities such as power dispatching and troubleshooting that rely on instant information to respond.

[0066] Limitations of existing big data synchronization technology: Although DataX, Sqoop and other components can be used for processing, DataX runs on a single node, which is unstable, and the configuration of DataX is complex; Sqoop components have low timeliness because Sqoop's underlying layer is MapReduce, which is slow, and Sqoop has low integration for multiple data sources. Existing big data synchronization technology has low integration for multiple data sources and poor stability, and cannot meet the real-time requirements of line loss data synchronization.

[0067] In order to solve the above problems, timely discover and effectively deal with line loss anomalies, and improve the stability and efficiency of power grid operation, the present application embodiment provides a line loss data synchronization method. The line loss data synchronization method, device, electronic device, and storage medium provided in the present application embodiment are described in detail below in conjunction with the accompanying drawings through specific embodiments and their application scenarios.

[0068] The line loss data synchronization method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the line loss data synchronization method. The electronic devices mentioned in the embodiment of the present application include but are not limited to computers and servers running bitsail data synchronization components. The line loss data synchronization method provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0069] Figure 1 is a flow chart of a line loss data synchronization method provided by some embodiments of the present application, such as Figure 1 As shown, the line loss data synchronization method includes: step 110, step 120, step 130, step 140 and step 150.

[0070] Step 110, extracting historical table code curve data from the Hive database;

[0071] It is easy to understand that the Hive database is a data warehouse system built on Hadoop, which provides data summary, query and analysis functions. The Hive database allows users to use a SQL-like query language (HiveQL) to access and process large-scale structured data, especially data stored in the Hadoop Distributed File System (HDFS).

[0072] The historical meter code curve data includes data information such as the voltage, current, power factor, and the amount of electricity used by the user's electric energy meter (also called an electric meter).

[0073] It should be noted that the energy meter terminal is used to collect voltage, current, and power factor data information of the user's energy meter, and upload it to the Hive database in the form of curve data.

[0074] Furthermore, the electronic device extracts historical table code curve data from the Hive database.

[0075] Exemplarily, the electronic device uses a nohup (no hang up) command to run a shell script to extract historical table code curve data from a Hive database.

[0076] Step 120: Determine the power consumption value of each user in the preset power supply area or area according to the historical meter curve data and the real-time meter curve data, as well as the archive information of the user, the device and the area;

[0077] It is easy to understand that the equipment includes the user's electric energy meter, terminal and other equipment, wherein the terminal is the equipment in the transformer that copies the electric energy meter data.

[0078] In the embodiment of the present application, a substation refers to the range powered by a distribution transformer (also referred to as a "distribution station"). Each substation will be divided according to its load conditions and geographical location for the convenience of management and maintenance. Different power supply areas can be divided through substations. The archival information of the substation is used to query which specific user the electric energy meter belongs to and the substation where the user is located.

[0079] Optionally, the electronic device obtains real-time meter curve data, which includes current, voltage, meter reading timestamp and user ID.

[0080] Among them, the archive information is used to divide the electricity meter into which substation and which user. By matching the meter code curve data with the archive information, it is possible to determine which substation and which user the specific meter code curve data belongs to, and then determine the electricity value used by each user in the preset power supply area based on the historical meter code curve data and the real-time meter code curve data, that is, determine the electricity value used by each user in the preset power supply area based on the historical meter code curve data and the real-time meter code curve data, as well as the archive information of the user, equipment and substation.

[0081] For example, the electronic device extracts historical meter curve data from the Hive database, combines it with real-time meter curve data, and calculates the power consumption change of each user through the spark data execution engine and HqlSql data processing. The power consumption change of each user, that is, the power value used by the user, is the difference between the real-time meter data and the historical meter curve data.

[0082] It should be noted that the preset power supply area may be one substation or multiple substations.

[0083] Step 130: Perform regionalized aggregation analysis on the power value to obtain a line loss rate of a preset power supply area or substation;

[0084] It is easy to understand that the power grid will cause power loss due to technical reasons and poor management during the transmission of electric energy. The line loss rate refers to the ratio of power loss caused by resistance, conductivity and capacitance loss of components such as wires, cables, transformers, switchgear, or poor contact, equipment aging, etc. during the power transmission process of the power system to the actual power supply. It is an important indicator for measuring the operating efficiency of the power system. By accurately calculating the line loss rate of the preset power supply area or substation, potential line faults, power theft or other inefficiencies can be discovered in a timely manner, so as to take measures to optimize the operation of the power grid and improve energy efficiency.

[0085] Optionally, the electronic device performs regionalized aggregation analysis on the power value, summarizes and compiles statistics on the power usage data of the preset power supply area or substation, and obtains the total power supply and total power consumption of the preset power supply area or substation. Based on the total power supply and total power consumption of the preset power supply area or substation, the line loss rate of the preset power supply area or substation can be calculated.

[0086] Step 140: Determine the line loss type corresponding to the preset power supply area or substation according to the line loss rate and preset line loss range of the preset power supply area or substation;

[0087] In one embodiment of the present application, determining the line loss type corresponding to the preset power supply area or substation according to the line loss rate and the preset line loss range of the preset power supply area or substation includes:

[0088] When the line loss rate of the preset power supply area or substation is greater than the maximum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is high line loss;

[0089] When the line loss rate of the preset power supply area or substation is less than the minimum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is low line loss;

[0090] The abnormal line loss includes the high line loss and the low line loss.

[0091] It is easy to understand that the line loss type corresponding to the preset power supply area or substation is determined based on the line loss rate and the preset line loss range of the preset power supply area or substation. When the line loss rate of the preset power supply area or substation is greater than the maximum value of the preset line loss range, the line loss type corresponding to the preset power supply area or substation is determined to be high line loss.

[0092] Exemplarily, the line loss rate of the substation is 10%, the preset line loss range is 2% to 5%, and the line loss rate of the substation is greater than the maximum value of the preset line loss range, and the line loss type corresponding to the substation is determined to be high line loss.

[0093] When the line loss rate of the preset power supply area or substation is less than the minimum value of the preset line loss range, it is determined that the line loss type corresponding to the preset power supply area or substation is low line loss.

[0094] Exemplarily, the line loss rate of the preset power supply area is 1%, the preset line loss range is 2% to 5%, the line loss rate of the preset power supply area is less than the minimum value of the preset line loss range, and the line loss type corresponding to the preset power supply area is determined to be low line loss.

[0095] It should be noted that when the line loss rate of the preset power supply area or substation is within the preset line loss range, the line loss type corresponding to the preset power supply area or substation is normal line loss, and high line loss and low line loss are both abnormal line losses.

[0096] In the above technical scheme, by comparing the line loss rate of the preset power supply area or substation with the preset line loss range, the line loss type corresponding to the preset power supply area or substation can be determined, and the preset power supply area or substation corresponding to the abnormal line loss can be located, which facilitates the subsequent synchronization and analysis of abnormal line loss data.

[0097] Step 150: When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, synchronize the table code curve data corresponding to each user in the preset power supply area or substation to the data center through the bitsail platform.

[0098] Furthermore, when the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the corresponding table code curve data for each user in the preset power supply area or substation corresponding to the abnormal line loss is synchronized to the data middle station through the bitsail platform, and the data middle station performs line loss analysis based on the table code curve data corresponding to each user in the preset power supply area or substation.

[0099] It should be noted that the Bitsail data synchronization component is an open source data synchronization and integration platform, which is often used for data flow and processing in big data environments.

[0100] Optionally, the data middle platform can be understood as a platform for processing data, and data monitoring, application, and operational decision-making can be performed on the data middle platform. This application uses bitsail, which has distributed execution capabilities. When abnormal line loss data needs to be synchronized in the business system, the yarn resource management system can be used to process and transmit data, improve the data transmission rate, solve the problem of slow data synchronization, and support the synchronization of multiple data sources.

[0101] The line loss data synchronization method provided in the embodiment of the present application extracts historical table code curve data from the Hive database, determines the electricity value used by each user in the preset power supply area or substation according to the historical table code curve data and the real-time table code curve data, as well as the archival information of users, equipment and substations, performs regionalized aggregation analysis on the electricity value, and obtains the line loss rate of the preset power supply area or substation. When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the table code curve data corresponding to each user in the preset power supply area or substation is synchronized to the data middle station through the bitsail platform, and the abnormal line loss data in the Hive database can be synchronized to the data middle station in time, and can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time performance of the synchronization of abnormal line loss data, so that the line loss anomaly can be analyzed in time on the data middle station, thereby improving the stability of power grid operation.

[0102] In one embodiment of the present application, the method further includes:

[0103] Acquire the file information, and store the file information in a relational database;

[0104] The real-time table code curve data is obtained, and the real-time table code curve data is synchronized to a Hive database.

[0105] It is easy to understand that a relational database is a database system that uses a relational model to store and manage data, uses tables (rows and columns) to organize data, and uses structured query language (SQL) to query and manage data. Relational databases include MySQL, PostgreSQL, and Oracle databases. Among them, Oracle database can provide high performance and high availability, supports a variety of advanced functions (such as partitioning and parallel query), and is widely used in enterprise-level applications and large-scale data processing. Since the amount of archival information is not large, using a relational database to store archival information can achieve structured storage of archival information.

[0106] Exemplarily, the electronic device obtains archive information, including user archive information, substation archive information, meter archive information and other archive information, and stores the archive information in an Oracle database.

[0107] In one embodiment of the present application, the real-time meter curve data includes current, voltage, meter reading, timestamp and user ID, and the real-time meter curve data is obtained through 96-point synchronous sampling and phase-by-phase statistics.

[0108] It is easy to understand that the electronic equipment performs daily statistical line loss operations. The daily statistical line loss is used to calculate the electricity meter and meter curve freeze electricity at 96 points per day (current, voltage, meter reading, current timestamp and user ID are sampled once a day at an interval of 15 minutes, for a total of 96 points), obtain 96-point synchronous sampling curves and phase statistics (phase statistics are used to monitor and analyze the operating status of the three-phase power system, and the current, voltage, power and other parameters of each phase are separately counted), and store them in the form of curve data to obtain real-time meter code curve data.

[0109] In the above technical solution, real-time meter code curve data is obtained through 96-point synchronous sampling and phase statistics, and current, voltage, and meter readings at multiple time points can be collected, which improves the real-time and accuracy of data acquisition and is conducive to timely detection of abnormal data.

[0110] Furthermore, since the amount of real-time code curve data is large, the electronic device synchronizes the real-time code curve data to the Hive database. The Hive database can efficiently process large-scale data sets and is suitable for storing a large amount of code curve data in this application. In addition, Hive supports reading data from multiple data sources, which facilitates data synchronization between different databases.

[0111] In the above technical solution, the archive information is stored in a relational database, and the real-time table code curve data is obtained and synchronized to the Hive database. The synchronization of the real-time table code curve data can be realized, and it can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time performance of data synchronization and improving the operating efficiency and service quality of the power system.

[0112] In one embodiment of the present application, synchronizing the real-time table code curve data to a Hive database includes:

[0113] The real-time table code curve data is stored in the Kafka message queue in JSON format;

[0114] Extracting the real-time code curve data in JSON format from the Kafka message queue through the bitsail platform, parsing the real-time code curve data in JSON format, and obtaining structured data corresponding to the real-time code curve data;

[0115] Matching the structured data with the archive information to determine the user and station area corresponding to the real-time table code curve data;

[0116] The real-time table code curve data and the users and stations corresponding to the real-time table code curve data are synchronized to the Hive database.

[0117] It is easy to understand that Kafka, as a distributed message queue system, plays the role of a middleware for data temporary storage and transmission. It is widely used in real-time data stream processing scenarios. Real-time meter curve data is stored in the Kafka message queue in JSON format. The real-time meter curve data includes current, voltage, meter readings, timestamps, user IDs and other data.

[0118] Furthermore, the bitsail platform connects to the Kafka queue, extracts the real-time table code curve data in JSON format from the Kafka message queue, parses the real-time table code curve data in JSON format, and obtains the structured data corresponding to the real-time table code curve data.

[0119] Finally, the electronic device matches the structured data with the archival information, determines the users and stations corresponding to the real-time table code curve data, and synchronizes the real-time table code curve data and the users and stations corresponding to the real-time table code curve data to the Hive database according to the preset mapping rules. As a data warehouse tool built on Hadoop, Hive is very suitable for storing and querying large-scale data sets, which can facilitate subsequent data analysis.

[0120] In the above technical solution, the bitsail platform extracts the real-time table code data from the Kafka message queue for analysis, obtains the structured data corresponding to the real-time table code data, matches the structured data with the archival information, determines the users and substations corresponding to the real-time table code curve data, and synchronizes the real-time table code curve data and the users and substations corresponding to the real-time table code curve data to the Hive database, thereby improving the speed and accuracy of the synchronization of the real-time table code curve data.

[0121] In one embodiment of the present application, the method further includes:

[0122] Capturing archive change data from the relational database using Flink’s change data capture function;

[0123] Write the archive change data into the Hive archive table through Flink's Hive connector;

[0124] The archive change data is synchronized from the Hive archive table to the Hive database through the bitsail platform.

[0125] It is easy to understand that the Flink CDC (Flink Change Data Capture) function can monitor the logs of the relational database in real time and capture the file change data in the relational database (additions, deletions, and modifications to the file information of users, stations, and devices).

[0126] Furthermore, Flink CDC captures every add, delete, and modify operation of the relational database, which can be transmitted to the Hive archive table in real time through Flink's Hive connector.

[0127] Finally, the archive change data is synchronized from the Hive archive table to the Hive database through the bitsail platform.

[0128] Figure 2 is a schematic diagram of a structure for implementing data synchronization using the bitsail platform provided in some embodiments of the present application, such as Figure 2 As shown, at the data display level, data monitoring, application and operational decision-making can be achieved through the database, and at the data processing level, the spark processing engine or Hql Sql data processing can be used.

[0129] The data sources include real-time table code data stored in the Kafka queue, historical table code curve data stored in the Hive database, and user profile data stored in the relational database (such as Oracle). The bitsail platform synchronizes the real-time table code curve data to the Hive database by connecting to the Kafka queue. Flink CDC captures profile change data from the relational database and writes the profile change data to the Hive profile table through the Hive connector. The bitsail platform synchronizes the profile change data from the Hive profile table to the Hive database.

[0130] Optionally, data storage can use a customized database system that supports OLAP (Online Analytical Processing) and OLTP (Online Transaction Processing) scenarios, and can be used for data monitoring, data application, and data decision-making, etc., to facilitate relevant personnel to quickly query and analyze data.

[0131] Optionally, the Bitsail platform can also use Flink and YARN (Yet Another Resource Navigato) as the underlying data synchronization engine and resource scheduling, giving it distributed execution capabilities. When data needs to be synchronized to other business systems, the YARN resource management system can be used to process and transmit data to improve data transmission speed.

[0132] In the above technical solution, the archive change data is captured from the relational database through Flink's change data capture function, and the bitsail platform synchronizes the archive change data from the Hive archive table to the Hive database, thereby realizing synchronization between different data sources. It is suitable for data integration of multiple data sources and can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time performance of data synchronization.

[0133] In one embodiment of the present application, performing regionalized aggregate analysis on the power value to obtain the line loss rate of a preset power supply area or substation includes:

[0134] According to the power consumption value of each user in the preset power supply area or substation area, the total power consumption value of all users in the preset power supply area or substation area is calculated;

[0135] Obtaining the difference between the power value used by the total meter of a preset power supply area or area and the total power value used by all users in the preset power supply area or area;

[0136] The line loss rate of the preset power supply area or substation is determined according to the difference and the power value used by the total meter of the preset power supply area or substation.

[0137] It is easy to understand that the electronic device calculates the total power value (X) used by all users in the preset power supply area or area based on the power value used by each user in the preset power supply area or area, and obtains the power value (Y) used by the total meter of the preset power supply area or area. The power value (Y) used by the total meter of the preset power supply area or area is the total power supply corresponding to the preset power supply area or area.

[0138] Furthermore, the difference (YX) between the total power value (X) used by all users in the preset power supply area or substation and the power value (Y) used by the total meter in the preset power supply area or substation is calculated.

[0139] Finally, according to the difference (YX) and the power value (Y) used by the total meter of the preset power supply area or substation, the calculation formula for determining the line loss rate of the preset power supply area or substation is as follows:

[0140]

[0141] Wherein, Z represents the line loss rate of the preset power supply area or substation, X represents the total power value used by all users in the preset power supply area or substation, and Y represents the power value used by the total meter of the preset power supply area or substation.

[0142] In the above technical solution, the difference is obtained by the power value used by the total meter of the preset power supply area or substation and the total power value used by all users in the preset power supply area or substation. According to the difference and the power value used by the total meter of the preset power supply area or substation, the line loss rate of the preset power supply area or substation is obtained. The loss occurring during the power transmission process can be quickly identified, the line loss abnormality can be discovered in time and corresponding measures can be taken, thereby improving the level of line loss management.

[0143] Optionally, the entire process of the above-mentioned line loss data synchronization method embodiment can be executed in the background, and scheduled and executed using Crontab or Dolphin Scheduler. Crontab is a scheduled task scheduling tool in Unix and Unix-like systems, which allows users to set scheduled tasks and automatically execute them at the specified time. Dolphin Scheduler is a distributed visual workflow task scheduling system that can provide rich scheduling functions and powerful scalability.

[0144] Figure 3 is a schematic diagram of the structure of a line loss data synchronization device provided in some embodiments of the present application, such as Figure 3As shown, the line loss data synchronization device 30 includes a table code data acquisition unit 301, an electric quantity determination unit 302, an aggregation analysis unit 303, a line loss type determination unit 304 and an abnormal line loss synchronization unit 305.

[0145] Table code data acquisition unit 301, used to extract historical table code curve data from the Hive database;

[0146] The power determination unit 302 is used to determine the power value used by each user in the preset power supply area or substation according to the historical meter curve data and the real-time meter curve data, as well as the archive information of the user, the device and the substation;

[0147] An aggregate analysis unit 303 is used to perform regional aggregate analysis on the power value to obtain a line loss rate of a preset power supply area or substation;

[0148] A line loss type determination unit 304, configured to determine a line loss type corresponding to the preset power supply area or substation according to a line loss rate and a preset line loss range of the preset power supply area or substation;

[0149] The abnormal line loss synchronization unit 305 is used to synchronize the table code curve data corresponding to each user in the preset power supply area or substation to the data center through the bitsail platform when the line loss type corresponding to the preset power supply area or substation is abnormal line loss.

[0150] Optionally, the table code data acquisition unit 301 is further used for:

[0151] Acquire the file information, and store the file information in a relational database;

[0152] The real-time table code curve data is obtained, and the real-time table code curve data is synchronized to a Hive database.

[0153] Optionally, the abnormal line loss synchronization unit 305 is further configured to:

[0154] The real-time table code curve data is stored in the Kafka message queue in JSON format;

[0155] Extracting the real-time code curve data in JSON format from the Kafka message queue through the bitsail platform, parsing the real-time code curve data in JSON format, and obtaining structured data corresponding to the real-time code curve data;

[0156] Matching the structured data with the archive information to determine the user and station area corresponding to the real-time table code curve data;

[0157] The real-time table code curve data and the users and stations corresponding to the real-time table code curve data are synchronized to the Hive database.

[0158] Optionally, the abnormal line loss synchronization unit 305 is further configured to:

[0159] Capturing archive change data from the relational database using Flink’s change data capture function;

[0160] Write the archive change data into the Hive archive table through Flink's Hive connector;

[0161] The archive change data is synchronized from the Hive archive table to the Hive database through the bitsail platform.

[0162] Optionally, the aggregation analysis unit 303 is used to:

[0163] According to the power consumption value of each user in the preset power supply area or substation area, the total power consumption value of all users in the preset power supply area or substation area is calculated;

[0164] Obtaining the difference between the power value used by the total meter of a preset power supply area or area and the total power value used by all users in the preset power supply area or area;

[0165] The line loss rate of the preset power supply area or substation is determined according to the difference and the power value used by the total meter of the preset power supply area or substation.

[0166] Optionally, the line loss type determination unit 304 is configured to:

[0167] When the line loss rate of the preset power supply area or substation is greater than the maximum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is high line loss;

[0168] When the line loss rate of the preset power supply area or substation is less than the minimum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is low line loss;

[0169] The abnormal line loss includes the high line loss and the low line loss.

[0170] Optionally, the real-time meter curve data includes current, voltage, meter reading, timestamp and user ID, and the real-time meter curve data is obtained through 96-point synchronous sampling and phase-by-phase statistics.

[0171] In the above technical scheme, historical table code curve data is extracted from the Hive database, and the electricity value used by each user in the preset power supply area or substation is determined according to the historical table code curve data and the real-time table code curve data, as well as the archival information of users, equipment and substations. The electricity value is regionalized and aggregated for analysis to obtain the line loss rate of the preset power supply area or substation. When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the table code curve data corresponding to each user in the preset power supply area or substation is synchronized to the data middle platform through the bitsail platform, so that the abnormal line loss data in the Hive database can be synchronized to the data middle platform in time, and can be flexibly used in different business systems or different storage structures, thereby improving the speed and real-time synchronization of abnormal line loss data, so that the line loss anomaly can be analyzed in time at the data middle platform, thereby improving the stability of power grid operation.

[0172] The line loss data synchronization device 30 in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or a device other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0173] The line loss data synchronization device 30 in the embodiment of the present application may be a device having an operating system. The operating system may be a Microsoft (Windows) operating system, an Android (Android) operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0174] The line loss data synchronization device 30 provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0175] In some embodiments, Figure 4 As shown, an embodiment of the present application further provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, each process of the above-mentioned line loss data synchronization method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0176] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0177] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned line loss data synchronization method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0178] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0179] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned line loss data synchronization method when executed by a processor.

[0180] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0181] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned line loss data synchronization method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0182] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0183] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0185] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0186] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0187] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A line loss data synchronization method, characterized in that: include: Extract historical table code curve data from Hive database; Determine the power consumption value of each user in the preset power supply area or area according to the historical meter curve data and the real-time meter curve data, as well as the archive information of the user, the equipment and the area; Performing regionalized aggregation analysis on the power value to obtain a line loss rate in a preset power supply area or substation; Determine the line loss type corresponding to the preset power supply area or substation according to the line loss rate and preset line loss range of the preset power supply area or substation; When the line loss type corresponding to the preset power supply area or substation is abnormal line loss, the table code curve data corresponding to each user in the preset power supply area or substation is synchronized to the data center through the bitsail platform.

2. The line loss data synchronization method according to claim 1, characterized in that: The method further comprises: Acquire the file information, and store the file information in a relational database; The real-time table code curve data is obtained, and the real-time table code curve data is synchronized to a Hive database.

3. The line loss data synchronization method according to claim 2, characterized in that: The step of synchronizing the real-time table code curve data to the Hive database includes: The real-time table code curve data is stored in the Kafka message queue in JSON format; Extracting the real-time code curve data in JSON format from the Kafka message queue through the bitsail platform, parsing the real-time code curve data in JSON format, and obtaining structured data corresponding to the real-time code curve data; Matching the structured data with the archive information to determine the user and station area corresponding to the real-time table code curve data; The real-time table code curve data and the users and stations corresponding to the real-time table code curve data are synchronized to the Hive database.

4. The line loss data synchronization method according to claim 2, characterized in that: The method further comprises: Capturing archive change data from the relational database using Flink’s change data capture function; Write the archive change data into the Hive archive table through Flink's Hive connector; The archive change data is synchronized from the Hive archive table to the Hive database through the bitsail platform.

5. The line loss data synchronization method according to claim 1, characterized in that: The performing regionalized aggregation analysis on the power value to obtain the line loss rate of a preset power supply area or substation includes: According to the power consumption value of each user in the preset power supply area or substation area, the total power consumption value of all users in the preset power supply area or substation area is calculated; Obtaining the difference between the power value used by the total meter of a preset power supply area or area and the total power value used by all users in the preset power supply area or area; The line loss rate of the preset power supply area or substation is determined according to the difference and the power value used by the total meter of the preset power supply area or substation.

6. The line loss data synchronization method according to claim 1, characterized in that: The determining, according to the line loss rate and the preset line loss range of the preset power supply area or substation, the line loss type corresponding to the preset power supply area or substation includes: When the line loss rate of the preset power supply area or substation is greater than the maximum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is high line loss; When the line loss rate of the preset power supply area or substation is less than the minimum value of the preset line loss range, determining that the line loss type corresponding to the preset power supply area or substation is low line loss; The abnormal line loss includes the high line loss and the low line loss.

7. The line loss data synchronization method according to claim 2, characterized in that: The real-time meter code curve data includes current, voltage, meter reading, timestamp and user ID, and the real-time meter code curve data is obtained through 96-point synchronous sampling and phase-by-phase statistics.

8. A line loss data synchronization device, characterized in that: include: Table code data acquisition unit, used to extract historical table code curve data from the Hive database; The power determination unit is used to determine the power value used by each user in the preset power supply area or substation according to the historical meter code curve data and the real-time meter code curve data, as well as the archive information of the user, the equipment and the substation; An aggregate analysis unit, used to perform regional aggregate analysis on the power value to obtain a line loss rate in a preset power supply area or substation; A line loss type determination unit, configured to determine a line loss type corresponding to the preset power supply area or substation according to a line loss rate and a preset line loss range of the preset power supply area or substation; The abnormal line loss synchronization unit is used to synchronize the table code curve data corresponding to each user in the preset power supply area or substation to the data center through the bitsail platform when the line loss type corresponding to the preset power supply area or substation is abnormal line loss.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the line loss data synchronization method as described in any one of claims 1-7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the line loss data synchronization method as described in any one of claims 1 to 7 is implemented.

Images