Power data processing method, device, and computer equipment

Abstract

The present application relates to a method, device, computer equipment and storage medium for processing power data. The method comprises: obtaining power business processing requirements; retrieving power parameters related to the power business processing requirements from a kafka message queue; wherein, a consumer group is pre-registered with a kafka message queue through a flink stream processor, and based on the consumer group, the power parameters generated by power consumption in a set size window are obtained through the kafka message queue, and the power parameters are written into the kafka message queue; and power business processing is performed on the power parameters. This method can overcome the lag in power data and better support decisions that require real-time power data support.

Description

Technical Field

[0001] The present application relates to the technical field of big data computing and processing, and in particular to a method, device, computer equipment and storage medium for processing power data. Background Art

[0002] With the development of big data computing and processing technology, power data processing technology has emerged in order to extract the required valuable data from massive data more quickly.

[0003] In traditional technology, the data center built by the power enterprise regularly collects basic data from the source system into the warehouse, and supports the internal business decision-making of the enterprise through batch scheduling data processing, which has extremely high data asset value. In data calculation, the commonly used methods are: writing database stored procedures for processing, writing shell (a scripting language) or perl scripts (a scripting language) for data processing, using tools to connect to the database, and realizing data association conversion processing in the tool.

[0004] However, the current traditional method uses a timed acquisition method to obtain various power data when processing power data. It needs to rely on the scheduling cycle of each power data to complete the daily scheduling task of power data processing. When some power data has a large lag, some decisions that require real-time power data support will not be fully supported. Summary of the invention

[0005] Based on this, it is necessary to provide a power data processing method, device, computer equipment and storage medium that can overcome the lag in power data and better support decisions that require real-time power data to address the above technical problems.

[0006] A method for processing power data, the method comprising:

[0007] Obtain power business processing requirements;

[0008] Retrieving power parameters related to the power business processing requirements from the Kafka message queue; wherein, a consumer group is pre-registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written into the Kafka message queue;

[0009] Perform power business processing on the power parameters.

[0010] In one of the embodiments, before writing the power parameters into the Kafka message queue, the method further includes:

[0011] Verifying whether the power parameters are successfully associated with each other according to the power business processing requirements;

[0012] When the power parameters are successfully associated with each other, executing the step of writing the power parameters into the Kafka message queue;

[0013] When the association between the power parameters fails, the power parameters are written into the Hbase table.

[0014] In one embodiment, after writing the incremental power parameter into the Kafka message queue, the method further includes:

[0015] Clean up the incremental power parameters written into the Kafka message queue from the Hbase table.

[0016] In one embodiment, after writing the incremental power parameter into the Kafka message queue, the method further includes:

[0017] Clean up the incremental power parameters written into the Kafka message queue from the Hbase table.

[0018] In one embodiment, the consumer group includes multiple subscription topics, and one power parameter corresponds to one subscription topic; writing the power parameter into an Hbase table includes:

[0019] According to different subscription topics, the power parameters that fail to be associated are written into the corresponding Hbase tables respectively, and one Hbase table stores the power parameters corresponding to one subscription topic.

[0020] In one of the embodiments, after registering the consumer group with the Kafka message queue in advance through the Flink stream processor, the method further includes:

[0021] Determining the primary key of the Hbase according to the generation time point of the power parameter and the measurement point of the power parameter;

[0022] Cleaning the incremental power parameters written into the Kafka message queue from the Hbase table includes:

[0023] When the primary keys in the Hbase table are the same, only the power parameters obtained at the latest generation time point are retained, and the incremental power parameters written into the Kafka message queue from the Hbase table are cleared.

[0024] A power data processing device, the device comprising:

[0025] A business processing requirement acquisition module is used to acquire power business processing requirements;

[0026] The power parameter acquisition module is used to retrieve the power parameters related to the power business processing requirements from the Kafka message queue; wherein, the consumer group is registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written to the Kafka message queue;

[0027] The power service processing module is used to perform power service processing on the power parameters.

[0028] In one embodiment, the device also includes: an association verification device, used to verify whether each of the power parameters is successfully associated according to the power business processing requirements; when the power parameters are successfully associated, executing the step of writing the power parameters to the kafka message queue; when the association between the power parameters fails, writing the power parameters into the Hbase table.

[0029] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0030] Obtain power business processing requirements;

[0031] Retrieving power parameters related to the power business processing requirements from the Kafka message queue; wherein, a consumer group is pre-registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written into the Kafka message queue;

[0032] Perform power business processing on the power parameters.

[0033] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0034] Obtain power business processing requirements;

[0035] Retrieving power parameters related to the power business processing requirements from the Kafka message queue; wherein, a consumer group is pre-registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written into the Kafka message queue;

[0036] Perform power business processing on the power parameters.

[0037] The above-mentioned power data processing method, device, computer equipment and storage medium obtain power business processing requirements; retrieve power parameters related to the power business processing requirements from the kafka message queue; wherein, register a consumer group with the kafka message queue through the flink stream processor, based on the consumer group, obtain the power parameters generated by the power consumption of the set size window through the kafka message queue, and write the power parameters into the kafka message queue; perform power business processing on the power parameters. The kafka message queue can perceive the changes of power parameters in real time. The power parameters are stored in the kafka message queue. It only needs to use the consumer to connect to the kafka message queue to quickly perceive the changes in power parameters. Each operation only needs to process the power parameters (power data) stored in the kafka message queue, and there is no need to continuously scan the underlying data of the database to obtain the changed new power parameters. Accordingly, after the terminal obtains the power business requirements, it can directly retrieve the relevant power parameters from the kafka message queue for power business processing, without the need to scan the underlying data of the database multiple times to obtain the changed power parameters. Therefore, the present application can overcome the hysteresis of power data and can better support decisions that require real-time power data support. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is an application environment diagram of a power data processing method in an embodiment;

[0039] Figure 2 is a flow chart of a power data processing method in one embodiment;

[0040] Figure 3 A schematic diagram of a process before retrieving power parameters related to power business processing requirements from a Kafka message queue in one embodiment;

[0041] Figure 4 is a structural block diagram of a power data processing device in one embodiment;

[0042] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0044] The power data processing method provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 may be, but is not limited to, various computers.

[0045] When retrieving power parameters, three different computer terminals are involved, one is the upstream production terminal 102, one is the downstream user terminal 106, and the other is the terminal 104 referred to in this application, which is located between the upstream production terminal 102 and the downstream user terminal 106, and plays the role of connecting the upstream production terminal 102 and the downstream user terminal 106. Specifically, the upstream production terminal 102 obtains various power parameters (power data) by collecting data from various meter measurement points. Before the terminal 104 obtains the power business demand sent by the downstream user terminal 106, the consumer group has been pre-registered to the kafka message queue through the flink stream processor, so that the kafka message queue and the flink stream processor are adapted, so that the subscription topic that the kafka message queue needs to perceive can be determined according to the consumer group. When the kafka message queue obtains the power parameters related to the power business processing demand, it will be stored in the kafka message queue. When terminal 104 obtains the electricity business demand sent by the downstream terminal 106, terminal 104 retrieves the electricity parameters related to the electricity business processing demand from the kafka message queue, and sends these electricity parameters to the downstream terminal 106 for the downstream terminal 106 to process the relevant electricity business.

[0046] The upstream party producing power data collects meter data from each measuring point. The measuring point refers to the location where the meter is installed corresponding to each user. By collecting meter data from each measuring point, the power consumption of each user can be known.

[0047] In one embodiment, Figure 2 As shown, a method for processing power data is provided, and the method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0048] Step S202, obtaining power business processing requirements.

[0049] Among them, the power business processing demand is an electrical signal sent by the "consumer" terminal according to the needs of power business processing, and the "consumer" is the downstream party that uses the power data. After the "consumer" terminal sends out the power business processing demand, the terminal located between the upstream party that produces the power data and the downstream party that uses the power data obtains the power business processing demand sent by the "consumer" terminal (the terminal in the middle is the execution subject of this application, hereinafter referred to as the terminal).

[0050] Step S204, retrieving power parameters related to the power business processing requirements from the Kafka message queue; wherein, a consumer group is pre-registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written to the Kafka message queue.

[0051] Specifically, after the terminal obtains the power service processing requirements, since the power parameters related to the power service processing requirements are stored in the Kafka message queue, the terminal will retrieve the power parameters related to the power service processing requirements from the Kafka message queue. For example, the power service processing requirements of the downstream terminal are used to obtain the real-time power value at each time point. Since the power value can be the product of the current value and the voltage value, the power parameters related to the power service processing requirements can be the voltage value and the corresponding current value at each time point.

[0052] Before storing the power parameters related to the power business processing requirements in the Kafka message queue, the terminal pre-registers the consumer group with the Kafka message queue through the Flink stream processor. This process adapts the Flink stream processor and the Kafka message queue. As a result, when the Kafka message queue obtains the real-time power parameters from the upstream, it can send the corresponding real-time power parameters to the Flink stream processor according to the registered consumer group. The consumer group is a scalable and fault-tolerant consumer mechanism provided by the Kafka message queue.

[0053] The number of power parameters obtained by the kafka message queue within a period of time can be adjusted and controlled by setting the size window. Among them, the size of the window is set according to the flink stream processor. The set size window can be understood as a time period during which the kafka message queue can obtain the power parameters generated by power consumption. The window size is determined by time. The longer the time, the larger the corresponding window, and the more power parameters pass through the window. For example, if the set window size for the flink stream processor to receive power parameters is 2 minutes, and the time required to retrieve the power parameters is 10 minutes, then the flink stream processor is equivalent to dividing it into 5 times in time order within the time length, and continuously receiving the real-time power parameters perceived by the kafka message queue. When obtaining the power parameters of the set size window, the terminal writes these power parameters to (stores them in) the kafka message queue.

[0054] Step S206: performing power business processing on the power parameters.

[0055] Specifically, when the terminal obtains the aforementioned power parameters, it will perform corresponding power business processing according to these power parameters, that is, send these power parameters to the downstream terminal for the downstream terminal to process related power business.

[0056] In the above-mentioned power data processing method, the kafka message queue can sense the changes in power parameters in real time. The power parameters are stored in the kafka message queue. It is only necessary to use consumers to connect to the kafka message queue to quickly sense the changes in power parameters. Each operation only requires processing the power parameters (power data) stored in the kafka message queue, and there is no need to continuously scan the underlying database data to obtain the changed new power parameters. Accordingly, when the terminal obtains the power business demand, it can directly retrieve the relevant power parameters from the kafka message queue for power business processing, without the need to scan the underlying database data multiple times to obtain the changed power parameters. Therefore, this application can overcome the lag in power data and can better support decisions that require real-time power data support.

[0057] In one embodiment, before writing the power parameters into the Kafka message queue, the method further includes:

[0058] Verify whether the power parameters are successfully associated according to the power business processing requirements; when the power parameters are successfully associated, execute the step of writing the power parameters into the kafka message queue; when the power parameters fail to be associated, write the power parameters into the Hbase table.

[0059] Specifically, the various power parameters are associated according to the power business processing requirements. When the power parameters are successfully associated, the power parameters are written into the kafka message queue. Since there may be some data delays in obtaining power parameters due to the set size window, these power parameters that could have been successfully associated will be temporarily written into the Hbase table. The Hbase table does not have a secondary index, and fast indexing can be achieved through the primary key. This application adopts but is not limited to the Hbase table, and other table structures that can achieve fast reading and writing can also be used.

[0060] In this embodiment, before writing the power parameters into the Kafka message queue, the terminal verifies whether the power parameters are successfully associated with each other according to the power business processing requirements to ensure that the power parameters written into the Kafka message channel are the data required by the downstream terminal.

[0061] In one embodiment, after writing the power parameters into the Hbase table, the method further includes:

[0062] According to the set interval threshold, the power parameters in the Hbase table are regularly scanned at high frequency every set interval threshold to obtain the incremental power parameters, and the incremental power parameters are written into the kafka message queue.

[0063] Specifically, the set interval threshold is a set time interval threshold, which is set by the terminal and can be changed. The terminal can scan the delayed power parameters in the Hbase table at every set interval threshold through a high-frequency scanning program, and write the power parameters that cannot be successfully associated due to delay into the kafka message queue. These power parameters that can be successfully associated but cannot be successfully associated due to delay are called incremental power parameters.

[0064] In this embodiment, the Hbase table is scanned periodically, so that the delayed but associated power parameters in the Hbase table are written into the Hbase table.

[0065] In one embodiment, after writing the incremental power parameters into the Kafka message queue, the method further includes: clearing the incremental power parameters written into the Kafka message queue from the Hbase table.

[0066] Specifically, after the incremental power parameters are written into the Kafka message queue, in order to reduce the amount of calculation of the terminal, the incremental power parameters written into the Kafka message queue in the Hbase table will be cleared.

[0067] In this embodiment, the incremental power parameters written into the Kafka message queue are cleaned up, thereby reducing redundant data in the Hbase table and reducing the amount of computer calculation, thereby improving the efficiency of writing the power parameters associated with the delay energy into the Kafka message queue.

[0068] In one embodiment, the consumer group includes multiple subscription topics, and one power parameter corresponds to one subscription topic; writing the power parameter into an Hbase table includes:

[0069] According to different subscription topics, the power parameters that fail to be associated are written into the corresponding Hbase tables respectively, and one Hbase table stores the power parameters corresponding to one subscription topic.

[0070] Among them, a consumer group has multiple subscription topics, and one power parameter corresponds to one subscription topic. For example, if the power business processing requirement of the downstream terminal is to obtain the real-time power value at each time point, and the power parameters related to the power business processing requirement are the voltage value and the corresponding current value at each time point, then the voltage value can be regarded as a subscription topic, and the current value can be regarded as a subscription topic.

[0071] Specifically, there are multiple Hbase tables in the terminal, and the table data stored in each Hbase table is determined according to the subscription topic, that is, one Hbase table only stores the power parameters corresponding to one subscription topic. For example, if Hbase table (1) corresponds to the subscription topic of current value, then the corresponding Hbase table (1) only stores the power parameters corresponding to the subscription topic. The power parameters that cannot be associated temporarily due to delay will be written into the corresponding Hbase tables according to different subscription topics.

[0072] In this embodiment, according to different subscription topics, the power parameters that failed to be associated are written into the corresponding Hbase table respectively, so as to facilitate the grouping of power parameters and effectively manage and divide the data. When the terminal regularly scans the data that failed to be associated in the Hbase table through a high-frequency scanning program, it is beneficial to more quickly delay the power parameters that can be associated in the Hbase table.

[0073] In one embodiment, after registering the consumer group with the Kafka message queue in advance through the Flink stream processor, it also includes:

[0074] Determining the primary key of the Hbase according to the generation time point of the power parameter and the measurement point of the power parameter;

[0075] Cleaning the incremental power parameters written into the Kafka message queue from the Hbase table includes:

[0076] When the primary keys in the Hbase table are the same, only the power parameters obtained at the latest generation time point are retained, and the incremental power parameters written into the Kafka message queue from the Hbase table are cleared.

[0077] Specifically, the primary key in the Hbase table is determined based on the generation time point of the power parameter (the generation time point does not include year / month / day, but only general hours / minutes / seconds) and the measurement point of the power parameter. The unique corresponding power parameter can be obtained by combining the measurement point and generation time of the power parameter. Therefore, the unique power parameter corresponding to each generation time point and measurement point can be obtained according to the primary key of the Hbase table.

[0078] When the primary keys in the Hbase table are the same, there may be at least two identical power parameters, whose generation time points and measurement points are consistent. The high-frequency scanning program will clean up the power parameters obtained at historical moments.

[0079] In this embodiment, when the primary keys in the Hbase table are the same, only the power parameters obtained at the latest generation time point are retained, and the incremental power parameters written to the Kafka message queue in the Hbase table are cleared, thereby effectively preventing data redundancy and repeated scanning of expired data.

[0080] In one embodiment, before retrieving the power parameters related to the power service processing requirements from the Kafka message queue, steps S302 to S314 are included, wherein:

[0081] Step S302, register the consumer group to the Kafka message queue through the Flink stream processor.

[0082] Step S304: Based on the consumer group, power parameters generated by power consumption in a set size window are obtained through the Kafka message queue.

[0083] Step S306, verify whether the power parameters are successfully associated according to the power business processing requirements. If the power parameters are successfully associated, execute step S308 to write the power parameters into the Kafka message queue. Then execute step S310 to perform power business processing on the power parameters.

[0084] When the association between the power parameters fails, step S312 is executed to write the power parameters into the Hbase table.

[0085] After executing step S312, execute step S314, and regularly perform high-frequency scanning on the power parameters in the Hbase table every set interval threshold according to the set interval threshold, obtain the incremental power parameters, and write the incremental power parameters into the kafka message queue.

[0086] After the incremental power parameters are written into the Kafka message queue, step S316 is executed to clean up the incremental power parameters written into the Kafka message queue in the Hbase table.

[0087] In this embodiment, based on the rapid data change perception of the kafka message queue, there is no need to scan and judge the underlying data of the database through SQL (database), which reduces the database pressure. Based on the rapid data change perception of the kafka message queue, there is no need to scan and judge the underlying data of the database through SQL, which reduces the database pressure. It can also make full use of the batch and stream integrated data processing capabilities of the Flink stream processor to divide the changing power data into data windows, drive data correlation analysis within the time window with data, and provide real-time result data for decision analysis.

[0088] It should be understood that, although the steps in each flow chart involved in the above-described embodiment are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a portion of the steps in each flow chart involved in the above-described embodiment may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the steps or stages in other steps.

[0089] In one embodiment, Figure 4 As shown, a power data processing device is provided, including: a business processing demand acquisition module 402, a power parameter acquisition module 404 and a power business processing module 406, wherein:

[0090] The business processing requirement acquisition module 402 is used to acquire the power business processing requirement;

[0091] The power parameter acquisition module 404 is used to retrieve the power parameters related to the power business processing requirements from the Kafka message queue; wherein, the consumer group is registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written into the Kafka message queue;

[0092] The power service processing module 406 is used to perform power service processing on the power parameters.

[0093] In one embodiment, the power data processing device also includes: an association verification device, used to verify whether each of the power parameters is successfully associated according to the power business processing requirements; when the power parameters are successfully associated, executing the step of writing the power parameters to the kafka message queue; when the power parameters fail to be associated, writing the power parameters into the Hbase table.

[0094] In one embodiment, the power parameter acquisition module also includes: an incremental power parameter acquisition module, which is used to regularly perform high-frequency scanning on the power parameters in the Hbase table at intervals of a set interval threshold according to a set interval threshold, obtain the incremental power parameters, and write the incremental power parameters into the kafka message queue.

[0095] In one embodiment, the power data processing device further includes: a power parameter cleaning module for cleaning the incremental power parameters written into the Kafka message queue in the Hbase table after writing the incremental power parameters into the Kafka message queue.

[0096] For the specific definition of the power data processing device, please refer to the definition of the power data processing method above, which will not be repeated here. Each module in the above power data processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0097] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for processing power data is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a cleaned touch layer on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0098] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0099] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0100] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the steps in the above-mentioned method embodiments. It can be understood by a person skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium, and the computer program can include the processes of the embodiments of the above-mentioned methods when it is executed. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0101] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for processing power data, characterized in that: The method comprises: Obtain power business processing requirements; Retrieving power parameters related to the power business processing requirements from the Kafka message queue; wherein, a consumer group is pre-registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written into the Kafka message queue; Performing power business processing on the power parameters, wherein before writing the power parameters into the Kafka message queue, it also includes: Verifying whether the power parameters are successfully associated with each other according to the power business processing requirements; When the power parameters are successfully associated with each other, executing the step of writing the power parameters into the Kafka message queue; When the association between the power parameters fails, the power parameters are written into the Hbase table, wherein the Hbase table stores power parameters that can be successfully associated with the power business processing requirements but fail to be successfully associated due to delays caused by the set size window.

2. The method according to claim 1, characterized in that After writing the power parameters into the Hbase table, it also includes: according to a set interval threshold, regularly performing high-frequency scanning on the power parameters in the Hbase table at intervals of a set interval threshold to obtain incremental power parameters, and writing the incremental power parameters into the kafka message queue.

3. The method according to claim 2, characterized in that after writing the incremental power parameters into the Kafka message queue, it also includes: Clean up the incremental power parameters written into the Kafka message queue from the Hbase table.

4. The method according to claim 1, characterized in that: The consumer group includes a plurality of subscription topics, and one power parameter corresponds to one subscription topic; The power parameters are written into the Hbase table, including: According to different subscription topics, the power parameters that fail to be associated are written into the corresponding Hbase tables respectively, and one Hbase table stores the power parameters corresponding to one subscription topic.

5. The method according to claim 3, characterized in that: After registering the consumer group with the Kafka message queue in advance through the Flink stream processor, it also includes: Determining the primary key of the Hbase according to the generation time point of the power parameter and the measurement point of the power parameter; Cleaning the incremental power parameters written into the Kafka message queue from the Hbase table includes: When the primary keys in the Hbase table are the same, only the power parameters obtained at the latest generation time point are retained, and the incremental power parameters written into the Kafka message queue from the Hbase table are cleared.

6. A power data processing device, characterized in that: The device comprises: A business processing requirement acquisition module is used to acquire power business processing requirements; The power parameter acquisition module is used to retrieve the power parameters related to the power business processing requirements from the Kafka message queue; wherein, the consumer group is registered with the Kafka message queue through the Flink stream processor, and based on the consumer group, the power parameters generated by the power consumption of the set size window are obtained through the Kafka message queue, and the power parameters are written to the Kafka message queue; An electric power business processing module is used to perform electric power business processing on the electric power parameters, wherein the electric power data processing device also includes an association verification device, and the association verification device is used to verify whether each of the electric power parameters is successfully associated according to the electric power business processing requirements; when each of the electric power parameters is successfully associated, the step of writing the electric power parameters to the Kafka message queue is executed; when the association between the electric power parameters fails, the electric power parameters are written into the Hbase table, wherein the Hbase table stores electric power parameters that can be successfully associated with the electric power business processing requirements but fail to be successfully associated due to delays caused by the set size window.

7. The device according to claim 6, characterized in that The power parameter acquisition module also includes an incremental power parameter acquisition module, which is used to regularly perform high-frequency scanning on the power parameters in the Hbase table at intervals of a set interval threshold according to a set interval threshold, obtain the incremental power parameters, and write the incremental power parameters into the Kafka message queue.

8. The device according to claim 6, characterized in that The power data processing device also includes a power parameter cleaning module, which is used to clean up the incremental power parameters written into the Kafka message queue in the Hbase table after writing the incremental power parameters into the Kafka message queue.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images