Distributed processing and real-time pushing method and system for data of intelligent equipment of power grid

Abstract

The embodiment of the invention provides a power grid intelligent equipment data distributed processing and real-time pushing method and system, and belongs to the technical field of power grid data processing. The method comprises the following steps: collecting electric power data, and transmitting the data to an Internet of Things gateway; the Internet of Things gateway directly stores the collected data into the message queue without processing based on a publishing attribute mode of the message queue; persistently storing the relationship between the collected data and the gateway register point location in a relational database, and pre-loading associated data in the relational database into a cache database Redis; associating the obtained data with the mapping relation in the Redis cache, and packaging a unified update message structure; according to the method, updating messages are pushed to different themes, and the system pushing pressure is reasonably distributed to all pushing services by appointing the themes subscribed by different pushing services. According to the method, the operation data of the power system can be timely and effectively acquired, and real-time interaction and updating with the client can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of power grid data processing, and in particular to a method and system for distributed processing and real-time push of power grid intelligent device data. Background Art

[0002] At present, the power grid is facing unprecedented challenges and profound changes. With the continuous growth of global energy demand and the large-scale grid connection of renewable energy, the scale of the power system has expanded dramatically, posing a severe test to the traditional operation and maintenance management model. In order to cope with this complex situation and promote the transformation of the power system to modernization, building a new power system has become an industry consensus. In this process, the introduction of a large number of intelligent sensing devices has not only enhanced the intelligence level of the power system, but also greatly enriched the data source, but also put forward new and higher requirements for the efficient collection, accurate processing and in-depth application of data. At present, the system status monitoring of the power system is mainly based on static charts and query data records. However, this method has some problems and limitations. For example, their data push lacks real-time performance and cannot quickly adapt to the complex changes in the power system, resulting in delayed monitoring and decision-making of the system status, which limits the optimization control of the system operation. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a method and system for distributed processing and real-time push of data of power grid smart devices, which can timely and effectively obtain the operation data of the power system, and interact and update with the client in real time.

[0004] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides a method for distributed processing and real-time push of data of power grid smart devices, the method comprising:

[0005] Collecting power data and transmitting the data to an IoT gateway;

[0006] The IoT gateway uses the publishing mode based on the message queue to store the collected data directly into the message queue without processing;

[0007] The relationship between the collected data and the gateway register points is persistently stored in a relational database, and the associated data in the relational database is pre-loaded into the cache database Redis;

[0008] Associate the acquired data with the mapping relationship in the Redis cache and encapsulate the unified update message structure;

[0009] Push update messages to different topics. By specifying topics that different push services subscribe to, the system push pressure is reasonably distributed to all push services. The topic content includes UpdateMsg and region id.

[0010] The client establishes communication with the push server to which the message content to be subscribed belongs through gateway forwarding, and the push server pushes the acquired update message to the client that has established the connection.

[0011] Optionally, collecting power data and transmitting the data to an IoT gateway device includes:

[0012] Acquire data from power equipment and sensors in substations and transmission lines, where the data includes operating status, power, temperature and pressure;

[0013] Transmit the collected data to the data center;

[0014] Distributed microservices build data collection clusters and adjust the load of all collection service nodes through the gateway;

[0015] The collection service completes the data format conversion according to the transmission protocol supported by the gateway and verifies the data during the collection and transmission process.

[0016] Optionally, based on the publishing mode of the message queue, the collected data is directly stored in the message queue without processing, including:

[0017] The data after data format conversion is stored in the message queue with the RcvData topic as the keyword and as the data content;

[0018] The data processing service sends a message subscription about the topic RcvData to the message queue.

[0019] Optionally, persistently storing the relationship between the collected data and the gateway register point in a relational database, and pre-loading the associated data in the relational database into a cache database Redis includes:

[0020] Configure the association information between different sensor information and gateways, and store them persistently in a relational database;

[0021] Convert and store the associated data between sensors and gateways into an in-memory database to support high-concurrency, low-latency requests for the system;

[0022] The register point number is mapped with the converted associated information. The mapping relationship between the register point number and the associated information is expressed as: X→Y j , X is the current register point number, Y i Indicates the unit, accuracy, collection element id, sensor node id, gateway id, and region id of the current register point mapping;

[0023] The continuous associated information of the mapped register points is stored in batches to the cache, and the fob information of the points with continuous register point numbers is stored in the cache in the form of key-value pairs, with the gateway IP and the transmission transaction ID as keywords.

[0024] Optionally, the acquired data is associated with the mapping relationship in the Redis cache, and the encapsulated unified update message structure includes:

[0025] Get the converted data from the message queue, and get the unit, accuracy, collection element id, sensor node id, gateway id, and region id in batches from the cache according to the keyword of the message;

[0026] After acquiring the perception data and associated information, the collected data is stored in a relational database and pushed to the data warehouse at regular intervals;

[0027] Compare the current collected element value with the historical value in the database to determine whether the difference between the current collected element value and the historical value in the database is 0;

[0028] When the difference is not 0, determine whether the difference exceeds the alarm range;

[0029] When the difference exceeds the alarm range, determine whether the current node status is disconnected;

[0030] When the current node state is in a disconnected state, an alarm message is generated, and the format of the alarm message is: 1, sensor node id, collection element id, message content;

[0031] When the difference does not exceed the alarm range, format information is generated, and the format of the format information is: 0, sensor node id, collection element id, message content.

[0032] Optionally, the acquired data is associated with the mapping relationship in the Redis cache, and the encapsulated unified update message structure includes:

[0033] If the difference is 0, no update is performed.

[0034] Optionally, update messages are pushed to different topics. By specifying topics to which different push services subscribe, the system push pressure is reasonably distributed to all push services. The topic content includes UpdateMsg and the region id includes:

[0035] The push service node load is adjusted by adjusting the subscription topic of the push service on the message queue. The push message uses UpdateMsg and the region id as the publishing topic.

[0036] Assign push services to update messages in a many-to-many fashion.

[0037] Optionally, the collection service completes the data format conversion according to the transmission protocol supported by the gateway, including:

[0038] Get data of sensor information;

[0039] Calculate the register value according to formula (1):

[0040] j=αx+d

[0041] Among them, j represents the register value, α represents the conversion coefficient, x represents the data of the sensor information, and d represents the offset.

[0042] On the other hand, the present invention also provides a distributed processing and real-time push system for power grid smart device data, the system comprising:

[0043] Multiple collection gateways for acquiring power data;

[0044] A data acquisition module, used for receiving the power data transmitted by the acquisition gateway;

[0045] A message queue cluster module, used for receiving the power data from the data acquisition module and arranging the data in a queue form;

[0046] A data processing module, used for receiving power data in the message queue cluster module of a specified task, mapping the power data and encapsulating it into a unified structure;

[0047] Data push module, which pushes update messages to different services based on different topics;

[0048] Multiple clients receive update messages from different services based on topics.

[0049] Through the above technical scheme, the invention provides a method and system for distributed processing and real-time push of data of smart power grid devices, which collects power data and transmits the data to the Internet of Things gateway. The Internet of Things gateway can directly store the collected data in the message queue without processing based on the publishing attributive mode of the message queue. The relationship between the collected data and the gateway register point can be persistently stored in the relational database, and the associated data in the relational database can be pre-loaded into the cache database Redis for easy calling at any time. The acquired data is associated with the mapping relationship in the Redis cache, and can be encapsulated into a unified update message structure. The update message can be pushed to different topics respectively, and then the system push pressure can be reasonably distributed to all push services by specifying the topics subscribed by different push services. The topic content can include UpdateMsg and regional id. The client can establish communication with the push server to which the message content to be subscribed belongs through gateway forwarding, and the push server can push the acquired update message to the client that builds the connection, thereby realizing interaction with the client. The method can obtain the operation data of the power system in a timely and effective manner, and interact and update with the client in real time.

[0050] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0052] Figure 1 is a flow chart of a method for distributed processing and real-time push of data of smart devices in power grids according to an embodiment of the present invention;

[0053] Figure 2 It is a flow chart of transmitting data to an Internet of Things gateway device in a distributed processing and real-time push method of power grid smart device data according to an embodiment of the present invention;

[0054] Figure 3 It is a flowchart of storing data into a message queue according to a method for distributed processing and real-time push of data of smart devices in a power grid according to an embodiment of the present invention;

[0055] Figure 4 It is a flowchart of storing data in a relational database according to a method for distributed processing and real-time push of data of smart devices in a power grid according to an embodiment of the present invention;

[0056] Figure 5It is a flow chart of a unified update message structure encapsulated in a method for distributed processing and real-time push of data of smart devices in a power grid according to an embodiment of the present invention;

[0057] Figure 6 It is a flowchart of a method for distributed processing and real-time push of data of smart devices in power grids according to an embodiment of the present invention, in which update messages are pushed to different topics;

[0058] Figure 7 It is a flow chart of completing data format conversion of a method for distributed processing and real-time push of data of smart devices in power grid according to an embodiment of the present invention;

[0059] Figure 8 It is a connection diagram of a system for distributed processing and real-time push of data of smart devices in a power grid according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0061] Figure 1 The flowchart of a method for distributed processing and real-time push of data of a power grid smart device according to an embodiment of the present invention. In the present invention, the push process may include:

[0062] In step S1, power data is collected and transmitted to the Internet of Things gateway.

[0063] In step S2, the IoT gateway stores the collected data directly into the message queue without processing based on the publishing mode of the message queue.

[0064] In step S3, the relationship between the collected data and the gateway register points is persistently stored in a relational database, and the associated data in the relational database is pre-loaded into the cache database Redis.

[0065] In step S4, the acquired data is associated with the mapping relationship in the Redis cache, and a unified update message structure is encapsulated.

[0066] In step S5, the update message is pushed to different topics respectively. By specifying the topics subscribed by different push services, the system push pressure is reasonably distributed to all push services. The topic content includes UpdateMsg and region id.

[0067] In step S6, the client establishes communication with the push server to which the message content to be subscribed belongs through gateway forwarding, and the push server pushes the acquired update message to the client to which the connection is established.

[0068] In the present invention, when pushing messages, power data can be collected by intelligent power monitoring equipment, and then the data can be transmitted to the Internet of Things gateway. The data collection service cluster can collect the data transmission in the gateway to the local through the network. In order to avoid the collection service collecting data and then processing the data, a large amount of processing data causes the processor to be highly occupied and unable to process the remaining gateway data transmission, the Internet of Things gateway can store the collected data directly in the message queue without processing based on the publish-subscribe mode of the message queue. The data processing service subscribes to data from the specified topic. In order to improve the efficiency of data promotion, the gateway only transmits necessary data, such as: voltage, current and other values, and stores the relationship between the gateway register point and the collected data of the sensor in a persistent manner in a relational database. The mapping relationship between the gateway register point and the sensor collected data will not change frequently. In order to improve the speed of querying the associated data during data mapping, the associated data in the database is pre-loaded into the cache database Redis. The data processing service can associate the acquired data with the mapping relationship in the Redis cache, encapsulate it into a unified update message structure, so that the client can parse it uniformly. Considering that after accessing a large number of sensing devices, the push service needs to push update messages to all subscribed clients, and the push service is under great pressure. Therefore, through the message queue release, the update messages of different sensing devices are processed separately through multiple push services. The update messages are pushed to different topics respectively, and by specifying the topics subscribed by different push services, the system push pressure is reasonably distributed to all push services. Among them, the topic content includes UpdateMsg and regional id. In order to push data and status information to the client in real time, WebSocket technology can be used to build a communication channel between the push service and the client. The client establishes communication with the push server to which the message content to be subscribed belongs through gateway forwarding. When the push service obtains a new message, it pushes it to the connected client. This method can obtain the operating data of the power system in a timely and effective manner, and interact and update with the client in real time.

[0069] In one embodiment of the present invention, Figure 2 As shown, the process of transmitting data to an IoT gateway device may include:

[0070] In step S7, data of power equipment and sensors in the substation and transmission lines are obtained, wherein the data includes operating status, power, temperature and pressure.

[0071] In step S8, the collected data is transmitted to the data center.

[0072] In step S9, the distributed microservices build a data collection cluster and adjust the load of all collection service nodes through the gateway.

[0073] In step S10, the collection service completes the data format conversion according to the transmission protocol supported by the gateway, and verifies the data during the collection and transmission process.

[0074] In the present invention, when collecting data, data such as various power equipment, sensors, etc. in scenes such as substations and transmission lines can be collected. These data include indicators such as the operating status, power, temperature, and pressure of the power equipment. After the data is collected, data transmission can be performed through communication technologies such as optical fiber and WIFI, and the collected data can be transmitted to the data center. In the Internet of Things system, it is often necessary to face massive data. In order to solve the task of completing data collection within the specified time, a data collection cluster can be built through distributed microservices, and the load of all collection service nodes can be adjusted through the gateway. The collection service needs to complete the data format conversion according to the transmission protocol supported by the gateway. Since the device obtains data directly from the gateway, the data length of each communication transmission must be consistent with the mapping relationship stored in the server and the number of collection elements. Since the data in the data gateway cannot be transmitted once by a single data packet in each collection batch, it is necessary to verify whether the transmission transaction id of each communication is legal, that is, the data can be verified during the collection process.

[0075] In one embodiment of the present invention, Figure 3 As shown, the process of storing data in a message queue may include:

[0076] In step S11, the data after the data format conversion is completed is stored in the message queue as data content using the RcvData subject as a keyword.

[0077] In step S12, the data processing service sends a message subscription about the topic RcvData to the message queue.

[0078] In the present invention, in order to improve the overall security of the system, the message queue parameters can be tuned to ensure zero loss of collected data. After the collection service completes the conversion of the collected data, the data can be keyworded with the RcvData topic, and the converted data can be stored in the message queue as data content. By building a data processing service cluster, the data processing speed is improved. The data processing service can send a message subscription about the topic RcvData to the message queue. All data processing services are in the same consumer group, thereby avoiding repeated consumption of data.

[0079] In one embodiment of the present invention, Figure 4 As shown, the process of storing data in a relational database may include:

[0080] In step S13, the association information between different sensor information and the gateway is configured and persistently stored in a relational database.

[0081] In step S14, the associated data between the sensor and the gateway is converted and stored in the memory database to support high concurrency and low latency requests of the system.

[0082] In step S15, the register point number is mapped with the converted associated information. The mapping relationship between the register point number and the associated information is expressed as: X→Y j , X is the current register point number, Y i Indicates the unit, accuracy, collection element id, sensor node id, gateway id, and region id of the current register point mapping.

[0083] In step S16, the continuous associated information of the mapped register points is stored in batches to the cache, and the fob information of the points with continuous register point numbers is stored in the cache in the form of key-value pairs, with the gateway IP and the transmission transaction ID as keywords.

[0084] In the present invention, when sensor data is connected to the data gateway, the gateway often only transmits the collected perception data, and the server needs certain association information to match the perception data with the sensor. Therefore, the user can configure the association information of different sensor information and the gateway through the client, and then store it persistently in a relational database, such as Oracle and MySQL. These associated data usually do not change and are used frequently. Therefore, when the service is initialized, the associated data between the sensor and the gateway can be converted and stored in the memory database, so as to support the high concurrency and low latency requests of the system. After the client modifies the association information between the sensor and the gateway, the corresponding information in the cache database should be modified synchronously to avoid abnormalities in the data parsing process. After association, the register point number can be mapped to the converted association information. The mapping relationship between the register point number and the association information can be expressed as: X→Y j , X is the current register point number, Y i Indicates the unit, accuracy, acquisition element id, sensor node id, gateway id, and region id of the current register point mapping. Since each collected data consists of different acquisition elements of multiple sensors, the register points of these collected data in the gateway are usually continuous. In order to save storage space and reduce the usage of keywords in the cache database, the continuous association information of the register points can be stored in batches to the cache. The cache stores the association information of the points with continuous register point numbers in the form of key-value pairs, with the gateway IP and the transmission transaction id as keywords.

[0085] In one embodiment of the present invention, Figure 5 As shown, the process of encapsulating the unified update message structure may include:

[0086] In step S17, the converted data is obtained from the message queue, and the unit, accuracy, collection element id, sensor node id, gateway id, and region id are obtained in batches from the cache according to the keyword of the message.

[0087] In step S18, after the perception data and associated information are acquired, the collected data is stored in a relational database and pushed to a data warehouse at regular intervals.

[0088] In step S19, the current acquisition element value is compared with the historical value in the database to determine whether the difference between the current acquisition element value and the historical value in the database is zero.

[0089] In step S20, when the difference is not 0, it is determined whether the difference exceeds the warning range.

[0090] In step 21, when the difference exceeds the alarm range, it is determined whether the current node state is in a disconnected state.

[0091] In step S22, when the current node state is in a disconnected state, an alarm message is generated, and the format of the alarm message is: 1, sensor node id, collection element id, message content.

[0092] In step S23, when the difference does not exceed the alarm range, format information is generated, and the format of the format information is: 0, sensor node id, collection element id, message content.

[0093] In the present invention, the converted data can be obtained from the message queue, and the unit, accuracy, acquisition element id, sensor node id, gateway id, and region id can be obtained in batches from the cache according to the keyword of the message. After the perception data and related information are obtained, the collected data can be stored in a relational database and can be pushed to the data warehouse at a regular time, so as to avoid data loss. After the data is mapped, the generated numerical data and alarm information need to be encapsulated into a unified format (message type, sensor node id, acquisition element id, message content). Compare the current acquisition element value with the historical value in the database to determine whether the difference between the current acquisition element value and the historical value in the database is 0. If the difference is not 0, that is, the value changes, it can be determined whether the difference exceeds the alarm range. If it does not exceed, the data can be encapsulated into an alarm message of (0, sensor node id, acquisition element id, message content). If it exceeds, the current node state can be further determined to determine whether the current node state is in a disconnected state. If it is in a disconnected state, an additional alarm message (1, sensor node id, acquisition element id, message content) can be generated. Otherwise, it can be returned directly. When a sensor node fails, it will be disconnected. In order to detect whether the device is disconnected, the data collection service will periodically detect the collection flag of the sensor to see whether it has uploaded data within a specific collection cycle. If the flag is 1, the sensor is uploading data normally within the current collection cycle. Otherwise, it is disconnected and sends a status update message (2, sensor node id, collection element id, message content).

[0094] In one embodiment of the present invention, Figure 5 As shown, the process of encapsulating the unified update message structure may include:

[0095] In step S24, when the difference is 0, no updating is performed.

[0096] In one embodiment of the present invention, Figure 6 As shown, the process of pushing update messages to different topics may include:

[0097] In step S25, the push service node load is adjusted by adjusting the subscription topic of the push service on the message queue, and the push message uses UpdateMsg and the region id as the publishing topic.

[0098] In step S26, the push service and the update message are distributed in a many-to-many manner.

[0099] In the present invention, update messages need to be pushed to the client in real time. Considering that a single service is difficult to bear the update load of all perception nodes, the push service node load can be adjusted by adjusting the subscription topic of the push service on the message queue. Client real-time requests are usually in units of regions, so push messages are published with (UpdateMsg, region id) as the publishing topic. The push service can achieve the purpose of adjusting the load by allocating update information subscribed to different topics. The push service and the update message topic are allocated in a many-to-many form. One service can consume multiple update message topics, and one message topic can also be consumed by multiple services. All services need to be set in different consumer groups, so as to avoid the defect of incomplete update messages obtained by a single service. After the push service starts or modifies the consumer subscription configuration, the configuration information is stored in the cache database Redis, which is convenient for the client to establish a WebSocket connection, and the gateway obtains the push service information from Redis for forwarding.

[0100] In one embodiment of the present invention, Figure 7 As shown, the process of completing data format conversion may include:

[0101] In step S27, data of sensor information is acquired.

[0102] In step S28, the register value is calculated according to formula (1):

[0103] j=αx+d

[0104] Among them, j represents the register value, α represents the conversion coefficient, x represents the data of the sensor information, and d represents the offset.

[0105] In the present invention, when completing the data format conversion according to the transmission protocol supported by the gateway, the data of the sensor information can be obtained first, and then the register value can be calculated according to formula (1), so as to complete the data format conversion.

[0106] On the other hand, the present invention also provides a distributed processing and real-time push system for power grid smart device data, such as Figure 8As shown, the system may include a collection gateway, a data collection module, a message queue cluster module, a data processing module, a data push module and a client. The number of collection gateways can be multiple, so as to obtain power data. The data collection module can be used to receive the power data transmitted by the collection gateway, and the message queue cluster module can be used to receive the power data of the data collection module, and can be arranged in the form of a queue. The data processing module can be used to receive the power data in the message queue cluster module of the specified task, and the power data can be mapped and encapsulated into a unified structure. The data push module can push update messages to different services according to different topics. The number of the client can be multiple, and update messages of different services can be received according to the topic.

[0107] Through the above technical scheme, the invention provides a method and system for distributed processing and real-time push of data of smart power grid devices, which collects power data and transmits the data to the Internet of Things gateway. The Internet of Things gateway can directly store the collected data in the message queue without processing based on the publishing attributive mode of the message queue. The relationship between the collected data and the gateway register point can be persistently stored in the relational database, and the associated data in the relational database can be pre-loaded into the cache database Redis for easy calling at any time. The acquired data is associated with the mapping relationship in the Redis cache, and can be encapsulated into a unified update message structure. The update message can be pushed to different topics respectively, and then the system push pressure can be reasonably distributed to all push services by specifying the topics subscribed by different push services. The topic content can include UpdateMsg and regional id. The client can establish communication with the push server to which the message content to be subscribed belongs through gateway forwarding, and the push server can push the acquired update message to the client that builds the connection, thereby realizing interaction with the client. The method can obtain the operation data of the power system in a timely and effective manner, and interact and update with the client in real time.

[0108] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0112] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0113] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0114] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0115] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0116] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for distributed processing and real-time push of data of power grid smart devices, characterized in that: The method comprises: Collecting power data and transmitting the data to an IoT gateway; The IoT gateway uses the publishing mode based on the message queue to store the collected data directly into the message queue without processing; The relationship between the collected data and the gateway register points is persistently stored in a relational database, and the associated data in the relational database is pre-loaded into the cache database Redis; Associate the acquired data with the mapping relationship in the Redis cache and encapsulate the unified update message structure; Push update messages to different topics. By specifying topics that different push services subscribe to, the system push pressure is reasonably distributed to all push services. The topic content includes UpdateMsg and region id. The client establishes communication with the push server to which the message content to be subscribed belongs through gateway forwarding, and the push server pushes the acquired update message to the client that has established the connection.

2. The method according to claim 1, characterized in that Collecting power data and transmitting the data to IoT gateway devices includes: Acquire data from power equipment and sensors in substations and transmission lines, where the data includes operating status, power, temperature and pressure; Transmit the collected data to the data center; Distributed microservices build data collection clusters and adjust the load of all collection service nodes through the gateway; The collection service completes the data format conversion according to the transmission protocol supported by the gateway and verifies the data during the collection and transmission process.

3. The method according to claim 2, characterized in that Based on the publishing mode of the message queue, the collected data is directly stored in the message queue without processing, including: The data after data format conversion is stored in the message queue with the RcvData topic as the keyword and as the data content; The data processing service sends a message subscription about the topic RcvData to the message queue.

4. The method according to claim 2, characterized in that: Persistently storing the relationship between the collected data and the gateway register point in a relational database, and pre-loading the associated data in the relational database into the cache database Redis includes: Configure the association information between different sensor information and gateways, and store them persistently in a relational database; Convert and store the associated data between sensors and gateways into an in-memory database to support high-concurrency, low-latency requests for the system; The register point number is mapped with the converted associated information. The mapping relationship between the register point number and the associated information is expressed as: X→Y j , X is the current register point number, Y i Indicates the unit, accuracy, collection element id, sensor node id, gateway id, and region id of the current register point mapping; The continuous associated information of the mapped register points is stored in batches to the cache, and the fob information of the points with continuous register point numbers is stored in the cache in the form of key-value pairs, with the gateway IP and the transmission transaction ID as keywords.

5. The method according to claim 4, characterized in that Associate the acquired data with the mapping relationship in the Redis cache, and encapsulate the unified update message structure including: Get the converted data from the message queue, and get the unit, accuracy, collection element id, sensor node id, gateway id, and region id in batches from the cache according to the keyword of the message; After acquiring the perception data and associated information, the collected data is stored in a relational database and pushed to the data warehouse at regular intervals; Compare the current collected element value with the historical value in the database to determine whether the difference between the current collected element value and the historical value in the database is 0; When the difference is not 0, determine whether the difference exceeds the alarm range; When the difference exceeds the alarm range, determine whether the current node status is disconnected; When the current node state is in a disconnected state, an alarm message is generated, and the format of the alarm message is: 1, sensor node id, collection element id, message content; When the difference does not exceed the alarm range, format information is generated, and the format of the format information is: 0, sensor node id, collection element id, message content.

6. The method according to claim 5, characterized in that Associate the acquired data with the mapping relationship in the Redis cache, and encapsulate the unified update message structure including: If the difference is 0, no update is performed.

7. The method according to claim 5, characterized in that Push update messages to different topics. By specifying topics to which different push services subscribe, the system push pressure is reasonably distributed to all push services. The topic content includes UpdateMsg and the region id includes: The push service node load is adjusted by adjusting the subscription topic of the push service on the message queue. The push message uses UpdateMsg and the region id as the publishing topic. Assign push services to update messages in a many-to-many fashion.

8. The method according to claim 4, characterized in that The acquisition service completes the data format conversion according to the transmission protocol supported by the gateway, including: Get data of sensor information; Calculate the register value according to formula (1): j=αx+d Among them, j represents the register value, α represents the conversion coefficient, x represents the data of the sensor information, and d represents the offset.

9. A distributed processing and real-time push system for power grid smart device data, characterized in that: The system comprises: Multiple collection gateways for acquiring power data; A data acquisition module, used for receiving the power data transmitted by the acquisition gateway; A message queue cluster module, used for receiving the power data from the data acquisition module and arranging the data in a queue form; A data processing module, used for receiving power data in the message queue cluster module of a specified task, mapping the power data and encapsulating it into a unified structure; Data push module, which pushes update messages to different services based on different topics; Multiple clients receive update messages from different services based on topics.

Images