Data synchronization method, system, computer equipment and storage medium based on big data

Abstract

The present invention relates to the technical field of data synchronization, and discloses a data synchronization method, system, computer equipment and storage medium based on big data, including: obtaining synchronization data from a main channel and a secondary channel, and sending the synchronization data to a computing layer; performing calculation and analysis on the synchronization data, generating display data and sending it to a storage layer; obtaining a data call instruction, and sending the stored display data to a client for display based on the data call instruction; the present application has the effect of improving the reliability of communication between a power dispatching system and a substation.

Description

Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a data synchronization method, system, computer equipment and storage medium based on big data. Background Art

[0002] At present, power dispatching systems at all levels need to receive network information data such as assets, alarms, events, network security data, etc. from substations at the same level so that managers can know the operating conditions of substations at the same level and better manage substations at the same level.

[0003] The existing power dispatching system obtains the original network information data of the substation at this level through the collection device and transmits it to the power dispatching system through a dedicated communication channel. The dedicated communication channel may be destroyed due to network attacks or other factors.

[0004] With respect to the above-mentioned related technologies, the inventor believes that there is a problem that network information data cannot be transmitted between the power dispatching system and the substation after the dedicated communication channel is destroyed. Summary of the invention

[0005] In order to improve the reliability of communication between the power dispatching system and the substation, the present application provides a data synchronization method, system, computer equipment and storage medium based on big data.

[0006] The above-mentioned invention objective 1 of the present application is achieved by adopting the following technical solution:

[0007] A data synchronization method based on big data, comprising:

[0008] Get the synchronization data from the primary channel and the secondary channel, and send the synchronization data to the computing layer;

[0009] Performing calculation and analysis on the synchronization data, generating display data and sending it to the storage layer;

[0010] A data calling instruction is obtained, and based on the data calling instruction, the stored display data is sent to the client for display.

[0011] By adopting the above technical solution, synchronous data is obtained through the channels of the main channel and the secondary channel, which has higher communication reliability than the traditional transmission method that can only obtain synchronous data from a dedicated communication channel, so that when one of the channels is unavailable, other channels can be used to obtain synchronous data; the obtained synchronous data is sent to the computing layer to perform computational analysis on the synchronous data to generate display data that can be used for client display, and the display data after computational analysis is sent to the storage layer for storage; when a data call instruction is received, the stored display data is sent to the client based on the request of the data call instruction, so that the corresponding display data can be displayed on the client according to the actual needs of relevant personnel.

[0012] In a preferred example, the present application can be further configured as follows: the synchronization data includes master data and slave data, the master data is data obtained from the master channel, and the slave data is data obtained from the slave channel. The step of obtaining the synchronization data from the master channel and the slave channel and sending the synchronization data to the computing layer specifically includes:

[0013] Receive master data and send it to the computing layer;

[0014] If the communication of the main channel is abnormal, the slave data from the secondary channel is received and sent to the computing layer.

[0015] By adopting the above technical solution, the synchronized data includes the master data obtained from the main channel and the slave data obtained from the sub-channel. The master data is preferentially tried to be received. If the master data is successfully received, the master data is sent to the computing layer for subsequent computing and analysis. The master data from the main channel is preferentially tried to be received, so that the main channel can be used for data transmission when the main channel communication is normal, reducing the possible impact of using the sub-channel to transmit data on other power dispatching systems; if the main channel has a communication abnormality, the slave data from the sub-channel is received and sent to the computing layer for subsequent use, so that the sub-channel can be used as a backup communication channel when the main channel communication is abnormal.

[0016] In a preferred example, the present application can be further configured as follows: if the communication of the main channel is abnormal, the step of receiving slave data from the secondary channel specifically includes:

[0017] When the main channel communication is abnormal, the synchronous data collected by the power dispatching system is sent to the front-end module of the cooperative dispatching system;

[0018] Send a data import instruction to the cooperative scheduling system to receive slave data from the cooperative scheduling system.

[0019] By adopting the above technical scheme, when a communication abnormality occurs in the main channel, the synchronous data collected by the collection device of the present electric power dispatching system will be sent to the front-end module of the cooperative dispatching system, so that when a communication abnormality occurs in the main channel, the front-end module of the cooperative dispatching system can receive the synchronous data collected by the present electric power dispatching system; a data call-in instruction is sent to the cooperative dispatching system to receive slave data from the cooperative dispatching system for subsequent use, thereby achieving the effect of using the communication channel of the cooperative dispatching system to receive the synchronous data collected by the present electric power dispatching system, and using the cooperative dispatching system as a secondary channel for obtaining synchronous data, thereby improving the reliability of communication between the electric power dispatching system and the substation while reducing the cost of establishing the secondary channel.

[0020] In a preferred example, the present application may be further configured as follows: the step of performing calculation and analysis on the synchronization data specifically includes:

[0021] Receive and process master data through Kafka, and then perform rule operations on the master data through Spark;

[0022] Receive and process slave data through Logstash.

[0023] By adopting the above technical solution, the received synchronization data is calculated and analyzed according to the master data and the slave data respectively. If the master data is received, the master data is received and processed by the Kafka software, which has the advantage of high throughput. Then, the master data is subjected to rule operations by the Spark software to obtain display data that can be used for subsequent display. If the received data is the slave data, since the slave data is obtained from the cooperative scheduling system and has been processed by the Kafka software and the Spark software in the cooperative scheduling system, it only needs to be simply received and processed by the Logstash software to obtain the display data that can be used for subsequent display.

[0024] In a preferred example, the present application may be further configured as follows: after the step of obtaining a data call instruction and sending the stored display data to the client for display based on the data call instruction, the present application may further include:

[0025] Receive synchronous data collected by the cooperative scheduling system in real time;

[0026] When a data calling instruction is received, the communication module is controlled based on the data calling instruction to send the corresponding display data to the cooperative scheduling system.

[0027] By adopting the above technical solution, the synchronous data collected by the collection device of the cooperative scheduling system from the cooperative scheduling system is received in real time, so as to assist in receiving the synchronous data collected by the collection device of the cooperative scheduling system when the main channel of the cooperative scheduling system is unavailable, and the received synchronous data is used as the main data for calculation and analysis to generate display data and then saved; when a data call-out instruction is received from the cooperative scheduling system, the corresponding display data is sent to the cooperative scheduling system through the communication module based on the data call-out instruction as the slave data of the cooperative scheduling system, thereby achieving the effect of assisting the cooperative scheduling system in receiving and processing the synchronous data that the cooperative scheduling system cannot receive and process.

[0028] In a preferred example, the present application may be further configured as follows: the step of controlling the communication module to send the corresponding display data to the cooperative scheduling system based on the data call-out instruction specifically includes:

[0029] The display data corresponding to the data call-out instruction is obtained, and the display data is received and processed through Logstash, and sent to the cooperative scheduling system through the communication module.

[0030] By adopting the above technical solution, the corresponding display data is obtained from the storage layer according to the data call-out instruction, and after the display data is received and processed by the Logstash software, the display data is sent to the cooperative scheduling system through the communication module, so that the cooperative scheduling system uses the display data as slave data for subsequent use.

[0031] The second object of the invention of this application is achieved by the following technical solution:

[0032] A data synchronization system based on big data, comprising:

[0033] A data acquisition module is used to acquire the synchronous data from the primary channel and the secondary channel, and send the synchronous data to the computing layer;

[0034] A data calculation and analysis module, used to calculate and analyze the synchronized data, generate display data and send it to the storage layer;

[0035] The data display module is used to obtain a data call instruction and send the stored display data to the client for display based on the data call instruction.

[0036] By adopting the above technical solution, synchronous data is obtained through the channels of the main channel and the secondary channel, which has higher communication reliability than the traditional transmission method that can only obtain synchronous data from a dedicated communication channel, so that when one of the channels is unavailable, other channels can be used to obtain synchronous data; the obtained synchronous data is sent to the computing layer to perform computational analysis on the synchronous data to generate display data that can be used for client display, and the display data after computational analysis is sent to the storage layer for storage; when a data call instruction is received, the stored display data is sent to the client based on the request of the data call instruction, so that the corresponding display data can be displayed on the client according to the actual needs of relevant personnel.

[0037] The third object of the invention is achieved by the following technical solution:

[0038] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned data synchronization method based on big data when executing the computer program.

[0039] The fourth invention objective of the present application is achieved by the following technical solution:

[0040] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned data synchronization method based on big data.

[0041] In summary, the present application includes at least one of the following beneficial technical effects:

[0042] 1. Acquiring synchronous data through the main channel and the secondary channel has higher communication reliability than the traditional transmission method that can only obtain synchronous data from a dedicated communication channel, so that when one of the channels is unavailable, other channels can be used to obtain synchronous data.

[0043] 2. When communication abnormality occurs in the main channel, the synchronous data collected by the collection device of this organization is sent to the front-end module of the cooperative dispatching system, so that when communication abnormality occurs in the main channel, the synchronous data can be received with the help of the cooperative dispatching system block; a data transfer instruction is sent to the cooperative dispatching system to receive slave data from the cooperative dispatching system for subsequent use, thereby achieving the effect of using the cooperative dispatching system as a secondary channel for obtaining synchronous data, thereby improving the reliability of communication between the power dispatching system and the substation while reducing the cost of establishing a secondary channel.

[0044] 3. Receive the synchronous data collected from the cooperative scheduling system, so as to assist in receiving the synchronous data collected by the cooperative scheduling system when the main channel of the cooperative scheduling system is unavailable, and calculate and analyze the received synchronous data to generate display data and save it; when a data call-out instruction is received from the cooperative scheduling system, the corresponding display data is sent to the cooperative scheduling system through the communication module based on the data call-out instruction, thereby achieving the effect of assisting the cooperative scheduling system to receive and process the synchronous data of the cooperative scheduling system. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a structural diagram of the power dispatching system in this application;

[0046] Figure 2 is a flow chart of a data synchronization method based on big data in one embodiment of the present application;

[0047] Figure 3 is a flowchart of step S10 in a data synchronization method based on big data in one embodiment of the present application;

[0048] Figure 4 is another flow chart of step S10 in the data synchronization method based on big data in one embodiment of the present application;

[0049] Figure 5 is a flowchart of step S20 in a data synchronization method based on big data in an embodiment of the present application;

[0050] Figure 6 This is another flow chart of a data synchronization method based on big data in one embodiment of the present application;

[0051] Figure 7 is a flowchart of step S50 in a data synchronization method based on big data in an embodiment of the present application;

[0052] Figure 8 This is a principle block diagram of a data synchronization system based on big data in one embodiment of the present application;

[0053] Fig. 9 It is a schematic diagram of a device in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following is combined with Figure 1-9 This application is described in further detail.

[0055] This application provides a power dispatching system, such as Figure 1 As shown, the power dispatching system includes:

[0056] The front module is provided with a communication channel for receiving data from the substation so as to input the synchronous data from the substation into the power dispatching system.

[0057] The computing layer includes several preset algorithms and software, which are used to perform calculations and analysis on the master data and slave data received by the power dispatching system to generate display data, and to perform calculations and analysis on the display data specified based on the data call-out instruction and send it to the cooperative dispatching system through the communication module.

[0058] The storage layer is used to store presentation data or other data.

[0059] An application program interface is used to send data input instructions and receive data output instructions to control the power dispatching system to input or output corresponding data.

[0060] The communication module is used for transmitting data in multiple power dispatching systems, wherein the communication modules of each power dispatching system maintain a long connection.

[0061] The following are also used in conjunction with the power dispatching system:

[0062] Client is used to display the display data through hardware and the software that needs to perform subsequent processing and use of the display data.

[0063] The substation is used to transform current, receive and distribute electrical energy, and collect synchronous data and send it to the front-end module of the power dispatching system.

[0064] In one embodiment, if Figure 2 As shown, the present application discloses a data synchronization method based on big data, which specifically includes the following steps:

[0065] S10: Acquire the synchronization data from the primary channel and the secondary channel, and send the synchronization data to the computing layer.

[0066] In this embodiment, the main channel refers to the communication channel between the power dispatching system and the substation it controls; the secondary channel refers to the communication channel through which the power dispatching system sends the data collected by the substation it controls to other power dispatching systems; the synchronous data refers to the data collected by the substation that needs to be uploaded to the power dispatching system; the computing layer refers to the functional module used to perform computational analysis on the data received by the power dispatching system to generate data that is easy for personnel to understand or use later.

[0067] Specifically, an electric power dispatching system can be used to control one or more substations. A communication channel as the main channel is established between an electric power dispatching system and each substation it controls. Each substation can establish a secondary channel with multiple other electric power dispatching systems, and each electric power dispatching system can also establish secondary channels with multiple substations. The synchronized data includes assets, alarms, events, configurations and other data according to the recorded content.

[0068] Specifically, synchronous data is acquired through the main channel and the sub-channel. Compared with the traditional single-channel synchronous data acquisition mode, data can be acquired through the sub-channel when the main channel communication is abnormal, which has the characteristic of good communication stability.

[0069] Specifically, the establishment of a secondary channel needs to consider factors such as communication quality, communication security and channel establishment cost. For example, establishing a secondary channel by building a signal optical cable has the advantages of strong security and high communication quality, but has the disadvantage of high channel establishment cost. You can choose to build a secondary channel with a substation or other power dispatching system that is geographically close, or choose to cooperate with more substations and other power dispatching systems to improve the utilization rate of signal optical cables and reduce the cost of use. If the existing Internet is used to establish a secondary channel, it has the advantage of low channel establishment cost, but has the disadvantage of low communication security.

[0070] S20: Perform calculation and analysis on the synchronized data, generate display data and send it to the storage layer.

[0071] In this embodiment, the display data refers to data generated by calculation and analysis based on the synchronized data and is easy for people to understand.

[0072] Specifically, the synchronization data is analyzed and calculated, including classification and statistics of the received synchronization data containing various types of data to generate display data; for example, alarm data is filtered out from the synchronization data, and the number and duration of alarm data received in each statistical period are calculated in combination with the time corresponding to the alarm data, so as to obtain information such as the average number of network attacks on the substation or power dispatching system in each statistical period, the average loss per single network attack, and the average duration of network attacks, so that management personnel can provide suggestions for security upgrades of the network system built into the substation or power dispatching system by viewing these display data; the display data is sent to the storage layer to facilitate the retention of the display data so as to subsequently generate historical display data sets for future inspection.

[0073] S30: Obtain a data calling instruction, and send the stored display data to the client for display based on the data calling instruction.

[0074] In this embodiment, the data calling instruction refers to an instruction from the client for requesting to call the display data; the client refers to software for displaying the display data through hardware and for subsequent processing and use of the display data.

[0075] Specifically, the content of the data call instruction includes the index information of the display data that needs to be retrieved, so as to quickly find the required display data corresponding to the data call instruction from the storage layer; the client can be an email address, APP, official account, mini program or web page, etc.; based on the data call instruction, the corresponding display data is sent to the client, so that the display data can be displayed through hardware, providing personnel with observable and tangible page display and services.

[0076] In this embodiment, the synchronous data is obtained through the channels of the main channel and the secondary channel, which has higher communication reliability than the traditional transmission method that can only obtain synchronous data from a dedicated communication channel, so that other channels can be used to obtain synchronous data when one of the channels is unavailable; the obtained synchronous data is sent to the computing layer to perform computational analysis on the synchronous data to generate display data that can be used for client display, and the display data after computational analysis is sent to the storage layer for storage; when a data call instruction is received, the stored display data is sent to the client based on the request of the data call instruction, so that the corresponding display data can be displayed on the client according to the actual needs of relevant personnel.

[0077] In one embodiment, if Figure 3 As shown, in step S10, the synchronization data includes master data and slave data, the master data is data obtained from the main channel, and the slave data is data obtained from the secondary channel, specifically including:

[0078] S11: Receive the main data and send it to the computing layer.

[0079] In this embodiment, the main data refers to the data obtained through the main channel, that is, the synchronous data collected by the collection device installed in the substation under the power dispatching system and transferred to the power dispatching system.

[0080] Specifically, the communication status of the main channel is detected in real time and the main data is received. When the main data collected from the subordinate substation is received, the main data is input into the calculation layer for subsequent calculation and analysis.

[0081] Specifically, when the communication state of the main channel is normal, the sub-channel is closed to suspend receiving slave data from the sub-channel.

[0082] S12: If the communication of the main channel is abnormal, the slave data from the secondary channel is received and sent to the computing layer.

[0083] In this embodiment, slave data refers to data obtained through a secondary channel, that is, data collected by a collection device installed in a substation under this power dispatching system and then transmitted to other power dispatching systems, and then transmitted to this power dispatching system through a communication module.

[0084] Specifically, the communication status of the main channel is detected in real time. When the communication status of the main channel is detected to be abnormal, the secondary channel is opened to receive slave data from the secondary channel in real time; when slave data is received from the cooperative scheduling system, the slave data is input into the computing layer for subsequent computational analysis.

[0085] In one embodiment, if Figure 4 As shown, in step S12, it specifically includes:

[0086] S13: When the main channel communication is abnormal, the synchronous data collected by the power dispatching system is sent to the front-end module of the cooperative dispatching system.

[0087] Specifically, when a communication anomaly of the main channel is detected, the synchronous data collected by the substation under the power dispatching system is input into the front module of the cooperative dispatching system to receive the synchronous data through the communication channel of the cooperative dispatching system.

[0088] Specifically, when the cooperative dispatching system receives synchronization data from the substation under the power dispatching system, it uses the received synchronization data as the main data of the cooperative dispatching system for subsequent processing, generates display data and sends it to the storage layer of the cooperative dispatching system.

[0089] S14: Send a data import instruction to the cooperative scheduling system to receive slave data from the cooperative scheduling system.

[0090] Specifically, a data import instruction is sent to the cooperative dispatching system, wherein the data import instruction includes index information of display data to be imported, so as to control the cooperative dispatching system to send the stored display data to the power dispatching system through the communication module.

[0091] Specifically, the display data received by the power dispatching system from the cooperative dispatching system is used as slave data for subsequent processing.

[0092] In one embodiment, if Figure 5 As shown, in step S20, it specifically includes:

[0093] S21: Receive and process the master data through Kafka, and then perform rule operations on the master data through Spark.

[0094] Specifically, the master data is received and processed through Kafka software, and then the master data is operated according to rules through Spark software to obtain display data that is easy for people to understand; the operation rules should be preset according to actual needs.

[0095] S22: Receives and processes slave data through Logstash.

[0096] Specifically, since the slave data has been calculated and analyzed as the master data of the cooperative scheduling system in the cooperative scheduling system, it only needs to be simply received and processed by the Logstash software to obtain display data that is easy for personnel to understand.

[0097] In one embodiment, if Figure 6 As shown, after step S30, the data synchronization method based on big data further includes:

[0098] S40: receiving the synchronization data collected by the cooperative scheduling system in real time.

[0099] Specifically, this power dispatching system can also serve as a secondary channel of the cooperative dispatching system; it receives the synchronous data collected from the cooperative dispatching system in real time through the front-end module, and sends the received synchronous data as the main data of this power dispatching system to the computing layer for calculation and analysis, and generates display data and sends it to the storage layer.

[0100] S50: When a data calling instruction is received, the communication module is controlled to send corresponding display data to the cooperative scheduling system based on the data calling instruction.

[0101] Specifically, the data call-out instruction includes index information of the display data to be called out. When the data call-out instruction is received from the cooperative scheduling system, the corresponding display data is searched from the storage layer based on the index information of the data call-out instruction, and sent to the cooperative scheduling system corresponding to the issuing data call-out instruction through the communication module.

[0102] In one embodiment, if Figure 7 As shown, in step S50, it specifically includes:

[0103] S51: Obtain the display data corresponding to the data call-out instruction, receive and process the display data through Logstash, and send it to the cooperative scheduling system through the communication module.

[0104] Specifically, the corresponding display data is searched from the storage layer according to the index information of the data call-out instruction, and the display data is input into the computing layer. The display data is received and processed by the Logstash software, and the processed display data is sent to the cooperative scheduling system through the communication module as the slave data of the cooperative scheduling system for subsequent use.

[0105] It should be understood that the serial numbers of the steps in the above embodiments do not imply a sequence of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0106] In one embodiment, a data synchronization system based on big data is provided, and the data synchronization system based on big data corresponds one-to-one to the data synchronization method based on big data in the above-mentioned embodiment.

[0107] like Figure 8 As shown, a data synchronization system based on big data includes a data acquisition module, a data calculation and analysis module, and a data display module. The detailed description of each functional module is as follows:

[0108] A data acquisition module is used to acquire the synchronous data from the primary channel and the secondary channel, and send the synchronous data to the computing layer;

[0109] The data calculation and analysis module is used to calculate and analyze the synchronized data, generate display data and send it to the storage layer;

[0110] The data display module is used to obtain a data call instruction and send the stored display data to the client for display based on the data call instruction.

[0111] Optionally, the data calculation and analysis module includes:

[0112] The master data calculation and analysis submodule is used to receive and process master data through Kafka, and then perform rule operations on the master data through Spark;

[0113] The slave data calculation and analysis submodule is used to receive and process slave data through Logstash.

[0114] For the specific limitations of the data synchronization system based on big data, please refer to the limitations of the data synchronization method based on big data in the above text, which will not be repeated here; each module in the above-mentioned data synchronization system based on big data can be implemented in whole or in part through software, hardware and their combination; each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the corresponding operations of each of the above modules.

[0115] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as master data, slave data, display data, data call instructions, data call-in instructions and data call-out instructions. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data synchronization method based on big data is implemented.

[0116] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0117] S10: Obtain synchronization data from the primary channel and the secondary channel, and send the synchronization data to the computing layer;

[0118] S20: Calculate and analyze the synchronized data, generate display data, and send it to the storage layer;

[0119] S30: Obtain a data calling instruction, and send the stored display data to the client for display based on the data calling instruction.

[0120] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0121] S10: Obtain synchronization data from the primary channel and the secondary channel, and send the synchronization data to the computing layer;

[0122] S20: Calculate and analyze the synchronized data, generate display data, and send it to the storage layer;

[0123] S30: Obtain a data calling instruction, and send the stored display data to the client for display based on the data calling instruction.

[0124] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0125] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0126] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the features thereof may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A data synchronization method based on big data, characterized in that: The data synchronization method based on big data is implemented by the power dispatching system and the cooperative dispatching system; The power dispatching system controls one or more substations; a communication channel is established between the power dispatching system and each substation it controls as a main channel; a secondary channel is established between each substation and multiple cooperative dispatching systems for sending data collected by the substation to the cooperative dispatching system; the cooperative dispatching system is another power dispatching system that establishes a communication connection relationship with the substation of the power dispatching system; The electric power dispatching system and the cooperative dispatching system both include a communication module, and the communication module is used to transmit data between the electric power dispatching system and the cooperative dispatching system, wherein a long connection is maintained between the communication module of the electric power dispatching system and the communication module of the cooperative dispatching system; The steps of the data synchronization method based on big data include: Get the synchronization data from the primary channel and the secondary channel, and send the synchronization data to the computing layer; Performing calculation and analysis on the synchronization data, generating display data and sending it to the storage layer; Obtaining a data call instruction, and sending the stored display data to the client for display based on the data call instruction; After the step of obtaining the data calling instruction and sending the stored display data to the client for display based on the data calling instruction, the method further includes: Receive synchronous data collected by the cooperative scheduling system in real time; When receiving a data call-out instruction, the communication module is controlled to send corresponding display data to the cooperative scheduling system based on the data call-out instruction; The synchronization data includes at least one of assets, alarms, events, and network security data of the substation at this level; The synchronization data includes master data and slave data, the master data is data obtained from the master channel, and the slave data is data obtained from the slave channel. The step of obtaining the synchronization data from the master channel and the slave channel and sending the synchronization data to the computing layer specifically includes: Detect the communication status of the main channel in real time and receive the main data, and send the main data to the computing layer; If the communication status of the main channel is normal, close the secondary channel and stop receiving slave data from the secondary channel; If the communication of the main channel is abnormal, the secondary channel is opened to receive slave data from the secondary channel in real time, and the slave data is sent to the computing layer; the slave data is synchronous data transmitted to the cooperative dispatching system by the power dispatching system; if the communication of the main channel is abnormal, the step of receiving slave data from the secondary channel specifically includes: When the main channel communication is abnormal, the synchronous data collected by the power dispatching system is sent to the front-end module of the cooperative dispatching system; When the cooperative dispatching system receives the synchronization data from the power dispatching system, it processes the received synchronization data, generates display data and sends it to the storage layer of the cooperative dispatching system; Send a data import instruction to the cooperative scheduling system to receive slave data from the cooperative scheduling system.

2. The data synchronization method based on big data according to claim 1, characterized in that: The step of calculating and analyzing the synchronization data specifically includes: Receive and process master data through Kafka, and then perform rule operations on the master data through Spark; Receive and process slave data through Logstash.

3. The data synchronization method based on big data according to claim 1, characterized in that: The step of controlling the communication module to send the corresponding display data to the cooperative scheduling system based on the data call-out instruction specifically includes: The display data corresponding to the data call-out instruction is obtained, and the display data is received and processed through Logstash, and sent to the cooperative scheduling system through the communication module.

4. A data synchronization system based on big data, characterized by: The data synchronization system based on big data includes a power dispatching system and a cooperative dispatching system; The power dispatching system controls one or more substations; a communication channel is established between the power dispatching system and each substation it controls as a main channel; a secondary channel is established between each substation and multiple cooperative dispatching systems for sending data collected by the substation to the cooperative dispatching system; the cooperative dispatching system is another power dispatching system that establishes a communication connection relationship with the substation of the power dispatching system; The electric power dispatching system and the cooperative dispatching system both include a communication module, and the communication module is used to transmit data between the electric power dispatching system and the cooperative dispatching system, wherein a long connection is maintained between the communication module of the electric power dispatching system and the communication module of the cooperative dispatching system; The power dispatching system and the cooperative dispatching system both further include a data acquisition module for acquiring synchronization data from the primary channel and the secondary channel, and sending the synchronization data to the computing layer; A data calculation and analysis module, used to calculate and analyze the synchronized data, generate display data and send it to the storage layer; A data display module is used to obtain a data call instruction and send the stored display data to the client for display based on the data call instruction; After the step of obtaining the data calling instruction and sending the stored display data to the client for display based on the data calling instruction, the method further includes: Receive synchronous data collected by the cooperative scheduling system in real time; When receiving a data call-out instruction, the communication module is controlled based on the data call-out instruction to send the corresponding display data to the cooperative scheduling system; The synchronization data includes at least one of assets, alarms, events, and network security data of the substation at this level; The data acquisition module acquires the synchronous data from the primary channel and the secondary channel and sends the synchronous data to the computing layer, and is used to: Detect the communication status of the main channel in real time and receive the main data, and send the main data to the computing layer; If the communication status of the main channel is normal, close the secondary channel and stop receiving slave data from the secondary channel; If the communication of the main channel is abnormal, the secondary channel is opened to receive slave data from the secondary channel in real time, and the slave data is sent to the computing layer; the slave data is synchronous data transmitted from the power dispatching system to the cooperative dispatching system; When the data acquisition module receives slave data from the secondary channel due to abnormal communication in the primary channel, it is used to: When the main channel communication is abnormal, the synchronous data collected by the power dispatching system is sent to the front-end module of the cooperative dispatching system; When the cooperative dispatching system receives the synchronization data from the power dispatching system, it processes the received synchronization data, generates display data and sends it to the storage layer of the cooperative dispatching system; Send a data import instruction to the cooperative scheduling system to receive slave data from the cooperative scheduling system.

5. A data synchronization system based on big data according to claim 4, characterized in that: The data calculation and analysis module includes: The master data calculation and analysis submodule is used to receive and process master data through Kafka, and then perform rule operations on the master data through Spark; The slave data calculation and analysis submodule is used to receive and process slave data through Logstash.

6. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the data synchronization method based on big data as described in any one of claims 1 to 3 are implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the data synchronization method based on big data as described in any one of claims 1 to 3 are implemented.

Images