A High-Speed Data Interaction System and Method within a Container

Abstract

The present invention discloses a high-speed data interaction system and method within a container, which solves the problems in the prior art that the data interaction system and method rely too much on the cloud or real-time library, restricting the data interaction speed and having insufficient dynamic expansion ability. It includes a data processing service subsystem located within the container, which accesses the high-speed real-time data caching service subsystem to read and write and schedule power grid measurement cached data; a high-speed real-time data caching service subsystem located within the container, which realizes high-speed real-time data caching and performs time stamp verification and source verification on the cached data; a real-time data snapshot service subsystem that stores, rolls back, and views snapshot data; and a real-time data synchronization service subsystem located in the host machine, which writes the received real-time data into the real-time library. The present invention reduces the access to the real-time library and improves the data interaction speed and dynamic expansion ability.

Description

Technical Field

[0001] The present invention relates to the technical field of power dispatching automation, and specifically provides a system and method for high-speed data interaction within a container. Background Art

[0002] With the continuous growth of power load, new elements such as adjustable load, electric vehicles, distributed photovoltaics, new energy storage, microgrids, and resource aggregators have entered the stage of large-scale development. The data types and access methods show a trend of heterogeneous and complex development, and the data scale has increased exponentially.

[0003] Existing data interaction systems and methods store data feature information in multiple databases, relying too much on the cloud or real-time database. The high load of the database causes potential performance bottlenecks, affecting the overall performance and response time of the system, and thus restricting the data interaction speed. Moreover, the complexity of the system and the complexity of operation and maintenance deployment are difficult to meet the online dynamic expansion of data processing capabilities, and there are certain limitations in dealing with the flexible access of a large number of heterogeneous new business entities and high-reliable data processing capabilities, and the dynamic expansion ability for processing heterogeneous data is insufficient. Summary of the Invention

[0004] In order to solve the problems that the existing data interaction systems and methods rely too much on the cloud or real-time database, which restricts the data interaction speed and has insufficient dynamic expansion ability, the present invention provides a system and method for high-speed data interaction within a container, reducing the access to the real-time database and improving the data interaction speed and dynamic expansion ability.

[0005] The specific solution of the present invention is as follows.

[0006] In the first aspect, the present invention provides a system for high-speed data interaction within a container, including:

[0007] A data processing service subsystem, located within the container, accessing the high-speed real-time data cache service subsystem and reading and writing dispatching power grid measurement cache data;

[0008] A high-speed real-time data cache service subsystem, located within the container, implementing high-speed real-time data caching and performing time stamp verification and source verification on the cached data;

[0009] A real-time data snapshot service subsystem, connected to the high-speed real-time data cache service subsystem through a snapshot data update channel, storing, rolling back, and viewing snapshot data;

[0010] And a real-time data synchronization service subsystem, located in the host, connected to the high-speed real-time data cache service subsystem through a real-time data update channel, and writing the received real-time data into the real-time database.

[0011] The data processing service subsystem is connected to the high-speed real-time data caching service subsystem. The real-time data snapshot service subsystem, the real-time data synchronization service subsystem, and the real-time database are integrated in the host computer. The real-time data snapshot service subsystem and the real-time data synchronization service subsystem are respectively connected to the real-time database.

[0012] The high-speed real-time data caching service subsystem is directly integrated in the data processing container, which can capture and respond to frequent data access requests, reduce the number of direct accesses to the underlying real-time database, thereby reducing the load on the database and avoiding potential performance bottlenecks. The real-time data synchronization service subsystem is integrated in the host computer. This read-write separation design allows the system to efficiently process data writing without affecting data reading performance, further improving the overall performance.

[0013] Furthermore, it also includes: a scheduling real-time database service subsystem, connected to the high-speed real-time data caching service subsystem, which constructs power grid model data for missing page data when the device type and measurement type accessed by the data processing service subsystem are not in the cache; a model change monitoring service subsystem, connected to the high-speed real-time data caching service subsystem through the model data update channel, which triggers the reconstruction of the high-speed real-time data cache model when the real-time database has a download of the real-time database or the power grid model changes.

[0014] Both the scheduling real-time database service subsystem and the model change monitoring service subsystem are integrated in the host computer. The scheduling real-time database service subsystem is connected to the real-time database. The high-speed real-time data caching service subsystem in the container listens to the model data update channel. When it receives a model update message, it checks whether there is such a model cache in the cache. If there is, it delays the reconstruction of the model.

[0015] Furthermore, the high-speed real-time data caching service subsystem includes:

[0016] Data access interfaces exposed to the data processing service subsystem, including data reading interfaces, data update interfaces, and data addition and deletion interfaces;

[0017] A data storage module. Data reading and data update are directly implemented by accessing the data storage module. When updating data, it is necessary to verify whether the time stamp of the updated data is the latest;

[0018] A data update upstream module, which listens to the real-time data update channel and updates the real-time data processed by other containers to the data storage module of this container;

[0019] A data update downstream module, which updates the real-time data in the data storage module of this container to the real-time data update channel or the snapshot data update channel;

[0020] And a model synchronization module, which listens to the model data update channel and provides an interface for model data update.

[0021] The data reading interface and data updating interface determine whether there is a page missing in the high-speed real-time data cache. If so, the model synchronization module is called to build the cache and write the data to the data storage module; the data adding and deleting interface directly implements the cache model update through the model synchronization module.

[0022] Furthermore, the high-speed real-time data cache supports dynamic scaling: when the power grid model adds measurement types, it triggers the horizontal scaling of the high-speed real-time data cache in the container; when a device is put into operation or decommissioned, it triggers the vertical scaling of the high-speed real-time data cache in the container.

[0023] High-speed real-time data cache is implemented using file-mapped memory technology (mmap). The power grid model breaks the idea of ​​modeling by application and realizes dynamic scaling of high-speed real-time data cache by modeling by device measurement type. The power grid measurement type can be timely tailored according to the data processing service subsystem to increase the memory resource usage in the container. The cache mechanism supports dynamic data management and can dynamically adjust the cache size and content according to the access frequency and timeliness of the data to optimize the use of memory and storage resources.

[0024] Furthermore, the real-time data snapshot service subsystem listens to the snapshot data update channel, and after receiving the snapshot data, it rolls back and stores the snapshot data and deletes expired snapshot data; the real-time data snapshot service subsystem provides a snapshot viewing function, and the snapshot data of the specified measurement point can be accessed through a standardized interface; the real-time data snapshot service subsystem also provides a snapshot rollback function, and the measurement information of the specified measurement point at a specified time can be rolled back through a standardized interface.

[0025] The real-time data snapshot service subsystem stores the historical change data of several frames of real-time scheduling data, facilitates the rollback of historical data and improves the means of detecting problems after scheduling accidents, provides heterogeneous data query and data recovery functions at the time of power grid accidents, and the snapshot storage structure occupies small resources, takes a short time to view data and rolls back data quickly.

[0026] In a second aspect, the present invention provides a method for high-speed data interaction in a container, which relies on a high-speed data interaction system in a container, including:

[0027] The high-speed real-time data cache service subsystem in the container reads the power grid measurement data. If the cache has been established, the measurement data in the cache is directly returned. If the cache has not been established, the missing page data is used to build the power grid model data.

[0028] Perform timestamp verification and source verification on the cached data, send the data that passes the timestamp verification to the real-time data update channel, and perform snapshot storage on the heterogeneous source data;

[0029] The high-speed real-time data caching service subsystem in other containers reads the real-time data in the real-time data update channel.

[0030] The data processing service subsystem calls the read interface of the high-speed real-time data caching service subsystem to read power grid measurement data, and calls the write interface of the high-speed real-time data caching service subsystem to update power grid measurement data; when there is an update to the power grid real-time data in this container, it is necessary to distribute the updated real-time data to the real-time data update channel. The real-time data synchronization service subsystem listens to the real-time data update channel and writes the updated real-time data into the real-time database of the dispatching monitoring panel.

[0031] Furthermore, after constructing the power grid model data for the page fault data, return the constructed cached data to the data processing service subsystem within the container. The data processing service subsystem directly updates the high-speed real-time data cache within the container, and then compares the timestamps of the corresponding real-time database data in the cache, and saves the data with the latest timestamp in the high-speed real-time data cache. Performing timestamp verification on the cached data refers to verifying the cache data timestamp and the write timestamp to ensure that the measurement data in the cache is data with the latest timestamp.

[0032] Furthermore, after the high-speed real-time data caching service subsystem updates the data of the data processing service subsystem in this container, it distributes the data with the latest timestamp to other containers through the real-time data update channel; the high-speed real-time data caching service subsystem distributes the data with the latest timestamp to the real-time database through the real-time data synchronization service subsystem, providing a real-time data source for the dispatching monitoring panel to ensure that the real-time data for power grid monitoring is data with the latest timestamp. Performing source verification on the cached data refers to verifying the source of the cached data and the source of the write timestamp. If it is found that the service names written by the two are inconsistent, that is, a heterogeneous source scenario occurs, and this cached data is heterogeneous source data.

[0033] Furthermore, performing snapshot storage on the heterogeneous source data includes: converting the heterogeneous source data into snapshot data and sending it to the real-time data snapshot service subsystem for storage through the snapshot data update channel.

[0034] Furthermore, after the high-speed real-time data caching service subsystem in other containers reads the real-time data in the real-time data update channel, it determines whether the cache of the read real-time data is established. If the cache is not established, no processing is performed. If the cache is established, timestamp verification is performed on the cached data, and the data with the latest timestamp is updated to the cache and the real-time database. The display of the dispatching monitoring panel monitoring data obtains the measurement data in the real-time database through the dispatching real-time database service subsystem for monitoring, thereby ensuring the data consistency and stability of the entire system.

[0035] Therefore, the present invention has the following beneficial effects:

[0036] (1) By introducing the technology of high-speed data interaction within the container, the direct access to the network real-time library service is reduced, the latency is lowered, the data access speed is increased, and the data access performance is optimized;

[0037] (2) The caching mechanism supports dynamic data management, and can dynamically adjust the cache size and content according to the access frequency and timeliness of the data, optimizing the use of memory and storage resources;

[0038] (3) The high-speed real-time data caching service subsystem is directly integrated into the data processing container, which can capture and respond to frequent data access requests, reduce the direct access times to the underlying real-time library, thereby reducing the load on the database and avoiding potential performance bottlenecks;

[0039] (4) The real-time data synchronization service subsystem is integrated into the host computer. This read-write separation design allows the system to efficiently process data writing without affecting the data reading performance, further improving the overall performance;

[0040] (5) Based on the containerized efficient data storage scheme and the data access mechanism between containers, it supports historical tracing of the changes in cached data, ensures the consistency, efficiency, and data storage stability of the data exchanged between containers. According to the decoupling of the business layer and the data access layer, it solves the problem of efficient sharing and synchronization of real-time service data during container scheduling, and fully exerts the advantages of container technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0042] Figure 1 It is the architecture diagram of the high-speed data interaction system within the container of the present invention.

[0043] Figure 2 It is the architecture diagram of the high-speed real-time data caching service subsystem within the container of the present invention.

[0044] Figure 3 It is the design diagram of the high-speed implementation plan for data storage within the container of the present invention.

[0045] Figure 4 It is the architecture diagram of the relevant modules for high-speed real-time data interaction and access of the present invention.

[0046] Figure 5This is the implementation architecture diagram of the real-time data snapshot service subsystem of the present invention. Detailed implementation manners

[0047] The embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described by referring to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0048] As Figure 1 shown is the architecture diagram of the in-container data high-speed interaction system of the present invention, which is applicable to the flexible access of multiple new business format entities. The in-container data high-speed interaction system includes a data processing service subsystem and a high-speed real-time data caching service subsystem integrated in the container and connected to each other, and also includes a real-time data snapshot service subsystem, a real-time data synchronization service subsystem, a scheduling real-time library service subsystem, a model change monitoring service subsystem, and a real-time library integrated in the host. The real-time data snapshot service subsystem, the real-time data synchronization service subsystem, and the scheduling real-time library service subsystem are respectively connected to the real-time library, and the scheduling real-time library service subsystem is connected to the high-speed real-time data caching service subsystem. The high-speed real-time data caching service subsystem is connected to the real-time data synchronization service subsystem through a real-time data update channel, connected to the model change monitoring service subsystem through a model data update channel, and connected to the real-time data snapshot service subsystem through a snapshot data update channel.

[0049] Embodiment 1.

[0050] The data processing service subsystem in the container for power grid real-time monitoring accesses the high-speed real-time data caching service subsystem to perform data read and write requests. The specific steps are as follows:

[0051] Step 1, call the read interface of the high-speed real-time data caching service subsystem to read power grid measurement data;

[0052] Step 2, if the high-speed real-time data cache has been established, directly return the measurement data in the cache;

[0053] Step 3, if the high-speed real-time data cache has not been established, that is, the power grid model data is "page fault", and this power grid model data is page fault data, then the high-speed real-time data caching service subsystem calls the scheduling real-time library service subsystem to construct the power grid model data and returns the constructed cache data to the in-container data processing service subsystem;

[0054] Step 4, call the write interface of the high-speed real-time data caching service subsystem to update the power grid measurement data;

[0055] Step 5. If the high-speed real-time data cache is not established, i.e., the power grid model data has a "page fault", and this power grid model data is page fault data, then the high-speed real-time data cache service subsystem calls the scheduling real-time library service subsystem to construct the power grid model data;

[0056] Step 6. Check the cache data timestamp and the write timestamp to ensure that the measured data in the cache is the latest timestamp data. After passing the verification, update the measured data to the cache and send it to the real-time data update channel;

[0057] Step 7. Check the source of the cache data and the source of the write timestamp. If it is found that the service names written by the two are inconsistent, i.e., the "heterogeneous source" scenario appears, and this cache data is heterogeneous source data, then convert the heterogeneous source data into snapshot data and send it to the snapshot data update channel.

[0058] After returning the constructed cache data to the in-container data processing service subsystem, the data processing service subsystem directly updates the in-container high-speed real-time data cache, and then compares the corresponding real-time library data timestamps in the cache, and saves the data with the latest timestamp in the high-speed real-time data cache.

[0059] Embodiment 2.

[0060] High-speed real-time data sharing and consistency guarantee between containers are as follows:

[0061] Step 1. The high-speed real-time data cache service subsystem listens to the messages on the real-time data update channel;

[0062] Step 2. When the high-speed real-time data cache service subsystem receives the power grid measurement data updated from other sources, it determines whether the cache in this container is established. If it is not established, it will not process;

[0063] Step 3. If the cache in this container has been established, it is necessary to check the cache data timestamp and the write timestamp to ensure that the measured data in the cache is the latest timestamp data. After passing the verification, update the measured data to the cache;

[0064] Step 4. The real-time data synchronization service subsystem listens to the messages on the real-time data update channel;

[0065] Step 5. After the real-time data synchronization service subsystem receives the updated measurement data, it is necessary to check the cache data timestamp and the write timestamp to ensure that the measured data in the cache is the latest timestamp data. After passing the verification, update the measured data to the cache and update it to the real-time library;

[0066] Step 6. The display of the dispatching monitoring data is obtained through the dispatching real-time library service subsystem to obtain the measurement data in the real-time library for monitoring, so as to ensure the data consistency and stability of the entire system.

[0067] Embodiment III.

[0068] The real-time data snapshot service subsystem is used for storing, rolling back, and viewing snapshot data. The specific steps are as follows:

[0069] Step 1: The real-time data snapshot service subsystem listens to the snapshot data update channel.

[0070] Step 2: After receiving the snapshot data, the real-time data snapshot service subsystem performs rollback storage on the snapshot data and deletes the expired snapshot data.

[0071] Step 3: The real-time data snapshot service subsystem provides a snapshot viewing function. Through a standardized interface, the snapshot data of a specified measurement point can be accessed. The data information includes: measurement ID, measurement name, measurement value, measurement quality code, timestamp, and write source information.

[0072] Step 4: The real-time data snapshot service subsystem provides a snapshot rollback function. Through a standardized interface, the measurement information of a specified measurement point at a specified time can be rolled back.

[0073] The real-time data snapshot service subsystem will schedule the storage of historical change data of real-time data frames, which is convenient for historical data rollback and improves the means of detecting problems after dispatching accidents, and provides the function of querying heterogeneous source data and data recovery at the time point of power grid accidents.

[0074] Embodiment IV.

[0075] The model change monitoring service subsystem monitors model change messages and triggers the construction of a high-speed real-time data cache model in the container. The specific steps are as follows:

[0076] Step 1: The model change monitoring service subsystem monitors power grid model change messages. If there are model change messages, it sends model update messages to the model data update channel. The model update messages include: application number, table name, and model update time.

[0077] Step 2: The model change monitoring service subsystem monitors the action of downloading the power grid real-time library on the host. If there is a download of the power grid real-time library on the application host, it sends model update messages to the model data update channel.

[0078] Step 3: The high-speed real-time data cache service subsystem in the container listens to the model data update channel.

[0079] Step 4: When the high-speed real-time data cache service subsystem receives the model update message, it checks whether there is a cache of this model in the cache. If so, it performs a delayed reconstruction on this model.

[0080] The high-speed real-time data caching service subsystem is directly integrated into the data processing container, which can capture and respond to frequent data access requests, reduce the number of direct accesses to the underlying real-time library, thereby reducing the load on the database and avoiding potential performance bottlenecks. The real-time data synchronization service subsystem is integrated into the host computer. This read-write separation design allows the system to efficiently process data writing without affecting the data reading performance, further improving the overall performance of the system.

[0081] As Figure 2 shown is the architecture diagram of the high-speed real-time data caching service subsystem within the container of the present invention. The high-speed real-time data caching service subsystem includes a data access interface exposed to the data processing service subsystem within the container, a data storage module, a data update upstream module, a data update downstream module, a model synchronization module, etc. The model synchronization module is connected to the scheduling real-time library service subsystem. The data addition and deletion interface is connected to the real-time data synchronization service subsystem and the real-time data snapshot service subsystem. The data update upstream module is connected to the real-time data update channel. The data update downstream module is connected to the real-time data update channel and the snapshot data update channel.

[0082] Embodiment Five.

[0083] The data access interface of the high-speed real-time data caching service subsystem includes: a data reading interface, a data update interface, and a data addition and deletion interface. The data reading interface and the data update interface determine whether there is a "page fault" in the high-speed real-time data cache. If so, the model synchronization module is called to construct the cache and write the data into the data storage module. Data reading and data update are directly implemented by accessing the data storage module, where data update requires verifying whether the time stamp of the updated data is the latest. The data addition and deletion interface directly updates the cache model through the model synchronization module, and the data addition and deletion interface sends a real-time library data addition and deletion request to the real-time data synchronization service subsystem or the real-time data snapshot service subsystem.

[0084] Embodiment Six.

[0085] When there is an update of the power grid real-time data in this container, the data update upstream module and the data update downstream module share the updated data at high speed to other containers and synchronize it to the real-time library for dispatching and monitoring, and process the heterogeneous data. The specific steps are as follows:

[0086] Step 1, when there is an update of the power grid real-time data in this container, the data update downstream module distributes the updated real-time data to the real-time data update channel; the real-time data synchronization service listens to the real-time data update channel and writes the updated real-time data into the real-time library for dispatching and monitoring;

[0087] Step 2: When there is a "heterogeneous source" phenomenon in data update, that is, when there is heterogeneous data in the updated data, the data update downstream module distributes the heterogeneous data to the snapshot data update channel. The real-time data snapshot service subsystem listens to the snapshot data update channel and stores the updated snapshot data as a snapshot;

[0088] Step 3: The data update upstream module listens to the real-time data update channel and updates the real-time data processed by other containers to the data storage module of this container.

[0089] Embodiment 7.

[0090] The model synchronization module is used to listen to the model data update channel and provide an interface for model update. The specific steps are as follows:

[0091] Step 1: When the data processing service subsystem accesses the high-speed real-time data cache service subsystem and there is a "page fault" in the data storage cache, the model synchronization module calls the scheduling real-time library service subsystem to construct the cache;

[0092] Step 2: The model synchronization module listens to the model data update channel. When receiving a model change message, it checks whether the changed model is cached in this container. If it is cached, it actively calls the scheduling real-time library service subsystem to reconstruct the cache.

[0093] Embodiment 8.

[0094] As Figure 3 shown is the design diagram of the high-speed implementation plan for in-container data storage of the present invention. The high-speed real-time data cache is implemented using the file mapping memory technology (mmap). This technology maps a file to the address space of a process, realizing a one-to-one mapping relationship between the file disk address and a section of virtual addresses in the process virtual address space. After realizing the mapping relationship, the process can read and write this section of memory in the way of pointers. The system will automatically write back the dirty pages to the corresponding file disk, that is, complete the file operation without having to call system call functions such as read / write. On the contrary, the modification of this area in the kernel space is also directly reflected in the user space, thus realizing file sharing between different processes.

[0095] The power grid model breaks the thinking of modeling according to applications. By synchronizing the model on demand, the power grid model can be trimmed in a timely manner according to the data processing service subsystem to reduce the occupancy of in-container memory resources. As Figure 3 shown, if only the AC line terminal table and the bus table are used in the data processing service subsystem, then this container will only synchronize the power grid models of the AC line terminal table and the bus table.

[0096] The power grid model breaks the idea of ​​modeling by application. By modeling by device measurement type, it realizes the dynamic scaling of high-speed real-time data cache. It can timely tailor the power grid measurement type according to the data processing service subsystem to improve the memory resource usage in the container. When the power grid model adds measurement types, it triggers the horizontal scaling of the high-speed real-time data cache in the container; when a device is put into operation or decommissioned, it triggers the vertical scaling of the high-speed real-time data cache in the container. Figure 3 As shown, if the data processing service subsystem only uses the active power, reactive power, and current of the AC line end, then the container only synchronizes the active power, reactive power, current, and other measurement type data of the AC line end meter.

[0097] The power grid model creates a memory mapping file according to the measurement type. First, an index file is created. When multiple measurement types are queried by keyword, the record row where the keyword is located can be quickly located through the index file. Then, the location of the relevant measurement type can be quickly located through the memory mapped by the relevant measurement file, and the real-time measurement records of the power grid of multiple measurement types can be quickly spliced ​​together.

[0098] For power grid measurement type files, it is necessary to store information such as the measured value, quality code, measurement source author, and measurement value update time stamp. The measurement update time stamp can ensure that the measurement data in the high-speed real-time data cache is the latest updated measurement data in the system; the measurement source author can trigger the storage mechanism of snapshot data.

[0099] The data storage module exposes the data update interface and the data reading interface, which supports the common interface of heterogeneous systems and can adapt to data reading and writing of different systems.

[0100] like Figure 4 The figure shows the high-speed real-time data interactive access related module architecture diagram of the present invention, which includes a real-time data synchronization service subsystem, a real-time data snapshot service subsystem, a model change monitoring service subsystem, and a snapshot data operation tool.

[0101] Embodiment 9.

[0102] The real-time data synchronization service subsystem includes a time-stamp data storage module, and the real-time data snapshot service subsystem includes a data snapshot storage module. The data of the real-time data update channel, snapshot data update channel, and model data update channel are stored in Figure 4 In the architecture shown, interactive access can be achieved through the following steps:

[0103] Step 1: The real-time data synchronization service subsystem listens to the real-time data update channel, and when receiving the real-time power grid measurement data after data processing of each container, performs time stamp verification through the time stamp data storage module, and writes the time stamp data into the local host real-time library after passing the verification;

[0104] Step 2: After the snapshot data update channel receives the heterogeneous data messages determined by each container high-speed real-time data caching service subsystem, it stores the heterogeneous data in the data snapshot storage module;

[0105] Step 3: The model change monitoring service subsystem monitors the model changes in the real-time database and the download actions of the real-time database. After detection, it sends relevant model change messages to the model data update channel.

[0106] Embodiment Ten.

[0107] As Figure 5 shown is the implementation architecture diagram of the real-time data snapshot service subsystem of the present invention. The real-time data snapshot service subsystem shown in the figure includes a data snapshot storage module and a standardized interface. The standardized interface includes a snapshot storage interface, a snapshot recovery interface, and a snapshot viewing interface. The snapshot storage structure of the real-time data snapshot service subsystem occupies less resources, has a short data viewing time, and can quickly roll back data. The real-time data snapshot service subsystem receives messages from the snapshot data update channel and stores the snapshot data in the data storage module.

[0108] There is a base class snapshot in the snapshot storage, which stores the most original measured real-time data in the snapshot, that is, it stores the full amount of "sector" data; the snapshots are stored frame by frame according to the 10-second level time. When the snapshot data changes, it is stored. The "sectors" without changes no longer occupy storage resources. That is, the real-time storage size of each frame of snapshot is different. Snapshots with more data changes occupy more storage resources of the "sector", as Figure 5 shown, in each snapshot, the "sectors" in the dotted part do not occupy actual storage space.

[0109] In addition to the basic snapshot, ordinary snapshots are merged in the order of snapshot time according to the limit of storing "sector" resources, as Figure 5 shows the storage occupancy of the "sector" after merging the snapshots at 01:01:10 and 01:01:00. The merged snapshot time is the snapshot at 01:01:00, and the "sector" size is the union of the snapshots at the two time points. The data in the overlapping part is subject to the snapshot at the latest time.

[0110] As Figure 5 shown is the snapshot viewing result. When viewing the data of a "sector", when viewing the specified snapshot data, the snapshot real-time data service will drill down and search starting from the starting point of this frame of snapshot until it finds the snapshot with data for this "sector", and returns its data.

[0111] For the real-time measurement data of the power grid at a specified time, find the storage location of the "sector" where the measurement data is located according to the real-time measurement data of the power grid, find the "frame position" of the snapshot according to the specified time, perform "sector" data search on the snapshot, and after obtaining the real-time measurement data of the power grid to be searched at this time, write the measurement data into the real-time database in the form of a dispatching real-time database service request, and the dispatching monitoring can read the data rolled back by the snapshot.

[0112] In a preferred embodiment, the in-container data high-speed interaction system further includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is loaded into the processor, it implements the in-container data high-speed interaction method in power grid real-time monitoring; the in-container data high-speed interaction system further includes a computer-readable storage medium storing a computer program, and when the computer program is executed by the processor, it implements the in-container data high-speed interaction method in power grid real-time monitoring.

[0113] The present invention reduces the direct access to the network real-time database service by introducing the in-container data high-speed interaction technology, reduces latency, improves data access speed, and optimizes data access performance; the caching mechanism supports dynamic data management, and can dynamically adjust the cache size and content according to the access frequency and timeliness of the data, optimizing the use of memory and storage resources; the high-speed real-time data caching service subsystem is directly integrated into the data processing container, which can capture and respond to frequent data access requests, reducing the direct access times to the underlying real-time database, thus reducing the load on the database and avoiding potential performance bottlenecks; the real-time data synchronization service subsystem is integrated into the host computer. This read-write separation design allows the system to efficiently process data writing without affecting the data reading performance, further improving the overall performance; the efficient data storage scheme based on containerization and the data access mechanism between containers support historical tracing of the changes in cached data, ensuring data consistency, efficiency, and data storage stability in the interaction between containers. According to the decoupling of the business layer and the data access layer, it solves the problem of efficient sharing and synchronization of real-time service data between containers, and gives full play to the advantages of container technology.

[0114] The above are only preferred embodiments of the present invention, and do not impose any limitations on the present invention. Any simple modifications, changes, and equivalent structural transformations made to the above embodiments according to the technical essence of the present invention still fall within the protection scope of the technical solution of the present invention.

Claims

1. A high-speed data interaction system in a container, characterized in that: include: The data processing service subsystem is located in the container and accesses the high-speed real-time data cache service subsystem to read and write the cache data of the dispatching power grid measurement; The high-speed real-time data cache service subsystem is located in the container, realizes high-speed real-time data cache, and performs time stamp verification and source verification on the cached data; A real-time data snapshot service subsystem is connected to the high-speed real-time data cache service subsystem through a snapshot data update channel to store, roll back and view snapshot data; And the real-time data synchronization service subsystem is located in the host machine, connected with the high-speed real-time data cache service subsystem through the real-time data update channel, and writes the received real-time data into the real-time library.

2. A high-speed data interaction system in a container according to claim 1, characterized in that: Also includes: The dispatching real-time library service subsystem is connected to the high-speed real-time data cache service subsystem, and when the device type and measurement type accessed by the data processing service subsystem are not in the cache, the power grid model data is constructed for the missing page data; The model change monitoring service subsystem is connected to the high-speed real-time data cache service subsystem through a model data update channel, and triggers the reconstruction of the high-speed real-time data cache model when the real-time library is downloaded or the power grid model changes.

3. A high-speed data interaction system in a container according to claim 2, characterized in that: The high-speed real-time data cache service subsystem includes: The data access interface exposed to the data processing service subsystem includes a data reading interface, a data updating interface, and a data adding and deleting interface; Data storage module, data reading and data updating are realized by directly accessing the data storage module. Data updating needs to verify whether the time stamp of the updated data is the latest; A data update uplink module listens to the real-time data update channel and updates the real-time data processed by other containers to the data storage module of the current container; A data update downstream module, which updates the real-time data of the container data storage module to the real-time data update channel or the snapshot data update channel; And a model synchronization module, which listens to the model data update channel and provides an interface for model data update.

4. A high-speed data interaction system in a container according to claim 3, characterized in that: High-speed real-time data cache supports dynamic scaling: when the power grid model adds measurement types, it triggers the horizontal scaling of the high-speed real-time data cache in the container; when a device is put into operation or decommissioned, it triggers the vertical scaling of the high-speed real-time data cache in the container.

5. The high-speed data interaction system in a container according to claim 1, characterized in that: The real-time data snapshot service subsystem listens to the snapshot data update channel, and after receiving the snapshot data, rolls back and stores the snapshot data and deletes expired snapshot data; the real-time data snapshot service subsystem provides a snapshot viewing function, accessing the snapshot data of a specified measurement point through a standardized interface; the real-time data snapshot service subsystem also provides a snapshot rollback function, rolling back the measurement information of a specified measurement point at a specified time through a standardized interface.

6. A method for high-speed data interaction in a container, characterized in that: A high-speed data interaction system in a container according to any one of claims 2 to 5, comprising: The high-speed real-time data cache service subsystem in the container reads the power grid measurement data. If the cache has been established, the measurement data in the cache is directly returned. If the cache has not been established, the missing page data is used to build the power grid model data. Perform time stamp verification and source verification on cached data, send the data that passes the time stamp verification to the real-time data update channel, and perform snapshot storage on heterogeneous source data; The high-speed real-time data cache service subsystem in other containers reads the real-time data in the real-time data update channel.

7. A method for high-speed data interaction in a container according to claim 6, characterized in that: After constructing the power grid model data with the missing page data, the constructed cache data is returned to the data processing service subsystem in the container. The data processing service subsystem directly updates the high-speed real-time data cache in the container, and then compares the corresponding real-time library data timestamp in the cache, and saves the real-time data in the high-speed real-time data cache with the data with the latest timestamp.

8. A method for high-speed data interaction in a container according to claim 7, characterized in that: After updating the data of the data processing service subsystem in this container, the high-speed real-time data caching service subsystem distributes the latest time-stamped data to other containers through the real-time data update channel; the high-speed real-time data caching service subsystem distributes the latest time-stamped data to the real-time library through the real-time data synchronization service subsystem.

9. A method for high-speed data interaction in a container according to claim 6, characterized in that: Storing heterogeneous data in snapshot form includes: converting heterogeneous data into snapshot data, and sending the data to the real-time data snapshot service subsystem for storage through a snapshot data update channel.

10. A method for high-speed data interaction in a container according to claim 8, characterized in that: After the high-speed real-time data cache service subsystem in other containers reads the real-time data in the real-time data update channel, it determines whether the cache of the read real-time data is established. If the cache is not established, no processing is performed. If the cache is established, the cache data is timestamped and the latest timestamp data is updated to the cache and real-time library.

Images