A data recording method and device based on a multi-core processor

Abstract

The application discloses a data recording method and device based on a multi-core processor. A non-real-time system and a real-time system are respectively run on different cores of the multi-core processor, and the two systems use shared memory as a medium. The real-time system side acquires high-frequency sampling data and low-frequency sampling data required for data recording; the two kinds of data are compressed and stored by using a dynamic lossless compression method; continuous recording wave initial information and dynamic recording wave initial information are respectively generated according to the two kinds of data, and are transmitted to the shared memory; the non-real-time system side generates recording wave description files according to the dynamic recording wave initial information and the continuous recording wave initial information in the shared memory; the real-time system side directly stores compressed real-time data into files by using a shared file physical address mode, and informs the non-real-time system side to synchronize the memory to a storage medium by using a method of triggering an inter-core interrupt.

Description

Technical Field

[0001] This invention relates to the field of power, and more particularly to a method and apparatus for synchronous data recording based on a multi-core processor. Background Technology

[0002] With the development of new energy sources, more and more new energy sources are being connected to the power system, triggering a series of subsynchronous and supersynchronous oscillations and high-order harmonics, leading to accidents such as power plant tripping and impacting the stability of the power system. At the same time, the grid connection of new energy sources introduces a large number of power electronic devices, causing changes in grid characteristics. However, the impact of high-proportion power electronic devices on grid characteristics is still under investigation. Traditional PMUs (Phasor Measurement Units) and waveform recorders can only record short waveforms, missing a large amount of high-frequency information during oscillations. Therefore, long-term continuous waveform recording of grid characteristic quantities and transient event recording are particularly important.

[0003] Current broadband measurement and control technology has introduced the requirement of continuous waveform recording on top of the existing transient event waveform recording. However, the generation of long-term continuous waveforms, compared to the original transient event waveform recording, requires a large amount of memory and large-capacity storage peripherals from real-time data acquisition to file generation, and must run continuously without interruption to ensure continuous waveform recording. In addition, transient event analysis requires higher frequency sampling data, which requires the system to record data at two sampling frequencies simultaneously. Therefore, the ability to perform both transient event waveform recording and continuous waveform recording places high demands on processor performance. Summary of the Invention

[0004] The main objective of this invention is to provide a data recording method and apparatus based on a multi-core processor, which can complete continuous waveform recording and transient event waveform recording functions with relatively small memory and system overhead, effectively solving the problem of high processor performance requirements for data recording.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] In a first aspect, the present invention provides a data recording method based on a multi-core processor. The method is implemented using a real-time system and a non-real-time system running on a multi-core processor, with the real-time system and the non-real-time system interacting using shared memory. The method includes:

[0007] The real-time system acquires high-frequency and low-frequency sampling data; it identifies transient events based on the high-frequency sampling data to form transient event status identifiers; it compresses and stores the high-frequency and low-frequency sampling data into the high-frequency and low-frequency sampling data areas of the real-time system respectively using a dynamic lossless compression method; based on the transient event status identifiers, it records the pointers of the high-frequency sampling data areas before and after the occurrence of the corresponding transient event to generate dynamic waveform recording initial information; based on the sampling time and the corresponding pointers of the low-frequency sampling data areas, it generates continuous waveform recording initial information; and it transmits the generated dynamic waveform recording initial information and continuous waveform recording initial information to shared memory.

[0008] The non-real-time system generates corresponding waveform description files based on the initial dynamic waveform recording information and the initial continuous waveform recording information in the shared memory. Based on the waveform description files, a raw waveform recording data file is created, and the address of the raw waveform recording data file is shared to the real-time system via shared memory through a memory-mapped file.

[0009] The real-time system writes the compressed high-frequency or low-frequency sampled data to the address of the corresponding waveform recording raw data file and triggers an inter-core interrupt.

[0010] The non-real-time system responds to inter-core interrupts, performs file data synchronization operations, and refreshes virtual memory file data to the storage medium;

[0011] If it is necessary to read the waveform recording file, the non-real-time system will decompress the compressed data file corresponding to the storage medium to generate a standard format dynamic waveform recording file or continuous waveform recording file.

[0012] Furthermore, the high-frequency sampling data area and the low-frequency sampling data area form a circular buffer, and the dynamic lossless compression method includes:

[0013] Using multiple consecutive sampling points as a sampling group, based on the differential algorithm, high-frequency sampling data and low-frequency sampling data are compressed in groups according to the increment of data change, and the compressed grouped sampling data is stored in the corresponding circular buffer.

[0014] Furthermore, the step of performing group compression of high-frequency and low-frequency sampled data based on the differential algorithm and according to the increment of data change includes:

[0015] Each sampling group structure includes one descriptor and M sampling channels. Each sampling channel consists of N 16-bit groups, where N = 2, 3, ..., which can store N to 2N-1 sampling point data. The descriptor is used to represent the number of data points in each channel of the sampling group. For each sampling channel, data mutation is judged based on a differential algorithm, with each group consisting of 2N-1 adjacent sampling points. If no data mutation is determined in any of the M sampling channels, then for each sampling channel, the first sampling point data in the 2N-1 adjacent sampling points is stored in a 16-bit group, and the data change increment obtained by comparing each subsequent sampling point data with the previous sampling point data is stored in an 8-bit group. If one or more of the M sampling channels are determined to have data mutation, then for each sampling channel, N independent sampling point data are stored in N 16-bit groups.

[0016] Furthermore, the method for determining whether there is a data mutation is as follows:

[0017] Each sampling channel is judged by grouping the data of 2N-1 adjacent sampling points. The data of the next 2(N-1) sampling points after the first sampling point are compared with the data of the previous sampling point in turn. If the absolute value of the data change is less than or equal to the set value, it is considered that there is no data mutation. If the absolute value of the data change is greater than the set value, it is considered that there is a data mutation.

[0018] Furthermore, the step of decompressing the compressed data file corresponding to the storage medium by the non-real-time system to generate a standard format dynamic waveform recording file or continuous waveform recording file includes:

[0019] The corresponding compressed data file in the storage medium is decompressed and mirrored to the virtual memory file system. The decompressed grouped sampling data is then extracted and decompressed again into single-point sampling data.

[0020] The total number of sampling points is calculated based on the information in the waveform description file. The data is then segmented and converted into a standard data format. The converted data is appended to a temporary waveform file in the virtual memory file system. The temporary waveform file is then forwarded or transferred to the corresponding storage medium in one go.

[0021] Secondly, the present invention provides a data recording device based on a multi-core processor, comprising: a digital sampling unit and a dynamic data recording, processing and management unit.

[0022] The digital sampling unit is used to receive raw sampling messages from an external network, parse the received raw sampling messages, and transmit the parsed data to the dynamic data processing and management unit through the internal network.

[0023] The dynamic data recording, processing, and management unit includes an internal sampling message processing module, a data synchronization and interpolation module, a transient event identification module, a data compression module, a data sharing module, a dynamic waveform recording initial information generation module, and a continuous waveform recording initial information generation module located on the real-time system side; and a waveform recording storage module, a data decompression module, a dynamic waveform recording file generation module, and a continuous waveform recording file generation module located on the non-real-time system side. The real-time and non-real-time systems run on different cores of a multi-core processor, and the real-time and non-real-time systems interact using shared memory.

[0024] The internal sampling message processing module obtains the original sampling message from the internal network bus, parses it, and generates the original sampling data with the sampling sequence number and sampling time.

[0025] The data synchronization and interpolation module synchronizes the original sampled data and resamples it to generate high-frequency and low-frequency sampled data based on the sampling sequence number of the original sampled data and the external time synchronization signal.

[0026] The transient event identification module identifies transient events based on high-frequency sampled data and generates transient event status identifiers.

[0027] The data compression module compresses high-frequency and low-frequency sampled data using a dynamic lossless compression method and stores them separately in the high-frequency sampled data area and the low-frequency sampled data area on the real-time system side.

[0028] The dynamic waveform recording information generation module records the pointers of the high-frequency sampling data area before and after the occurrence of the corresponding transient event based on the transient event identifier, and generates the initial information of dynamic waveform recording.

[0029] The continuous waveform recording information generation module generates initial information for continuous waveform recording based on the original sampling time and the pointer of the corresponding low-frequency sampling data area.

[0030] The data sharing module transmits the initial information of dynamic waveform recording and the initial information of continuous waveform recording to the shared memory; and writes the compressed high-frequency sampling data and low-frequency sampling data to a file through the shared file address and triggers an inter-core interrupt.

[0031] The waveform recording and storage module generates corresponding waveform recording description files based on the dynamic waveform recording initial information and the continuous waveform recording initial information in the shared memory; it creates a waveform recording raw data file based on the waveform recording description file, and shares the address of the waveform recording raw data file to the real-time system side through shared memory via a memory-mapped file; and it responds to inter-core interrupts, performs file data synchronization operations, and refreshes the virtual memory file data to the storage medium.

[0032] The data decompression module decompresses the compressed data file corresponding to the storage medium and mirrors it to the virtual memory file system. It then extracts the decompressed high-frequency and low-frequency sampling data, decompresses it a second time to generate high-frequency and low-frequency sampling data with sampling sequence numbers and sampling times, and stores them in the data cache interval in seconds.

[0033] The dynamic waveform recording file generation module converts the decompressed high-frequency sampling data into a standard format dynamic waveform recording file.

[0034] The continuous waveform recording file generation module converts the decompressed low-frequency sampling data into a standard format continuous waveform recording file.

[0035] Thirdly, the present invention provides a data recording device based on a multi-core processor, characterized in that it includes: an analog sampling unit and a dynamic data recording, processing and management unit.

[0036] The analog sampling unit is used to collect the analog signals required for data recording, digitally discretize the analog signals, obtain high-frequency sampling data and low-frequency sampling data, and transmit them to the dynamic data processing and management unit through the internal network.

[0037] The dynamic data recording, processing, and management unit includes an internal sampling message processing module, transient event identification module, data compression module, data sharing module, dynamic waveform recording initial information generation module, and continuous waveform recording initial information generation module located on the real-time system side; and a waveform recording storage module, data decompression module, dynamic waveform recording file generation module, and continuous waveform recording file generation module located on the non-real-time system. The real-time and non-real-time systems run on different cores of a multi-core processor, and the real-time and non-real-time systems interact using shared memory.

[0038] The internal sampling message processing module is used to obtain high-frequency sampling data and low-frequency sampling data from the internal network bus, and generate raw sampling data with sampling sequence number and sampling time after parsing.

[0039] Transient event identification module: Identifies transient events based on high-frequency sampled data and generates transient event status identifiers;

[0040] The data compression module compresses high-frequency and low-frequency sampled data using a dynamic lossless compression method and stores them separately in the high-frequency sampled data area and the low-frequency sampled data area on the real-time system side.

[0041] The dynamic waveform recording information generation module records the pointers of the high-frequency sampling data area before and after the occurrence of the corresponding transient event based on the transient event identifier, and generates the initial information of dynamic waveform recording.

[0042] The continuous waveform recording information generation module generates initial information for continuous waveform recording based on the original sampling time and the pointer to the low-frequency sampling data area.

[0043] The data sharing module transmits the initial information of dynamic waveform recording and the initial information of continuous waveform recording to the shared memory; and writes the compressed high-frequency sampling data and low-frequency sampling data to a file through the shared file address and triggers an inter-core interrupt.

[0044] The waveform recording and storage module generates corresponding waveform recording description files based on the dynamic waveform recording initial information and the continuous waveform recording initial information in the shared memory; it creates a waveform recording raw data file based on the waveform recording description file, and shares the address of the waveform recording raw data file to the real-time system side through shared memory via a memory-mapped file; and it responds to inter-core interrupts, performs file data synchronization operations, and refreshes the virtual memory file data to the storage medium.

[0045] The data decompression module decompresses and mirrors the compressed data file corresponding to the storage medium to the virtual memory file system, extracts the decompressed high-frequency sampling data and low-frequency sampling data, decompresses it a second time to generate high-frequency sampling data and low-frequency sampling data with sampling sequence number and sampling time, and stores them in the data cache interval in seconds.

[0046] The dynamic waveform recording file generation module converts the decompressed high-frequency sampling data into a standard format dynamic waveform recording file.

[0047] The continuous waveform recording file generation module converts the decompressed low-frequency sampling data into a standard format continuous waveform recording file.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] This invention utilizes a multi-core processor as the medium and operates in a dual-system mode. Shared memory is used between the real-time and non-real-time systems. By rationally planning the data flow, and employing a simple and easy-to-use method for dynamic lossless compression of real-time data and memory-mapped files, the real-time system directly stores compressed real-time data to files, saving memory space, reducing data copying overhead, and minimizing data response latency. Inter-core interrupts improve the real-time performance of data recording and storage, enabling the system to stably and efficiently complete continuous waveform recording and transient event recording. Direct file manipulation on the real-time system side reduces data copying. For stored compressed data files, decompression and conversion are performed as needed during external transmission, improving the system's storage medium utilization. Attached Figure Description

[0050] Figure 1 This is a structural block diagram of the device provided in Embodiment 1 of the present invention;

[0051] Figure 2This is a structural block diagram of the dynamic data recording, processing, and management unit provided in Embodiment 1 of the present invention;

[0052] Figure 3 This is a structural block diagram of the device provided in Embodiment 2 of the present invention;

[0053] Figure 4 This is a structural block diagram of the dynamic data recording, processing, and management unit provided in Embodiment 2 of the present invention;

[0054] Figure 5 This is a flowchart of the method provided in Embodiment 3 of the present invention;

[0055] Figure 6 This is a schematic diagram of real-time data compression provided by the present invention;

[0056] Figure 7 A detailed data flow diagram is provided for this invention;

[0057] Figure 8 A flowchart of the file storage process for the dual-system provided by this invention. Detailed Implementation

[0058] The present invention will be further described below with reference to specific embodiments. These embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0059] Example 1

[0060] like Figure 1 As shown, the present invention provides a data recording device based on a multi-core processor, comprising: a digital sampling unit and a dynamic data recording, processing and management unit.

[0061] The digital sampling unit includes a message processing module, which is used to parse SV messages, GOOSE messages, etc. received from the external network, and package the parsed data to transmit it to the dynamic data processing and management unit through the internal network.

[0062] like Figure 2 As shown, the dynamic data recording, processing, and management unit is designed based on a multi-core processor, including a real-time system side and a non-real-time system side, and performs continuous waveform recording and transient event waveform recording functions. The real-time system and the non-real-time system interact using shared memory.

[0063] The real-time system includes an internal sampling message processing module, a data synchronization and interpolation module, a transient event identification module, a data compression module, a dynamic waveform recording initial information generation module, a continuous waveform recording initial information generation module, a data sharing module, and a time management module.

[0064] Internal sampling message processing module: Obtains the corresponding sampling message from the internal network bus, parses it, and generates a raw sampling data window with sampling sequence number and sampling time.

[0065] Data synchronization and interpolation module: Based on the sampling sequence number of the original acquired data and the external time synchronization signal, the original sampled data is synchronized and resampled (interpolated or decimation filter) to generate two different sampling rates, namely high-frequency sampling data and low-frequency sampling data. The sampling rate of the original sampled data is not lower than the sampling rate of the high-frequency sampling data.

[0066] Transient event identification module: Identifies transient events based on high-frequency sampled data and generates relevant transient event status identifiers.

[0067] Data compression module: Compresses high-frequency and low-frequency sampled data using dynamic lossless compression methods and stores them separately in the high-frequency sampled data area and low-frequency sampled data area on the real-time system side, and generates sampled data description information based on the original sampling time and sampling sequence number.

[0068] The high-frequency sampling data area and the low-frequency sampling data area adopt a ring-shaped data buffer.

[0069] Accordingly, the sampling data description information includes the location and time information of the circular buffer corresponding to the start and end data per second, as well as the sampling counter and data window pointer at the moment of data mutation.

[0070] Dynamic waveform recording information generation module: Based on the transient event identifier, it records the data window pointer of the high-frequency sampling data area before and after the corresponding event occurs, and generates the initial information of dynamic waveform recording.

[0071] Continuous waveform recording information generation module: Based on the original sampling time and the pointer to the corresponding low-frequency sampling data area, it generates initial information for continuous waveform recording.

[0072] Data sharing module: transmits dynamic waveform initial information and continuous waveform initial information to shared memory; and writes compressed high-frequency sampling data and low-frequency sampling data to a file through a shared file address and triggers an inter-core interrupt.

[0073] Time management module: Performs clock synchronization based on external clock source signals.

[0074] The non-real-time system side includes a waveform recording and storage module, a data decompression module, a dynamic waveform recording file generation module, a continuous waveform recording file generation module, a communication module, and a management module.

[0075] Waveform recording storage module: Based on the initial dynamic waveform recording information and the initial continuous waveform recording information in shared memory, it generates corresponding waveform recording description files that meet the requirements, such as cfg, hdr, etc. (excluding dat files); based on the generated waveform recording description files, it creates waveform recording raw data files, and shares the address of the waveform recording raw data files to the real-time system side through shared memory via memory-mapped files; and, in response to inter-core interrupts, it performs file data synchronization operations, synchronizing the data files (sdat files) in the virtual memory file system operated on the real-time system side to the storage medium.

[0076] Data decompression module: When it is necessary to retrieve or view dynamic waveform files, the data file (sdat file) corresponding to the storage medium is decompressed and mirrored to the virtual memory file system. The decompressed high-frequency sampling data and low-frequency sampling data and related status are extracted. The data is decompressed again to generate high-frequency sampling data and low-frequency sampling data with sampling sequence number and sampling time, and stored in the data cache interval in seconds.

[0077] Dynamic waveform recording file generation module: Generates dynamic waveform recording files that meet the standard format from the high-frequency sampling data obtained after decompression by the data decompression module.

[0078] Continuous waveform recording file generation module: Generates waveform recording files that meet standard formats from the low-frequency sampling data obtained after decompression by the data decompression module.

[0079] Communication module: Sends fault alarm information to the station control layer monitoring backend or remote dispatch master station in accordance with IEC61850 or other standards; transmits relevant dynamic data messages to the management unit of the dynamic monitoring system in accordance with GBT26865.2.

[0080] Management module: Manages the storage media; displays relevant electrical quantities.

[0081] Example 2

[0082] like Figure 3 As shown, the present invention provides a data recording device based on a multi-core processor, comprising: an analog sampling unit and a dynamic data recording, processing and management unit.

[0083] The analog sampling unit includes an analog sampling board (Potential Transformer and Current Transformer, PTCT) and an AD (Analog-to-Digital) board. The AD board is used to digitally discretize the analog signal, package the data, and transmit it to the dynamic data processing and management unit through the internal network. The AD board can directly acquire high-frequency and low-frequency sampled data by setting different sampling rates.

[0084] The dynamic data processing and management unit module is basically the same as the device in Embodiment 1, such as... Figure 4 As shown, the dynamic data recording, processing, and management unit includes a real-time system side and a non-real-time system side, with the real-time system and the non-real-time system interacting using shared memory.

[0085] The real-time system includes an internal sampling message processing module, a transient event identification module, a data compression module, a data sharing module, a dynamic waveform recording initial information generation module, a continuous waveform recording initial information generation module, and a time management module.

[0086] The internal sampling message processing module is used to obtain high-frequency sampling data and low-frequency sampling data from the internal network bus, and after parsing, generate a raw sampling data window with sampling sequence number and sampling time.

[0087] The transient event identification module, data compression module, data sharing module, dynamic waveform recording initial information generation module, continuous waveform recording initial information generation module, and time management module are all the same as in Example 1.

[0088] The non-real-time system side includes the same functional modules as in Embodiment 1.

[0089] Example 3

[0090] This invention provides a data recording method based on a multi-core processor. This method uses a multi-core processor as the data recording processor, where the number of physical cores required for management and real-time business operations can be planned in various combinations according to needs. The multi-core processor operates in a dual-system mode based on business requirements. The management business module operates on the non-real-time system side (applicable to systems such as Linux), while the real-time business module operates on the real-time system side. The two systems use memory-mapped files, utilizing shared memory interaction and inter-core interrupts to implement file write operations on the real-time system side, completing continuous waveform recording and transient event waveform recording functions.

[0091] This embodiment uses the apparatus described in Embodiment 1, primarily taking a power grid signal with a sampling rate of 12800Hz or higher as an example. Specifically, in conjunction with... Figure 5 and Figure 7 As shown, the method includes:

[0092] Step S1: The digital sampling unit on the real-time system side obtains the power grid signal with a sampling rate of 12800Hz or higher from the external SV network, parses it, packages the parsed data, and transmits it to the dynamic data processing and management unit through the internal network.

[0093] Step S2: The internal sampling message processing module on the real-time system side obtains the corresponding sampling message from the internal network bus, parses it, and generates a raw sampling data window with sampling sequence number and sampling time.

[0094] The real-time system synchronizes the local second pulse according to the external clock signal, and the 78.125μs sampling interrupt is synchronized with the external second pulse. During the sampling interrupt, the UTC time and sampling sequence number are sampled on the identifier of the power grid sampling data collected by the SV network.

[0095] Step S3: The real-time system side data synchronization and interpolation module synchronizes the original sampled data according to the sampling sequence number of the original acquired data and the external time synchronization signal, and generates high-frequency sampled data of 12800Hz / s and low-frequency sampled data of 6400Hz / s through a multi-phase decimation filter.

[0096] Step S4: The transient event identification module on the real-time system side identifies transient events based on high-frequency sampling data and forms transient event status identifiers.

[0097] Step S5: The data compression module on the real-time system side compresses the high-frequency sampling data and low-frequency sampling data into the high-frequency sampling data area and low-frequency sampling data area on the real-time system side respectively using a dynamic lossless compression method. It also forms sampling data description information based on the original sampling time and sampling sequence number, including the position and time information of the circular buffer corresponding to the start and end data per second, the sampling counter and data window pointer at the moment of data change.

[0098] The real-time system and the non-real-time system interact using shared memory. The non-real-time system uses the Linux operating system, while the real-time system uses PNOS. The sampling rate is 12,800 points per second, which is a large amount of data considering 16 analog channels and 32 digital channels.

[0099] This invention is designed from two aspects: memory usage overhead and operating overhead. The first is the arrangement of memory data structures, and the second is the simplification of shared information, so as to reduce the memory pressure and operating pressure of the system, reduce the data copying and latency of the system, and increase the system throughput.

[0100] To address this, the real-time system uses a data compression module to dynamically and losslessly compress the sampled data, compressing and storing both high-frequency and low-frequency sampled data separately, thus achieving high-speed real-time data caching across multiple channels with minimal memory usage.

[0101] First, define the data structure for dynamic lossless compression and establish a data cache.

[0102] On the real-time system side, a low-frequency sampling data area and a high-frequency sampling data area are established. Groups of low-frequency sampling data and high-frequency sampling data are stored in a circular buffer, respectively. The corresponding UTC time per second is recorded, along with the information at the beginning of the grouped sampling data and the information corresponding to the data mutation time per second.

[0103] To minimize memory usage on the real-time system side, each analog channel is grouped into 16-bit groups, while the digital channels are arranged bit-by-bit, with 16 digital channels per group. Based on a differential algorithm, multiple sampling point data are compressed and stored in groups, and these groups are stored in a circular buffer.

[0104] like Figure 6 As shown, the original single-point sampled data uses 16 bits to represent a sample point. Each sample group structure consists of one descriptor (8 bits) and M sample channel data. Each sample channel has N 16-bit segments (N = 2, 3...), which can store N to 2N-1 sample point data. The descriptor represents the number of data points in each channel of the sample group.

[0105] For each sampling channel, data mutation detection is performed using a differential algorithm, grouping the data from 2N-1 adjacent sampling points together. If no data mutation is detected in any of the M sampling channels, then for each sampling channel, the first sampling point from the 2N-1 adjacent sampling points is stored as a 16-bit byte, and the increment of each subsequent sampling point is stored as an 8-bit byte, obtained by comparing it with the previous sampling point. If one or more of the M sampling channels are found to have data mutation, then for each sampling channel, N independent 16-bit sampling point data are stored as N 16-bit byte groups. This method can compress the sampling data and output it to the corresponding memory area with a compression ratio of approximately (2N-1) / N.

[0106] To facilitate searching for data within a whole second, starting from the zero-sampling counter at point t1, data is grouped into sets of five samples. The subsequent four samples are compared sequentially with the previous sample. A change in absolute value less than 127 is considered steady state (i.e., no data mutation); a change in absolute value greater than 127 is considered transient state (i.e., with data mutation). In steady state, the five interrupted sample data points are compressed into three 16-bit groups for transmission. In transient state, three independent sample data points are stored in three 16-bit groups. A pointer to the data group corresponding to the zero-sampling counter is recorded every second. For easy traceability, the data from the zero-sampling counter every second serves as the starting point for a new sample group.

[0107] When the transient event identification module detects a transient event, it records the fault time and the pointer to the corresponding data group.

[0108] Step S6: The dynamic waveform recording information generation module on the real-time system side records the pointers of the high-frequency sampling data area before and after the occurrence of the corresponding transient event according to the transient event status identifier, and generates the initial information of dynamic waveform recording; the continuous waveform recording information generation module generates the initial information of continuous waveform recording according to the sampling time and the pointer of the corresponding low-frequency sampling data area; the initial information of dynamic waveform recording and the initial information of continuous waveform recording are transmitted to the shared memory through the data sharing module.

[0109] To reduce data duplication between the real-time system side and the non-real-time system side, the two systems do not share real-time sampling data. Instead, they share the dynamic waveform initial information and continuous waveform initial information generated by the real-time system side, as well as the physical address information of the corresponding waveform raw data file mapped by the non-real-time system side.

[0110] On the real-time system side, low-frequency and high-frequency sampled data are processed by a dynamic waveform recording information generation module and a continuous waveform recording information generation module, respectively. Specifically, after identifying a transient fault in the power grid, the pointers of the corresponding high-frequency sampled data areas before and after the fault occurrence are recorded in the initial information of the transient waveform recording. The transient fault state and related initial information are shared to the non-real-time system side via shared memory. Based on the sampling time, the pointers of the low-frequency sampled data areas corresponding to the first and last seconds of a minute are recorded in the initial information of the continuous waveform recording, and this initial information is shared to the non-real-time system side via shared memory.

[0111] To reliably record data, the real-time system records a pointer every second in a fixed sampling group's circular buffer on the real-time sampled data buffer. In this embodiment, to accommodate both one-minute continuous waveform recording and transient event waveform recording, the raw data points for both types of waveform recordings are based on their respective sampling group's circular data buffer (capable of buffering 64 seconds of data). When a waveform recording condition is triggered, the second position and sampling counter corresponding to the triggering event are recorded. When waveform recording is completed, the second position and sampling counter corresponding to the end time are recorded, and the waveform recording write pointer is updated.

[0112] The continuous waveform information generation module on the real-time system side judges the waveform recording based on the UTC time, while the dynamic waveform information generation module judges the waveform recording of transient events based on the transient fault marker and the sampled UTC time.

[0113] More specifically, the process by which the continuous waveform information generation module generates initial information for continuous waveform recording is as follows:

[0114] By converting the sampled UTC time to local time (year-month-day-hour-minute-second), it is determined whether the current second is the starting second of a minute. If the current second is the starting second of a minute, the position PTR1 of the first second of the current minute in the sampling group's circular buffer is recorded. When the first second of the next minute arrives, the position PTR2 of the first second of the next minute in the sampling group's circular buffer is recorded. The data between PTR1 and PTR2 constitutes a complete minute of data. The recorded position information is stored in the temporary information of the continuous waveform recording information group, recording the waveform recording trigger time, initial point time, and end time. The continuous waveform recording write pointer is incremented by 1, and this waveform recording information is transmitted to the non-real-time system side via shared memory. In this way, the problem of uncertainty in the total number of seconds in a minute caused by leap seconds can be reliably handled.

[0115] The process by which the dynamic waveform recording information generation module generates the initial dynamic waveform recording information is as follows:

[0116] The transient event recording is temporarily set to one second before and one second after startup. These times can be adjusted as needed. When the real-time system detects a transient event, it initiates the transient event recording process, recording the sampling counter and the corresponding second position at the time of the fault, and recording the position of the previous second in the transient event recording information group. Once the number of continuous recording points after the transient event starts exceeds the set one second, dynamic recording information (recording trigger time, initial point time, and end time) is recorded. The transient event recording write pointer is incremented by 1, and the dynamic recording information is transmitted to the non-real-time system.

[0117] Step S7: The non-real-time system side generates corresponding waveform description files based on the initial dynamic waveform recording information and the initial continuous waveform recording information in the shared memory; based on the generated waveform description files, a waveform raw data file to be stored is created; the virtual address of the waveform raw data file is obtained through memory-mapped files; the corresponding physical address is calculated; and the physical address is shared to the real-time system side through shared memory.

[0118] Step S8: After the real-time system converts the physical address of the file into a virtual address, it directly writes the compressed sampled data into the corresponding virtual address and triggers an inter-core interrupt.

[0119] In step S9, the non-real-time system side responds to the inter-core interrupt by performing a file data synchronization operation and refreshing the virtual memory file data to the storage medium.

[0120] The non-real-time system obtains transient waveform initial information and continuous waveform initial information from shared memory, and generates corresponding cfg, hdr and other files based on the waveform initial information.

[0121] When the waveform recording storage module on the non-real-time system side detects a change in the write pointer of the continuous waveform recording group or the transient event waveform recording group, it retrieves the name of the waveform recording file and generates a waveform recording description file, such as a cfg file.

[0122] The name of the waveform recording file is obtained based on the waveform recording configuration information (site name, etc.) and the initial waveform recording information in the waveform recording group (wave recording start time, trigger time).

[0123] Based on the waveform recording configuration information (wave recording channel name, waveform conversion factor, etc.) ) The initial information of the waveform recording group (wave recording start time, trigger time, number of waveform recording points) is used to generate a cfg file, which is encoded in ASCII.

[0124] The waveform recording configuration information is obtained from files and the real-time system side during system initialization on the non-real-time system side.

[0125] In particular, this part can reduce system memory and operating pressure, reduce system data copying and response latency, and increase system throughput by optimizing data flow.

[0126] The dual-system architecture employs memory-mapped files, shared file physical addresses, and interrupt triggering to reduce data copying overhead on the Linux side while ensuring system stability and real-time performance. On the non-real-time system side, based on the initial waveform recording information in shared memory, a waveform recording description file is generated. Based on this description file, a raw waveform recording data file (SDAT file) to be stored is created. The virtual address of the SDAT file to be operated on is obtained, and the corresponding physical address is calculated. This physical address is then shared with the real-time system side via shared memory. On the real-time system side, after mapping the file's physical address to the virtual address, the compressed, grouped sampled data is directly written to the corresponding virtual address. Upon completion of the operation, an inter-core interrupt is triggered. The non-real-time system side responds to the inter-core interrupt, performs synchronization operations, and flushes the memory to the storage medium.

[0127] like Figure 8 As shown, the specific steps are as follows:

[0128] [1] The non-real-time system generates the name and file size of the waveform recording file based on the initial waveform recording information, opens the file to be stored, and obtains the handle fd;

[0129] [2] On the non-real-time system side, mmap is used to map the file descriptor to the memory address fd_va;

[0130] [3] The non-real-time system obtains the fd_pa corresponding to fd_va through the / proc / pid / pagemap file and puts fd_va into the shared memory area shm_1;

[0131] [4] The real-time system obtains fd_pa from the shared memory area shm_1 and maps fd_pa to the process's address space rt_va;

[0132] [5] The real-time system writes compressed grouped sampled data to rt_va from the side, and triggers an inter-core interrupt after the writing is completed;

[0133] [6] After the non-real-time system responds to the inter-core interrupt, it performs a sync operation, refreshes the memory content to the storage medium, and then unmaps the previously mapped fd_va.

[0134] Step S10: When it is necessary to retrieve or view the waveform recording file, the non-real-time system side decompresses the compressed data file corresponding to the storage medium to generate a dynamic waveform recording file or a continuous waveform recording file that meets the standard format.

[0135] When there is a need for external transmission or retrieval, the compressed sdat file in the storage medium is decompressed and converted into a dat file that meets the required format through the dynamic waveform recording file generation module and the continuous waveform recording file generation module, respectively, for transmission. The specific conversion process is as follows:

[0136] [1] First, decompress and mirror the compressed real-time data file stored in the storage medium to the virtual memory file system;

[0137] [2] Take out the decompressed group sampling data and decompress the group sampling data into single-point sampling data;

[0138] [3] Calculate the number of sampling points based on the information in the waveform description file, such as the cfg file. By calculating the number of sampling points for the waveform recording duration, use limited memory to convert the data into the required data format of the waveform recording dat file in batches in streaming mode and store it in the cache.

[0139] [4] The cached content is appended to the temporary recording file in the virtual memory file system one by one, and then forwarded or transferred to the corresponding storage medium in one go through communication.

[0140] In a specific implementation, since there are many sampling points per minute, considering a sampling frequency of 12800Hz, 16 analog channels, and 32 digital channels, the memory size for 60 seconds of sampled data is (16 + 32 / 16) * 2 * 60 * 12800 = 27,648,000 bytes. If converted to binary at once, the temporary waveform recording buffer occupies approximately 28MB of space. If the number of channels increases or the sampling rate is higher, the memory space occupied by the temporary waveform recording buffer may be even larger; if converted to ASCII code, it may require hundreds of megabytes of memory.

[0141] To better utilize memory space, a segmented conversion method is adopted. The temporary waveform recording buffer is set to a fixed size size_min (which can be 1M or smaller). Before the waveform recording file is format converted, the required total space size_total is calculated based on the total number of sampling points in the waveform recording description file. The number of times the file needs to be written in segments n is obtained based on the ratio of size_total to size_min.

[0142] Specifically, to minimize operations on the hardware storage medium, once the cumulative number of converted bytes reaches size_min, the temporary waveform recording buffer is appended to the virtual memory file system. After all sampling points have been converted, the waveform recording .dat file in the virtual memory file system is moved to the storage medium.

[0143] Using a virtual memory file system can improve read performance; during the waveform generation process, the waveform file is written through the virtual memory file system, and the complete waveform file is transferred to the storage medium, reducing the operation on the storage medium.

[0144] In a further embodiment, the method of the present invention further includes: managing and optimizing the storage medium space through a management module on the non-real-time system side.

[0145] To ensure continuous waveform recording over extended periods, at least 72 hours of recording data must be available. Storage media space must be managed accordingly. The used and remaining space on the storage media should be checked periodically. Files older than 3 days should be deleted once the used space exceeds a certain threshold.

[0146] To facilitate analysis, transient event recordings and continuous recordings are placed in separate folders. When analyzing power grid transient events caused by a high proportion of power electronic equipment, the corresponding time period can be found in the transient event recording folder to pinpoint the time of the transient event. When monitoring and analyzing the operating characteristics of power grids with a high proportion of power electronic equipment, the recordings in the continuous recording folder should be consulted.

[0147] Meanwhile, due to the large storage space of the storage medium, and the fact that continuous waveform recordings are stored in files per minute, the number of files increases significantly with storage time. For example, there are 1440 DAT files and 1440 cfg files per day, totaling 2880 files. Using traditional file system retrieval methods, reading the daily file list from the storage medium would take several minutes. To solve this problem, a hierarchical directory structure was implemented, and an index directory file was created for faster searching. The continuous waveform recording folder uses dates as the first-level directory, and the daily directory uses hours as the second-level directory. More specifically, the continuous waveform recording folder on the storage medium was initially divided into several fixed-size folders, labeled with dates, and each daily folder was further divided into folders corresponding to 24 hours. An index directory file was also created in the root directory. When a relevant file needs to be retrieved, the corresponding folder can be found directly using the index directory, with the indexing time for a single file being in the millisecond range.

[0148] The dynamic waveform recording folder is numbered as its first-level directory.

[0149] By rationally partitioning the storage medium and establishing hierarchical and index directories, the file can be quickly located in the storage medium through the index directory when reading the recorded file, and the file is cached in the virtual memory file system, thus optimizing the reading speed.

[0150] In the embodiments of this invention, to save memory usage, reduce system operating pressure, and improve the lifespan of storage media, some designs have been made in data structure design, process design, and file operation. The embodiments of this invention are designed for a sampling rate of 12800Hz, but are not limited to 12800Hz. If the sampling rate is higher than 12800Hz, the shared memory space used will increase. However, due to the use of a segmented, sequential appending method, the buffer used for waveform generation does not change with the number of channels or the sampling rate.

[0151] The embodiments of the present invention are for digital sampling scenarios, but are also applicable to local analog sampling scenarios.

[0152] This invention improves storage medium utilization by storing compressed real-time sampled data. When storage medium space is sufficient, the real-time sampled data from a single point can be directly stored as a standard-format .dat file via a shared file physical address between the two systems, eliminating the need for conversion during use and thus reducing data latency and conversion delay.

[0153] The storage medium involved in this embodiment of the invention is an external solid-state drive, but the storage medium is not limited to a solid-state drive and can be any other scalable storage medium, such as a disk storage device.

[0154] This invention enables multi-core processors to operate in both non-real-time and real-time systems, using shared memory as a medium to acquire initial waveform recording information and file physical addresses. Through rational data structure design and optimized data flow design, it completes the storage of continuous waveform recording files and transient event waveform recording files on the storage medium from real-time data acquisition. By performing dynamic lossless compression on real-time data, it achieves interaction between the two systems with minimal shared memory and good operating performance. By using shared file physical addresses and inter-core interrupts, the real-time system directly writes the compressed real-time data to the corresponding file virtual address, and the non-real-time system synchronizes the data from memory to the storage medium, achieving minimal system overhead, ensuring stable system operation, and solving the problem of high processor performance requirements for continuous waveform recording files. During the waveform recording file conversion process, temporary waveform recording files are generated by caching in the virtual memory file system and appended in stages to reduce the caching required for waveform recording file generation. The temporary waveform recording files are migrated to the storage medium in one go, reducing the operation time on the storage medium. In terms of file reading, the reading speed is optimized by establishing hierarchical directories and index directories, and by transferring files transmitted or converted through communication from the storage medium to the virtual memory file system.

[0155] The present invention has been disclosed above with reference to preferred embodiments, but it is not intended to limit the present invention. All technical solutions obtained by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the present invention.

Claims

1. A data recording method based on a multi-core processor, characterized in that, The method is implemented based on a real-time system and a non-real-time system running on a multi-core processor. The real-time system and the non-real-time system interact using shared memory. The method includes: The real-time system acquires high-frequency and low-frequency sampling data; it identifies transient events based on the high-frequency sampling data to form transient event status identifiers; it compresses and stores the high-frequency and low-frequency sampling data into the high-frequency and low-frequency sampling data areas of the real-time system respectively using a dynamic lossless compression method; based on the transient event status identifiers, it records the pointers of the high-frequency sampling data areas before and after the occurrence of the corresponding transient event to generate dynamic waveform recording initial information; based on the sampling time and the corresponding pointers of the low-frequency sampling data areas, it generates continuous waveform recording initial information; and it transmits the generated dynamic waveform recording initial information and continuous waveform recording initial information to shared memory. The non-real-time system generates corresponding waveform description files based on the initial dynamic waveform recording information and the initial continuous waveform recording information in the shared memory. Based on the waveform description files, a raw waveform recording data file is created, and the address of the raw waveform recording data file is shared to the real-time system via shared memory through a memory-mapped file. The real-time system writes the compressed high-frequency or low-frequency sampled data to the address of the corresponding waveform recording raw data file and triggers an inter-core interrupt. The non-real-time system responds to inter-core interrupts, performs file data synchronization operations, and refreshes virtual memory file data to the storage medium; If it is necessary to read the waveform recording file, the non-real-time system will decompress the compressed data file corresponding to the storage medium to generate a standard format dynamic waveform recording file or continuous waveform recording file.

2. The data recording method based on a multi-core processor according to claim 1, characterized in that, The high-frequency sampling data area and the low-frequency sampling data area form a circular buffer. The dynamic lossless compression method includes: Using multiple consecutive sampling points as a sampling group, based on the differential algorithm, high-frequency sampling data and low-frequency sampling data are compressed in groups according to the increment of data change, and the compressed grouped sampling data is stored in the corresponding circular buffer.

3. The data recording method based on a multi-core processor according to claim 2, characterized in that, The differential algorithm-based method performs group compression on high-frequency and low-frequency sampled data according to the increment of data changes, including: Each sampling group structure includes one descriptor and M sampling channels, each sampling channel being... N 16-bit groups N =2,3..., can be stored N ~2 N -1 sample point data, the descriptor is used to characterize the number of data points in each channel of this sampling group; for each sampling channel, in adjacent 2 N -1 sampling points are grouped together, and data mutation detection is performed based on the difference algorithm; if it is determined that there is no data mutation in M ​​sampling channels, then for each sampling channel, the adjacent 2 N The first sample point in a set of -1 sample points is stored as a 16-bit byte, and the increment of each subsequent sample point is stored as an 8-bit byte, obtained by comparing it with the previous sample point. If it is determined that one or more of the M sample channels have a data mutation, then for each sample channel, ... N Each 16-bit group stores N Data from individual sampling points.

4. The data recording method based on a multi-core processor according to claim 3, characterized in that, The method for determining whether there is a data mutation is as follows: Each sampling channel is adjacent to 2 N -1 sampling points are grouped together for judgment, and the 2 ( ) after the first sampling point data are used. N -1) The data of each sampling point is compared with the data of the previous sampling point in turn. If the absolute value of the data change is less than or equal to the set value, it is considered that there is no data mutation. If the absolute value of the data change is greater than the set value, it is considered that there is a data mutation.

5. The data recording method based on a multi-core processor according to claim 2, characterized in that, The step of decompressing the compressed data file corresponding to the storage medium by the non-real-time system to generate a standard format dynamic waveform file or continuous waveform file includes: The corresponding compressed data file in the storage medium is decompressed and mirrored to the virtual memory file system. The decompressed grouped sampling data is then extracted and decompressed again into single-point sampling data. The total number of sampling points is calculated based on the information in the waveform description file. The data is then segmented and converted into a standard data format. The converted data is appended to a temporary waveform file in the virtual memory file system. The temporary waveform file is then forwarded or transferred to the corresponding storage medium in one go.

6. A data recording device based on a multi-core processor, characterized in that, include: Digital sampling unit and dynamic data recording, processing and management unit, The digital sampling unit is used to receive raw sampling messages from an external network, parse the received raw sampling messages, and transmit the parsed data to the dynamic data processing and management unit through the internal network. The dynamic data recording, processing, and management unit includes an internal sampling message processing module, a data synchronization and interpolation module, a transient event identification module, a data compression module, a data sharing module, a dynamic waveform recording initial information generation module, and a continuous waveform recording initial information generation module located on the real-time system side; and a waveform recording storage module, a data decompression module, a dynamic waveform recording file generation module, and a continuous waveform recording file generation module located on the non-real-time system side. The real-time and non-real-time systems run on different cores of a multi-core processor, and the real-time and non-real-time systems interact using shared memory. The internal sampling message processing module obtains the original sampling message from the internal network bus, parses it, and generates the original sampling data with the sampling sequence number and sampling time. The data synchronization and interpolation module synchronizes the original sampled data and resamples it to generate high-frequency and low-frequency sampled data based on the sampling sequence number of the original sampled data and the external time synchronization signal. The transient event identification module identifies transient events based on high-frequency sampled data and generates transient event status identifiers. The data compression module compresses high-frequency and low-frequency sampled data using a dynamic lossless compression method and stores them separately in the high-frequency sampled data area and the low-frequency sampled data area on the real-time system side. The dynamic waveform recording information generation module records the pointers of the high-frequency sampling data area before and after the occurrence of the corresponding transient event based on the transient event identifier, and generates the initial information of dynamic waveform recording. The continuous waveform recording information generation module generates initial information for continuous waveform recording based on the original sampling time and the pointer of the corresponding low-frequency sampling data area. The data sharing module transmits the initial information of dynamic waveform recording and the initial information of continuous waveform recording to the shared memory; and writes the compressed high-frequency sampling data and low-frequency sampling data to a file through the shared file address and triggers an inter-core interrupt. The waveform recording and storage module generates corresponding waveform recording description files based on the dynamic waveform recording initial information and the continuous waveform recording initial information in the shared memory; it creates a waveform recording raw data file based on the waveform recording description file, and shares the address of the waveform recording raw data file to the real-time system side through shared memory via a memory-mapped file; and it responds to inter-core interrupts, performs file data synchronization operations, and refreshes the virtual memory file data to the storage medium. The data decompression module decompresses the compressed data file corresponding to the storage medium and mirrors it to the virtual memory file system. It then extracts the decompressed high-frequency and low-frequency sampling data, decompresses it a second time to generate high-frequency and low-frequency sampling data with sampling sequence numbers and sampling times, and stores them in the data cache interval in seconds. The dynamic waveform recording file generation module converts the decompressed high-frequency sampling data into a standard format dynamic waveform recording file. The continuous waveform recording file generation module converts the decompressed low-frequency sampling data into a standard format continuous waveform recording file.

7. A data recording device based on a multi-core processor, characterized in that, include: Simulation sampling unit and dynamic data recording, processing and management unit, The analog sampling unit is used to collect the analog signals required for data recording, digitally discretize the analog signals, obtain high-frequency sampling data and low-frequency sampling data, and transmit them to the dynamic data processing and management unit through the internal network. The dynamic data recording, processing, and management unit includes an internal sampling message processing module, transient event identification module, data compression module, data sharing module, dynamic waveform recording initial information generation module, and continuous waveform recording initial information generation module located on the real-time system side; and a waveform recording storage module, data decompression module, dynamic waveform recording file generation module, and continuous waveform recording file generation module located on the non-real-time system. The real-time and non-real-time systems run on different cores of a multi-core processor, and the real-time and non-real-time systems interact using shared memory. The internal sampling message processing module is used to obtain high-frequency sampling data and low-frequency sampling data from the internal network bus, and generate raw sampling data with sampling sequence number and sampling time after parsing. Transient event identification module: Identifies transient events based on high-frequency sampled data and generates transient event status identifiers; The data compression module compresses high-frequency and low-frequency sampled data using a dynamic lossless compression method and stores them separately in the high-frequency sampled data area and the low-frequency sampled data area on the real-time system side. The dynamic waveform recording information generation module records the pointers of the high-frequency sampling data area before and after the occurrence of the corresponding transient event based on the transient event identifier, and generates the initial information of dynamic waveform recording. The continuous waveform recording information generation module generates initial information for continuous waveform recording based on the original sampling time and the pointer to the low-frequency sampling data area. The data sharing module transmits the initial information of dynamic waveform recording and the initial information of continuous waveform recording to the shared memory; and writes the compressed high-frequency sampling data and low-frequency sampling data to a file through the shared file address and triggers an inter-core interrupt. The waveform recording and storage module generates corresponding waveform recording description files based on the dynamic waveform recording initial information and the continuous waveform recording initial information in the shared memory; it creates a waveform recording raw data file based on the waveform recording description file, and shares the address of the waveform recording raw data file to the real-time system side through shared memory via a memory-mapped file; and it responds to inter-core interrupts, performs file data synchronization operations, and refreshes the virtual memory file data to the storage medium. The data decompression module decompresses and mirrors the compressed data file corresponding to the storage medium to the virtual memory file system, extracts the decompressed high-frequency sampling data and low-frequency sampling data, decompresses it a second time to generate high-frequency sampling data and low-frequency sampling data with sampling sequence number and sampling time, and stores them in the data cache interval in seconds. The dynamic waveform recording file generation module converts the decompressed high-frequency sampling data into a standard format dynamic waveform recording file. The continuous waveform recording file generation module converts the decompressed low-frequency sampling data into a standard format continuous waveform recording file.

Images