Millisecond network source data analysis and intelligent optimization system based on high-speed database

Abstract

This invention discloses a millisecond-level network source data analysis and intelligent optimization system based on a high-speed database, belonging to the field of power grid automation and data analysis technology. The system includes a data acquisition module, a database module, an analysis module, a control module, and an execution module. The acquisition module acquires power grid measurement data. The database module adds absolute time stamps to the data and generates data version information and revocation logs. The analysis module uses recursive sliding discrete Fourier transform to extract real-time state features and recalibrates during abrupt changes. The control module generates optimized control commands, establishes a time-anchored association index, and forms a command-data mapping table; when revocation feedback is received, non-blocking logical rollback is implemented based on the mapping table and version information. The execution module issues commands and reports feedback. This invention solves the problems of large high-frequency data write latency and data inconsistency during command revocation, achieving millisecond-level closed-loop optimization and strong data consistency.

Description

Technical Field

[0001] This invention relates to the field of power grid automation and data analysis technology, specifically to a millisecond-level network-source data analysis and intelligent optimization system based on a high-speed database. Background Technology

[0002] With the large-scale grid connection of new energy sources, the requirements for real-time and consistency of data processing in power grid dispatching are increasing. Existing power grid data acquisition and optimization systems typically use relational databases or general-purpose time-series databases to store measurement data, with the analysis engine and control decision-maker deployed independently, achieving data interaction through repeated database reads and writes. This storage-compute separation architecture suffers from significant write latency and query blocking in high-frequency data writing scenarios, and the end-to-end latency from data acquisition to control command issuance is insufficient to meet millisecond-level response requirements.

[0003] On the other hand, dispatch instructions may be revoked during execution. Because there is no unified index linking the status of measurement data records to the progress of dispatch instructions, when an instruction is revoked, the system cannot accurately locate the historical measurement data version corresponding to that instruction in the database, leading to inconsistencies between the data records and the actual physical operation status. While existing technologies include time-window-based data matching methods and multi-version concurrency control-based transaction rollback mechanisms, these are applied to data fusion and database transaction management respectively. No solution has yet been found to integrate these technologies into the synchronous rollback scenario of power grid dispatch instructions and measurement data.

[0004] Therefore, building a system that can simultaneously satisfy millisecond-level closed-loop optimization and data consistency guarantee during instruction revocation is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a millisecond-level network source data analysis and intelligent optimization system based on a high-speed database, so as to solve the problems of large high-frequency data write latency, slow response due to storage and computing separation, and inability to synchronously roll back measurement data when instructions are canceled in the prior art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A millisecond-level network source data analysis and intelligent optimization system based on a high-speed database is applied in a power grid dispatching environment. The system includes a data acquisition module, a database module, an analysis module, a control module, and an execution module, among which: The acquisition module is connected to the phasor measurement unit on the power grid side and is configured to acquire power grid measurement data streams, perform anomaly filtering on the received data, and output a first data set. The database module, connected to the acquisition module, includes a hot memory area and a persistent cold memory area. The database module is configured as follows: Receive the first data set, and attach an absolute time label aligned with the time axis to each measurement data point to form a second data set with time node information; The memory hot zone and the persistent cold zone are maintained in a hierarchical storage manner; wherein the memory hot zone caches the data of the second data set within the most recent preset time period as a real-time copy, and is provided with a shared memory read interface for the analysis module to access; the persistent cold zone is used to persistently store the second data set in compressed form. When performing write or update operations on the second data set, corresponding data version information and reversal logs are generated and maintained; The analysis module is directly connected to the memory hotspot through the shared memory read interface and is configured to read real-time measurement data from the real-time copy in the memory hotspot and extract real-time status features that reflect the power grid operation status. The control module is connected to both the analysis module and the database module, and is configured to receive scheduling instructions issued by the scheduling center. The control module includes: The decision unit is configured to generate optimized control instructions based on the real-time state characteristics, and the decision unit maintains a historical sample set, which records the real-time state characteristics at previous times and the optimized control instructions generated at that time. The association mapping unit is configured to match the target data record of the corresponding time node in the second data set according to the time characteristics of the scheduling instruction, and establish a time anchoring association index between the scheduling instruction and the target data record to form an instruction-data mapping table that can be shared and accessed. The rollback unit is configured to: when receiving the cancellation feedback of the scheduling instruction, locate the target data record according to the instruction-data mapping table, and restore the target data record to the version state before the scheduling instruction was issued based on the data version information and the cancellation log; The execution module is connected to the control module and the controlled equipment on the power grid side, and is configured to send the optimized control command to the controlled equipment and report the execution status returned by the controlled equipment back to the control module.

[0007] Furthermore, the acquisition module includes a frequency monitoring unit and a path selection unit; The frequency monitoring unit is used to count the number of data arrivals per unit time to determine the current frequency value; The path selection unit is used to compare the current frequency value with a first preset threshold, and select a receiving path accordingly: When the current frequency value is greater than the first preset threshold, a lock-free circular queue is enabled to receive the first data set and deliver it to the database module via the queue; otherwise, the first data set is delivered directly to the database module via a straight path; wherein, the lock-free circular queue uses compare-and-swap atomic operations to control the movement of read and write pointers in a multi-threaded environment to avoid mutex lock overhead; The acquisition module uses an outlier detection algorithm to perform anomaly filtering on the received data and remove abnormal measurement values.

[0008] Furthermore, the capacity of the memory hotspot is fixed as the total data within the last N milliseconds, where N is a preset positive integer; The persistent cold zone employs an adaptive compression strategy oriented towards power frequency waveforms, the compression strategy including: The data difference is calculated based on the measurement data corresponding to adjacent absolute time tags in the second dataset; When the data difference is less than or equal to a preset steady-state threshold, it is determined to be steady-state data, and dead-zone compression combined with run-length encoding is used. When the data difference is greater than a preset transient threshold, it is determined to be transient data and stored using Fourier coefficients.

[0009] Furthermore, the shared memory read interface is configured to provide zero-copy access to the memory hotspot; The analysis module directly reads the data in the memory hotspot through the shared memory pointer of the shared memory read interface to reduce serialization and copy latency.

[0010] Furthermore, the analysis module includes a feature extraction unit and a state feature extraction unit, wherein: The feature extraction unit is configured to use a recursive sliding discrete Fourier transform algorithm, with a sliding time window as the unit, to extract the amplitude and phase of the fundamental wave and each harmonic from the real-time measurement data as electrical feature quantities, wherein there is a recursive relationship between adjacent time windows. The state feature extraction unit has a built-in lightweight classification model for classifying the electrical feature quantities and outputting real-time state features that reflect the power grid's operating status.

[0011] Furthermore, the feature extraction unit also includes a mutation detection subunit, configured as follows: Based on the real-time measurement data, calculate the rate of change of voltage or current within the current time window; When the rate of change exceeds a preset mutation threshold, the feature extraction unit is triggered to pause the use of the result of the recursive sliding discrete Fourier transform algorithm, and re-execute the discrete Fourier transform calculation based on the complete sampled data within the current time window, so as to use the calculation result to cover or calibrate the electrical feature quantity of the current output.

[0012] Furthermore, the instruction-data mapping table generated by the association mapping unit resides in the memory hot zone in the form of key-value pairs; wherein, the time-anchored association index is specifically implemented as follows: the key includes the hash value obtained by hashing the identification information and time information of the scheduling instruction, and the value is the record version information or location information pointing to the target data record; The association mapping unit further includes a conflict handling subunit, which is configured to organize conflicting items using chaining when the hash processing generates key conflicts, in order to maintain constant-level expected query performance. The instruction-data mapping table can be used by the decision unit to query the status of associated measurement data during the generation of optimization control instructions, and can be used by the rollback unit to locate the record to be rolled back when the rollback unit receives the revocation feedback.

[0013] Furthermore, the rollback unit includes a positioning subunit and a reversal subunit, wherein: The positioning subunit is used to obtain the index of the target data record according to the instruction-data mapping table, and locate the previous version data from the revocation log according to the position indicated by the data version information; The reversal subunit is configured to: point the current version pointer of the target data record to the previous version data, and modify the corresponding commit flag to achieve logical rollback, and the logical rollback process does not block concurrent read operations on other data records.

[0014] Furthermore, the decision-making unit includes a reinforcement learning decision-maker, which is configured to: Reward data is calculated based on the degree of deviation between the real-time status characteristics and the preset optimization target; the preset optimization target includes at least one of voltage deviation, network loss, and harmonic distortion rate. The reward data is associated with the corresponding real-time status features and the optimization control instructions and stored in the historical sample set; Data is read from the historical sample set using a periodic sampling method to update the decision-maker parameters online; The online update employs a dual-buffer hot-switching mechanism, ensuring that the output of control commands is not interrupted during the update process.

[0015] Furthermore, the execution module also includes a filtering unit and a dead-time determination unit, wherein: The filtering unit is used to perform first-order low-pass filtering on the optimized control command; The dead zone determination unit is used to prohibit issuing commands to the controlled device when the change in the optimized control command is less than a preset dead zone threshold.

[0016] Compared with existing technologies, this invention has achieved significant progress in millisecond-level response speed, data consistency guarantee, and online adaptive optimization, and has the following beneficial effects: I. By setting up memory hotspots and configuring shared memory read interfaces, the analysis module can directly access real-time measurement data with zero copying, eliminating serialization and network transmission delays in traditional storage-compute separation architectures, and significantly reducing the response time from data acquisition to feature extraction.

[0017] Second, by establishing a time-anchored association index between scheduling instructions and measurement data in the control module, an instruction data mapping table is formed, which enables the decision-making unit to quickly query the associated measurement status during the execution phase, and the rollback unit to accurately locate the data record to be rolled back when the instruction is canceled.

[0018] Third, by maintaining data version information and revocation logs, and by implementing logical rollback of target data records based on a multi-version concurrency control mechanism, the measurement data can be restored to the version state before the instruction was issued after the instruction is revoked, thus ensuring the consistency of data and instructions during power grid dispatching.

[0019] Fourth, the decision-making unit of this invention maintains a historical sample set internally and uses a dual-buffer hot-switching technology to achieve online updates, so that the optimization strategy can be continuously improved according to the real-time state of the power grid without interrupting the output of control commands. Attached Figure Description

[0020] Figure 1 This is a system block diagram of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0022] This embodiment provides a millisecond-level network source data analysis and intelligent optimization system based on a high-speed database, applied in a power grid dispatching environment. (See attached document.) Figure 1 The system includes a data acquisition module, a database module, an analysis module, a control module, and an execution module.

[0023] This embodiment uses a grid-connected point equipped with a photovoltaic power station and an energy storage system as an example to illustrate the specific implementation of the millisecond-level grid-source data analysis and intelligent optimization system based on a high-speed database provided by this invention. The acquisition module is connected to the phasor measurement unit (PMU) deployed at the grid-connected point. The PMU sampling rate is set to 1000 Hz, meaning it generates a set of instantaneous three-phase voltage and current values ​​every 1 millisecond. The database module, analysis module, and control module are deployed on the same industrial control computer, equipped with 16GB of memory and an NVMe solid-state drive, running a customized time-series database engine and real-time analysis program. The execution module connects to the controlled equipment on the grid side, such as energy storage converters, via the GOOSE communication protocol. This co-location ensures that memory hotspots can be directly accessed by the analysis module through shared memory pointers.

[0024] The acquisition module is connected to the phasor measurement unit (PMU) on the power grid side, such as... Figure 1 As shown, it is configured to acquire power grid measurement data streams, perform anomaly filtering on the received data, and output a first data set. Specifically, the acquisition module continuously acquires power grid measurement data streams from the PMU. Let the data packet at the i-th sampling time be... ,in These are the instantaneous values ​​of the three-phase voltage. These are the instantaneous values ​​of the three-phase current. The original timestamp provided for the PMU.

[0025] The acquisition module includes a frequency monitoring unit and a path selection unit. The frequency monitoring unit is used to count the number of data arrivals per unit time to determine the current frequency value. In this embodiment, this unit counts the number of data packets arriving per second. Let C be the number of data packets received in the current second, then the current frequency value f = C / 1 second. A first preset threshold is set. Set it to 20000 Hz.

[0026] The path selection unit is used to compare the current frequency value f with a first preset threshold. The comparison is performed, and the receiving path is selected accordingly, specifically as follows: When the current frequency value is greater than the first preset threshold, that is The system enables a lock-free circular queue to receive the first data set and delivers it to the database module via the queue. When the current frequency value is less than or equal to the first preset threshold, i.e. The first data set is delivered directly to the database module via a direct path; The lock-free circular queue employs compare-and-swap (CAS) atomic operations to control the movement of read and write pointers in a multi-threaded environment, thus avoiding the overhead of mutex locks. The lock-free circular queue is implemented by allocating contiguous space in memory and initializing both the read and write pointers to 0. In a multi-threaded environment, write threads use CAS to move the write pointer forward, and read threads use CAS to move the read pointer forward, thereby avoiding the context switching overhead caused by mutex locks. Data processed by this queue or data passing through a direct path is ultimately delivered to the write entry point of the database module.

[0027] The acquisition module employs an outlier detection algorithm to filter out abnormal measurements from the received data. For each newly arrived data packet, the rolling mean and standard deviation of its three-phase voltage instantaneous values ​​are calculated. If any phase voltage value deviates from the mean by more than three times the standard deviation, it is determined to be an abnormal measurement and is removed from further processing. The filtered data forms the first dataset. .

[0028] The database module, connected to the acquisition module, includes a hot memory area and a persistent cold memory area. The database module is configured as follows: (1) Receive the first data set Each measurement data point is then appended with an absolute time label aligned with the time axis, forming a second dataset with time node information. Specifically, for For each measurement data point, the database module calls a high-precision timestamp generation function to append locally synchronized Coordinated Universal Time (UTC) as an absolute time stamp to the data. The original sampling time of a certain measurement data point is... The absolute time after synchronization via Network Time Protocol is Then the data entries after appending absolute time labels are All data entries with attached absolute time tags constitute the second data set described above. .

[0029] (2) The memory hot zone and the persistent cold zone are maintained in a hierarchical storage manner; wherein the memory hot zone caches the data of the second data set within the most recent preset time period as a real-time copy, and is provided with a shared memory read interface for the analysis module to access; the persistent cold zone is used to persistently store the second data set in a compressed form.

[0030] The memory hotspot employs a circular buffer structure with a fixed capacity equal to the total data within the last N milliseconds; in this embodiment, N is set to 100 milliseconds. When a new data entry is written, the memory hotspot automatically overwrites the oldest data exceeding the 100-millisecond time window, caching the second set of data as a real-time copy. The memory hotspot also exposes a shared memory read interface to the analysis module. This interface, implemented through a memory-mapped file, is configured to provide zero-copy access to the memory hotspot. The analysis module can directly read the data stored in the memory hotspot by obtaining a shared memory pointer. To prevent read / write conflicts, the memory hotspot uses a copy-on-write or version counter mechanism: the write thread increments the version number when updating data, and the analysis module checks the version number consistency before and after reading to ensure that a complete single record is read without requiring a copy operation in kernel space. The version number verification is performed on a single data record. When the analysis module reads the metadata (including the version number) of a record, it then reads the payload of that record. If a version number change is detected during the reading process, the read result is discarded and the process is retried.

[0031] The persistent cold storage area adopts a columnar storage format and implements an adaptive compression strategy for power frequency waveforms. The database module calculates the data difference based on the measurement data corresponding to adjacent absolute time stamps in the second data set. Taking a voltage signal as an example, suppose two consecutive sampling times... and The corresponding effective voltage values ​​are respectively and Calculate the difference .

[0032] like Data less than or equal to a preset steady-state threshold, such as 0.5 volts, is considered steady-state data. Dead-zone compression combined with run-length encoding is used to record only data points whose changes exceed the dead-zone threshold, and the number of repetitions of continuous, unchanging data segments is recorded.

[0033] like Data exceeding a preset transient threshold, such as 5 volts, is considered transient data and stored using Fourier coefficients. The waveform within this data segment is then subjected to a Fourier transform, retaining the amplitude and phase of the fundamental wave and harmonics up to the 25th order as stored content.

[0034] (3) When performing write or update operations on the second data set, generate and maintain corresponding data version information and rollback logs. The version information includes a transaction identifier and a version creation timestamp, and the rollback log records the complete old value of the data record before this write and the type of change operation. This information is stored in a separate log file in an append-only manner to provide a basis for subsequent rollback operations.

[0035] The analysis module, directly connected to the memory hotspot via the shared memory read interface, is configured to read real-time measurement data from the real-time replica in the memory hotspot and extract real-time status features reflecting the power grid's operating state. The analysis module obtains a shared memory pointer pointing to the latest data in the memory hotspot through the shared memory read interface and directly reads the real-time measurement data from the real-time replica. The analysis module includes a feature extraction unit and a status feature extraction unit.

[0036] The feature extraction unit processes data in units of sliding time windows. Let the time window width W = 20 milliseconds, and the sliding step size... Milliseconds. Within each window, the feature extraction unit performs a recursive sliding discrete Fourier transform to extract the amplitude and phase of the fundamental frequency and each harmonic from the real-time measurement data as electrical characteristics, where there is a recursive relationship between adjacent time windows. For a 50 Hz power frequency signal, the fundamental angular frequency... Let the voltage sampling sequence within the m-th window be... ,in N is the number of sampling points within a window. The formula for calculating the fundamental phasor of the Discrete Fourier Transform is: Where j is the imaginary unit, satisfying ; The fundamental angular frequency; The sampling period is 1 millisecond in this embodiment.

[0037] Due to the sliding step size milliseconds, sampling period Milliseconds, therefore there are 10 sampling points between adjacent windows ( Data overlap. Recursive relationships utilize the data overlap between adjacent windows: in, This represents the new sampling point that slides into the m-th window. This indicates the old sampling point that slid out. The fundamental integer rotation factor, This is the window duration.

[0038] This recursive calculation reduces the computational complexity of harmonic phasors for each window from... Reduce to There is a recursive relationship between adjacent time windows. The feature extraction unit outputs the fundamental effective value, the third harmonic content, and the fifth harmonic content as electrical characteristic quantities every 2 milliseconds.

[0039] The feature extraction unit also includes a mutation detection subunit. This subunit calculates the rate of change of voltage within the current time window in real time, specifically by taking the fundamental RMS values ​​of two consecutive windows, such as the voltage RMS value. and Calculate the rate of change .when When the mutation threshold is exceeded, such as 0.2 volts per millisecond, the mutation detection subunit triggers a calibration mechanism: suspends the use of the results of the recursive sliding discrete Fourier transform algorithm, re-executes the standard FFT calculation based on the complete sampled data within the current time window, uses the calculation result to overwrite or calibrate the electrical characteristic quantity of the current output, and resets the initial state of the recursive algorithm to ensure the accuracy of subsequent recursions.

[0040] The state feature extraction unit incorporates a lightweight classification model; in this embodiment, the extreme gradient boosting tree model is used. This model takes the fundamental effective value, third harmonic content, fifth harmonic content, and voltage change rate as input features, and outputs probabilities for three categories: normal operation, transient disturbance, and fault. The state feature extraction unit merges the classification results with electrical feature quantities to form a real-time state feature vector. ,in Total harmonic distortion (THD) The rate of change of frequency, This represents the failure probability. This vector reflects the real-time state characteristics of the power grid's operation.

[0041] The control module is connected to both the analysis module and the database module, and is configured to receive scheduling instructions from the scheduling center. The control module includes a decision-making unit, an association mapping unit, and a rollback unit. In this embodiment, a scheduling instruction is set to "The energy storage system discharges at a rate of 0.2C for 5 seconds," and the instruction issuance time is... The instruction identifier is CMD_001.

[0042] The decision unit is configured to generate optimized control instructions based on the real-time state characteristics, and the decision unit maintains a historical sample set, which records the real-time state characteristics at previous times and the optimized control instructions generated at that time.

[0043] The decision-making unit within the control module includes a reinforcement learning decision-maker. In this embodiment, the reinforcement learning decision-maker employs a dual-delay deep deterministic policy gradient algorithm. The decision-maker receives the real-time state feature vector F from the analysis module, inputs F into the current policy network, and outputs optimized control commands. ,in For energy storage charging and discharging power commands, This refers to the power command for the reactive power compensation device. The decision-making unit also maintains a historical sample set to store the real-time state characteristics of previous moments, the optimized control commands generated at those moments, and the reward data calculated based on the actual control effect. The formula for calculating the reward data R is: in, and These are reference values ​​for voltage and frequency, respectively. and These are measured values; THD is the total harmonic distortion rate, expressed as a percentage, such as 5% is represented by 5. The rated voltage is 220V in this embodiment; The rated frequency is 50Hz in this embodiment; This represents the corresponding penalty weight coefficient. In this embodiment, let... =0.01 (corresponding to the penalty for every 1% deviation). =0.5, =0.2. Through the above normalization process, physical quantities with different dimensions are unified into dimensionless percentage values, ensuring the convergence stability of the reinforcement learning model under multi-objective optimization. Through the above weighted summation, the multi-dimensional power grid state is mapped to a single reward scalar, which guides the policy optimization of the reinforcement learning model.

[0044] After each five control decisions, the decision maker randomly samples a batch of data from the historical sample set to update the gradients of the policy network and the evaluation network. The update process employs a dual-buffer hot-swap technique: the system maintains two model parameter buffers in memory, one primary and one backup. The inference thread always points to the primary buffer for real-time decision-making; once sufficient data has accumulated in the historical sample set, the training thread completes gradient updates and parameter refreshes in the backup buffer, and then atomically switches the model pointer to the backup buffer, marking the original primary buffer as the new backup buffer. This process ensures that the inference thread does not interrupt the output of control commands during the microsecond-level switching.

[0045] The association mapping unit is configured to match the target data record of the corresponding time node in the second data set according to the time characteristics of the scheduling instruction, and establish a time anchoring association index between the scheduling instruction and the target data record to form an instruction-data mapping table that can be shared and accessed.

[0046] Specifically, the association mapping unit determines the time characteristics of the scheduling instructions. In the second data set The system matches target data records with corresponding time nodes. Specifically, the association mapping unit uses... Set a time window centered on the time window. ,exist The filter selects all measurement data records whose time stamps fall within this window, and takes the record with the highest correlation to the instruction identifier as the target data record, whose storage location is denoted as [location to be filled in]. .

[0047] The association mapping unit then establishes a time-anchored association index between the scheduling instruction and the target data record. This is done by linking the instruction identifier CMD_001 with the time tag. The strings are concatenated, and a hash value is calculated using the SHA-256 secure hash algorithm. The first 64 bits are then used as the key. This key is used as the key for key-value pairs, and the target data record's storage address is used as the key. The record's version chain pointer is used as the value to generate a key-value pair entry. All key-value pair entries reside in the memory hotspot as key-value pairs, forming an instruction-data map table. When hash key collisions occur, the collision handling subunit uses chaining to organize multiple colliding entries into a linked list within the same hash bucket to maintain constant-level expected query performance. The instruction-data map table can be used by the decision unit to query the status of associated measurement data during the generation of optimization control instructions, and is used by the rollback unit to locate the record to be rolled back when it receives reversal feedback.

[0048] The rollback unit is configured to: upon receiving a cancellation feedback for the scheduling instruction, locate the target data record according to the instruction-data mapping table, and restore the target data record to its version state before the scheduling instruction was issued, based on the data version information and the cancellation log. The rollback unit continuously monitors the execution status feedback reported by the execution module. When the rollback unit receives cancellation feedback for instruction CMD_001, it performs the following operations: The location subunit looks up the corresponding value in the instruction-data map table using the hash key CMD_001 to obtain the storage address of the target data record. and version chain pointer; The positioning subunit reads the old version of the target data record from the reversal log, prior to the issuance of instruction CMD_001, based on the location indicated by the data version information. The reversal log records the transaction identifier. and a complete snapshot of the corresponding data rows; The reversal sub-unit executes a logic rollback: it sets the version pointer of the current data row to the previous version data in the undo log, and at the same time modifies the transaction commit flag of the data row to the uncommitted state.

[0049] This process is achieved by modifying the internal transaction management structure of the database, without overwriting actual physical data. The logical rollback operation uses a non-blocking method, allowing other concurrent read requests to continue accessing other versions of the same data row without affecting the system's normal data services.

[0050] The execution module is connected to the control module and the controlled equipment on the power grid side, and is configured to send the optimized control command to the controlled equipment and report the execution status returned by the controlled equipment back to the control module.

[0051] The execution module receives the optimization control instructions output by the decision-making unit. The execution module internally includes a filtering unit and a dead-time detection unit. The filtering unit performs a first-order low-pass filter on the command value. Let the currently received active power command be... The actual instruction issued at the previous moment was Filtered instructions ,in This represents the filter coefficient, such as 0.3. The dead-time determination unit calculates the current filtered instruction. Compared with the last time the instruction was actually issued The absolute value of the difference. If the difference is less than the preset dead zone threshold (set to 0.01 MW in this embodiment), the grid state is determined to be stable, and new commands are prohibited from being issued to the energy storage converter to avoid frequent equipment wear due to quantization errors or small fluctuations; otherwise, the command issuance process is triggered. The execution module transmits the command via the GOOSE protocol. and The data is sent to the energy storage converter, and the execution status feedback returned by the energy storage converter, including the actual response power, response timestamp, and execution success flag, is packaged and reported to the control module.

[0052] In summary, this invention constructs a complete millisecond-level closed-loop optimization system through dynamic path selection in the acquisition module, hierarchical storage and version management in the database module, recursive sliding discrete Fourier transform (DFT) and mutation calibration in the analysis module, and time-anchored association index and logical rollback based on multi-version concurrent control (MVCC) in the control module.

[0053] In simulation tests of grid-connected points equipped with photovoltaic and energy storage systems, the system simulated a high-frequency write scenario with multiple phasor measurement units connected, achieving a total data throughput of 50,000 points per second. Test data showed that the end-to-end latency from grid-side sampling to the execution module issuing optimized control commands was approximately 45 milliseconds; when a scheduling command was revoked, the rollback unit, based on the command-data mapping table and the revocation log, completed the logical rollback of the target data record in approximately 30 milliseconds. This process did not block concurrent read operations on other data records, verifying the system's ability to ensure data consistency in high-frequency scenarios.

[0054] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0055] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0056] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0057] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0058] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A millisecond-level network source data analysis and intelligent optimization system based on a high-speed database, characterized in that: Applied to power grid dispatching environments, this system includes a data acquisition module, a database module, an analysis module, a control module, and an execution module, among which: The acquisition module is connected to the phasor measurement unit on the power grid side and is configured to acquire power grid measurement data streams, perform anomaly filtering on the received data, and output a first data set. The database module, connected to the acquisition module, includes a hot memory area and a persistent cold memory area. The database module is configured as follows: Receive the first data set, and attach an absolute time label aligned with the time axis to each measurement data point to form a second data set with time node information; The memory hot zone and the persistent cold zone are maintained in a hierarchical storage manner; wherein the memory hot zone caches the data of the second data set within the most recent preset time period as a real-time copy, and is provided with a shared memory read interface for the analysis module to access; the persistent cold zone is used to persistently store the second data set in compressed form. When performing write or update operations on the second data set, corresponding data version information and reversal logs are generated and maintained; The analysis module is directly connected to the memory hotspot through the shared memory read interface and is configured to read real-time measurement data from the real-time copy in the memory hotspot and extract real-time status features that reflect the power grid operation status. The control module is connected to both the analysis module and the database module, and is configured to receive scheduling instructions issued by the scheduling center. The control module includes: The decision unit is configured to generate optimized control instructions based on the real-time state characteristics, and the decision unit maintains a historical sample set, which records the real-time state characteristics at previous times and the optimized control instructions generated at that time. The association mapping unit is configured to match the target data record of the corresponding time node in the second data set according to the time characteristics of the scheduling instruction, and establish a time anchoring association index between the scheduling instruction and the target data record to form an instruction-data mapping table that can be shared and accessed. The rollback unit is configured to: when receiving the cancellation feedback of the scheduling instruction, locate the target data record according to the instruction-data mapping table, and restore the target data record to the version state before the scheduling instruction was issued based on the data version information and the cancellation log; The execution module is connected to the control module and the controlled equipment on the power grid side, and is configured to send the optimized control command to the controlled equipment and report the execution status returned by the controlled equipment back to the control module.

2. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that: The acquisition module includes a frequency monitoring unit and a path selection unit; The frequency monitoring unit is used to count the number of data arrivals per unit time to determine the current frequency value; The path selection unit is used to compare the current frequency value with a first preset threshold, and select a receiving path accordingly: When the current frequency value is greater than the first preset threshold, a lock-free circular queue is enabled to receive the first data set and deliver it to the database module via the queue; otherwise, the first data set is delivered directly to the database module via a straight path; wherein, the lock-free circular queue uses compare-and-swap atomic operations to control the movement of read and write pointers in a multi-threaded environment to avoid mutex lock overhead; The acquisition module uses an outlier detection algorithm to perform anomaly filtering on the received data and remove abnormal measurement values.

3. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that: The capacity of the memory hotspot is fixed as the total data within the last N milliseconds, where N is a preset positive integer; The persistent cold zone employs an adaptive compression strategy oriented towards power frequency waveforms, the compression strategy including: The data difference is calculated based on the measurement data corresponding to adjacent absolute time tags in the second dataset; When the data difference is less than or equal to a preset steady-state threshold, it is determined to be steady-state data, and dead-zone compression combined with run-length encoding is used. When the data difference is greater than a preset transient threshold, it is determined to be transient data and stored using Fourier coefficients.

4. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that: The shared memory read interface is configured to provide zero-copy access to the memory hotspot; The analysis module directly reads the data in the memory hotspot through the shared memory pointer of the shared memory read interface to reduce serialization and copy latency.

5. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that, The analysis module includes a feature extraction unit and a state feature extraction unit, wherein: The feature extraction unit is configured to use a recursive sliding discrete Fourier transform algorithm, with a sliding time window as the unit, to extract the amplitude and phase of the fundamental wave and each harmonic from the real-time measurement data as electrical feature quantities, wherein there is a recursive relationship between adjacent time windows. The state feature extraction unit has a built-in lightweight classification model for classifying the electrical feature quantities and outputting real-time state features that reflect the power grid's operating status.

6. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 5, characterized in that, The feature extraction unit further includes a mutation detection subunit, configured as follows: Based on the real-time measurement data, calculate the rate of change of voltage or current within the current time window; When the rate of change exceeds a preset mutation threshold, the feature extraction unit is triggered to pause the use of the result of the recursive sliding discrete Fourier transform algorithm, and re-execute the discrete Fourier transform calculation based on the complete sampled data within the current time window, so as to use the calculation result to cover or calibrate the electrical feature quantity of the current output.

7. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that: The instruction-data mapping table generated by the association mapping unit resides in the memory hot zone in the form of key-value pairs; wherein, the time anchor association index is specifically implemented as follows: the key includes the hash value obtained by hashing the identification information and time information of the scheduling instruction, and the value is the record version information or location information pointing to the target data record; The association mapping unit further includes a conflict handling subunit, which is configured to organize conflicting items using chaining when the hash processing generates key conflicts, in order to maintain constant-level expected query performance. The instruction-data mapping table can be used by the decision unit to query the status of associated measurement data during the generation of optimization control instructions, and can be used by the rollback unit to locate the record to be rolled back when the rollback unit receives the revocation feedback.

8. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that, The rollback unit includes a positioning subunit and a reversal subunit, wherein: The positioning subunit is used to obtain the index of the target data record according to the instruction-data mapping table, and locate the previous version data from the revocation log according to the position indicated by the data version information; The reversal subunit is configured to: point the current version pointer of the target data record to the previous version data, and modify the corresponding commit flag to achieve logical rollback, and the logical rollback process does not block concurrent read operations on other data records.

9. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that, The decision-making unit includes a reinforcement learning decision-maker, which is configured to: Reward data is calculated based on the degree of deviation between the real-time status characteristics and the preset optimization target; the preset optimization target includes at least one of voltage deviation, network loss, and harmonic distortion rate. The reward data is associated with the corresponding real-time status features and the optimization control instructions and stored in the historical sample set; Data is read from the historical sample set using a periodic sampling method to update the decision-maker parameters online; The online update employs a dual-buffer hot-switching mechanism, ensuring that the output of control commands is not interrupted during the update process.

10. The millisecond-level network source data analysis and intelligent optimization system based on a high-speed database according to claim 1, characterized in that, The execution module further includes a filtering unit and a dead-time determination unit, wherein: The filtering unit is used to perform first-order low-pass filtering on the optimized control command; The dead zone determination unit is used to prohibit issuing commands to the controlled device when the change in the optimized control command is less than a preset dead zone threshold.

Images