A multi-source information access method and system based on real-time microclimate data

Abstract

The application relates to the technical field of power grids, and particularly discloses a multi-source information access method and system based on real-time micro-meteorological data, which comprises the following steps: acquiring time sequence characteristics of multiple data sources in a power grid environment; dynamically calculating the allowed duration of each data source; calculating the data aging duration of each data source; dividing the state of each data source into one of the following states: newly-arrived valid, continuously valid and overdue invalid; if there is an overdue invalid data source, extracting valid data in the newly-arrived valid and continuously valid data sources and integrating and outputting the valid data; otherwise, extracting valid data in each data source and integrating and outputting the valid data; when the integrated output is obtained, attaching a multi-dimensional state identifier to the data of each data source; and adaptively adjusting the time sequence characteristics and the allowed duration of each data source according to the state of each data source. The application can avoid actual delay caused by waiting for a low-frequency data source and can also avoid missing of key information caused by abandoning due to timeout.

Description

Technical Field

[0001] This invention relates to the field of power grid technology, and in particular to a multi-source information access method and system based on real-time micro-meteorological data. Background Technology

[0002] In the power grid operating environment, to achieve the access and integration of multi-source information, the system typically triggers integration operations at a preset time interval, checking the latest arriving messages from each data source within the period. When new messages arrive from designated data sources such as substation-built micro-weather stations, radar networks, and regional weather stations, the system packages and outputs the data to form a real-time data stream for use by the early warning model. Under normal conditions without severe convective weather, the system maintains a relatively long integration cycle, the update interval of each data source is usually shorter than this cycle, the cycle readiness judgment logic can be normally satisfied, and the data stream is output stably at a fixed pace.

[0003] However, when severe convective weather approaches, early warning models require a significant reduction in the data integration and refresh cycle from minutes to seconds to capture rapidly changing convective characteristics. At this point, the condition in the cycle readiness judgment logic—that "all data sources have arrived with the latest data"—begins to face challenges. Because the data update cycles of low-frequency data sources such as radar and regional weather stations are much longer than the shortened integration window, these data sources will not receive new data within the vast majority of the integration windows.

[0004] Existing conventional methods typically rely on preset timeout rules. One approach is to continue waiting for low-frequency data sources until their new data arrives, which can lead to actual delays of several minutes in the integration process. The real-time output stream degenerates into a non-periodic, long-delayed output, severely impacting the timeliness of early warnings. Another approach is to immediately abandon the wait after the timeout, resulting in an incomplete stream containing only real-time data from substations and omitting radar echo characteristics. This leads to the loss of crucial information and undermines the integrity of subsequent early warnings. This readiness determination and timeout strategy based on fixed time windows is ill-suited to the significant difference between the requirements for accelerated early warning and the inherent update frequency of data sources, failing to balance timeliness with information integrity. Summary of the Invention

[0005] This invention proposes a multi-source information access method and system based on real-time micro-meteorological data, aiming to solve the technical problems that existing technologies struggle to ensure both timeliness and information integrity during the multi-source information access and integration process when severe convective weather is approaching in the power grid operation environment. Furthermore, the fixed time window readiness judgment and timeout strategies cannot adapt to the huge difference between the requirements for accelerated early warning and the inherent update frequency of the data source.

[0006] In a first aspect, the present invention provides a multi-source information access method based on real-time micro-meteorological data, applied to the multi-source information access and integration during the approach of severe convective weather in the power grid operation environment. The method includes the following steps: The time-series characteristics of multiple data sources in the power grid environment are obtained. The data sources include regional meteorological stations, radar networks, and self-built micro meteorological stations in substations. The time-series characteristics include the average update cycle and the standard deviation of the update interval. Based on the aforementioned timing characteristics, the allowable connection duration for each data source is dynamically calculated; The observation timestamp of the latest data from each data source is calculated, and the time difference between it and the integrated target time of the second-level tick driven by the severe convective weather warning model is used as the data aging duration for the corresponding data source. Compare the data aging time of each data source with the corresponding allowed acceptance time, and combine this with whether new data has arrived within the current integration cycle to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, or expired and invalid. If there are expired data sources, replace the data from the expired data sources with the preset missing data markers, and extract the valid data from the newly arrived valid and the continued valid data sources and integrate them into the output; otherwise, extract the valid data from each data source and integrate them into the output. When integrating the output, a multi-dimensional status identifier is attached to the data from each data source. The multi-dimensional status identifier includes a participation mode identifier to indicate the data source type, a time decay attribute to indicate the staleness of the data, and a boundary margin to indicate the remaining valid time of the data. Based on the status of each data source in the integration output of each integration cycle, the timing characteristics and allowable duration of each data source are adaptively adjusted.

[0007] According to some embodiments of the present invention, the timing features are obtained through the following process: Maintain a fixed-length historical time window for each data source and record the observation timestamps of the most recent preset number of message arrivals for the corresponding data source; Calculate the difference between two adjacent observation timestamps to obtain the update interval; Based on the update interval, the average update cycle and the standard deviation of the update interval are calculated as the time-series characteristics of the corresponding data source.

[0008] According to some embodiments of the present invention, the step of calculating the time difference between the observation timestamp of the latest data from each data source and the integrated target time of the second-level tick driven by the severe convective weather warning model, as the data aging duration of the corresponding data source, includes: Extract the observation timestamp and arrival timestamp of the latest data from the cache queues of each data source; The time difference between the observation timestamp and the integrated target time is calculated and used as the data aging duration of the corresponding data source; wherein, the integrated target time is continuously generated by the severe convective weather warning model at a second-level interval and is used as the time point at which the downstream warning model expects to obtain a complete meteorological snapshot; The network transmission delay is determined based on the difference between the arrival timestamp and the observation timestamp. The corrected data aging time is obtained by subtracting the network transmission delay from the data aging time.

[0009] According to some embodiments of the present invention, the step of comparing the data aging time of each data source with the corresponding allowed acceptance time, and combining this with whether new data has arrived within the current integration cycle, to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired invalid, includes: If new data arrives during the current integration cycle and the corresponding data aging time does not exceed the corresponding allowed acceptance time, then the status of the corresponding data source will be classified as newly arrived and valid. If no new data arrives during the current integration cycle and the corresponding data aging time does not exceed the corresponding allowed acceptance time, then the status of the corresponding data source will be classified as continued valid. If the corresponding data aging time exceeds the corresponding allowed acceptance time, the corresponding data source will be classified as expired.

[0010] According to some embodiments of the present invention, the step of comparing the data aging time of each data source with the corresponding allowed acceptance time, and combining this with whether new data has arrived within the current integration cycle, to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired invalid, further includes: A register is maintained for each data source to record the observation timestamp of the latest valid physical observation. When a new message arrives, if the data aging time of the corresponding data source exceeds the corresponding allowed acceptance time, and the observation timestamp of the new message is later than the observation timestamp recorded in the register, then a marker for a late valid physical frame is added to the status of the corresponding data source as an identifier to trigger asynchronous backtracking compensation processing in subsequent steps.

[0011] According to some embodiments of the present invention, the steps of replacing expired data from data sources with preset missing identifiers and extracting valid data from newly arrived valid and continuing valid data sources for integrated output include: Replace the data from data sources that are in an expired or invalid state with a preset missing identifier, and extract the valid data from data sources that have just arrived in a valid state and data sources that continue to be valid, as the real-time integrated data stream for the current integration cycle. If any data source in the expired state has the tag of the late valid physical frame, then asynchronous backtracking compensation processing is triggered. The asynchronous backtracking compensation processing includes: Using the observation timestamp of the data source marked with the late valid physical frame as the search keyword, the corresponding historical integration target time is located from the historical integration output; wherein, the historical integration output is: the integration data stream generated in each integration cycle before the current integration cycle and the corresponding integration target time; Extract valid data from data sources other than the data source that generated the late valid physical frame at the target historical integration time, align and integrate it with the data of the late valid physical frame, and generate a historical compensation integration data packet; The historical compensation integration data packet is then asynchronously inserted into the output queue of the current integration cycle and pushed in combination with the real-time integration data stream.

[0012] According to some embodiments of the present invention, the step of extracting and integrating valid data from various data sources includes: Using the second-level tick as a step size, the latest data is extracted from the cache queue of the data source in the newly arrived valid state and marked as read, and the data that was last read is extracted from the data source in the continued valid state. The extracted data from each data source is assembled into an integrated data package according to a predetermined data model, and output as the real-time integrated data stream for the current integration cycle.

[0013] According to some embodiments of the present invention, the step of attaching a multi-dimensional status identifier to the data from each data source, wherein the multi-dimensional status identifier includes a participation mode identifier for indicating the data source type, a time decay attribute for indicating the staleness of the data, and a margin of tolerance for the remaining validity time of the data, includes: Assign a participation method identifier for newly arrived data categories to data sources in a newly arrived valid state; assign a participation method identifier for data sources in a continuously valid state to data categories that retain historical data categories; and assign a participation method identifier for data sources in an expired state to data categories that have missing data categories. Use the data aging duration of each data source as the corresponding time-decrease attribute; The difference between the allowed acceptance time and the data aging time for each data source is used as the corresponding acceptance boundary margin. The participation method identifier, the time-decrease attribute, and the acceptance boundary margin are used as multi-dimensional state identifiers and appended to the data of the corresponding data source.

[0014] According to some embodiments of the present invention, the step of adaptively adjusting the timing characteristics and allowable duration of each data source based on the status of each data source in the integration output of each integration cycle includes: The statistical distribution of the proportions of newly arrived valid, continued valid, and expired invalid data sources was analyzed in a preset number of integration cycles. If the proportion of data sources in the newly arrived valid state is lower than the preset threshold for the proportion of newly arrived valid data sources, the time-series characteristics and allowed acceptance time of the corresponding data sources will be recalculated. If the proportion of data sources that are in a continuing valid state is higher than the preset continuing valid proportion threshold, then the time-series characteristics and allowed acceptance duration of the corresponding data sources will be recalculated. If the proportion of data sources that are in an expired state is higher than the preset expired failure ratio threshold, the confidence coefficient of the dynamic tolerance margin of the corresponding data source will be increased to broaden the corresponding allowable acceptance period.

[0015] Secondly, the present invention also provides a multi-source information access system based on real-time micro-meteorological data, applied to the multi-source information access and integration during the approach of severe convective weather in the power grid operation environment, the system comprising: The information acquisition module is used to acquire the time-series characteristics of multiple data sources in the power grid environment. The data sources include regional meteorological stations, radar networks, and self-built micro meteorological stations in substations. The time-series characteristics include the average update cycle and the standard deviation of the update interval. The range calculation module is used to dynamically calculate the allowable duration of each data source based on the time series characteristics. The duration calculation module is used to calculate the time difference between the observation timestamp of the latest data from each data source and the integrated target time of the second-level beat driven by the severe convective weather warning model, which serves as the data aging duration for the corresponding data source. The status classification module is used to compare the data aging time of each data source with the corresponding allowed acceptance time, and combine it with whether new data has arrived in the current integration cycle to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired and invalid. The integrated output module is used to replace the data from expired data sources with preset missing identifiers if there are expired data sources, and extract the valid data from newly arrived valid and continuing valid data sources for integrated output; otherwise, it extracts the valid data from each data source for integrated output. An identifier attachment module is used to attach a multi-dimensional status identifier to the data from each data source during the integrated output. The multi-dimensional status identifier includes an participation mode identifier to indicate the data source type, a time decay attribute to indicate the staleness of the data, and a boundary margin to indicate the remaining valid time of the data. The parameter adjustment module is used to adaptively adjust the timing characteristics and allowable duration of each data source based on the status of each data source in the integration output of each integration cycle.

[0016] The technical solution according to the embodiments of the present invention has at least the following beneficial effects: The technical solution of this invention, with a second-level integration cycle, allows the most recent valid data from low-frequency data sources to continue participating in integration within the allowed acceptance time even when no new data arrives. This avoids actual delays caused by waiting for low-frequency data sources and also avoids the loss of key information due to timeouts. It ensures the timeliness of early warnings while taking into account the integrity of information, thereby improving the meteorological early warning capability under the power grid operating environment.

[0017] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0019] Figure 1 This is a flowchart illustrating a multi-source information access method based on real-time micrometeorological data, provided in an embodiment of the present invention.

[0020] Figure 2 This is a flowchart illustrating embodiment S130 of the present invention.

[0021] Figure 3 This is a schematic flowchart of embodiment S140 of the present invention.

[0022] Figure 4 This is a flowchart illustrating embodiment S170 of the present invention.

[0023] Figure 5 This is a schematic diagram of the architecture of a multi-source information access system based on real-time micro-meteorological data, provided for an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The components of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0025] In the power grid operating environment, to achieve the access and integration of multi-source information, the system typically triggers integration operations at a preset time interval, checking the latest arriving messages from each data source within the period. When new messages arrive from designated data sources such as substation-built micro-weather stations, radar networks, and regional weather stations, the system packages and outputs the data to form a real-time data stream for use by the early warning model. Under normal conditions without severe convective weather, the system maintains a relatively long integration cycle, and the update interval of each data source is usually less than this cycle. The cycle readiness judgment logic can be satisfied normally, and the data stream is output stably at a fixed pace. However, when severe convective weather approaches, the early warning model requires a significant reduction in the data integration refresh cycle from minutes to seconds to capture rapidly changing convective characteristics. At this time, the condition "all data sources have arrived with the latest data" in the cycle readiness judgment logic begins to face challenges. Because the data update cycle of low-frequency data sources such as radar and regional weather stations is much longer than the shortened integration window, these data sources will not receive new data within most integration windows. Existing conventional methods typically rely on preset timeout rules, which are difficult to adapt to the huge difference between the requirements for accelerated early warning and the inherent update frequency of data sources, and cannot ensure both timeliness and information integrity.

[0026] In this regard, such as Figure 1 As shown, this invention discloses a multi-source information access method based on real-time micro-meteorological data, applied to the access and integration of multi-source information during the approach of severe convective weather in the power grid operation environment. The method includes the following steps: S110, Obtain time-series characteristics of multiple data sources in the power grid environment. The data sources include regional meteorological stations, radar networks, and self-built micro-meteorological stations in substations. The time-series characteristics include the average update cycle and the standard deviation of the update interval. S120, Based on the time sequence characteristics, dynamically calculate the allowable connection time for each data source; S130, calculate the observation timestamp of the latest data from each data source and the time difference between it and the integrated target time of the second-level tick driven by the severe convective weather warning model, and use it as the data aging duration of the corresponding data source; S140, compare the data aging time of each data source with the corresponding allowed acceptance time, and combine whether new data has arrived in the current integration cycle to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired and invalid. S150, if there are expired data sources, replace the data of the expired data sources with the preset missing identifier, and extract the valid data from the newly arrived valid and the continued valid data sources and integrate the output; otherwise, extract the valid data from each data source and integrate the output. S160, when integrating the output, a multi-dimensional status identifier is attached to the data from each data source. The multi-dimensional status identifier includes a participation mode identifier for indicating the data source type, a time decay attribute for indicating the staleness of the data, and a boundary margin for indicating the remaining valid time of the data. S170, based on the status of each data source in the integration output of each integration cycle, adaptively adjust the timing characteristics and allowed acceptance duration of each data source.

[0027] In this invention, "data source" refers to the entity that provides meteorological information, which periodically generates timestamped meteorological data. "Time series characteristics" are statistics describing the update pattern of the data source, reflecting its stability and volatility. "Allowable consolidation duration" is a dynamically set time threshold for each data source, indicating how long the data is considered valid; exceeding this time may render it invalid. "Data aging duration" refers to the time elapsed from observation to the current integration target time, used to measure the data's "freshness." "Integration target time" is a second-level tick driven by the severe convective weather warning model, representing the time point at which the downstream warning model expects to obtain a complete meteorological snapshot. "Multidimensional status identifiers" are metadata attached to the data, providing detailed status information during the integration process.

[0028] Time-series characteristics can be obtained in several ways. For example, the average update cycle and standard deviation of the update interval for each data source can be manually configured. Another approach is to preset a fixed time window and continuously monitor the arrival of data from each data source within this window, recording the timestamp of each data arrival. Then, these timestamps can be statistically analyzed to calculate the average update cycle and standard deviation of the update interval. For instance, the timestamps of the most recent N data arrivals can be recorded, the difference between two adjacent arrival timestamps can be calculated, and then the average and standard deviation of these differences can be calculated.

[0029] The allowable latency can be dynamically adjusted based on the average update cycle and standard deviation of the update interval of the data source. Specifically, the allowable latency equals the average update cycle of the corresponding data source plus a dynamic tolerance margin. The dynamic tolerance margin is equal to the standard deviation of the update interval of the corresponding data source multiplied by a preset confidence coefficient. The preset confidence coefficient can be set according to the needs of the actual scenario. For example, under the high real-time requirements of severe convective weather warnings, a smaller confidence coefficient can be set to tighten the allowable latency and ensure data timeliness; while in environments with large fluctuations in data source updates, the confidence coefficient can be appropriately increased to relax the allowable latency and avoid data being mistakenly judged as expired due to normal fluctuations. In this way, the allowable latency can be adaptively adjusted according to the update cycle and fluctuation characteristics of each data source, thereby maximizing the tolerance for reasonable delays from low-frequency data sources while ensuring data timeliness.

[0030] The observation timestamp is typically the time when the data was actually observed, while the integration target time is the desired point in time for obtaining a data snapshot. The data aging duration can be calculated directly by taking the time difference between the observation timestamp of the latest data and the integration target time. For example, the system can retrieve the observation timestamp of the latest data from the data source's cache queue and compare it with the current integration target time to obtain the time difference.

[0031] The status of each data source is categorized into one of the following: newly arrived and valid, continuing to be valid, or expired and invalid. For example, a simple logical judgment can be set: if the data aging period is less than the allowed retention period, the data is valid; otherwise, the data is invalid. Based on this, the status can be further subdivided according to whether new data arrives within the current integration cycle. For example, if new data arrives within the current integration cycle and is valid, it is considered newly arrived and valid; if no new data arrives but the data remains valid, it is considered continuing to be valid; if the data aging period exceeds the allowed retention period, it is considered expired and invalid.

[0032] When data from a certain data source is determined to be expired, the data fields of that data source can be left blank or filled with a specific error code to indicate that the data is missing. Then, the latest valid data is extracted from those data sources that have been determined to be newly arrived or still valid, and integrated to form a complete data packet for output.

[0033] When integrating the output, attach multi-dimensional status identifiers to the data from each data source. For example, a string or enumeration value can be appended to the data from each data source to indicate "newly arrived data," "using historical data," or "missing data." The timeliness decay attribute can be directly represented using the data aging duration. The acceptance margin can be obtained by calculating the difference between the allowed acceptance duration and the data aging duration. These identifiers can be appended as metadata to the data from each data source and output along with the integration package.

[0034] Finally, based on the status of each data source in the integration output of each integration cycle, the time-series characteristics and allowed uptime of each data source are adaptively adjusted. For example, the proportion of each data source in the newly arrived valid, continued valid, and expired invalid states over a period of time can be periodically calculated. If the proportion of expired invalid states for a certain data source is too high, it may mean that its allowed uptime is set too short, and the corresponding allowed uptime can be extended based on this proportion. Conversely, if the proportion of newly arrived valid states is too low, it may mean that the time-series characteristics are estimated inaccurately, and the time-series characteristics can be recalculated.

[0035] This invention acquires the time-series characteristics of multiple data sources in the power grid environment and dynamically calculates the allowable uptime for each data source based on these characteristics, thus flexibly adapting to the update frequency of different data sources. By calculating the data aging time and comparing it with the allowable uptime, and considering whether new data has arrived within the current integration cycle, the status of each data source is classified into newly arrived valid, continued valid, and expired invalid, achieving refined management of data validity. When there is an expired invalid data source, its data is replaced with a preset missing identifier, and valid data is extracted for integration output, ensuring the continuity of the data flow. During integration output, multi-dimensional status identifiers are attached to the data of each data source, providing richer data status information. Furthermore, the method of this invention can adaptively adjust the time-series characteristics and allowable uptime of each data source based on the status of each data source in the integration output of each integration cycle.

[0036] In summary, the technical solution of this invention effectively solves the problem that fixed-time window readiness judgment and timeout strategies are difficult to adapt to the difference between the requirements for accelerated early warning and the inherent update frequency of data sources when strong convective weather is approaching, by dynamically calculating the allowable acceptance time based on the average update cycle and standard deviation of the data source and introducing a "continued validity" state. Under a second-level integration cycle, even when no new data arrives, the most recently valid data from low-frequency data sources can still participate in the integration within the allowable acceptance time, thus avoiding actual delays caused by waiting for low-frequency data sources and avoiding the loss of key information due to timeout abandonment. This ensures both the timeliness of early warnings and the integrity of information, improving the meteorological early warning capability under the power grid operating environment.

[0037] In a specific embodiment of the present invention, the timing features are preferably obtained through the following process: Maintain a fixed-length historical time window for each data source and record the observation timestamps of the most recent preset number of message arrivals for the corresponding data source; Calculate the difference between two adjacent observation timestamps to obtain the update interval; Based on the update interval, the average update cycle and the standard deviation of the update interval are calculated as the time-series characteristics of the corresponding data source.

[0038] The historical time window is used to continuously record the observation timestamps of the most recent preset number of messages arriving from the corresponding data source. It can be understood as a sliding window or a circular buffer. For data sources with high update frequencies (such as self-built micro-weather stations in substations, with an update cycle of approximately 1-2 seconds), the fixed duration can be set to a shorter value, such as 30 minutes, to quickly respond to changes in the data source update cycle. For data sources with low update frequencies (such as radar networks, with an update cycle of approximately 5-6 minutes; regional weather stations, with an update cycle of approximately 10-15 minutes), the fixed duration can be set to a longer value, such as 2 hours, to ensure that the historical time window contains sufficient message samples for statistical calculations. The preset number of messages refers to the number of messages to be recorded within the historical time window, for example, it can be set to the observation timestamps of the most recent 100 or 200 messages, aiming to provide sufficient data samples for subsequent time-series feature calculations. The observation timestamp refers to the time point when the data source actually observes the meteorological phenomenon and generates data, rather than the time point when the data arrives at the system. This helps to more accurately reflect the actual update behavior of the data source.

[0039] By calculating the difference between two adjacent observation timestamps, a series of update intervals can be obtained, reflecting the frequency and pattern of data updates from the data source.

[0040] The average update cycle refers to the arithmetic mean of all update intervals, which characterizes the average frequency of data updates from the data source. The standard deviation of the update interval measures the volatility or stability of the data source's update frequency; a larger standard deviation indicates greater volatility and less stable data updates, while a smaller standard deviation indicates more stable updates. Using these two statistics as time-series characteristics of the corresponding data source allows for a comprehensive and quantitative description of the data update behavior.

[0041] The solution of this invention, through the maintenance of historical time windows and statistical calculations, ensures that the standard deviation of the obtained average update cycle and update interval accurately reflects the true update pattern and stability of the data source. This allows subsequent steps such as comparing data aging duration, classifying data states, and adaptively adjusting the allowable duration to be performed based on more accurate time-series information, thereby improving the accuracy and reliability of multi-source information access and integration. Especially in scenarios with extremely high requirements for data timeliness, such as the approach of severe convective weather, it enables more effective management and utilization of real-time micrometeorological data.

[0042] In some embodiments of the present invention, such as Figure 2 As shown, the step of calculating the observation timestamp of the latest data from each data source and the time difference between it and the integrated target time of the second-level tick driven by the severe convective weather warning model, as the data aging duration of the corresponding data source, preferably includes: S131, extract the observation timestamp and arrival timestamp of the latest data from the cache queues of each data source; S132, calculate the time difference between the observation timestamp and the integrated target time, and use it as the data aging duration of the corresponding data source; wherein, the integrated target time is continuously generated by the severe convective weather warning model at a second-level pace, and is used as the time point at which the downstream warning model expects to obtain a complete meteorological snapshot; S133, determine the network transmission delay based on the difference between the arrival timestamp and the observation timestamp; S134, subtract the network transmission delay from the data aging time to obtain the corrected data aging time.

[0043] During the data access process, the latest received data is extracted from the cache queues of each data source. For each piece of data, not only is an observation timestamp recorded (the time when the data is actually observed or generated in the physical world), but also an arrival timestamp (the time when the data message is successfully received). These two timestamps form the basis for subsequent calculations of data aging time. The severe convective weather warning model is a meteorological warning model used to monitor and predict severe convective weather (such as thunderstorms, strong winds, and hail). It has extremely high real-time requirements for data input, needing to acquire multi-source meteorological data from the power grid operating environment at a second-level frequency to promptly capture the rapid evolution characteristics of severe convective weather. The integration target time is continuously generated by the severe convective weather warning model at a second-level interval, serving as the time reference point for the warning model to obtain a complete meteorological snapshot. Specifically, at the arrival of each second-level interval, the warning model sends an integration request to the data access system. The time point corresponding to this request is the integration target time. Based on this, the system extracts the latest valid data from each data source and assembles it into a meteorological snapshot corresponding to the integration target time, which is then used by the warning model for the next round of severe convective weather analysis and forecasting. This ensures that downstream early warning models can acquire the latest integrated meteorological data at a high frequency. The initial data aging time is the time difference between the data's observation timestamp and the target time for integration. It reflects the "natural" aging of the data from the time it is observed until the early warning model expects to capture a snapshot.

[0044] Network transmission delay is specifically determined by calculating the difference between the arrival timestamp and the observation timestamp of the data. This difference directly reflects the network transmission time that the data takes from being sent from the observation point to being received by the system.

[0045] The revised data aging duration more accurately reflects the actual degree of data aging within the "effective" time window from when it is observed until the integration target time, eliminating false aging caused by network transmission.

[0046] The solution of this invention quantifies and compensates for network transmission delay, avoiding the problem of inflated data aging time caused by transmission delay, thus enabling the system to more accurately assess the timeliness of data. This is particularly crucial for applications with extremely high real-time data requirements, such as severe convective weather warnings, effectively reducing the probability of valid data being misjudged as expired, thereby ensuring that the warning model can fully utilize the latest and most accurate meteorological data, improving the timeliness and accuracy of warnings, and enhancing the safety and reliability of power grid operation.

[0047] The following is a specific example to illustrate this.

[0048] Suppose a self-built micro-weather station at a substation observes a set of meteorological data with an observation timestamp of T0. Due to network transmission, this data packet is successfully received by the system at T0 + 5 seconds, i.e., its arrival timestamp is T0 + 5 seconds. At this time, the integration target time driven by the severe convective weather warning model is T0 + 10 seconds. According to the scheme of this invention, firstly, the observation timestamp T0 and the arrival timestamp T0 + 5 seconds of the data are extracted from the cache queue. Next, the initial data aging time is calculated: the time difference between the observation timestamp T0 and the integration target time T0 + 10 seconds, i.e., (T0 + 10 seconds) - T0 = 10 seconds. Then, based on the difference between the arrival timestamp T0 + 5 seconds and the observation timestamp T0, the network transmission delay is determined to be (T0 + 5 seconds) - T0 = 5 seconds. Finally, the initial calculated data aging time of 10 seconds is subtracted from the network transmission delay of 5 seconds to obtain the corrected data aging time of 10 seconds - 5 seconds = 5 seconds. This correction allows the system to accurately identify the actual aging time of the data from observation to the integration target time as 5 seconds, instead of the uncorrected 10 seconds. This enables the system to more accurately determine the timeliness of the data, avoiding misjudging it as over-aged due to network transmission delays, thus ensuring that the valid data can be integrated and output, providing a more reliable input for the severe convective weather warning model.

[0049] In embodiments of the present invention, such as Figure 3 As shown, the preferred step of comparing the data aging time of each data source with the corresponding allowed acceptance time, and considering whether new data has arrived within the current integration cycle, to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired invalid, includes: S141, If ​​new data arrives during the current integration cycle and the corresponding data aging time does not exceed the corresponding allowed acceptance time, then the status of the corresponding data source will be classified as newly arrived and valid. S142, if no new data arrives during the current integration cycle and the corresponding data aging time does not exceed the corresponding allowed acceptance time, then the status of the corresponding data source will be classified as continued valid. S143, if the corresponding data aging time exceeds the corresponding allowed acceptance time, the status of the corresponding data source will be classified as expired.

[0050] When a data source is determined to be "newly arrived and valid," it means the data is up-to-date and timely, and can be directly used for integration. When a data source is determined to be "continuously valid," although there is no new data, the existing data is still considered to have sufficient timeliness and can continue to be used. When a data source is determined to be "expired," it means the data is outdated, no longer meets the real-time requirements, and should not be directly used for integration.

[0051] This invention achieves refined management of the timeliness and effectiveness of multi-source micrometeorological data by clearly defining three states of data sources (newly arrived and valid, continuing valid, and expired invalid) and precisely classifying them based on data aging duration, allowable acceptance time, and the arrival of new data. This classification mechanism ensures that the power grid operation system can always obtain the most reliable and timely data snapshots in the rapidly changing environment of approaching severe convective weather. Specifically, by distinguishing between "newly arrived and valid" and "continuing valid," it can flexibly respond to possible short-term interruptions or update delays in data sources, and can still utilize valid historical data when the latest data is unavailable, avoiding data loss. At the same time, by identifying "expired invalid" data, outdated information can be promptly removed, preventing old data from negatively impacting the accuracy of early warning models.

[0052] In a further embodiment of the present invention, the step of comparing the data aging time of each data source with the corresponding allowed acceptance time, and combining this with whether new data has arrived within the current integration cycle, to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired invalid, preferably further includes: A register is maintained for each data source to record the observation timestamp of the latest valid physical observation. When a new message arrives, if the data aging time of the corresponding data source exceeds the corresponding allowed acceptance time, and the observation timestamp of the new message is later than the observation timestamp recorded in the register, then a marker for a late valid physical frame is added to the status of the corresponding data source as an identifier to trigger asynchronous backtracking compensation processing in subsequent steps.

[0053] The register for the latest valid physical observation timestamp can be understood as a storage unit designed to continuously track and record data frames with the latest observation timestamps successfully processed or received by each data source. This register ensures that even if a data packet arrives late, it can still determine whether it carries physical observation information newer than currently recorded data. Specifically, a new packet's observation timestamp being later than the timestamp recorded in the register means that the actual observation time point contained in the newly arrived data packet is later than the latest observation time point previously effectively received and recorded by the system from that data source. The marker for a late valid physical frame is an identifier attached to a late data packet when certain conditions are met (i.e., the data aging time exceeds the allowed acceptance time but the new packet's observation timestamp is updated). This identifier clearly indicates that although the data packet is late, the physical observation content is still valid and up-to-date, and should not be simply discarded. This marker serves as an identifier for triggering asynchronous backtracking compensation processing in subsequent steps, meaning the system will initiate an independent, non-real-time processing flow based on this marker to correct or supplement the historical integrated output using the late but valid data.

[0054] The present invention maintains a register of the latest valid physical observation timestamps and introduces a marker for late valid physical frames, enabling the system to identify and retain these valuable late data points and use them as identifiers to trigger asynchronous backtracking compensation processing. This improves the robustness and data utilization of the multi-source information access method, especially in complex scenarios where network transmission is unstable or data source updates are irregular. Consequently, it ensures that the power grid severe convective weather early warning model can acquire more comprehensive and accurate real-time micro-meteorological snapshots, thereby improving the timeliness and accuracy of early warnings and providing more reliable data support for the safe and stable operation of the power grid.

[0055] The following is a specific example to illustrate this.

[0056] Suppose that a self-built micro-weather station at a substation experiences a momentary communication link failure, causing it to fail to send data on time during a certain integration cycle. Its data aging time exceeds the allowed reception time, thus it is initially judged as expired. However, in a subsequent integration cycle, the micro-weather station restores communication and sends a new data message. Although the arrival time of this new message is later than its expected integration target time, its internally recorded observation timestamp is later than the latest observation timestamp successfully received and recorded by the system previously. At this point, the system first checks the register of the latest valid physical observation timestamp of the data source and finds that the new message's observation timestamp is indeed later than the timestamp recorded in the register. Even though the data aging time of the data source has exceeded the allowed reception time, the system will still add a marker for a late valid physical frame to the new message. This marker will serve as an identifier to trigger asynchronous backtracking compensation processing, enabling the system to use this delayed but valid physical frame to correct or supplement the historical integrated output in subsequent compensation processing. This ensures that even in the event of data transmission anomalies, critical micro-meteorological information can be effectively utilized, avoiding information loss.

[0057] In a further embodiment of the present invention, the step of replacing the data from expired data sources with a preset missing identifier and extracting valid data from newly arrived valid and continuing valid data sources for integrated output preferably includes: Replace the data from data sources that are in an expired or invalid state with a preset missing identifier, and extract the valid data from data sources that have just arrived in a valid state and data sources that continue to be valid, as the real-time integrated data stream for the current integration cycle. If any data source in the expired state has the tag of the late valid physical frame, then asynchronous backtracking compensation processing is triggered. The asynchronous backtracking compensation processing includes: Using the observation timestamp of the data source marked with the late valid physical frame as the search keyword, the corresponding historical integration target time is located from the historical integration output; wherein, the historical integration output is: the integration data stream generated in each integration cycle before the current integration cycle and the corresponding integration target time; Extract valid data from data sources other than the data source that generated the late valid physical frame at the target historical integration time, align and integrate it with the data of the late valid physical frame, and generate a historical compensation integration data packet; The historical compensation integration data packet is then asynchronously inserted into the output queue of the current integration cycle and pushed in combination with the real-time integration data stream.

[0058] During the integration and output process, data from data sources deemed expired or invalid are replaced with preset missing data markers to ensure the integrity and consistency of the real-time integrated data stream. Simultaneously, the latest valid data is extracted from data sources newly arriving in a valid state and those continuing to be valid. This data is then aggregated to form the real-time integrated data stream for the current integration cycle. This data stream primarily serves to meet the real-time weather snapshot requirements of downstream early warning models.

[0059] When a data source that is in an expired state is found to contain a data source marked as a late valid physical frame, an asynchronous backtracking compensation mechanism is triggered. This mechanism aims to correct and supplement historical integrated data to ensure that even late valid data can be included in the final integration result.

[0060] Using the observation timestamp of the late valid physical frame as the search keyword, representing the actual time the data was observed, this timestamp is used to search the historical integration output to locate the historical integration target time closest to or corresponding to that observation timestamp. In this way, it can be determined in which historical snapshot the late data should have originally been integrated into.

[0061] Once the corresponding historical integration target time is located, valid data from other data sources besides the data source that generated the late valid physical frame are extracted at that historical integration target time. This data represents the data that was successfully integrated and output at that time. Subsequently, this historical valid data is aligned and integrated with the data of the currently received late valid physical frame. The purpose of alignment and integration is to ensure that all data corresponds to the same physical observation time point, thereby generating a complete and corrected historical compensation integration data packet.

[0062] The generated historical compensation integration data packets do not immediately affect the current real-time integration data stream; instead, they are asynchronously inserted into the output queue of the current integration cycle. These compensation data packets are then pushed to downstream systems along with the current real-time integration data stream. This asynchronous push method ensures the continuity of the real-time data stream while providing correction and supplementation to historical data. This allows downstream systems to obtain a more accurate and complete view of meteorological data, especially useful when historical backtracking analysis or model training is required, as they can utilize this more precise, compensated data.

[0063] Through the above technical solution, this invention can improve the integrity and accuracy of multi-source information integration, especially in processing delayed but valid physical observation data. Traditional real-time integration schemes often simply discard or replace expired data, even if the data is still physically valid. However, the asynchronous backtracking compensation processing introduced in this invention allows the system to effectively utilize these delayed but valid physical frames to correct and supplement historical integration outputs, avoiding the loss of valuable data. This not only ensures the continuity and timeliness of the real-time data stream but also provides a more accurate and complete dataset for downstream early warning models or historical analysis, thereby improving the accuracy and reliability of severe convective weather warnings. Its value is particularly prominent in scenarios requiring refined backtracking analysis or model retraining.

[0064] The following is a specific example to illustrate this.

[0065] Suppose that in a certain integration period T1, data from regional weather station A expires due to network latency, exceeding the allowed data aging time. However, the observation timestamp of this data packet (e.g., T_obs_A) is later than the latest recorded valid observation timestamp, and therefore a late valid physical frame marker is added. In the real-time integrated data stream of integration period T1, the data from regional weather station A is replaced with the missing marker. Subsequently, the system triggers asynchronous backtracking compensation processing. First, using T_obs_A as the search keyword, the system searches in the historical integration output to locate the historical integration target time corresponding to T_obs_A, such as T_target_A. Next, the system extracts valid data from other data sources besides regional weather station A (such as radar networks or self-built micro-weather stations in substations) at time T_target_A. Then, these historical valid data are aligned and integrated with the late valid physical frame data from regional weather station A to generate a historical compensation integration data packet. Finally, this historical compensation integration data packet is asynchronously inserted into the output queue of the current integration period T1, and pushed in sync with the real-time integration data stream. In this way, when the downstream system receives the output of period T1, it not only receives a real-time meteorological snapshot (in which data from regional weather station A may be missing), but also simultaneously receives a corrected compensation data packet for time T_target_A, containing late but valid data from regional weather station A. This allows the downstream system to make decisions or update historical records using more complete data as needed, thereby improving the overall quality and reliability of the data service.

[0066] In some embodiments of the present invention, the step of extracting and integrating valid data from each data source preferably includes: Using the second-level tick as a step size, the latest data is extracted from the cache queue of the data source in the newly arrived valid state and marked as read, and the data that was last read is extracted from the data source in the continued valid state. The extracted data from each data source is assembled into an integrated data package according to a predetermined data model, and output as the real-time integrated data stream for the current integration cycle.

[0067] Second-level ticks refer to continuously generated time points, measured in seconds, driven by severe convective weather warning models. These serve as the time reference for downstream warning models to obtain complete meteorological snapshots. Data extraction using second-level ticks ensures real-time data integration and synchronization with downstream warning models. For newly arrived valid data sources, their cache queues typically store the latest received data. Retrieving the latest data from the cache queue and marking it as read ensures the latest observation information from the data source is obtained in each integration cycle, avoiding redundant processing. For data sources in a continuing valid state, since no new data may arrive in the current integration cycle, retrieving the previously read data ensures a valid and continuous data stream even without the latest data, avoiding data loss. In practical applications, assembling the extracted data from various data sources into an integrated data package according to a predetermined data model refers to the unified encapsulation and organization of valid data from different data sources (such as regional meteorological stations, radar networks, and self-built micro-meteorological stations in substations) according to a predefined data structure and format. This predetermined data model ensures structural consistency across data from different sources, facilitating analysis and use by downstream early warning models. The integrated data packet, as the real-time integrated data stream output of the current integration cycle, means that it will serve as a real-time meteorological snapshot, provided to severe convective weather early warning models for analysis and decision-making.

[0068] This invention intelligently extracts data from data sources in different states using a second-level tick size and assembles it according to a unified data model. This ensures that the power grid operating environment can continuously and stably acquire high-quality multi-source meteorological information when severe convective weather is approaching. Specifically, by prioritizing the extraction of the latest data from newly arrived valid data sources, the real-time nature of the data is guaranteed. Simultaneously, by using previously read data from continuing valid data sources, gaps that may be caused by untimely data updates are compensated for, maintaining the continuity of the data stream. Therefore, the accuracy and timeliness of the severe convective weather warning model are avoided due to missing or outdated data.

[0069] In a specific embodiment of the present invention, the step of attaching a multi-dimensional status identifier to the data from each data source, wherein the multi-dimensional status identifier includes a participation mode identifier for indicating the data source type, a time decay attribute for indicating the staleness of the data, and a boundary margin for indicating the remaining valid time of the data, preferably includes: Assign a participation method identifier for newly arrived data categories to data sources in a newly arrived valid state; assign a participation method identifier for data sources in a continuously valid state to data categories that retain historical data categories; and assign a participation method identifier for data sources in an expired state to data categories that have missing data categories. Use the data aging duration of each data source as the corresponding time-decrease attribute; The difference between the allowed acceptance time and the data aging time for each data source is used as the corresponding acceptance boundary margin. The participation method identifier, the time-decrease attribute, and the acceptance boundary margin are used as multi-dimensional state identifiers and appended to the data of the corresponding data source.

[0070] The participation method identifier indicates the type of data source, i.e., the acquisition status of the data within the current integration cycle. Specifically, the "New Arrival Data" category means that new data packets have arrived from the data source within the current integration cycle and are deemed valid; the "Continued Historical Data" category means that no new data packets have arrived from the data source within the current integration cycle, but data from the previous integration cycle remains valid and is being continued; the "Missing Data" category means that the data from the data source has exceeded the allowed retention period, or has not arrived within the current cycle and cannot be reused, thus being considered invalid. The "Time-Degradation" attribute directly reflects the staleness of the data; a larger value indicates older data. The "Retention Margin" represents the remaining time that the data can be retained in its current valid state, calculated as the difference between the allowed retention period and the data aging period; a larger value indicates higher data freshness and a lower risk of expiration. These identifiers and attributes are integrated into a multi-dimensional status identifier and appended to the data from the corresponding data source so that downstream applications can fully understand the data status.

[0071] This invention attaches multi-dimensional status identifiers to the data from each data source, ensuring that the integrated output data not only contains raw meteorological information but also carries rich metadata. This metadata clearly describes the data's source status, timeliness, and reliability boundaries. The participation method identifier clarifies whether the data is newly acquired in real-time, historically used, or missing, providing transparency about the data source for downstream models. The timeliness decay attribute intuitively quantifies the data's "freshness," allowing data users to assess its usability based on actual needs. The bearing margin provides a forward-looking indicator, indicating how long the data can still be considered valid in the future, aiding downstream systems in risk assessment and decision-making. Therefore, when processing integrated data, downstream early warning models can make refined judgments on the reliability and timeliness of different data sources based on these multi-dimensional status identifiers.

[0072] In the above embodiments of the present invention, such as Figure 4 As shown, the step of adaptively adjusting the timing characteristics and allowable duration of each data source based on the status of each data source in the integration output of each integration cycle preferably includes: S171, Statistically analyze the proportion of each data source in three states: newly arrived valid, continued valid, and expired invalid in a preset number of integration cycles; S172, If the proportion of data sources in the newly arrived valid state is lower than the preset newly arrived valid proportion threshold, then the time series characteristics and allowed acceptance duration of the corresponding data source are recalculated. S173, If the proportion of data sources in a continuing valid state is higher than the preset continuing valid proportion threshold, then the time series characteristics and allowed acceptance duration of the corresponding data source are recalculated. S174 If the proportion of data sources that are in an expired state is higher than the preset expired failure ratio threshold, the confidence coefficient of the dynamic tolerance margin of the corresponding data source is increased to broaden the corresponding allowable acceptance time.

[0073] By accumulating statistics on the number of times each data source is judged as "newly arrived and valid," "continuing to be valid," or "expired" within these periods, the respective proportions are calculated. The preset number of integration periods can be configured according to the actual application scenario and the characteristics of the data source, such as the most recent 100 integration periods. The purpose is to obtain the long-term trend of the data source's behavior pattern, rather than judging it based solely on the instantaneous state.

[0074] If the proportion of data sources in a newly arrived valid state is lower than the preset threshold for the proportion of newly arrived valid data sources, the time-series characteristics and allowable reception duration of the corresponding data sources are recalculated. This indicates that the frequency of new data arrivals from this data source may be lower than expected, or that its data quality and timeliness have declined. In this case, the system will trigger a recalculation of the time-series characteristics of the data source (such as the average update cycle and the standard deviation of the update interval), and dynamically adjust its allowable reception duration based on the new time-series characteristics to more accurately reflect its current data supply capacity. The threshold for the proportion of newly arrived valid data sources is used to determine whether the frequency of new data arrivals from the data source has abnormally decreased. Its specific value can be set according to the inherent update frequency of the data source and the actual needs of the power grid operating environment. For example, for self-built micro-weather stations in substations with high update frequencies, new data should arrive in most integration cycles under a second-level integration cycle, so the threshold for the proportion of newly arrived valid data sources can be set to a higher value, such as 80%; while for radar networks or regional weather stations with low update frequencies, no new data arrives in most integration cycles under a second-level integration cycle, so the threshold for the proportion of newly arrived valid data sources should be set to a lower value, such as 5%.

[0075] If the proportion of data sources in a continuously valid state exceeds a preset continuous valid proportion threshold, the time-series characteristics and allowed acceptance duration of the corresponding data source are recalculated. This means that the new data arrival frequency of this data source is too low, the system frequently uses old data, or its data update cycle becomes longer. To avoid over-reliance on old data, the system will re-evaluate the time-series characteristics of the data source and adjust the allowed acceptance duration accordingly, prompting the system to more actively wait for new data or more accurately reflect its actual update frequency. The continuous valid proportion threshold is used to determine whether the data source is overly reliant on old data, i.e., whether its new data arrival frequency is too low, causing the system to frequently use historical data. Its specific value can be set according to the inherent update frequency of the data source and the actual needs of the power grid operating environment. For example, for self-built micro-weather stations in substations with high update frequencies, their continuous valid proportion is usually low under a second-level integration cycle, so the continuous valid proportion threshold can be set to a low value, such as 20%; while for radar networks or regional weather stations with low update frequencies, their continuous valid proportion is usually high under a second-level integration cycle, so the continuous valid proportion threshold can be set to a high value, such as 80%.

[0076] If the proportion of data sources in an expired state exceeds a preset expired failure ratio threshold, the confidence coefficient of the corresponding data source's dynamic tolerance margin is increased to broaden the allowable uptime. This indicates that the data from this data source frequently expires beyond the allowable uptime, possibly due to longer update cycles or increased transmission delays. To improve the effective data utilization of this data source, the system will appropriately relax its data timeliness requirements by increasing the confidence coefficient of the dynamic tolerance margin to broaden the allowable uptime, thereby tolerating data aging to a certain extent and reducing data loss. The expired failure ratio threshold can be set according to the inherent update frequency of the data source and the actual needs of the power grid operating environment. For example, for self-built micro-weather stations in substations with high update frequency, the proportion of failures due to expiration is usually very low under the second-level integration cycle. Therefore, the threshold for the proportion of failures due to expiration can be set to a low value, such as 5%. For radar networks or regional weather stations with low update frequency, the proportion of failures due to expiration is usually low under the second-level integration cycle. However, due to factors such as transmission delay, failures due to expiration may occasionally occur. Therefore, the threshold for the proportion of failures due to expiration can be set to 10%-15%.

[0077] Through the above technical solution, this invention enables intelligent adaptive management of multi-source micrometeorological data access parameters. Compared to fixed or manually adjusted parameters, this invention dynamically optimizes the temporal characteristics and allowable access duration based on the actual operating status and historical performance of the data source, significantly improving the robustness of data integration and its adaptability to environmental changes. Especially in power grid operating environments, when severe convective weather is approaching, the behavior of micrometeorological data sources may fluctuate due to network congestion, equipment failures, and other factors. The adaptive adjustment mechanism of this invention can respond promptly to these changes, effectively reducing the risk of data failure and ensuring that downstream severe convective weather early warning models can continuously acquire high-quality, timely integrated data, thereby improving the accuracy and reliability of early warnings.

[0078] The following is a specific example to illustrate this.

[0079] Suppose that a self-built micro-weather station at a substation experiences increased delays in data packet arrival and irregular update intervals due to unstable communication links over a period of time. Without an adaptive adjustment mechanism, the system may continue to use the previously calculated shorter allowable reception time, resulting in the micro-weather station's data frequently being judged as "overdue and invalid," with a large amount of valid data being discarded or replaced with missing identifiers.

[0080] However, in the solution of this invention, the system continuously analyzes the proportion distribution of the three states of "newly arrived and valid," "continued valid," and "expired and invalid" of the micro-weather station over a preset number of integration cycles. When the proportion of the "expired and invalid" state is detected to be continuously higher than the preset expiration and invalidation proportion threshold, the system identifies that the data source has a persistent data aging problem. At this time, the system will automatically increase the confidence coefficient of the dynamic tolerance margin of the data source, thereby widening its allowable acceptance time. For example, if the original allowable acceptance time was 5 seconds, the system may adjust it to 8 seconds. In this way, even if the data of the micro-weather station has a certain delay, as long as its data aging time is still within the new allowable acceptance time range, its data can still be determined as valid and participate in the integration output.

[0081] Meanwhile, if the system detects that the proportion of newly arrived valid data from a certain weather station remains consistently below a preset threshold, it indicates that its update frequency may be decreasing. The system will recalculate the average update cycle and standard deviation of the update interval for that weather station, and adjust its allowable reception duration based on the new time-series characteristics to more accurately reflect its current actual update capacity, avoiding missing valid data or excessive waiting due to improper parameter settings. Through this adaptive adjustment, the system can dynamically optimize its data access parameters according to the actual operating conditions of each data source, thereby maximizing the effective integration and utilization of multi-source micro-meteorological data in complex and ever-changing environments.

[0082] like Figure 5 As shown, this invention also discloses a multi-source information access system based on real-time micro-meteorological data, applied to the multi-source information access and integration during the approach of severe convective weather in the power grid operation environment. The system includes: The information acquisition module 210 is used to acquire the time-series characteristics of multiple data sources in the power grid environment. The data sources include regional meteorological stations, radar networks, and self-built micro meteorological stations in substations. The time-series characteristics include the average update cycle and the standard deviation of the update interval. The range calculation module 220 is used to dynamically calculate the allowable duration of each data source based on the time series characteristics. The duration calculation module 230 is used to calculate the time difference between the observation timestamp of the latest data of each data source and the integrated target time of the second-level beat driven by the severe convective weather warning model, which serves as the data aging duration of the corresponding data source. The status division module 240 is used to compare the data aging time of each data source with the corresponding allowed acceptance time, and combine whether new data arrives in the current integration cycle to divide the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired and invalid. The integrated output module 250 is used to replace the data of the expired data source with a preset missing identifier if there is an expired data source, and extract the valid data from the newly arrived valid and the continued valid data sources for integrated output; otherwise, it extracts the valid data from each data source for integrated output. The identification attachment module 260 is used to attach a multi-dimensional status identifier to the data of each data source during the integrated output. The multi-dimensional status identifier includes a participation mode identifier for indicating the data source type, a time decay attribute for indicating the staleness of the data, and a boundary margin for indicating the remaining valid time of the data. The parameter adjustment module 270 is used to adaptively adjust the timing characteristics and allowable duration of each data source based on the status of each data source in the integration output of each integration cycle.

[0083] The information acquisition module 210 can be implemented as a standalone software service, periodically pulling or receiving data from various data sources (such as regional weather stations, radar networks, and self-built micro-weather stations in substations) through a preset API interface or data bus. It performs preliminary parsing and timestamp extraction on the received data, and then stores this data in an internal cache or database. Alternatively, the information acquisition module 210 can be integrated into a data acquisition gateway, performing data preprocessing and preliminary calculations of time-series characteristics directly on the data source side, and uploading the processed data and characteristics to the central processing unit.

[0084] The range calculation module 220 can be implemented as an independent calculation unit, with a pre-built algorithm model for calculating the allowed uptime. The range calculation module 220 can periodically retrieve the latest time-series feature data from the information acquisition module and adjust the allowed uptime of each data source in real time based on these features (such as the average update cycle and the standard deviation of the update interval). For example, the range calculation module 220 can use a simple linear function or a lookup table to determine the allowed uptime. Alternatively, the range calculation module 220 can be configured to set the allowed uptime by receiving external configuration parameters, such as manual input by operations personnel or loading through a configuration file.

[0085] The duration calculation module 230 can be implemented as a timestamp processing service, its function being to calculate time differences. The duration calculation module 230 can extract the observation timestamp of the latest data from the data source's cache queue and compare it with the integrated target time of the second-level ticks driven by the severe convective weather warning model. For example, the duration calculation module 230 can simply perform a timestamp subtraction operation to obtain the data aging duration. As one implementation, the duration calculation module 230 can be configured to rely solely on the observation timestamps for calculation, regardless of network latency during data transmission.

[0086] The state classification module 240 can be implemented as a logic judgment engine, containing a series of preset rules. The state classification module 240 receives the data aging duration from the duration calculation module 230, the allowed acceptance duration from the range calculation module 220, and a new data arrival flag from the data source. Based on these input conditions, it performs judgments to accurately classify the state of each data source as newly arrived and valid, continued valid, or expired and invalid. For example, the state classification module 240 can use simple if-else statements or decision tree logic for state determination.

[0087] The integration output module 250 can be implemented as a data assembly and publishing service. The integration output module 250 is responsible for collecting status information and corresponding valid data from each data source in the status segmentation module. When an expired data source is detected, the integration output module 250 can simply fill the data fields of that data source with a preset missing identifier (e.g., a specific null value or error code). Subsequently, the integration output module 250 assembles all valid data (including newly arrived valid and continuing valid data) and the replaced missing identifier data according to a predetermined data model, forming a complete integration data package, and pushes it to the downstream early warning model.

[0088] The identifier appending module 260 can be implemented as a metadata injector. Before or after the integration output module 250 assembles the data packet, the identifier appending module 260 appends multi-dimensional status identifiers to the data from each data source, based on the status information of each data source. For example, the identifier appending module 260 can simply add participation mode identifiers, time-limited decay attributes, and acceptance boundary margins as independent fields to the header of the data packet or to the data structure of each data source. These identifiers can be encoded as simple enumeration values, integers, or strings.

[0089] The parameter adjustment module 270 can be implemented as a feedback control unit. The parameter adjustment module 270 periodically receives status information from each data source in the integrated output and performs statistical analysis on it. For example, the parameter adjustment module 270 can simply count the number of times each data source is in a newly arrived valid, continued valid, or expired invalid state within a fixed time window, and based on these statistical results, adjust the timing characteristics or allowed validity period of the corresponding data source according to preset simple rules (e.g., increasing the allowed validity period if the number of expired invalid states exceeds a certain threshold).

[0090] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0091] The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention.

Claims

1. A multi-source information access method based on real-time micro-meteorological data, applied to the multi-source information access and integration during the approach of severe convective weather in the power grid operation environment, characterized in that... The method includes the following steps: The time-series characteristics of multiple data sources in the power grid environment are obtained. The data sources include regional meteorological stations, radar networks, and self-built micro meteorological stations in substations. The time-series characteristics include the average update cycle and the standard deviation of the update interval. Based on the aforementioned timing characteristics, the allowable connection duration for each data source is dynamically calculated; The observation timestamp of the latest data from each data source is calculated, and the time difference between it and the integrated target time of the second-level tick driven by the severe convective weather warning model is used as the data aging duration for the corresponding data source. Compare the data aging time of each data source with the corresponding allowed acceptance time, and combine this with whether new data has arrived in the current integration cycle to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, or expired and invalid. If there are expired data sources, replace the data from the expired data sources with the preset missing data markers, and extract the valid data from the newly arrived valid and the continued valid data sources and integrate them into the output; otherwise, extract the valid data from each data source and integrate them into the output. When integrating the output, a multi-dimensional status identifier is attached to the data from each data source. The multi-dimensional status identifier includes a participation mode identifier to indicate the data source type, a time decay attribute to indicate the staleness of the data, and a boundary margin to indicate the remaining valid time of the data. Based on the status of each data source in the integration output of each integration cycle, the timing characteristics and allowable duration of each data source are adaptively adjusted.

2. The multi-source information access method based on real-time micro-meteorological data according to claim 1, characterized in that, The time-series features are obtained through the following process: Maintain a fixed-length historical time window for each data source and record the observation timestamps of the most recent preset number of message arrivals for the corresponding data source; Calculate the difference between two adjacent observation timestamps to obtain the update interval; Based on the update interval, the average update cycle and the standard deviation of the update interval are calculated as the time-series characteristics of the corresponding data source.

3. The multi-source information access method based on real-time micro-meteorological data according to claim 1, characterized in that, The step of calculating the observation timestamp of the latest data from each data source and the time difference between it and the integrated target time of the second-level tick driven by the severe convective weather warning model, as the data aging duration for the corresponding data source, includes: Extract the observation timestamp and arrival timestamp of the latest data from the cache queues of each data source; The time difference between the observation timestamp and the integrated target time is calculated and used as the data aging duration of the corresponding data source; wherein, the integrated target time is continuously generated by the severe convective weather warning model at a second-level interval and is used as the time point at which the downstream warning model expects to obtain a complete meteorological snapshot; The network transmission delay is determined based on the difference between the arrival timestamp and the observation timestamp. The corrected data aging time is obtained by subtracting the network transmission delay from the data aging time.

4. The multi-source information access method based on real-time micrometeorological data according to claim 1, characterized in that, The step of comparing the data aging time of each data source with the corresponding allowed acceptance time, and considering whether new data has arrived within the current integration cycle, to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired and invalid, includes: If new data arrives during the current integration cycle and the corresponding data aging time does not exceed the corresponding allowed acceptance time, then the status of the corresponding data source will be classified as newly arrived and valid. If no new data arrives during the current integration cycle and the corresponding data aging time does not exceed the corresponding allowed acceptance time, then the status of the corresponding data source will be classified as continued valid. If the corresponding data aging time exceeds the corresponding allowed acceptance time, the corresponding data source will be classified as expired.

5. The multi-source information access method based on real-time micro-meteorological data according to claim 4, characterized in that, The step of comparing the data aging time of each data source with the corresponding allowed acceptance time, and combining this with whether new data has arrived within the current integration cycle, to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired and invalid, also includes: A register is maintained for each data source to record the observation timestamp of the latest valid physical observation. When a new message arrives, if the data aging time of the corresponding data source exceeds the corresponding allowed acceptance time, and the observation timestamp of the new message is later than the observation timestamp recorded in the register, then a marker for a late valid physical frame is added to the status of the corresponding data source as an identifier to trigger asynchronous backtracking compensation processing in subsequent steps.

6. The multi-source information access method based on real-time micrometeorological data according to claim 5, characterized in that, The steps of replacing expired data from data sources with preset missing data markers and extracting valid data from newly arrived and continuing valid data sources for integrated output include: Replace the data from data sources that are in an expired or invalid state with a preset missing identifier, and extract the valid data from data sources that have just arrived in a valid state and data sources that continue to be valid, as the real-time integrated data stream for the current integration cycle. If any data source in the expired state has the tag of the late valid physical frame, then asynchronous backtracking compensation processing is triggered. The asynchronous backtracking compensation processing includes: Using the observation timestamp of the data source marked with the late valid physical frame as the search keyword, the corresponding historical integration target time is located from the historical integration output; wherein, the historical integration output is: the integration data stream generated in each integration cycle before the current integration cycle and the corresponding integration target time; Extract valid data from data sources other than the data source that generated the late valid physical frame at the target historical integration time, align and integrate it with the data of the late valid physical frame, and generate a historical compensation integration data packet; The historical compensation integration data packet is then asynchronously inserted into the output queue of the current integration cycle and pushed in combination with the real-time integration data stream.

7. The method for multi-source information access based on real-time micrometeorological data according to claim 1, characterized in that, The steps for extracting and integrating valid data from various data sources include: Using the second-level tick as a step size, the latest data is extracted from the cache queue of the data source in the newly arrived valid state and marked as read, and the data that was last read is extracted from the data source in the continued valid state. The extracted data from each data source is assembled into an integrated data package according to a predetermined data model, and output as the real-time integrated data stream for the current integration cycle.

8. The method for multi-source information access based on real-time micrometeorological data according to claim 1, characterized in that, The steps of attaching a multidimensional status identifier to the data from each data source, wherein the multidimensional status identifier includes a participation mode identifier indicating the data source type, a time decay attribute indicating the staleness of the data, and a margin of tolerance indicating the remaining validity time of the data, include: Assign a participation method identifier for newly arrived data categories to data sources in a newly arrived valid state; assign a participation method identifier for data sources in a continuously valid state to data categories that retain historical data categories; and assign a participation method identifier for data sources in an expired state to data categories that have missing data categories. Use the data aging duration of each data source as the corresponding time-decrease attribute; The difference between the allowed acceptance time and the data aging time for each data source is used as the corresponding acceptance boundary margin. The participation method identifier, the time-decrease attribute, and the acceptance boundary margin are used as multi-dimensional state identifiers and appended to the data of the corresponding data source.

9. A multi-source information access method based on real-time micrometeorological data according to claim 1, characterized in that, The step of adaptively adjusting the timing characteristics and allowable duration of each data source based on the status of each data source in the integration output of each integration cycle includes: The statistical distribution of the proportions of newly arrived valid, continued valid, and expired invalid data sources was analyzed in a preset number of integration cycles. If the proportion of data sources in the newly arrived valid state is lower than the preset threshold for the proportion of newly arrived valid data sources, the time-series characteristics and allowed acceptance time of the corresponding data sources will be recalculated. If the proportion of data sources that are in a continuing valid state is higher than the preset continuing valid proportion threshold, then the time-series characteristics and allowed acceptance duration of the corresponding data sources will be recalculated. If the proportion of data sources that are in an expired state is higher than the preset expired failure ratio threshold, the confidence coefficient of the dynamic tolerance margin of the corresponding data source will be increased to broaden the corresponding allowable acceptance period.

10. A multi-source information access system based on real-time micro-meteorological data, applied to the multi-source information access and integration during the approach of severe convective weather in the power grid operation environment, characterized in that... The system includes: The information acquisition module is used to acquire the time-series characteristics of multiple data sources in the power grid environment. The data sources include regional meteorological stations, radar networks, and self-built micro meteorological stations in substations. The time-series characteristics include the average update cycle and the standard deviation of the update interval. The range calculation module is used to dynamically calculate the allowable duration of each data source based on the time series characteristics. The duration calculation module is used to calculate the time difference between the observation timestamp of the latest data from each data source and the integrated target time of the second-level beat driven by the severe convective weather warning model, which serves as the data aging duration for the corresponding data source. The status classification module is used to compare the data aging time of each data source with the corresponding allowed acceptance time, and combine it with whether new data has arrived in the current integration cycle to classify the status of each data source into one of the following states: newly arrived and valid, continued valid, and expired and invalid. The integrated output module is used to replace the data from expired data sources with preset missing identifiers if there are expired data sources, and extract the valid data from newly arrived valid and continuing valid data sources for integrated output; otherwise, it extracts the valid data from each data source for integrated output. An identifier attachment module is used to attach a multi-dimensional status identifier to the data from each data source during the integrated output. The multi-dimensional status identifier includes an participation mode identifier to indicate the data source type, a time decay attribute to indicate the staleness of the data, and a boundary margin to indicate the remaining valid time of the data. The parameter adjustment module is used to adaptively adjust the timing characteristics and allowable duration of each data source based on the status of each data source in the integration output of each integration cycle.

Images