An edge collection-based water affair industrial control data aggregation storage method and system

Abstract

This invention provides a method and system for aggregating and storing water management control data based on edge acquisition, relating to the field of data processing technology. The method includes: deploying edge acquisition terminals to collect multi-source water management control data; performing quality cleaning to obtain standardized water management control data; performing real-time event detection and feature value extraction on the standardized water management control data at the edge side to generate target water management control data; based on the target water management control data and network perception, executing local reliable caching, online sending, and breakpoint resumption control according to preset priorities to obtain a data reporting stream; establishing a hierarchical message buffer and asynchronous decoupling channel on the central platform to persistently access and aggregate the data reporting stream to obtain a data aggregation stream; performing fine-grained classification and hierarchical storage processing on the data aggregation stream to obtain traceable hierarchical storage results; and providing a unified data service interface for the traceable hierarchical storage results to support on-demand subscription and distribution, completing the data value closed loop.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for summarizing and storing water management control data based on edge acquisition. Background Technology

[0002] With the continuous advancement of smart water management, the scale of industrial control data generated during water plant production and operation is constantly expanding. The data types encompass multi-source information such as PLC control quantities, smart instrument measurements, pump unit status quantities, chemical dosing parameters, and water quality parameters. To support real-time monitoring, trend analysis, process optimization, equipment health diagnosis, and centralized group-level supervision of water plants, there is an urgent need to build a data infrastructure capable of highly reliable acquisition, aggregation, storage, and sharing of multi-source heterogeneous data from the field. This will enable the end-to-end availability, reliability, and traceability of water plant industrial control data.

[0003] Currently, most common information systems or SCADA systems in water plants adopt a centralized data flow architecture, where field devices (PLCs, instruments, sensors, etc.) are connected to the host computer or central SCADA system through an industrial network, and the management platform then completes data entry, analysis, and alarm processing.

[0004] However, existing waterworks industrial control data acquisition and storage solutions still have significant shortcomings. The numerous brands of equipment at water plants and the highly heterogeneous industrial communication protocols result in inconsistent data access interfaces, making system integration, expansion, and maintenance difficult and time-consuming. Furthermore, existing secure data acquisition methods are heavily reliant on continuous and stable network connections. Network fluctuations or interruptions can easily lead to data loss and gaps in historical data sequences, making it difficult to guarantee data integrity and temporal continuity. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a method for aggregating and storing water management industrial control data based on edge acquisition. This method can solve the problems of numerous brands of equipment at water plant sites and highly heterogeneous industrial communication protocols, which lead to inconsistent data access interfaces, making system integration, expansion, and maintenance difficult and time-consuming. Furthermore, existing secure acquisition methods are highly dependent on continuous and stable network connections. Once the network fluctuates or is interrupted, data transmission loss and gaps in historical data sequences are easily caused, making it difficult to guarantee data integrity and temporal continuity.

[0006] A first aspect of this invention proposes a method for aggregating and storing water management control data based on edge acquisition, comprising: S1: Deploy multi-protocol adaptive edge acquisition terminals to collect multi-source water management control data; S2: Perform quality cleaning on the multi-source water control data to obtain standardized water control data; S3: Based on the sliding window model and water affairs rule base, perform edge-side real-time event detection and feature value extraction on the standardized water affairs industrial control data to generate target water affairs industrial control data including alarm event data and event summary data; S4: Based on the target water affairs industrial control data and network perception, execute local reliable caching, online transmission and breakpoint resume control according to preset priority to obtain a data reporting stream that can be received by the central platform; S5: Establish a hierarchical message buffer and asynchronous decoupling channel on the central platform to persistently access and aggregate the data reporting stream, thereby obtaining a data aggregation stream; S6: Perform fine-grained classification and hierarchical storage processing on the data aggregation stream to obtain traceable hierarchical storage results.

[0007] S7: Provide a unified data service interface for the traceable hierarchical storage results to support on-demand subscription and distribution, and complete the data value closed loop.

[0008] A second aspect of this invention provides a water resources industrial control data aggregation and storage system based on edge acquisition, comprising: a processor and a memory; The memory stores programs or instructions that can run on the processor, and when the programs or instructions are executed by the processor, they implement the steps of the water resources industrial control data aggregation and storage method based on edge acquisition as described in the first aspect.

[0009] A third aspect of the present invention provides a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the water resources industrial control data aggregation and storage method based on edge acquisition as described in the first aspect.

[0010] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, multi-protocol adaptive edge acquisition terminals are deployed to achieve unified acquisition and protocol adaptation of multi-source industrial control data. The acquired data is then cleaned and standardized. This is complemented by a hierarchical message buffer and asynchronous decoupling channel established on the central platform, along with a unified data service interface. This enables unified data access, aggregation, and service output, thereby reducing system integration complexity and improving expansion and maintenance efficiency. Simultaneously, network awareness and priority control mechanisms are introduced to implement local reliable caching, online transmission, and breakpoint resumption. Combined with persistent buffer access and traceable hierarchical storage processing on the central side, this ensures that data can be cached, retransmitted, and traced in the event of network anomalies, guaranteeing data integrity and temporal continuity. Attached Figure Description

[0011] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0012] Figure 1 This is a flowchart illustrating a water resources industrial control data aggregation and storage method based on edge acquisition, provided in an embodiment of the present invention.

[0013] Figure 2 This is a schematic diagram of the structure of a water affairs industrial control data aggregation and storage system based on edge acquisition provided in an embodiment of the present invention. Detailed Implementation

[0014] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. It should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0015] The following description, in conjunction with the accompanying drawings, details the water resources industrial control data aggregation and storage method based on edge acquisition provided by the present invention through specific embodiments and application scenarios.

[0016] Reference manual attached Figure 1 The diagram illustrates a flowchart of a water resources industrial control data aggregation and storage method based on edge acquisition, provided by an embodiment of the present invention.

[0017] This invention provides a method for aggregating and storing water management control data based on edge acquisition, which may include the following steps: S1: Deploy multi-protocol adaptive edge acquisition terminals to collect multi-source water management control data.

[0018] In one possible implementation, S1 specifically includes: S101: Deploy embedded intelligent data acquisition terminals in PLC cabinets, pump station control boxes, and groundwater monitoring wells at water sites.

[0019] Specifically, the data acquisition terminal integrates multiple analog / digital acquisition interfaces as well as industrial communication interfaces such as RS485 and Ethernet, which are used to establish data links with field controllers, instruments and sensors.

[0020] S102: Load the pluggable protocol driver library into the intelligent acquisition terminal, and automatically identify the access device protocol based on port characteristics, handshake messages, register probes or device identification information. After identifying the protocol type, call the corresponding protocol driver to obtain water industrial control data.

[0021] S103: Map multi-source water management control data into a unified standard data object.

[0022] Specifically, standard data objects include at least the location identifier tag_id, device identifier device_id, site identifier site_id, timestamp ts, collected value value, unit unit, and data source protocol identifier proto_id, in order to form a consistent data input format.

[0023] S104: Generates a sequence number for standard data objects for subsequent local caching and resume interrupted downloads.

[0024] Specifically, the intelligent data acquisition terminal establishes a sequence counter locally for each station, device, and point, and persistently saves the current value of the counter. Whenever a new standard data object is generated, the terminal first reads the value of the corresponding counter as the sequence number of this data, then increments the counter by 1 and writes it back to local storage, thereby ensuring that the data sequence numbers of the same measurement point are continuous and non-repeating.

[0025] S105: Encapsulate and aggregate multiple standard data objects carrying serial numbers according to preset encapsulation rules to obtain multi-source water management control data.

[0026] The encapsulation and aggregation process includes sorting and encapsulating according to timestamps and serial numbers, or batch encapsulating according to preset time windows.

[0027] In this embodiment of the invention, by deploying multi-protocol adaptive edge acquisition terminals at water sites, local access and stable acquisition of heterogeneous devices such as PLCs, instruments and sensors are achieved. Pluggable protocol drivers and automatic identification mechanisms are used to reduce manual configuration costs and improve the versatility and scalability of new device access.

[0028] S2: Perform quality cleaning on multi-source water control data to obtain standardized water control data.

[0029] In one possible implementation, quality cleaning specifically involves: The system invokes a pre-defined set of quality rules to perform real-time verification and filtering on multi-source water management control data.

[0030] Optionally, the quality rule set includes upper and lower limits of the measurement range, a rate of change threshold, a dead zone range, and an abnormal jump threshold to eliminate invalid values ​​and spike jump values.

[0031] In this embodiment of the invention, by calling a pre-set quality rule set at the edge to perform real-time verification and filtering of multi-source water management control data, invalid or unreliable data such as out-of-range measurements, abnormal rates of change, dead-zone jitter, and spike jumps can be eliminated in advance before the data enters subsequent event detection and central aggregation. This reduces the proportion of noisy data from the source and reduces meaningless data transmission and storage overhead. At the same time, it ensures that the output standardized water management control data has higher numerical continuity, stability, and reliability, making subsequent sliding window feature calculation and threshold determination more accurate.

[0032] S3: Based on the sliding window model and water affairs rule base, perform real-time event detection and feature extraction on the edge side of standardized water affairs industrial control data to generate target water affairs industrial control data including alarm event data and event summary data.

[0033] The sliding window model is an online processing method for continuous time series data. Its basic idea is to maintain a fixed-length window (defined by time or by the number of samples) on the data stream. Only the most recent data segment is retained in the window, and the window is continuously updated by sliding forward as new data arrives.

[0034] Among them, the water affairs rule base is a set of rules that are structured and precipitated from water affairs operation experience, process constraints and management standards. It is mainly used to guide the edge side or the center side to make threshold judgments and event recognition on data.

[0035] In one possible implementation, S3 specifically includes: S301: Standardized water management control data is used as window input data, and the window input data is diverted according to the point location to obtain diverted window input data.

[0036] Specifically, a point identifier is an ID used to uniquely identify a collection point / measuring point / variable, which is equivalent to assigning "each sensor parameter" in a water treatment site.

[0037] S302: Perform digital filtering and statistical calculations on the input data of the split window within each sliding window to obtain the window feature values.

[0038] Optionally, digital filtering includes one of low-pass filtering, band-pass filtering, median filtering, and exponential smoothing. Statistical calculations include moving mean, moving variance, maximum value, and minimum value.

[0039] S303: Compare the window feature value with the threshold rule in the water affairs rule base. When the window feature value meets the threshold rule, generate alarm event data.

[0040] It's important to note that threshold rules are essentially a type of judgment rule that uses numerical boundaries to determine whether something is normal or whether an event has been triggered. They explicitly define the allowable range, rate of change, or fluctuation amplitude of a specific point (such as water level, pressure, flow rate, turbidity, residual chlorine, etc.) under specific operating conditions using thresholds (upper limit / lower limit / rate of change / duration). When real-time data or window characteristic values ​​exceed / fall below / continuously meet these boundary conditions, an anomaly is considered to have occurred or an alarm is required.

[0041] Optionally, if the threshold rule is not met, no event is generated, and window scrolling continues. Optionally, "No event flag / statistic" is written to the state cache for trend analysis, but it does not enter the event flow.

[0042] S304: Extract event feature triples from alarm event data and generate event summary data.

[0043] Optionally, the event feature triple includes duration, average amplitude, and peak value.

[0044] S305: Encapsulate alarm event data and event summary data into target water management control data.

[0045] In this embodiment of the invention, the advantage of this design is that by using a sliding window model to retain only the "most recent data segment" at the edge and calculating window feature values ​​such as mean, variance, and extreme values ​​in real time, abnormal fluctuations in water level, pressure, flow rate, and water quality indicators can be quickly captured without relying on full historical data. Simultaneously, digital filtering suppresses noise and spike interference, improving the stability of event judgment. Furthermore, comparing the window feature values ​​with threshold rules in the water management rule base solidifies operational experience and process constraints into configurable and reusable judgment logic, thereby achieving differentiated alarm identification for different stations and locations, reducing false alarms and missed alarms.

[0046] Furthermore, after an alarm is triggered, event summary triples such as "duration, average amplitude, and peak value" are extracted. This allows the central side to quickly complete analysis and graded response without transmitting and processing the full time series, significantly reducing bandwidth and storage overhead. At the same time, alarm event data and event summary data are uniformly packaged into target water management industrial control data, which facilitates subsequent reliable caching, breakpoint resumption, and central aggregation and storage according to priority. This ensures that critical alarm information is prioritized, data is traceable, and reconciliation is possible in weak network / network outage scenarios.

[0047] S4: Based on the target water management control data and network perception, execute local reliable caching, online transmission and breakpoint resume control according to preset priorities to obtain a data reporting stream that can be received by the central platform.

[0048] Among them, network perception is the ability of edge acquisition terminals to monitor and judge in real time whether the current network is usable, how well it works, and which network is more suitable.

[0049] Specifically, the terminal will periodically detect the currently available network type (such as Ethernet / 4G / 5G / private network) and link quality indicators (such as packet loss rate, latency, bandwidth, retransmission count, connection stability, etc.), and form a network status identifier from these results.

[0050] In one possible implementation, S4 specifically includes: S401: Enable network status monitoring on the edge terminal, periodically detect available network type, link quality, packet loss rate, latency and retransmission count to form a network status identifier.

[0051] S402: Configure a non-volatile local cache library to write regular time-series data, alarm events, and event summaries into independent cache queues according to different priorities.

[0052] Optionally, the non-volatile local cache library includes one of MicroSD, eMMC, Flash, or local file system queues.

[0053] Specifically, the different priorities are: alarm events > event summaries > regular timing.

[0054] S403: Through an adaptive transmission strategy, regular time-series data, alarm event data, and event summary data in each cache queue are transmitted to form a data reporting stream that can be received by the central platform.

[0055] In one possible implementation, the adaptive transmission strategy specifically includes: When the network status indicator shows online and the link is available, alarm events are sent immediately. Event summary data is sent first. Regular time-series data is sent at set intervals.

[0056] When the network status indicator indicates a network outage or unstable link, the data to be sent is reliably written to disk and the breakpoint is frozen. After the network is restored, the breakpoint is resumed according to the priority order of alarm events, event summaries, and regular time-series data, and the breakpoint is updated after each transmission is completed.

[0057] Optionally, the breakpoint information includes the sequence number and offset.

[0058] It should be noted that after each transmission is completed, the breakpoint is updated specifically by the center returning confirmation information, and the edge acquisition terminal updates the breakpoint based on the confirmation information.

[0059] In this embodiment of the invention, by acquiring link availability and quality status in real time through network sensing, the edge terminal can dynamically select sending or caching strategies based on different network states such as "online stable / unstable / interrupted," thereby avoiding bandwidth waste and data congestion caused by blind retransmission in weak network environments. Simultaneously, alarm events, event summaries, and regular time-series data are managed in priority queues, ensuring that critical alarms are immediately sent when the network is normal and are still prioritized when the network is restricted, thus improving the response time to abnormal conditions.

[0060] Furthermore, by combining non-volatile local caching to disk and breakpoint freezing mechanisms, data to be transmitted can be reliably saved during network outages or link fluctuations. Once the network is restored, the data can be resumed in the order of "alarm event > event summary > normal timing". The breakpoints are advanced through central confirmation receipts, which not only ensures that no data is lost or the order is broken during the retransmission process, but also helps to prevent duplicate retransmissions with the central side's idempotent deduplication. Ultimately, this significantly enhances the data integrity, continuity and traceability of water affairs sites under complex network conditions.

[0061] S5: Establish a hierarchical message buffer and asynchronous decoupling channel on the central platform to persistently access and aggregate the data reporting stream, thereby obtaining the data aggregation stream.

[0062] The central platform is a "unified access and management hub" for water industry control data deployed in a data center or cloud. It is used to centrally receive data reporting streams from multiple edge acquisition terminals and complete subsequent aggregation, buffering, parsing, hierarchical storage, and external services.

[0063] In one possible implementation, S5 specifically includes: S501: Deploy a message queue cluster on the central platform. The message queue cluster is used to perform hierarchical buffering and persistent access to data reporting streams from multiple edge acquisition terminals.

[0064] It should be noted that the message queue cluster is configured with persistent storage policies, replication mechanisms, and retry mechanisms to ensure reliable message access.

[0065] S502: Establishes independent partitions for each edge acquisition terminal, and configures message category identifiers for alarm event data, event summary data and regular time series data respectively, so as to realize hierarchical bearing and isolation buffering according to data type and priority.

[0066] S503: The edge acquisition terminal constructs the data reporting stream according to the preset encapsulation rules and pushes it to the corresponding partition.

[0067] S504: The central platform subscribes to each partition and pulls the data reporting stream to write to the central side memory buffer queue, forming a data aggregation stream.

[0068] It should be noted that this also includes commit and rollback control for message queue consumption points to enable data replay in case of abnormal restarts or fault recovery. Furthermore, traceability of the aggregation process is achieved based on sequence numbers and consumption point records.

[0069] In this embodiment of the invention, by introducing a message queue cluster as a hierarchical message buffer and asynchronous decoupling channel into the central platform, the "edge reporting" and "central processing / database entry" can be decoupled in terms of timing. This avoids direct impact on the database and business services when edge terminals send concurrent data or when the network recovers and retransmits data centrally, thereby improving the system's peak resistance capability and overall stability. Simultaneously, the message queue employs persistent storage, a replication mechanism, and a retry mechanism, ensuring that messages are not lost and can be recovered in the event of node failure or short-term anomalies, enhancing access reliability.

[0070] S6: Perform fine-grained classification and hierarchical storage processing on the data aggregation stream to obtain traceable hierarchical storage results.

[0071] In one possible implementation, S6 specifically includes: S601: The central platform parses the obtained data aggregation stream, extracts the sequence number, timestamp, reference key and data type from the data aggregation stream, and forms structured data entry records.

[0072] Specifically, the message header and body are read according to the data packet encapsulation format (e.g., JSON, binary frames, or key-value pair messages). The site identifier, location identifier, and protocol identifier carried in the message header are used to locate the data source. Then, the sequence number and timestamp are extracted from the message body or message header. If the data packet contains a reference key, the reference key is extracted simultaneously. The data type is also parsed to distinguish between regular time-series data, alarm event data, and event summary data. Finally, the fields obtained from the above parsing are written into a structured database record.

[0073] S602: Perform idempotent deduplication on the structured inbound records to obtain refined structured inbound records.

[0074] Among them, idempotent deduplication refers to the processing mechanism that ensures "the same data only takes effect once and the data entry results are consistent" even when data may be reported or consumed repeatedly.

[0075] It should be noted that the idempotent deduplication process is based on (timestamp, sequence number, reference key) to determine duplicates and discards or overwrites duplicate data to support duplicate retransmission in the breakpoint resume scenario of the previous steps.

[0076] S603: Based on data type, perform fine-grained category classification on the finely structured inbound records and generate corresponding category data streams. Categories include four types: raw time-series data, cleaned time-series data, alarm event data, and event summary data, and generate corresponding category data streams for each.

[0077] S604: Perform dual-database separate storage for the original time-series data and the cleaned time-series data. Write the time-series data carrying the original quality labels into the time-series history database. Write the cleaned time-series data into the high-quality time-series database.

[0078] S605: Write alarm event data and event summary data into a relational database and establish an associated index with device metadata and location metadata.

[0079] S606: Perform aggregation calculations on the cleaned time-series data according to a preset aggregation cycle, write the aggregation results into the storage layer, and save the event identifier and corresponding reference key in the event table to obtain traceable hierarchical storage results.

[0080] The preset aggregation period is 1 minute, 5 minutes, or a custom window.

[0081] It should be noted that the reference key is used to associate the original waveform or original time series data of the corresponding time period.

[0082] In this embodiment of the invention, the central platform first parses the data aggregation stream and extracts the sequence number, timestamp, reference key, and data type to form a structured data entry record. This ensures that data with different encapsulation formats is uniformly "structured" before being entered into the database, facilitating subsequent consistent processing and source location. Furthermore, an idempotent deduplication mechanism is used to prevent duplicate transmissions caused by breakpoint resumption, message retries, or repeated consumption, ensuring consistent data entry results and preventing duplicate accumulation of statistics.

[0083] Furthermore, records are categorized into four types based on data type: raw time series, cleaned time series, alarm events, and event summaries. A hierarchical storage strategy of "dual database separation + event insertion into a relational database" is adopted, ensuring that raw data is auditable and traceable, cleaned data is efficiently analyzed, and events and summaries can be quickly retrieved and associated with device / location metadata. Finally, the cleaned time series data is aggregated at the minute level or in a custom window, and the original waveform / original time period data is associated in the event table using event identifiers and reference keys. This achieves a closed loop of "lightweight indicators can be used quickly, and original evidence can be traced at any time," thereby simultaneously improving storage and query efficiency.

[0084] S7: Provides a unified data service interface for traceable and hierarchical storage results to support on-demand subscription and distribution, thus completing the data value loop.

[0085] Specifically, the central platform constructs a unified data service layer based on the traceable, hierarchical storage results generated by the above steps. It uniformly catalogs and encapsulates the raw time-series data, cleaned time-series data, alarm event data, event summary data, and aggregated analysis results, forming a unified data service interface. This unified data service interface includes a RESTful interface for historical queries and a subscription interface for real-time subscriptions. The subscription interface can be implemented using WebSocket, MQTT subscriptions, or message push. Building upon this, the central platform configures on-demand subscription strategies for different business applications such as SCADA dashboards, mobile maintenance applications, AI model training platforms, and regulatory data transmission platforms, providing data services such as real-time data streams, historical interval queries, event notifications, and statistical aggregation results according to business needs. Finally, standardized water management control data, alarm information, and analysis results are converted and controlled according to the data format and permission requirements of the target system before being distributed to the group-level data lake or the superior regulatory platform. Data traceability is maintained through distribution logs and reference keys or event identifiers, thereby achieving cross-level collaboration and closed-loop delivery of data value.

[0086] In this embodiment of the invention, by constructing a unified data service layer on a central platform and uniformly cataloging and encapsulating the original time series, cleaned time series, alarm events, event summaries, and aggregation results, different business systems no longer need to connect to multiple databases or processing links separately, thereby significantly reducing data access and system integration costs. Simultaneously, during the distribution process, conversion and control are performed according to target formats and permission requirements, and end-to-end traceability is maintained through distribution logs combined with reference keys / event identifiers. This ensures that when sharing data across levels, the data source is traceable, the process is controllable, and the results are recalculated, thus truly transforming the data assets formed through collection and storage into a closed loop of sustainable business value output.

[0087] Reference manual attached Figure 2 The diagram shows a structural schematic of a water management control data aggregation and storage system based on edge acquisition, provided by an embodiment of the present invention.

[0088] This invention provides a water resources industrial control data aggregation and storage system 20 based on edge acquisition, comprising: a processor 201 and a memory 202; The memory 202 stores programs or instructions that can run on the processor 201. When the program or instructions are executed by the processor 201, they implement the steps of the above-described method for summarizing and storing water management control data based on edge acquisition, and can achieve the same technical effect. To avoid repetition, the present invention will not elaborate further.

[0089] It should be understood that the processor 201 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0090] It should also be understood that the memory 202 in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM).

[0091] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0092] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0093] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0095] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0098] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] This invention provides a readable storage medium comprising: storing a program or instructions on the readable storage medium, wherein when the program or instructions are executed by a processor, the program or instructions implement the steps of the above-described method for summarizing and storing water management industrial control data based on edge acquisition, and can achieve the same technical effect. To avoid repetition, this invention will not elaborate further.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for aggregating and storing water management control data based on edge acquisition, characterized in that, include: S1: Deploy multi-protocol adaptive edge acquisition terminals to collect multi-source water management control data; S2: Perform quality cleaning on the multi-source water control data to obtain standardized water control data; S3: Based on the sliding window model and water affairs rule base, perform edge-side real-time event detection and feature value extraction on the standardized water affairs industrial control data to generate target water affairs industrial control data including alarm event data and event summary data; S4: Based on the target water affairs industrial control data and network perception, execute local reliable caching, online transmission and breakpoint resume control according to preset priority to obtain a data reporting stream that can be received by the central platform; S5: Establish a hierarchical message buffer and asynchronous decoupling channel on the central platform to persistently access and aggregate the data reporting stream, thereby obtaining a data aggregation stream; S6: Perform fine-grained classification and hierarchical storage processing on the data aggregation stream to obtain traceable hierarchical storage results; S7: Provide a unified data service interface for the traceable hierarchical storage results to support on-demand subscription and distribution, and complete the data value closed loop.

2. The method for aggregating and storing water management control data based on edge acquisition according to claim 1, characterized in that, S1 specifically includes: S101: Deploy embedded intelligent data acquisition terminals in PLC cabinets, pump station control boxes, and groundwater monitoring wells at water sites; S102: Load the pluggable protocol driver library into the intelligent acquisition terminal, and automatically identify the access device protocol based on port characteristics, handshake messages, register probes or device identification information. After identifying the protocol type, call the driver of the corresponding protocol to obtain the water affairs industrial control data. S103: Map the multi-source water management control data into a unified standard data object; S104: Generate a sequence number for the standard data object for subsequent local caching and breakpoint resume; S105: According to the preset encapsulation rules, multiple standard data objects carrying serial numbers are encapsulated and aggregated to obtain the multi-source water affairs industrial control data.

3. The method for aggregating and storing water management control data based on edge acquisition according to claim 1, characterized in that, The quality cleaning specifically refers to: The system invokes a pre-set set of quality rules to perform real-time verification and filtering on multi-source water control data. The set of quality rules includes upper and lower limits of measurement range, threshold of rate of change, dead zone range, and threshold of abnormal jumps, in order to eliminate invalid values ​​and peak jump values.

4. The method for aggregating and storing water management control data based on edge acquisition according to claim 1, characterized in that, S3 specifically includes: S301: The standardized water control data is used as window input data, and the window input data is diverted according to the point identification to obtain diverted window input data; S302: Perform digital filtering and statistical calculations on the input data of the split window within each sliding window to obtain the window feature values; S303: Compare the window feature value with the threshold rule in the water affairs rule base. When the window feature value meets the threshold rule, generate alarm event data. S304: Extract the event feature triplet from the alarm event data and generate event summary data; S305: Encapsulate the alarm event data and the event summary data into the target water management control data.

5. The method for aggregating and storing water management control data based on edge acquisition according to claim 1, characterized in that, S4 specifically includes: S401: Enable network status monitoring at the edge terminal, periodically detect available network type, link quality, packet loss rate, latency and retransmission count to form a network status identifier; S402: Configure a non-volatile local cache library to write regular time-series data, alarm events, and event summaries into independent cache queues according to different priorities; S403: Through an adaptive transmission strategy, the regular time-series data, alarm event data and event summary data in each of the cache queues are transmitted to form a data reporting stream that can be received by the central platform.

6. The method for aggregating and storing water management control data based on edge acquisition according to claim 5, characterized in that, The adaptive transmission strategy specifically includes: When the network status indicator meets the conditions of being online and the link is available, alarm events are sent immediately; event summary data is sent with priority; and regular time-series data is sent according to a set period. When the network status indicator indicates a network outage or link instability, the data to be sent is reliably written to disk and the breakpoint is frozen. After the network is restored, the breakpoint is resumed according to the priority order of alarm events, event summaries, and regular time-series data, and the breakpoint is updated after each transmission is completed.

7. The method for aggregating and storing water management control data based on edge acquisition according to claim 1, characterized in that, S5 specifically includes: S501: Deploy a message queue cluster on the central platform. The message queue cluster is used for hierarchical buffering and persistent access to data reporting streams from multiple edge acquisition terminals. S502: Establish independent partitions for each of the edge acquisition terminals, and configure message category identifiers for alarm event data, event summary data and regular time series data respectively, so as to realize hierarchical bearing and isolation buffering according to data type and priority; S503: The edge acquisition terminal constructs a data reporting stream according to a preset encapsulation rule and pushes it to the corresponding partition; S504: The central platform subscribes to each of the partitions and pulls the data reporting streams into the central side memory buffer queue to form a data aggregation stream.

8. The method for aggregating and storing water management control data based on edge acquisition according to claim 1, characterized in that, S6 specifically includes: S601: The central platform parses the obtained data aggregation stream, extracts the sequence number, timestamp, reference key and data type from the data aggregation stream, and forms a structured database entry record; S602: Perform idempotent deduplication on the structured inbound records to obtain refined structured inbound records; S603: Based on the data type, perform fine-grained classification of the finely structured inbound records and generate corresponding classification data streams; the categories include four types: raw time-series data, cleaned time-series data, alarm event data, and event summary data, and generate corresponding classification data streams for each; S604: Perform dual-database separate storage on the original time series data and the cleaned time series data, respectively writing the time series data carrying the original quality label into the time series history database; and writing the cleaned time series data into the high-quality time series database; S605: Write the alarm event data and the event summary data into a relational database, and establish an associated index with the device metadata and the location metadata; S606: Perform aggregation calculations on the cleaned time-series data according to a preset aggregation cycle, write the aggregation results into the storage layer, and save the event identifier and corresponding reference key in the event table to obtain traceable hierarchical storage results.

9. A water management industrial control data aggregation and storage system based on edge acquisition, characterized in that, include: Processor and memory; The memory stores programs or instructions that can run on the processor, and when the program or instructions are executed by the processor, they implement the steps of the water resources industrial control data aggregation and storage method based on edge acquisition as described in any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of the water resources industrial control data aggregation and storage method based on edge acquisition as described in any one of claims 1 to 8.

Images