Data collaborative management method and system of multi-site water conservancy monitoring network

Abstract

This application relates to the field of data processing technology and discloses a data collaborative management method and system for a multi-site water conservancy monitoring network. The method includes: hierarchically identifying and encoding multiple monitoring stations to generate global identifiers and encapsulating them into standard data packets; scheduling transmission based on geographical location and station type using a time-division multiplexing mechanism; calculating inter-site correlation consistency indicators using a multi-dimensional data quality assessment algorithm for anomaly correction; storing data in time-division segments and constructing a three-dimensional collaborative index system; and extracting multi-site monitoring data to calculate fusion indicators and generate collaborative decision-making suggestions. This application solves the problems of inconsistent identification of multi-site water conservancy monitoring data, lack of coordinated transmission, difficulty in ensuring quality, low query efficiency, and inability to conduct collaborative decision-making in existing technologies, thereby improving the data management efficiency and the scientific nature of irrigation decisions in multi-site water conservancy monitoring networks.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data collaborative management method and system for a multi-site water conservancy monitoring network. Background Technology

[0002] Existing water conservancy monitoring systems typically deploy various types of monitoring stations within irrigation districts, such as soil moisture stations, field flow stations, and headworks flow stations, to collect key parameters like soil moisture content and irrigation flow. These stations transmit the collected data to a cloud management platform via wireless communication networks. Managers can then view the monitoring data from each station on the platform to understand the irrigation water usage in the district. For data transmission, each monitoring station typically uses wireless communication technologies such as NB-IoT, GPRS, or 4G to upload data to the cloud. For data storage, the cloud platform stores the received data in a database for subsequent querying and analysis. For data analysis, managers use the data reported by each station to evaluate irrigation effectiveness and develop irrigation plans.

[0003] However, existing technologies have the following shortcomings: First, in terms of data identification, different types of monitoring stations use their own independent coding rules, lacking a unified station identification system, resulting in inconsistent data formats and difficulty in cross-site data correlation analysis; second, in terms of data transmission, each monitoring station transmits data independently without a coordination mechanism, which can easily cause network congestion and data transmission delays when multiple stations transmit simultaneously, affecting the real-time performance of the data; third, in terms of data quality management, existing systems usually only independently verify the data of a single station, lacking a mechanism for cross-validation using the inherent correlation between data from multiple stations, making it difficult to accurately identify and correct abnormal data; in addition, in terms of data storage, existing systems usually adopt a centralized storage method, lacking sharding and index optimization for the characteristics of multi-site time-series data, resulting in a significant decrease in query efficiency when the data volume increases.

[0004] The lack of a unified site identification system and data encapsulation standards in existing technologies further exacerbates the problem. This leads to the inability to differentiate data transmission and scheduling based on site attributes, resulting in the mixed transmission of high-priority and low-priority data and compromising the real-time performance of critical data. Furthermore, the absence of a quality assessment mechanism based on inter-site data relationships makes it difficult to identify systemic anomalies by relying solely on single-site inspections. For example, when the headworks water supply is normal but the field flow is abnormally reduced, it's impossible to determine whether it's channel leakage or flow meter malfunction, leading to inaccurate location of abnormal data sources. The lack of a multi-dimensional indexing system for multi-site data necessitates traversing all data for filtering when comprehensively analyzing data from multiple sites in a specific irrigation district over a specific time period, resulting in low query efficiency and difficulty in supporting real-time decision-making needs. Finally, the inability to quickly and accurately extract and integrate multi-site data makes it difficult to calculate indicators reflecting the entire process of irrigation water utilization from the headworks to the field and then to the soil, hindering the generation of scientific decision-making recommendations based on multi-site collaborative analysis. Summary of the Invention

[0005] This application provides a data collaborative management method and system for a multi-site water conservancy monitoring network. It establishes a multi-level site identification coding system and standard data packet encapsulation rules to achieve unified identification and formatted processing of data from different types of monitoring sites. It avoids network conflicts through a time-division multiplexing transmission scheduling mechanism based on site attributes, utilizes a multi-dimensional data quality assessment algorithm to perform cross-validation and anomaly correction based on inter-site relationships, achieves rapid retrieval of multi-site data through time-division sharding storage and a three-dimensional collaborative index system, and calculates comprehensive indicators such as water utilization coefficient and soil irrigation efficiency through multi-site data fusion. This solves the problems of inconsistent data identification, uncoordinated transmission, difficulty in ensuring quality, low query efficiency, and inability to conduct collaborative decision-making in existing multi-site water conservancy monitoring technologies, thereby improving the data management efficiency and the scientific nature of irrigation decisions in multi-site water conservancy monitoring networks.

[0006] Firstly, this application provides a data collaborative management method for a multi-site water conservancy monitoring network, the data collaborative management method for the multi-site water conservancy monitoring network comprising:

[0007] Step S1: Perform hierarchical identification and encoding processing on multiple monitoring stations to generate global identifiers, and encapsulate the raw data collected by each station into a standard data packet containing a global identifier, timestamp, and data quality identifier;

[0008] Step S2: Divide the transmission scheduling area according to the geographical location information of the site, set the transmission priority according to the site type, allocate transmission time slots through time division multiplexing mechanism, and generate transmission scheduling instructions;

[0009] Step S3: Perform a rationality check on the standard data package, extract data sequences from multiple stations within the same irrigation district, calculate the correlation consistency index between stations using a multi-dimensional data quality assessment algorithm, locate and correct abnormal data based on the correlation consistency index, and generate a collaborative dataset.

[0010] Step S4: Store the collaborative dataset in shards according to the time dimension, establish spatial index, time index and type index, and construct a three-dimensional collaborative index system;

[0011] Step S5: Extract monitoring data from multiple stations within the irrigation area from the three-dimensional collaborative index system, calculate multi-site fusion indicators, and generate collaborative decision-making suggestions based on the multi-site fusion indicators.

[0012] Secondly, this application provides a data collaborative management system for a multi-site water conservancy monitoring network, the data collaborative management system for the multi-site water conservancy monitoring network comprising:

[0013] The encoding module is used to perform hierarchical identification encoding processing on multiple monitoring stations, generate global identifiers, and encapsulate the raw data collected by each station into a standard data packet containing a global identifier, timestamp, and data quality identifier.

[0014] The partitioning module is used to divide the transmission scheduling area according to the geographical location information of the site, set the transmission priority according to the site type, allocate transmission time slots through the time division multiplexing mechanism, and generate transmission scheduling instructions.

[0015] The correction module is used to perform reasonableness checks on standard data packages, extract data sequences from multiple stations within the same irrigation district, calculate the correlation consistency index between stations through a multi-dimensional data quality assessment algorithm, locate and correct abnormal data based on the correlation consistency index, and generate a collaborative dataset.

[0016] The sharding module is used to shard and store the collaborative dataset according to the time dimension, and to build spatial index, time index and type index to construct a three-dimensional collaborative index system.

[0017] The extraction module is used to extract monitoring data from multiple stations within the irrigation area from the three-dimensional collaborative index system, calculate multi-site fusion indicators, and generate collaborative decision-making suggestions based on the multi-site fusion indicators.

[0018] Thirdly, a data collaborative management device for a multi-site water conservancy monitoring network is provided, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor invokes the instructions in the memory to cause the data collaborative management device for the multi-site water conservancy monitoring network to execute the aforementioned data collaborative management method for the multi-site water conservancy monitoring network.

[0019] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored in the computer and, when executed on a computer, cause the computer to perform the aforementioned data collaborative management method for a multi-site water conservancy monitoring network.

[0020] The technical solution provided in this application generates a global identifier by performing hierarchical identification and encoding processing on multiple monitoring stations, and encapsulates the raw data collected by each station into a standard data packet containing the global identifier, timestamp, and data quality identifier. This fundamentally solves the problem of inconsistent data formats and difficulty in cross-site correlation analysis among different types of monitoring stations in existing technologies. It ensures that data from different functional stations such as soil moisture stations, field flow stations, and canal head flow stations have a unified structure and identification system, facilitating subsequent data transmission scheduling, quality assessment, and fusion analysis. By dividing transmission scheduling areas according to the geographical location information of the stations, setting transmission priorities according to station type, and allocating transmission time slots through a time-division multiplexing mechanism to generate transmission scheduling instructions, it effectively avoids network congestion and data transmission delays caused by independent transmission from multiple stations in existing technologies, ensuring the real-time performance of critical monitoring data. Simultaneously, by setting differentiated transmission cycle settings, it reduces the transmission frequency of non-critical data, saving communication resources and station energy consumption. By performing a rationality check on standard data packets, data sequences from multiple stations within the same irrigation district are extracted. A multi-dimensional data quality assessment algorithm is used to calculate the correlation consistency index between stations. Based on the correlation consistency index, abnormal data is located and corrected to generate a collaborative dataset. This fully utilizes the inherent correlation between data from multiple stations for cross-validation, overcoming the limitation of existing technologies where independent testing of a single station is difficult to identify systematic anomalies. This significantly improves the accuracy of locating abnormal data sources and the reliability of data correction.

[0021] By storing collaborative datasets in time-series fragments and establishing spatial, temporal, and type indexes, a three-dimensional collaborative index system is constructed. This system optimizes the storage structure and indexing mechanism to address the temporal characteristics and multi-dimensional query requirements of multi-site water monitoring data. This allows for quick location of the target data fragment and storage partition via the index when querying data from a specific type of station within a specific time period in an irrigation district, eliminating the need to traverse the entire dataset. This significantly reduces the time complexity of data queries and solves the problem of low query efficiency caused by centralized storage in existing technologies. By extracting monitoring data from multiple stations within the irrigation district from the three-dimensional collaborative index system, multi-site fusion indicators such as water utilization coefficient and soil rehydration efficiency are calculated. Based on these multi-site fusion indicators, collaborative decision-making suggestions are generated, enabling scientific decision-making based on multi-site data collaborative analysis. This overcomes the shortcomings of existing technologies where single-site data analysis cannot comprehensively reflect the entire process of irrigation water utilization. It provides managers with a comprehensive decision-making basis that considers canal water transport losses, field irrigation effects, and soil absorption, thereby improving the scientific nature of irrigation management and the efficiency of water resource utilization. This application constructs a complete collaborative management technology system for multi-site water conservancy monitoring network data by unifying the identification, coordinating the transmission, collaborative quality control, efficient retrieval, and fusion analysis of multi-site data, providing an effective technical solution for the intelligent management of large-scale distributed water conservancy monitoring systems. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of an embodiment of the data collaborative management method for a multi-site water conservancy monitoring network in this application.

[0024] Figure 2 This is a schematic diagram of an embodiment of the data collaborative management system for a multi-site water conservancy monitoring network in this application.

[0025] Figure 3 This is a schematic block diagram of the data collaborative management device for a multi-site water conservancy monitoring network in an embodiment of the present invention. Detailed Implementation

[0026] This application provides a data collaborative management method and system for a multi-site water conservancy monitoring network. The terms first, second, third, fourth, etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms include or have, and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0027] For ease of understanding, the specific process of the embodiments of this application is described below. Please refer to [link / reference]. Figure 1 One embodiment of the data collaborative management method for a multi-site water conservancy monitoring network in this application includes:

[0028] Step S1: Perform hierarchical identification and encoding processing on multiple monitoring stations to generate global identifiers, and encapsulate the raw data collected by each station into a standard data packet containing a global identifier, timestamp, and data quality identifier;

[0029] Step S2: Divide the transmission scheduling area according to the geographical location information of the site, set the transmission priority according to the site type, allocate transmission time slots through time division multiplexing mechanism, and generate transmission scheduling instructions;

[0030] Step S3: Perform a rationality check on the standard data package, extract data sequences from multiple stations within the same irrigation district, calculate the correlation consistency index between stations using a multi-dimensional data quality assessment algorithm, locate and correct abnormal data based on the correlation consistency index, and generate a collaborative dataset.

[0031] Step S4: Store the collaborative dataset in shards according to the time dimension, establish spatial index, time index and type index, and construct a three-dimensional collaborative index system;

[0032] Step S5: Extract monitoring data from multiple stations within the irrigation area from the three-dimensional collaborative index system, calculate multi-site fusion indicators, and generate collaborative decision-making suggestions based on the multi-site fusion indicators.

[0033] It is understood that the executing entity of this application can be a data collaborative management system for a multi-site water conservancy monitoring network, or it can be a terminal or a server; the specific implementation is not limited here. This application's embodiment uses a server as an example for illustration.

[0034] Specifically, the core of hierarchical identification and coding for multiple monitoring stations lies in establishing a unified station identification system, which adopts a three-layer coding structure: The first layer, the irrigation district level coding, consists of the province code and the irrigation district number. For example, the code for Irrigation District No. 1 in a certain province is 32001, where 32 represents the province and 001 represents the irrigation district number; The second layer, the type coding, is based on the station function, with soil moisture stations identified as SM, field flow stations as FF, and headworks flow stations as CF; The third layer, the equipment coding, consists of the equipment type abbreviation and a two-digit serial number. For example, the soil moisture sensor number 01 is identified as SS01. Combining the three layers of coding in the order of irrigation district level-type-equipment, we obtain the global identifier 32001-SM-SS01, which uniquely corresponds to the No. 1 soil moisture sensor in Irrigation District No. 1 of the province. During the data encapsulation phase, when the soil moisture station collects a soil moisture content of 45%, the system extracts this value as the raw data and obtains the Unix timestamp of the collection time. It then determines whether 45% is within the preset range of 0-100% for soil moisture content. Since the value is within a reasonable range, the data quality is marked as normal. Subsequently, the global identifier, timestamp, data type label soil moisture content, value 45, unit % and data quality label normal are organized into a standard data packet in JSON format. In step S2, the core of transmission scheduling is to avoid network conflicts caused by multiple stations transmitting data simultaneously. First, the irrigation district-level code of the global identifier is extracted from the standard data packet. For example, 32001 is extracted from 32001-SM-SS01. The geographical coordinates are obtained by querying the irrigation district basic information table based on this code. Multiple irrigation districts with similar geographical locations are divided into a transmission scheduling area. For example, five adjacent irrigation districts are divided into a scheduling area. Then, the type code is extracted from the global identifier. For example, SM represents a soil moisture station. The transmission priority and transmission cycle are set according to the data characteristics of the station type: the soil moisture station is set to low priority and transmission cycle of 30 minutes because the soil moisture content changes slowly; the mechanical water meter of the field flow station needs to report the cumulative flow periodically and is set to medium priority and transmission cycle of 15 minutes; the electromagnetic flow meter of the canal head flow station is involved in the allocation of critical water volume and is set to high priority and transmission cycle of 5 minutes.Next, time-division multiplexing is performed: the 5-minute transmission cycle is divided into 10 time slots, each 30 seconds long. Time slots 1 to 4 are allocated to high-priority headworks flow stations in the area. Each station occupies approximately 5.45 seconds to complete data transmission according to the device sequence number of the global identifier. Time slots 5 to 6 are reserved for medium-priority field flow stations when their 15-minute transmission cycle arrives. Time slots 7 to 8 are reserved for low-priority soil moisture stations when their 30-minute transmission cycle arrives. Time slots 9 to 10 are used as emergency channels for data retransmission. Based on this time slot allocation scheme, a transmission scheduling instruction is generated. The instruction includes the global identifier of the station, the allocated time slot number, and the time slot start time. The cloud platform sends the transmission scheduling instruction to the DTUs of each station in the scheduling area. After receiving the instruction, the DTUs maintain a low-power standby state during non-allocated time slots and only activate the communication module to transmit data when the allocated time slot arrives.

[0035] The core of the multi-dimensional data quality assessment algorithm lies in cross-validation using the inherent correlation between data from multiple stations. First, a single-site verification is performed on the received standard data packet, extracting the numerical fields from the data packet. For example, if a soil moisture station reports a water content of 120%, it is determined whether this value is within the preset reasonable range of 0-100%. Since 120% exceeds the upper limit, the standard data packet is marked as reasonable anomaly. At the same time, the rate of change is calculated for continuous data packets from the same station. For example, if an electromagnetic flowmeter reports a flow rate of 200 cubic meters per hour in the previous moment and 600 cubic meters per hour in the current moment, the rate of change is calculated as the flow rate difference divided by the flow rate in the previous moment, which equals 200%. It is then determined whether this rate of change exceeds the preset threshold of 150%. Since 200% exceeds the threshold, the data packet in the current moment is marked as a sudden change anomaly. After completing the single-site verification, the irrigation district-level code is extracted from the standard data package to filter out the data packages of all monitoring stations within the irrigation district, including data packages of one soil moisture station, one field flow station, and one headworks flow station. The data packages within the most recent hour are extracted and sorted by timestamp from smallest to largest to form a data sequence. This data sequence is then processed by a multi-dimensional data quality assessment algorithm: the soil moisture content from one hour ago and the current soil moisture content are extracted from the soil moisture station data sequence, the change in soil moisture content is calculated and then divided by the time interval to obtain the rate of change in soil moisture content; the cumulative water supply is extracted from the headworks flow station data sequence, and the cumulative flow is extracted from the field flow station data sequence, the difference between the two is calculated and then divided by the headworks water supply to obtain the flow difference; the timestamps are extracted from the data packages of the two flow stations, the timestamp difference is calculated and then divided by 3600 seconds to obtain the time deviation. Based on the rate of change in soil moisture content, the flow difference, and the time deviation, a weighted summation is used to calculate the correlation consistency index. When this index is less than a preset threshold of 0.6, it is determined that there is inconsistency in the data of this group of stations. For inconsistent site combinations, the frequency with which each monitoring site is marked as abnormal across all possible combinations is statistically analyzed. The monitoring site with the highest frequency is identified as the abnormal data source. Normal standard data packets from the past 30 days for that site are extracted to build a time series prediction model. The measured value and the smoothed predicted value from the previous moment are used to calculate the correction value for the current moment using exponential smoothing. This correction value replaces the abnormal data to form a collaborative dataset. The core of storing the collaborative dataset in time-divided format is to reduce the time complexity of data retrieval. First, the timestamps of the standard data packets are extracted from the collaborative dataset. The collaborative dataset is then divided into time-divided data packets according to the timestamps, with data packets generated at a granularity of 1 day. For example, all standard data packets with the timestamp corresponding to October 27, 2025 are stored in a data packet named "Water Resources Monitoring 20251027". Global identifiers are extracted from the standard data packets within each data packet. The collaborative dataset is then divided into two partitions based on the global identifiers, and data from the same monitoring site are clustered and stored in the same storage partition.Then, a spatial index is built by extracting the irrigation district-level code from the global identifier of the standard data packet. The spatial index is organized using a B+ tree structure, with the leaf nodes of the tree storing the irrigation district-level code and its corresponding data fragment location. A time index is built by extracting the timestamp from the standard data packet, organized according to the time stamp range, recording the data fragment set corresponding to each time range. A type index is built by extracting the type code from the global identifier, organized using a hash table structure, recording the data record location corresponding to each station type. The spatial index, time index, and type index are combined and associated to construct a three-dimensional collaborative index system based on the combination relationship of irrigation district-level code, timestamp, and type code. This allows the query to locate the irrigation district-level code of the target irrigation district first through the spatial index, then filter the data fragment set of the past 7 days through the time index, and finally filter the data records of the soil moisture stations through the type index. The combined use of the three-dimensional index reduces the query time complexity from traversing all data to only traversing the target data located by the index.

[0036] Monitoring data from multiple stations within an irrigation district were extracted and fused for analysis using a three-dimensional collaborative indexing system. First, the irrigation district-level code of the target irrigation district was located using the spatial index within the system. Based on this code, global identifiers for multiple monitoring stations within the district were extracted. Timestamps within a preset 24-hour evaluation period were filtered using the time index. Standard data packages for soil moisture stations, field flow stations, and headworks flow stations were filtered using the type index to obtain the monitoring data for the target irrigation district. From this data, the initial and final moisture contents of the soil moisture stations were extracted (e.g., initial moisture content of 45% and final moisture content of 68%). The cumulative flow value of the field flow station (425 cubic meters) and the cumulative water supply of the headworks flow station (500 cubic meters) were extracted. The water utilization coefficient was calculated by dividing the cumulative flow value of the field flow station by the cumulative water supply of the headworks flow station (425 divided by 500 equals 0.85). The soil moisture content difference was calculated by subtracting 45% from 68%. The percentage is 23%. Converting 23% to a decimal of 0.23, multiplying it by the irrigation area of ​​100,000 square meters, the root depth of the soil layer of 0.6 meters, and the soil bulk density of 1350 kg / m³, we get an increase in soil moisture content of 18,630 m³. Dividing this by the cumulative flow value of 425 m³ at the field flow station, we get a soil water replenishment efficiency of 43.84. The water utilization coefficient of 0.85 and the soil water replenishment efficiency of 43.84 are used as the multi-site integrated indicators. By comparing the multi-site integrated indicators with preset thresholds, a water utilization coefficient of 0.85 is higher than the first preset threshold of 0.75, indicating that water loss is within a reasonable range. A soil water replenishment efficiency of 43.84 is higher than the second preset threshold of 30, indicating that the irrigation method is reasonable. A final moisture content of 68% is lower than the field capacity limit of 85%, indicating that over-irrigation has not occurred. Since all three indicators are within the normal range, a collaborative decision-making suggestion is generated that the current irrigation system is reasonable and does not require adjustment. This collaborative decision-making suggestion is pushed to the management terminal's mobile APP. Managers can view the water utilization coefficient, soil water replenishment efficiency, and collaborative decision-making suggestion for the irrigation area on the mobile APP, thereby understanding the effective utilization of irrigation water in the irrigation area.

[0037] In one specific embodiment, step S1 includes:

[0038] Multiple monitoring stations are identified and coded in three layers. The first layer uses the combination of province code and irrigation district number to generate irrigation district-level code. The second layer generates type code based on station type. The third layer generates equipment code based on equipment type and equipment number. The three layers of codes are combined to obtain a global identifier.

[0039] Extract the raw data collected from each monitoring station, obtain the timestamp corresponding to each raw data, determine whether each raw data exceeds the preset value range, and mark the data quality label according to the judgment result;

[0040] The global identifier, timestamp, data type label, numerical values ​​and units of the original data, and data quality identifier are organized according to a preset format to form data encapsulation rules;

[0041] In the data transmission unit of each monitoring station, a global identifier corresponding to the station is preset, and the raw data is encapsulated according to the data encapsulation rules to obtain a standard data packet.

[0042] Specifically, in the process of three-layer identification coding for multiple monitoring stations, the first layer, irrigation district-level coding, adopts a combination of province code and irrigation district serial number. For example, the code for irrigation district No. 1 in a certain province is 32001, where 32 represents the province code and 001 represents the irrigation district serial number within that province. The second layer, type coding, distinguishes stations based on their monitoring functions: soil moisture stations are identified as SM, field flow stations as FF, and headworks flow stations as CF. The third layer, equipment coding, consists of an abbreviation of the equipment type and a two-digit serial number. For example, the code for the first soil moisture sensor is SS01. Combining these three layers of coding according to the format of irrigation district-type coding-equipment coding, we obtain the complete global identifier 32001-SM-SS01. This identifier uniquely corresponds to the first soil moisture sensor in irrigation district No. 1 of a certain province. Through this hierarchical coding structure, each monitoring device has a unique identity identifier containing hierarchical information throughout the entire system.

[0043] When extracting raw data collected from various monitoring stations, if the soil moisture sensor detects a soil moisture content of 45%, the system extracts this value as raw data and obtains the Unix timestamp of the data generation time, such as 1729987200. The system then determines whether the raw data of 45% is within the preset range of 0-100% for soil moisture content. If 45% is within a reasonable range, the data is marked as normal. If a moisture content value of 120% is collected, it exceeds the upper limit of 100%, and the data is marked as abnormal. Through this pre-judgment mechanism, the data quality is initially identified during the data encapsulation stage, facilitating the rapid identification of abnormal data during subsequent processing. The six elements—global identifier 32001-SM-SS01, timestamp 1729987200, data type label soil moisture content, original data value 45, unit % and data quality identifier normal—are organized in JSON format to form a unified data encapsulation rule. This rule stipulates that the structure of all standard data packets is a JSON object containing an identifier field, a time field, a type field, a value field, a unit field, and a quality field.

[0044] Each monitoring station's Data Transmission Unit (DTU) has a pre-set global identifier and data encapsulation rules. When the soil moisture sensor generates new raw data, the DTU automatically assembles the global identifier, current timestamp, data type, raw value, unit, and quality identifier into a standard data packet according to the pre-set data encapsulation rules. For example, 45% of the raw data is encapsulated into a formatted JSON data packet. This data packet has already undergone standardized encapsulation processing before leaving the monitoring station and being transmitted to the cloud platform via NB-IoT or GPRS network. This ensures that the data uploaded by different types of monitoring stations has a unified format structure, avoiding difficulties in parsing caused by the cloud platform receiving raw data with different formats.

[0045] In one specific embodiment, step S2 includes:

[0046] Extract the irrigation district-level code of the global identifier from the standard data packet, obtain the geographical location information of each monitoring station based on the irrigation district-level code, and divide multiple irrigation districts into transmission scheduling areas;

[0047] Extract the type code of the global identifier from the standard data packet, determine the site type of each monitoring site based on the type code, and set the transmission priority and transmission period for each monitoring site according to the site type;

[0048] The transmission period is divided into multiple time slots, and the time slots are allocated to different monitoring stations in sequence according to the transmission priority of each monitoring station, thus obtaining a time slot allocation scheme.

[0049] Based on the time slot allocation scheme, a transmission scheduling instruction is generated and sent to the data transmission unit of the monitoring station in each transmission scheduling area, so that each monitoring station transmits standard data packets in the allocated time slot.

[0050] Specifically, when extracting the irrigation district-level code of the global identifier from the standard data packet, the system parses the first five digits 32001 of the global identifier 32001-SM-SS01 as the irrigation district-level code. Based on this code, it queries the irrigation district basic information database to obtain the geographical location information of the irrigation district, including latitude and longitude coordinates, administrative region, and other attributes. Multiple irrigation districts with adjacent geographical locations are divided into the same transmission scheduling area. For example, five irrigation districts coded as 32001, 32002, 32003, 32004, and 32005 are divided into scheduling area A. Because these irrigation districts are geographically close, they usually share the same network base stations and communication resources. Regional management avoids signal interference and resource competition caused by cross-regional transmission.

[0051] When extracting the type code of the global identifier from the standard data packet, the system parses SM in 32001-SM-SS01 to determine that the station is a soil moisture station, and CF in 32001-CF-EM01 to determine that the station is a canal head flow station. Based on the data characteristics of different station types, the transmission priority and transmission cycle are set. The soil moisture content monitored by the soil moisture station changes slowly, so it is set to low priority and transmission cycle of 30 minutes. The mechanical water meter readings monitored by the field flow station need to be reported regularly, so it is set to medium priority and transmission cycle of 15 minutes. The electromagnetic flow meter data monitored by the canal head flow station is related to the critical water allocation, so it is set to high priority and transmission cycle of 5 minutes. The 5-minute transmission cycle is divided into 10 time slots, each lasting 30 seconds. Time slots are allocated according to the transmission priority of each monitoring station. Time slots 1 to 4 are allocated to high-priority headwater flow stations. Assuming there are 22 headwater flow stations in dispatch area A, each station will occupy approximately 5.45 seconds in these 4 time slots to complete data transmission. Time slots 5 to 6 are reserved for medium-priority field flow stations, which will transmit data in these two time slots when their 15-minute transmission cycle arrives. Time slots 7 to 8 are reserved for low-priority soil moisture monitoring stations, which will transmit data in these two time slots when their 30-minute transmission cycle arrives. Time slots 9 to 10 are used as emergency channels for data retransmission, resulting in the complete time slot allocation scheme.

[0052] Based on the time slot allocation scheme, a transmission scheduling instruction is generated for each monitoring station. The instruction includes the station's global identifier, the allocated time slot number, and the time slot start time. For example, the instruction for station 32001-CF-EM01 is to start transmission at 0 seconds in time slot 1, and the instruction for station 32001-SM-SS01 is to start transmission at 180 seconds in time slot 7. The cloud management platform then distributes these transmission scheduling instructions to each monitoring station within scheduling area A via the network. The data transmission unit (DTU) of the site receives the scheduling instruction and shuts down the communication module to enter a low-power standby state during non-allocated time slots to save solar battery power. It only activates the NB-IoT or GPRS communication module when the allocated time slot arrives and uploads the pre-encapsulated standard data packets to the cloud platform. For example, the DTU of site 32001-CF-EM01 activates the communication module to transmit data at the 0th second of each 5-minute transmission cycle, and the DTU of site 32001-SM-SS01 activates the communication module to transmit data at the 180th second of each 30-minute transmission cycle.

[0053] In one specific embodiment, step S3 includes:

[0054] The values ​​in the received standard data packets are extracted, and it is determined whether the values ​​exceed the preset reasonable value range. Standard data packets that exceed the range are marked as reasonable anomalies. The data change rate of continuous standard data packets at the same monitoring station is calculated, and it is determined whether the data change rate exceeds the preset change rate threshold. Standard data packets that exceed the threshold are marked as mutation anomalies, and the single-site test results are obtained.

[0055] Extract irrigation district-level codes from standard data packets, filter out standard data packets from multiple monitoring stations within the same irrigation district, sort the standard data packets according to timestamps, and extract standard data packets within a preset time period to form a data sequence.

[0056] The data sequence is input into a multi-dimensional data quality assessment algorithm for processing. The change in soil moisture content and time interval are extracted from the data sequence to calculate the rate of change in soil moisture content. The cumulative flow values ​​of the headworks flow station and the field flow station are extracted to calculate the flow difference. The timestamps of the two flow stations are extracted to calculate the time deviation. The correlation consistency index is calculated based on the rate of change in soil moisture content, the flow difference, and the time deviation. When the correlation consistency index is less than a preset threshold, it is determined that there is inconsistency in the data of this group of stations.

[0057] The frequency of each monitoring station being marked as abnormal in all possible combinations of inconsistent site combinations is statistically analyzed. The monitoring station with the highest frequency is identified as the abnormal data source. Normal standard data packets within a preset number of historical days from the abnormal data source are extracted to establish a time series prediction model. The measured value and the smoothed predicted value of the previous moment are input into the time series prediction model for exponential smoothing calculation to obtain the correction value of the current moment to replace the abnormal data and generate a collaborative dataset.

[0058] Specifically, when performing single-site verification on the received standard data packets, the system first extracts the numerical fields from the data packets. For example, if a soil moisture station reports a moisture content of 120%, it determines whether this value is within the preset reasonable range of 0-100% for soil moisture content. Since 120% exceeds the upper limit, the standard data packet is marked as having an abnormal reasonableness. At the same time, the system calculates the rate of change of data for continuous standard data packets from the same monitoring station. For example, if the electromagnetic flowmeter at a canal head flow station reported a flow rate of 200 cubic meters per hour in the previous moment and reports a flow rate of 600 cubic meters per hour in the current moment, the rate of change is calculated as the flow rate difference of 400 divided by the previous flow rate of 200, resulting in 200%. It then determines whether this rate of change exceeds the preset rate of change threshold of 150%. Since 200% exceeds the threshold, the standard data packet for the current moment is marked as having an abnormal sudden change. The single-site verification result is obtained through reasonableness verification and continuity verification.

[0059] Extract the irrigation district-level code from the standard data packets. For example, extract 32001 from the global identifier 32001-SM-SS01. Filter out all standard data packets with the irrigation district-level code 32001, including standard data packets reported by one soil moisture station, one field flow station, and one headworks flow station within the irrigation district. Sort the data packets according to the timestamps in the data packets from smallest to largest. Extract the standard data packets from the most recent hour to form a data sequence. This data sequence includes the soil moisture station's reported moisture content of 45% one hour ago and the currently reported moisture content of 68%, the cumulative flow reported by the field flow station of 425 cubic meters, and the cumulative water supply reported by the headworks flow station of 500 cubic meters. The data sequence was processed using a multi-dimensional data quality assessment algorithm. First, the change in soil moisture content was extracted from the soil moisture station data. The change was calculated as 23% (68% current moisture content minus 45% moisture content one hour prior). Dividing this by the one-hour time interval yielded a soil moisture content change rate of 23% per hour. Then, the cumulative flow value of 500 cubic meters was extracted from the headworks flow station and 425 cubic meters from the field flow station. The flow difference was calculated as 75 cubic meters divided by the headworks flow value of 500 cubic meters, resulting in 0.15. Next, timestamps were extracted from the standard data packets of the two flow stations. Assuming the headworks flow station's timestamp is 1729987200 and the field flow station's timestamp is 1729987180, the time deviation was calculated as the timestamp difference. Dividing 20 by 3600 seconds yields 0.0056. Based on the soil moisture content change rate of 23% per hour, the flow difference of 0.15, and the time deviation of 0.0056, a weighted summation method is used to calculate the correlation consistency index. Dividing the soil moisture content change rate by the preset maximum reasonable change rate of 15% per hour yields 1.53. Subtracting 1.53 from 1 and taking the absolute value gives 0.53. Multiplying this by a weight of 0.3 gives the first term 0.159. Multiplying the flow difference of 0.15 by a weight of 0.5 gives the second term 0.075. Subtracting the time deviation of 0.0056 from 1 gives 0.9944. Multiplying this by a weight of 0.2 gives the third term 0.199. The sum of these three terms gives the correlation consistency index of 0.433. When this index is less than the preset threshold of 0.6, inconsistencies are determined to exist in the data of this group of stations.

[0060] For inconsistent site combinations, the system counts the frequency with which soil moisture stations, field flow stations, and headworks flow stations within the irrigation district are marked as abnormal across all possible site combinations. For example, if a soil moisture station is marked as abnormal twice out of three combinations, a field flow station zero times, and a headworks flow station once, the soil moisture station with the highest frequency is identified as the abnormal data source. Normal standard data packets from the past 30 days for this soil moisture station are extracted, and soil moisture content values ​​are extracted from these historical data packets to build a time series prediction model. Exponential smoothing is used for prediction. The measured value of 45% and the smoothed predicted value of 43% from the previous time step are input into the model. The smoothing coefficient of 0.4 is multiplied by the measured value of 45% from the previous time step to get 18%. The remaining smoothing coefficient of 0.6 is multiplied by the smoothed predicted value of 43% from the previous time step to get 25.8%. The two are added together to get the corrected value of 43.8% for the current time step. This corrected value is used to replace 68% of the abnormal data. The numerical field in the standard data package is updated to 43.8%, and the data quality label field is updated to "corrected". All the corrected standard data packages are aggregated to generate a collaborative dataset.

[0061] In one specific embodiment, step S4 includes:

[0062] Extract the timestamps of standard data packets from the collaborative dataset, and divide the collaborative dataset into time-dimension shards according to the timestamps, generating data shards with a preset time length as the sharding granularity.

[0063] Extract global identifiers from the standard data packets within each data segment, perform secondary partitioning of the collaborative dataset according to the global identifiers, and aggregate and store data from the same monitoring station.

[0064] A spatial index is built by extracting the irrigation district-level code from the global identifier of the standard data packet, a time index is built by extracting the timestamp from the standard data packet, and a type index is built by extracting the type code from the global identifier.

[0065] By combining and associating spatial, temporal, and type indexes, a three-dimensional collaborative index system is constructed based on the combination relationship between irrigation district-level codes, timestamps, and type codes. This enables the target monitoring data to be located through a combination of irrigation district location, time filtering, and type filtering queries.

[0066] Specifically, when extracting timestamps from standard data packets in the collaborative dataset for time-dimensional sharding, the system reads the timestamp field from each standard data packet and divides the collaborative dataset according to a preset sharding granularity. The sharding granularity is a preset time length unit. Standard data packets with timestamps within the same time period are grouped into the same data shard. A unique shard name is generated for each data shard. Through time-dimensional sharding, the massive collaborative dataset is organized into multiple independent data sharding units in chronological order. Each data shard contains all standard data packets generated within a specific time range, facilitating subsequent data retrieval and management according to time range.

[0067] When extracting global identifiers from standard data packets within each data shard for secondary partitioning, the system iterates through each standard data packet in the data shard, reads its global identifier field, and groups the standard data packets within the data shard according to the global identifier. All standard data packets with the same global identifier are stored in the same storage partition, so that all data generated by the same monitoring station within the same time period are clustered and stored in adjacent physical storage locations. The purpose of secondary partitioning is to further organize data according to the station dimension based on time sharding. When it is necessary to query the data of a certain station in a specific time period, it is only necessary to locate the corresponding time shard and the storage partition of that station, and read the data in that partition sequentially, avoiding random searching for the data of that station in the entire data shard.

[0068] When building a three-dimensional index from standard data packets, the system first extracts irrigation district-level codes from global identifiers to create a spatial index. The spatial index records the mapping relationship between each irrigation district-level code and the data fragment and storage partition where its data resides, organized using a tree data structure. This allows for quick location of all data fragments containing data for that irrigation district based on the irrigation district-level code. Next, the system extracts timestamps from standard data packets to create a time index. The time index records the mapping relationship between time ranges and data fragments, organizing the data fragment set corresponding to each time period, allowing for quick filtering of relevant data fragments based on the time range. Finally, the system extracts type codes from global identifiers to create a type index. The type index records the data record position of each site type in each data fragment, organized using a hash table data structure, allowing for quick filtering of data records for specific site types based on the site type.

[0069] When constructing a three-dimensional collaborative index system by combining spatial, temporal, and type indexes, the system establishes a combined mapping relationship between irrigation district-level codes, timestamps, and type codes. When querying data of a certain type of station in a specific irrigation district within a specific time period, the system first locates the data fragment set of that irrigation district by inputting the irrigation district-level code through the spatial index. Then, it filters out the data fragment subset corresponding to that time period by inputting the time range through the temporal index. Finally, it filters out the data record location of specific type stations in these data fragments by inputting the type code through the type index. The combined use of the three-dimensional index reduces the query path from traversing the entire collaborative dataset to accessing only the target data fragment and storage partition pointed to by the index, significantly reducing the time overhead of data query and enabling quick location of target monitoring data through a combined query of irrigation district location, time filtering, and type filtering.

[0070] In one specific embodiment, a global identifier is extracted from the standard data packets within each data segment, and the collaborative dataset is stored in a two-level partition according to the global identifier, thereby aggregating and storing data from the same monitoring station, including:

[0071] Extract global identifiers from each standard data packet within the data fragment, perform hash calculation on the global identifiers to obtain hash values, and allocate the standard data packets to a preset number of storage partitions based on the hash values;

[0072] The distribution of global identifiers of standard data packets in each storage partition is statistically analyzed, the number of standard data packets corresponding to each global identifier is calculated, and the presence of data skew is determined based on the number. When the number of standard data packets in a single storage partition exceeds a preset multiple of the average value, partition rebalancing is triggered.

[0073] Global identifiers within the storage partition that triggers rebalancing are grouped according to site type, standard data packets of the same site type are migrated to the new storage partition, and the mapping relationship between global identifiers and storage partitions is updated.

[0074] An index table of global identifiers is established within each storage partition. The index table records the physical offset of the global identifier and its corresponding standard data packet within the storage partition, enabling the data aggregation location of the same monitoring station to be quickly located using the global identifier.

[0075] Specifically, when extracting global identifiers from each standard data packet within a data shard for hash allocation, the system reads the global identifier field of each standard data packet from the data shard, performs a hash calculation on each global identifier, converts the global identifier into a hash value using a hash algorithm, performs a modulo operation on the hash value, calculates the remainder based on the preset number of storage partitions to obtain the partition number, and allocates the standard data packet to the corresponding numbered storage partition. Through hash allocation, the standard data packets within the data shard are distributed to multiple storage partitions. Different global identifiers are mapped to different storage partitions after hash calculation, achieving a preliminary balanced distribution of data at the storage level.

[0076] When analyzing the distribution of standard data packets within each storage partition, the system iterates through all storage partitions to count the total number of standard data packets in each partition. It calculates the average number of data packets across all storage partitions as a baseline and compares the actual number of data packets in each storage partition with the average. When it finds that the number of data packets in a certain storage partition exceeds a preset multiple of the average, it determines that the partition has data skew and triggers the partition rebalancing mechanism. It further counts the number of data packets corresponding to each global identifier in the storage partition and analyzes which global identifiers cause the data concentration. For example, it finds that the global identifiers of a large number of soil moisture monitoring stations are mapped to the same storage partition after hash calculation, resulting in the data volume of this partition being much larger than that of other partitions.

[0077] When processing a storage partition that triggers rebalancing, the system extracts the global identifier of all standard data packets within that partition, extracts the type encoding portion from the identifier, groups standard data packets with the same type encoding into a group, and categorizes the data packets according to site type into soil moisture station group, field flow station group, and canal head flow station group. The number of data packets in each group is calculated, the site type group with the largest proportion is identified, and some standard data packets corresponding to global identifiers in that group are migrated to other storage partitions with lighter loads. During the migration process, all data packets with the same global identifier are migrated as a whole to avoid data from the same site being scattered across multiple partitions. After the migration is completed, the mapping relationship table between global identifiers and storage partitions is updated, and the mapping relationship of the migrated global identifiers is modified from the original partition to the new partition to ensure that the data location can be correctly located during subsequent queries.

[0078] When establishing an index table for global identifiers within each storage partition, the system traverses all standard data packets within that partition, recording the physical offset of the first occurrence of each global identifier. The physical offset represents the byte position of the standard data packet within the storage partition file. The correspondence between global identifiers and physical offsets is recorded in the index table, which is organized in a tree structure. The leaf nodes store key-value pairs of global identifiers and physical offsets. When querying the site data corresponding to a certain global identifier, the system first queries the mapping table to determine which partition the identifier is stored in. Then, it searches for the physical offset corresponding to the global identifier in the index table of that partition. Starting from that offset position, all standard data packets of that site are read sequentially. Since the data of the same site is already clustered and stored in the secondary partition, sequential reading is more efficient than random access. The establishment of the index table enables the rapid location of the data cluster position of the same monitoring site through the global identifier, avoiding traversing the entire storage partition.

[0079] In one specific embodiment, step S5 includes:

[0080] The irrigation district-level code of the target irrigation district is located by spatial index in the three-dimensional collaborative index system. The global identifiers of multiple monitoring stations in the irrigation district are extracted based on the irrigation district-level code. The timestamps within the preset evaluation period are filtered by time index. The standard data packages of soil moisture station, field flow station and canal head flow station are filtered by type index to obtain the monitoring data of the target irrigation district.

[0081] The initial and final soil moisture contents of the monitoring data are extracted from the soil moisture stations. The cumulative flow values ​​of the field flow stations and the cumulative water supply of the headworks flow stations are extracted. The ratio of the cumulative flow value of the field flow stations to the cumulative water supply of the headworks flow stations is calculated to obtain the water utilization coefficient. The soil water replenishment efficiency is obtained by multiplying the soil moisture content difference by the irrigation area, the soil root layer depth and the soil bulk density and then dividing by the cumulative flow value of the field flow stations. The water utilization coefficient and the soil water replenishment efficiency are used as multi-site fusion indicators.

[0082] The multi-site fusion index is compared with the preset threshold. When the water utilization coefficient is lower than the first preset threshold, the first decision suggestion to optimize the water conveyance channel is generated. When the soil water replenishment efficiency is lower than the second preset threshold, the second decision suggestion to adjust the irrigation system is generated. When the end moisture content exceeds the field water holding capacity limit, the third decision suggestion to reduce the irrigation water volume is generated.

[0083] The first, second, or third decision recommendations will be pushed to the management terminal as collaborative decision recommendations.

[0084] Specifically, when extracting monitoring data of a target irrigation district through a three-dimensional collaborative indexing system, the system first inputs the target irrigation district-level code through a spatial index. The B+ tree structure of the spatial index quickly locates the data fragment set corresponding to the irrigation district. From these data fragments, it extracts the identifiers of all monitoring stations whose global identifiers are prefixed with the irrigation district code, including the global identifiers of stations such as soil moisture stations, field flow stations, and canal head flow stations in the irrigation district. Then, it inputs the time range corresponding to the preset evaluation period through a time index and filters out the standard data packets generated within the time range of the evaluation period. Next, it inputs the type codes of soil moisture stations, field flow stations, and canal head flow stations through a type index and filters out the standard data packets of these three types of stations within the time range. The combined query of the three-dimensional index allows the system to obtain the monitoring data of the irrigation district within the evaluation period without traversing all the data, by only accessing the target data fragments and storage partitions pointed to by the index.

[0085] When calculating multi-site fusion indicators from monitoring data, the system extracts the moisture content value at the start of the assessment period from the standard data packet sequence of the soil moisture station as the initial moisture content, and extracts the moisture content value at the end of the assessment period as the final moisture content. It extracts the cumulative flow reading of the mechanical water meter from the standard data packet sequence of the field flow station. Since the mechanical water meter records a cumulative value, it extracts the cumulative reading at the start of the assessment period and the cumulative reading at the end of the assessment period, and subtracts the two to obtain the cumulative flow value within the assessment period. It extracts the instantaneous flow value of the electromagnetic flow meter from the standard data packet sequence of the canal head flow station, and adds up all the instantaneous flow values ​​within the assessment period to obtain the cumulative water supply. When calculating the water utilization coefficient, the cumulative flow value of the field flow station is divided by the cumulative water supply of the headworks flow station to obtain the ratio. This coefficient reflects the water loss from the headworks to the field, and the closer the values ​​are, the smaller the water loss. When calculating the soil water replenishment efficiency, the initial moisture content is subtracted from the final moisture content to obtain the moisture content difference. The moisture content is converted from a percentage to a decimal. The irrigation area, soil root layer depth, and soil bulk density of the irrigation area are extracted from the irrigation area basic information table. The moisture content difference is multiplied by the irrigation area, then by the soil root layer depth, and then by the soil bulk density to obtain the total increase in soil moisture content. After converting it to volume units, it is divided by the cumulative flow value of the field flow station to obtain the soil water replenishment efficiency. This indicator reflects the increase in soil moisture content per unit of irrigation water. The water utilization coefficient and soil water replenishment efficiency are used as multi-site integrated indicators.

[0086] When generating collaborative decision-making suggestions by comparing multi-site fusion indicators with preset thresholds, if the water utilization coefficient is lower than the first preset threshold, it indicates excessive water loss from the canal head to the field. The system generates the first decision suggestion to optimize the water conveyance channel, which includes checking for leaks, siltation, or damage in the channel and suggesting measures such as canal seepage prevention or dredging to reduce water loss. If the soil water replenishment efficiency is lower than the second preset threshold, it indicates that irrigation water has not been effectively converted into increased soil moisture content. The system generates the second decision suggestion to adjust the irrigation system, which includes changing the irrigation method from flood irrigation to drip irrigation or sprinkler irrigation, and adjusting the irrigation quota and irrigation cycle to improve water use efficiency. If the end moisture content exceeds the upper limit of field capacity, it indicates over-irrigation and that the soil is saturated and can no longer absorb water. The system generates the third decision suggestion to reduce the amount of irrigation water, which includes reducing the water supply or extending the irrigation cycle in the next irrigation to avoid water waste and potential soil salinization problems. The system will generate first, second, or third decision suggestions as collaborative decision suggestions and push them to the mobile APP or workstation computer terminal of the managers via the network. The managers can view the water utilization coefficient, soil water replenishment efficiency and corresponding decision suggestions of the irrigation area on the terminal, and take corresponding management measures according to the suggestions, thereby realizing collaborative management of effective irrigation water utilization based on multi-site data fusion.

[0087] The data collaborative management method for a multi-site water conservancy monitoring network in this application embodiment has been described above. The data collaborative management system for a multi-site water conservancy monitoring network in this application embodiment is described below. Please refer to [link / reference]. Figure 2 One embodiment of the data collaborative management system for a multi-site water conservancy monitoring network in this application includes:

[0088] The encoding module is used to perform hierarchical identification encoding processing on multiple monitoring stations, generate global identifiers, and encapsulate the raw data collected by each station into a standard data packet containing a global identifier, timestamp, and data quality identifier.

[0089] The partitioning module is used to divide the transmission scheduling area according to the geographical location information of the site, set the transmission priority according to the site type, allocate transmission time slots through the time division multiplexing mechanism, and generate transmission scheduling instructions.

[0090] The correction module is used to perform reasonableness checks on standard data packages, extract data sequences from multiple stations within the same irrigation district, calculate the correlation consistency index between stations through a multi-dimensional data quality assessment algorithm, locate and correct abnormal data based on the correlation consistency index, and generate a collaborative dataset.

[0091] The sharding module is used to shard and store the collaborative dataset according to the time dimension, and to build spatial index, time index and type index to construct a three-dimensional collaborative index system.

[0092] The extraction module is used to extract monitoring data from multiple stations within the irrigation area from the three-dimensional collaborative index system, calculate multi-site fusion indicators, and generate collaborative decision-making suggestions based on the multi-site fusion indicators.

[0093] above Figure 2 The data collaborative management system for the multi-site water conservancy monitoring network in this embodiment of the invention is described in detail from the perspective of modular functional entities. The data collaborative management device for the multi-site water conservancy monitoring network in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0094] Reference Figure 3 This invention also provides a data collaborative management device for a multi-site water conservancy monitoring network. This device can be a server, and its internal structure can be as follows: Figure 3 As shown, the data collaborative management device for the multi-site water conservancy monitoring network includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor in this computer design provides computing and control capabilities. The memory of the data collaborative management device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the data collaborative management device is used to store the data corresponding to this embodiment. The network interface of the data collaborative management device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the above-described method.

[0095] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the data collaborative management equipment of the multi-site water conservancy monitoring network to which the present invention is applied.

[0096] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the data collaborative management method of the multi-site water conservancy monitoring network.

[0097] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0098] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a data collaborative management device (which may be a personal computer, server, or network device, etc.) of a multi-site water conservancy monitoring network to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data collaborative management method for a multi-site water conservancy monitoring network, characterized in that, The method includes: Step S1: Perform hierarchical identification and encoding processing on multiple monitoring stations to generate global identifiers, and encapsulate the raw data collected by each station into a standard data packet containing a global identifier, timestamp, and data quality identifier; Step S2: Divide the transmission scheduling area according to the geographical location information of the site, set the transmission priority according to the site type, allocate transmission time slots through time division multiplexing mechanism, and generate transmission scheduling instructions; Step S3: Perform a rationality check on the standard data package, extract data sequences from multiple stations within the same irrigation district, calculate the correlation consistency index between stations using a multi-dimensional data quality assessment algorithm, locate and correct abnormal data based on the correlation consistency index, and generate a collaborative dataset. Step S4: Store the collaborative dataset in shards according to the time dimension, establish spatial index, time index and type index, and construct a three-dimensional collaborative index system; Step S5: Extract monitoring data from multiple stations within the irrigation area from the three-dimensional collaborative index system, calculate multi-site fusion indicators, and generate collaborative decision-making suggestions based on the multi-site fusion indicators.

2. The data collaborative management method for a multi-site water conservancy monitoring network according to claim 1, characterized in that, Step S1 includes: Multiple monitoring stations are identified and coded in three layers. The first layer uses the combination of province code and irrigation district number to generate irrigation district-level code. The second layer generates type code based on station type. The third layer generates equipment code based on equipment type and equipment number. The three layers of codes are combined to obtain a global identifier. Extract the raw data collected from each monitoring station, obtain the timestamp corresponding to each raw data, determine whether each raw data exceeds the preset value range, and mark the data quality label according to the judgment result; The global identifier, timestamp, data type label, numerical values ​​and units of the original data, and data quality identifier are organized according to a preset format to form data encapsulation rules; In the data transmission unit of each monitoring station, a global identifier corresponding to the station is preset, and the raw data is encapsulated according to the data encapsulation rules to obtain a standard data packet.

3. The data collaborative management method for a multi-site water conservancy monitoring network according to claim 1, characterized in that, Step S2 includes: Extract the irrigation district-level code of the global identifier from the standard data packet, obtain the geographical location information of each monitoring station based on the irrigation district-level code, and divide multiple irrigation districts into transmission scheduling areas; Extract the type code of the global identifier from the standard data packet, determine the site type of each monitoring site based on the type code, and set the transmission priority and transmission period for each monitoring site according to the site type; The transmission period is divided into multiple time slots, and the time slots are allocated to different monitoring stations in sequence according to the transmission priority of each monitoring station, thus obtaining a time slot allocation scheme. Based on the time slot allocation scheme, a transmission scheduling instruction is generated and sent to the data transmission unit of the monitoring station in each transmission scheduling area, so that each monitoring station transmits standard data packets in the allocated time slot.

4. The data collaborative management method for a multi-site water conservancy monitoring network according to claim 1, characterized in that, Step S3 includes: The values ​​in the received standard data packets are extracted, and it is determined whether the values ​​exceed the preset reasonable value range. Standard data packets that exceed the range are marked as reasonable anomalies. The data change rate of continuous standard data packets at the same monitoring station is calculated, and it is determined whether the data change rate exceeds the preset change rate threshold. Standard data packets that exceed the threshold are marked as mutation anomalies, and the single-site test results are obtained. Extract irrigation district-level codes from standard data packets, filter out standard data packets from multiple monitoring stations within the same irrigation district, sort the standard data packets according to timestamps, and extract standard data packets within a preset time period to form a data sequence. The data sequence is input into a multi-dimensional data quality assessment algorithm for processing. The change in soil moisture content and time interval are extracted from the data sequence to calculate the rate of change in soil moisture content. The cumulative flow values ​​of the headworks flow station and the field flow station are extracted to calculate the flow difference. The timestamps of the two flow stations are extracted to calculate the time deviation. The correlation consistency index is calculated based on the rate of change in soil moisture content, the flow difference, and the time deviation. When the correlation consistency index is less than a preset threshold, it is determined that there is inconsistency in the data of this group of stations. The frequency of each monitoring station being marked as abnormal in all possible combinations of inconsistent site combinations is statistically analyzed. The monitoring station with the highest frequency is identified as the abnormal data source. Normal standard data packets within a preset number of historical days from the abnormal data source are extracted to establish a time series prediction model. The measured value and the smoothed predicted value of the previous moment are input into the time series prediction model for exponential smoothing calculation to obtain the correction value of the current moment to replace the abnormal data and generate a collaborative dataset.

5. The data collaborative management method for a multi-site water conservancy monitoring network according to claim 1, characterized in that, Step S4 includes: Extract the timestamps of standard data packets from the collaborative dataset, and divide the collaborative dataset into time-dimension shards according to the timestamps, generating data shards with a preset time length as the sharding granularity. Extract global identifiers from the standard data packets within each data segment, perform secondary partitioning of the collaborative dataset according to the global identifiers, and aggregate and store data from the same monitoring station. A spatial index is built by extracting the irrigation district-level code from the global identifier of the standard data packet, a time index is built by extracting the timestamp from the standard data packet, and a type index is built by extracting the type code from the global identifier. By combining and associating spatial, temporal, and type indexes, a three-dimensional collaborative index system is constructed based on the combination relationship between irrigation district-level codes, timestamps, and type codes. This enables the target monitoring data to be located through a combination of irrigation district location, time filtering, and type filtering queries.

6. The data collaborative management method for a multi-site water conservancy monitoring network according to claim 5, characterized in that, The step of extracting global identifiers from standard data packets within each data segment, performing secondary partitioning of the collaborative dataset according to the global identifiers, and aggregating and storing data from the same monitoring station includes: Extract global identifiers from each standard data packet within the data fragment, perform hash calculation on the global identifiers to obtain hash values, and allocate the standard data packets to a preset number of storage partitions based on the hash values; The distribution of global identifiers of standard data packets in each storage partition is statistically analyzed, the number of standard data packets corresponding to each global identifier is calculated, and the presence of data skew is determined based on the number. When the number of standard data packets in a single storage partition exceeds a preset multiple of the average value, partition rebalancing is triggered. Global identifiers within the storage partition that triggers rebalancing are grouped according to site type, standard data packets of the same site type are migrated to the new storage partition, and the mapping relationship between global identifiers and storage partitions is updated. An index table of global identifiers is established within each storage partition. The index table records the physical offset of the global identifier and its corresponding standard data packet within the storage partition, enabling the data aggregation location of the same monitoring station to be quickly located using the global identifier.

7. The data collaborative management method for a multi-site water conservancy monitoring network according to claim 1, characterized in that, Step S5 includes: The irrigation district-level code of the target irrigation district is located by spatial index in the three-dimensional collaborative index system. The global identifiers of multiple monitoring stations in the irrigation district are extracted based on the irrigation district-level code. The timestamps within the preset evaluation period are filtered by time index. The standard data packages of soil moisture station, field flow station and canal head flow station are filtered by type index to obtain the monitoring data of the target irrigation district. The initial and final soil moisture contents of the monitoring data are extracted from the soil moisture stations. The cumulative flow values ​​of the field flow stations and the cumulative water supply of the headworks flow stations are extracted. The ratio of the cumulative flow value of the field flow stations to the cumulative water supply of the headworks flow stations is calculated to obtain the water utilization coefficient. The soil water replenishment efficiency is obtained by multiplying the soil moisture content difference by the irrigation area, the soil root layer depth and the soil bulk density and then dividing by the cumulative flow value of the field flow stations. The water utilization coefficient and the soil water replenishment efficiency are used as multi-site fusion indicators. The multi-site fusion index is compared with the preset threshold. When the water utilization coefficient is lower than the first preset threshold, the first decision suggestion to optimize the water conveyance channel is generated. When the soil water replenishment efficiency is lower than the second preset threshold, the second decision suggestion to adjust the irrigation system is generated. When the end moisture content exceeds the field water holding capacity limit, the third decision suggestion to reduce the irrigation water volume is generated. The first, second, or third decision recommendations will be pushed to the management terminal as collaborative decision recommendations.

8. A data collaborative management system for a multi-site water conservancy monitoring network, characterized in that, A data collaborative management method for a multi-site water conservancy monitoring network as described in any one of claims 1-7, wherein the data collaborative management system for the multi-site water conservancy monitoring network comprises: The encoding module is used to perform hierarchical identification encoding processing on multiple monitoring stations, generate global identifiers, and encapsulate the raw data collected by each station into a standard data packet containing a global identifier, timestamp, and data quality identifier. The partitioning module is used to divide the transmission scheduling area according to the geographical location information of the site, set the transmission priority according to the site type, allocate transmission time slots through the time division multiplexing mechanism, and generate transmission scheduling instructions. The correction module is used to perform reasonableness checks on standard data packages, extract data sequences from multiple stations within the same irrigation district, calculate the correlation consistency index between stations through a multi-dimensional data quality assessment algorithm, locate and correct abnormal data based on the correlation consistency index, and generate a collaborative dataset. The sharding module is used to shard and store the collaborative dataset according to the time dimension, and to build spatial index, time index and type index to construct a three-dimensional collaborative index system. The extraction module is used to extract monitoring data from multiple stations within the irrigation area from the three-dimensional collaborative index system, calculate multi-site fusion indicators, and generate collaborative decision-making suggestions based on the multi-site fusion indicators.

9. A data collaborative management device for a multi-site water conservancy monitoring network, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the data collaborative management method for a multi-site water conservancy monitoring network as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor, it causes the processor to execute the data collaborative management method for a multi-site water conservancy monitoring network as described in any one of claims 1 to 7.

Images