Sewage treatment operation method, equipment and cloud platform based on Internet of Things and GIS
By introducing Internet of Things and GIS technology into the sewage treatment system, a sewage treatment operation cloud platform has been established, which has solved the shortcomings of traditional systems in data collection, storage and analysis, and has achieved efficient and intelligent sewage treatment, improving management efficiency and data Processing capability.
Patent Information
- Application Number
- CN202411709910.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-02-11
- Estimated Expiration
- 2044-11-27
AI Technical Summary
Traditional sewage treatment systems have limited capabilities in data collection, storage and analysis, and are unable to cope with the growing amount and complexity of data, resulting in serious information silos, inefficient efficiency, and difficulty in achieving efficient and intelligent sewage treatment.
Adopt a sewage treatment operation cloud platform based on the Internet of Things and GIS, and obtain user water data and sewage pipeline data through the data acquisition layer, perform edge computing and pre-processing, and store and manage data through high-frequency and low-frequency storage chains, and use GIS system Conduct data analysis and prediction with the sewage treatment rectification analysis model to generate a sewage treatment rectification plan.
The entire process of sewage treatment has been digitized and visualized, greatly improved the monitoring and management efficiency of sewage pipelines, reduced the risks of sewage leakage and environmental pollution, improved data transmission and treatment efficiency, and supported the long-term planning and decision-making of sewage treatment. .
Smart Images

Figure CN119204697B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of sewage treatment, and specifically relates to a sewage treatment operation method, equipment and cloud platform based on the Internet of Things and GIS. Background Art
[0002] With the acceleration of urbanization and the growth of population, sewage treatment has become an important issue in urban management and environmental protection. Traditional sewage treatment technologies mainly focus on physical, chemical and biological treatment methods, which purify sewage to the level of standard discharge through a series of treatment processes. However, with the continuous improvement of environmental protection requirements and the continuous expansion of treatment scale, traditional technologies are facing many challenges. For example, how to more accurately monitor each link of sewage treatment, how to use energy and resources more effectively, how to deal with emergencies and abnormal situations, etc. At the same time, problems such as water shortage and water environment pollution are becoming increasingly prominent, requiring sewage treatment to not only meet discharge standards, but also consider the recycling of water resources.
[0003] In traditional sewage treatment systems, data collection is often scattered and incomplete, which leads to serious information islands between subsystems and the inability to effectively integrate and utilize data. Traditional systems have limited capabilities in data collection, storage and analysis, and are unable to cope with the growing volume and complexity of data. Traditional systems often require a lot of manual intervention, which is not only inefficient but also prone to human errors. In addition, traditional systems are usually static and passive, and difficult to adapt to changing environments and needs. The existence of these problems makes it difficult for traditional sewage treatment systems to meet the needs of modern cities for efficient, accurate and intelligent sewage treatment. Therefore, a new technical solution is urgently needed to solve these problems and realize the intelligence and efficiency of sewage treatment. Summary of the invention
[0004] The present invention provides a sewage treatment operation cloud platform, method and system based on the Internet of Things and GIS to solve the problem that it is difficult to achieve efficient and intelligent sewage treatment.
[0005] In a first aspect, the present invention provides a sewage treatment operation cloud platform based on the Internet of Things and GIS, the cloud platform comprising a data collection layer, a data storage layer, a data analysis layer and an application service layer, the data collection layer is deployed with a plurality of edge computing nodes, and the plurality of edge computing nodes are connected to each other through a hybrid network architecture communication; the data storage layer is configured with a high-frequency storage chain and a low-frequency storage chain, and the data storage layer is further configured with a cross-chain communication module for data interaction between the high-frequency storage chain and the low-frequency storage chain; the data analysis layer is deployed with a GIS system and a sewage treatment rectification analysis model, and the GIS system pre-stores the pipe network deployment data of the sewage pipe network and the environmental space data of the coverage area of the sewage pipe network;
[0006] The data collection layer is used to obtain the user water use data uploaded by all user clients and the sewage pipe network data collected by all IoT devices, and distribute all the user water use data and all the sewage pipe network data to each edge computing node. The edge computing node is used to pre-process all the user water use data and all the sewage pipe network data. All the IoT devices are pre-deployed at each sewage treatment process node of the sewage pipe network. All the user clients are held by all water users within the coverage of the sewage pipe network.
[0007] The data storage layer is used to temporarily store the user water use data and the sewage pipe network data preprocessed by the edge computing node in the high-frequency storage chain, and to evaluate, screen and aggregate all the data in the high-frequency storage chain to obtain the comprehensive data of the sewage pipe network, and to permanently store the comprehensive data of the sewage pipe network in the low-frequency storage chain;
[0008] The data analysis layer is used to extract the comprehensive data of the sewage pipe network in the low-frequency storage chain, and input the comprehensive data of the sewage pipe network into the GIS system, and use the GIS system to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental space data to build a multi-dimensional dynamic GIS model, analyze and predict the multi-dimensional dynamic GIS model through the sewage treatment rectification analysis model, and generate a sewage treatment rectification plan for the sewage pipe network according to the analysis and prediction results;
[0009] The application service layer is used to obtain all the sewage treatment and rectification plans output by the data analysis layer, and distribute all the sewage treatment and rectification plans to the management client held by the sewage pipe network manager.
[0010] Optionally, the data collection layer includes a data collection module and a data distribution module, the data collection module is used to obtain user water usage data uploaded by all user clients and sewage pipe network data collected by all Internet of Things devices, the user water usage data contains client location information of the user clients, and the sewage pipe network data contains device location information of the Internet of Things devices;
[0011] The data distribution module is used to receive the user water consumption data and the sewage network data, and obtain the node location information of all the edge computing nodes, and calculate the target distance between the user client or the Internet of Things device and any target edge computing node in combination with the client location information, the device location information and the node location information. If the target distance is less than or equal to a preset distance threshold, the corresponding user water consumption data or the sewage network data is distributed to the target edge computing node.
[0012] Optionally, the data storage layer further includes a first storage module, a storage data processing module, and a second storage module, wherein the first storage module is configured with a high-frequency storage chain, and the second storage module is configured with a low-frequency storage chain, and the first storage module is used to preliminarily screen the user water use data and the sewage network data according to a preset environmental contribution proof mechanism, and upload the preliminarily screened user water use data and the sewage network data to the high-frequency storage chain for temporary storage;
[0013] The storage data processing module is used to perform a quality score on the high-frequency storage chain data temporarily stored in the high-frequency storage chain according to a preset data quality assessment smart contract, obtain a data quality score for each piece of the high-frequency storage chain data, perform a secondary screening on the high-frequency storage chain data based on the data quality score, and perform data aggregation and homomorphic encryption processing on the high-frequency storage chain data after the secondary screening to obtain the comprehensive data of the sewage pipe network;
[0014] The second storage module is used to permanently store the comprehensive data of the sewage network in the low-frequency storage chain.
[0015] Optionally, the hybrid network architecture includes a 5G network, a LoRaWAN network and a satellite communication network.
[0016] In a second aspect, the present invention further provides a sewage treatment operation method based on the Internet of Things and GIS, which is applied to the sewage treatment operation cloud platform based on the Internet of Things and GIS described in the first aspect, and the method comprises the following steps:
[0017] Obtain user water usage data uploaded by all user clients and sewage pipe network data collected by all IoT devices, and distribute all the user water usage data and all the sewage pipe network data to each edge computing node;
[0018] Preprocessing all the user water use data and all the sewage pipe network data through the edge computing node;
[0019] Temporarily storing the user water consumption data and the sewage pipe network data preprocessed by the edge computing node in a high-frequency storage chain, and evaluating, screening and aggregating all the data in the high-frequency storage chain to obtain comprehensive sewage pipe network data, and permanently storing the comprehensive sewage pipe network data in a low-frequency storage chain;
[0020] Extracting the comprehensive data of the sewage pipe network in the low-frequency storage chain, and inputting the comprehensive data of the sewage pipe network into the GIS system;
[0021] The GIS system is used to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental space data to build a multi-dimensional dynamic GIS model, and the multi-dimensional dynamic GIS model is analyzed and predicted by the sewage treatment rectification analysis model, and a sewage treatment rectification plan of the sewage pipe network is generated according to the analysis and prediction results;
[0022] All the sewage treatment rectification plans are distributed to the management client held by the sewage network manager.
[0023] Optionally, the obtaining of user water usage data uploaded by all user clients and sewage pipe network data collected by all IoT devices, and distributing all the user water usage data and all the sewage pipe network data to each edge computing node includes the following steps:
[0024] Acquire user water usage data uploaded by all user clients and sewage pipe network data collected by all IoT devices, wherein the user water usage data includes client location information of the user clients, and the sewage pipe network data includes device location information of the IoT devices;
[0025] Obtaining node location information and current load of all edge computing nodes;
[0026] Calculate the target distance between the user client or the IoT device and any target edge computing node in combination with the client location information, the device location information and the node location information; if the target distance is less than or equal to a preset distance threshold, distribute the corresponding user water usage data or the sewage pipe network data to the target edge computing node;
[0027] If the distance between the user client or the IoT device and all edge computing nodes is greater than the distance threshold, the corresponding user water consumption data or sewage network data will be distributed to the idle edge computing nodes based on the current load and in accordance with the dynamic load balancing algorithm.
[0028] Optionally, temporarily storing the user water use data and the sewage pipe network data preprocessed by the edge computing node in a high-frequency storage chain, evaluating, screening and aggregating all data in the high-frequency storage chain to obtain comprehensive sewage pipe network data, and permanently storing the comprehensive sewage pipe network data in a low-frequency storage chain includes the following steps:
[0029] Preliminarily screening the user water use data and the sewage pipe network data according to a preset environmental contribution certification mechanism, and uploading the user water use data and the sewage pipe network data after the preliminary screening to the high-frequency storage chain for temporary storage;
[0030] Performing a quality score on the high-frequency storage chain data temporarily stored in the high-frequency storage chain according to a preset data quality assessment smart contract to obtain a data quality score for each piece of the high-frequency storage chain data;
[0031] Performing secondary screening on the high-frequency storage chain data based on the data quality score, and performing data aggregation and homomorphic encryption processing on the high-frequency storage chain data after the secondary screening to obtain the comprehensive data of the sewage pipe network;
[0032] The comprehensive data of the sewage pipe network is permanently stored in the low-frequency storage chain.
[0033] Optionally, the use of the GIS system in combination with the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental spatial data to construct a multi-dimensional dynamic GIS model, the multi-dimensional dynamic GIS model is analyzed and predicted by a sewage treatment rectification analysis model, and the sewage treatment rectification plan of the sewage pipe network is generated according to the analysis and prediction results, including the following steps:
[0034] Utilizing the GIS spatial analysis module in the GIS system to extract the spatial features of the pipe network data in the pipe network deployment data and the environmental spatial data, and extracting the temporal features of the pipe network data from the sewage pipe network comprehensive data;
[0035] Determine multiple comprehensive environmental impact indicators of the sewage pipe network by combining the spatial characteristics of the pipe network data and the temporal characteristics of the pipe network data;
[0036] Performing a dimensionality reduction comprehensive analysis on the sewage pipe network comprehensive data, the pipe network deployment data and the environmental space data by principal component analysis to generate a single environmental impact index of the sewage pipe network;
[0037] Constructing a multi-dimensional dynamic GIS model of the sewage pipe network based on the plurality of comprehensive environmental impact indicators and the environmental impact index;
[0038] The multi-dimensional dynamic GIS model is analyzed and predicted through the sewage treatment rectification analysis model, and a sewage treatment rectification plan for the sewage network is generated according to the analysis and prediction results.
[0039] Optionally, the multi-dimensional dynamic GIS model is analyzed and predicted by the sewage treatment rectification analysis model, and a sewage treatment rectification plan of the sewage pipe network is generated according to the analysis and prediction results, including the following steps:
[0040] Combining the pipe network deployment data and the environmental space data and using a Monte Carlo simulation method to generate a variety of different future sewage environment change scenarios;
[0041] After importing a variety of future sewage environment change scenarios into the multi-dimensional dynamic GIS model, the multi-dimensional dynamic GIS model is subjected to short-term prediction through the LSTM network model in the sewage treatment rectification analysis model to obtain the sewage short-term change prediction result;
[0042] The multi-dimensional dynamic GIS model is used to make long-term predictions through the GNN network model in the sewage treatment rectification analysis model to obtain the long-term change prediction results of sewage;
[0043] Construct a rectification plan decision model based on Markov decision process;
[0044] The short-term sewage change prediction results and the long-term sewage change prediction results are input into the rectification plan decision model to generate a sewage treatment rectification plan for the sewage network.
[0045] In the third aspect, the present invention also provides a sewage treatment operation device based on the Internet of Things and GIS, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the sewage treatment operation method based on the Internet of Things and GIS as described in the second aspect is implemented.
[0046] The beneficial effects of the present invention are:
[0047] The present invention realizes the digitization and visualization of the entire sewage treatment process by integrating the Internet of Things and GIS technology, greatly improving the monitoring and management efficiency of the sewage pipe network. Managers can grasp the operation status of the sewage pipe network in real time, discover and deal with abnormal situations in a timely manner, and effectively reduce the risk of sewage leakage and environmental pollution. The present invention also adopts edge computing technology, which can perform preprocessing at the source of data, which not only reduces the burden on the central server, but also improves the efficiency of data transmission and processing, allowing the system to respond to various situations more quickly.
[0048] The present invention also introduces a dual-chain storage structure of a high-frequency storage chain and a low-frequency storage chain, realizing hierarchical storage and management of data, which not only ensures rapid access to real-time data, but also ensures the long-term preservation of historical data, providing reliable data support for long-term planning and decision-making of sewage treatment. In addition, the present invention uses the GIS system and the sewage treatment rectification analysis model to construct a multi-dimensional dynamic GIS model, which can analyze and predict the operating status of the sewage network more comprehensively and accurately, and provide a scientific basis for the formulation of sewage treatment rectification plans. This data-driven decision-making model greatly improves the accuracy and effectiveness of sewage treatment. At the same time, the present invention also establishes a complete data closed loop, from data collection, storage, analysis to application services, forming an efficient and collaborative workflow, greatly improving the overall efficiency of sewage treatment.
[0049] On the other hand, the sewage treatment operation cloud platform constructed by the present invention also has certain openness and scalability. The cloud platform can be easily integrated with other systems, such as urban water management systems, environmental monitoring systems, etc., so as to achieve more extensive data sharing and collaborative management. It not only helps to improve the efficiency and quality of sewage treatment, but also reduces operating costs, reduces resource waste, and makes important contributions to the sustainable development of cities. In general, this sewage treatment operation cloud platform based on the Internet of Things and GIS represents a technological innovation in the field of sewage treatment, and provides a new and efficient solution to the problem of urban sewage treatment, which is expected to play an important role in future urban management and environmental protection. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 This is a cloud platform system topology diagram of a sewage treatment operation cloud platform based on the Internet of Things and GIS in one of the implementation modes of the present application.
[0051] Figure 2 This is a page schematic diagram of a web display page of a front-end display layer in one implementation manner of the present application.
[0052] Figure 3 This is a flow chart of a sewage treatment operation method based on the Internet of Things and GIS in one of the embodiments of the present application. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.
[0054] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.
[0055] Reference Figure 1The present invention discloses a sewage treatment operation cloud platform based on the Internet of Things and GIS. The sewage treatment operation cloud platform adopts a layered architecture design. From the bottom to the top, they are the physical layer, data acquisition layer, data storage layer, data analysis layer, application service layer and front-end display layer. The physical layer includes various IoT devices and clients, and communicates with the upper layer through A / D conversion and Modbus protocol. The data acquisition layer adopts a variety of network technologies such as 5G network, LoRaWAN network and satellite communication, and combines multiple edge computing nodes (NUC nodes) and Ubuntu Server for data acquisition and preprocessing. The data acquisition layer also integrates Docker containerization technology, as well as data processing libraries such as Scipy, Scikit-learn and Pandas. The data storage layer is configured with a high-frequency storage chain and a low-frequency storage chain. The low-frequency chain uses data quality assessment smart contracts and PoA consensus mechanism, while the high-frequency chain is based on Hyperledger Fabric blockchain technology and implements functions such as sorting services, homomorphic encryption, data aggregation, privacy protection and PoFC mechanism. The data storage layer is also configured with a cross-chain communication module for data interaction between the high-frequency storage chain and the low-frequency storage chain. The cross-chain communication module is implemented based on Polkadot cross-chain technology.
[0056] The data analysis layer is deployed with a GIS system and a sewage treatment rectification analysis model. The GIS system pre-stores the sewage pipe network deployment data and the environmental spatial data of the sewage pipe network coverage area. The GIS system uses ESRI ArcGIS Enterprise and PostgreSQL+PostGIS. The deep learning model includes LSTM network and GNN model. The entire analysis process is driven by the Drools rule engine. The application service layer is based on the Spring Cloud microservice architecture and provides functional modules such as system management, message management, and plan management. The front-end display layer uses Vue.js and HTML5 technology to implement a unified Web user interface.
[0057] Based on the above-mentioned cloud platform architecture, in this implementation, the data collection layer is used to obtain the user water usage data uploaded by all user clients and the sewage network data collected by all IoT devices, and distribute all user water usage data and all sewage network data to each edge computing node. The edge computing node is used to pre-process all user water usage data and all sewage network data. All IoT devices are pre-deployed at each sewage treatment process node of the sewage network, and all user clients are held by all water users within the coverage of the sewage network.
[0058] The data storage layer is used to temporarily store the user water consumption data and sewage network data pre-processed by the edge computing node in the high-frequency storage chain, and evaluate, screen and aggregate all data in the high-frequency storage chain to obtain the comprehensive data of the sewage network, and permanently store the comprehensive data of the sewage network in the low-frequency storage chain.
[0059] The data analysis layer is used to extract the comprehensive data of the sewage pipe network in the low-frequency storage chain and input the comprehensive data of the sewage pipe network into the GIS system. The GIS system is used to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental spatial data to build a multi-dimensional dynamic GIS model. The multi-dimensional dynamic GIS model is analyzed and predicted through the sewage treatment rectification analysis model, and the sewage treatment rectification plan of the sewage network is generated according to the analysis and prediction results.
[0060] The application service layer is used to obtain all sewage treatment rectification plans output by the data analysis layer, and distribute all sewage treatment rectification plans to the management client held by the sewage network manager.
[0061] In the above implementation, specifically, the implementation principle of the data collection layer involves multiple key links. First, the user client (such as a smart water meter or a mobile application) regularly collects water use data, including water consumption, water use time, water quality parameters, etc. These data are transmitted to the central server of the system through a secure network protocol (such as HTTPS or MQTT). At the same time, IoT devices (such as flow meters, water quality sensors, pressure sensors, etc.) distributed throughout the sewage network will monitor the operation of the sewage network in real time and collect data such as water flow, water quality indicators, and pipeline pressure. These devices transmit data to the system through 5G, LoRaWAN, or satellite communications. After the data arrives, the system distributes it to multiple edge computing nodes, and multiple edge computing nodes are also connected through 5G, LoRaWAN, or satellite communications. These nodes may be small servers deployed in sewage treatment plants or pumping stations, which perform preliminary data processing tasks. For example, for water use data, the edge node may calculate the average water consumption of a community or street; for sewage network data, outlier detection may be performed, such as identifying sudden flow changes or water quality anomalies. This distributed processing approach not only reduces the burden on central servers, but also enables faster responses to local events.
[0062] In specific implementation, Docker containerization technology can be used to deploy preprocessing algorithms on edge nodes. For example, Python's Pandas library is used to process time series data, and the Scikit-learn library is used for anomaly detection. A simple anomaly detection algorithm may be as follows: first calculate the mean (μ) and standard deviation (σ) of historical data, and then for a new data point x, if |x-μ|>3σ, mark it as an anomaly. This method can quickly identify obvious anomalies, such as a sudden increase in flow caused by a pipe leak. This architecture can significantly improve data processing efficiency and system response speed. In addition, by preprocessing data at the source, the amount of data transmitted to the central server can be greatly reduced, saving network bandwidth and storage space. This is particularly beneficial for managing complex sewage systems in large cities, enabling near real-time monitoring and rapid response.
[0063] The implementation principle of the data storage layer is based on blockchain technology, using a dual-chain structure to balance data processing efficiency and long-term storage needs. The high-frequency storage chain uses a fast blockchain similar to Ethereum to temporarily store pre-processed data from edge computing nodes. This chain is characterized by high throughput and low latency, and can quickly process large amounts of real-time data. After the data enters the high-frequency chain, the system will start the evaluation screening and data aggregation process. The evaluation screening includes data integrity checks, duplicate data removal, outlier marking, etc. Data aggregation involves calculating statistical indicators such as average, maximum / minimum, and standard deviation within the time period. The purpose of this process is to transform raw data into more valuable comprehensive information. The processed comprehensive data will be transferred to the low-frequency storage chain for permanent storage. The low-frequency chain uses a safer and more stable consensus mechanism (such as PoA, Proof of Authority) to ensure the long-term reliability and immutability of the data.
[0064] In terms of implementation, high-frequency chains can be implemented using Hyperledger Fabric, which supports high-concurrency transactions and flexible smart contracts. The data aggregation process can be written as a smart contract, for example, the average flow and water quality indicators of each node in the pipeline network are calculated once an hour. Low-frequency chains can use blockchains that focus more on security, such as optimized Ethereum private chains. Data migration between the two chains can be achieved through cross-chain technologies such as Polkadot. Data evaluation can use the isolation forest algorithm to detect outliers. This storage architecture is capable of handling large-scale real-time data streams and supports real-time monitoring and rapid response of sewage systems. Through data aggregation and evaluation, the amount of data that needs to be stored for a long time is greatly reduced while retaining key information. On the other hand, blockchain technology ensures the integrity and traceability of data, which is very important in environmental supervision and accountability. The dual-chain structure also balances performance and security, providing support for data analysis and decision-making at different time scales.
[0065] The implementation principle of the data analysis layer combines GIS technology and advanced analytical models. First, the system extracts comprehensive sewage network data from low-frequency storage chains, which contains long-term historical trends and patterns. Then, this data is input into a GIS system, such as ESRI ArcGIS Enterprise. The GIS system pre-stores the deployment data of the sewage network (such as pipe location, diameter, material, etc.) and environmental spatial data (such as topography, soil type, population distribution, etc.). The system spatially associates and integrates data from these different sources to build a multi-dimensional dynamic GIS model. This model not only includes the physical spatial characteristics of the pipeline network, but also includes operational data and environmental factors that change over time. Next, the sewage treatment rectification analysis model will analyze and predict this dynamic GIS model. This analysis model may contain multiple sub-models, such as hydraulic models (predicting water flow dynamics in the pipeline network), water quality models (predicting pollutant diffusion), pipeline network status prediction models (predicting pipeline aging and failure), etc.
[0066] In terms of implementation, PostgreSQL database can be used with PostGIS extension to store and process spatial data. The construction of dynamic GIS models can use ArcGIS's ModelBuilder tool to integrate different data sources into a unified geographic database. The analysis model can use machine learning methods, such as LSTM networks to predict short-term governance results, and use graph neural networks (GNNs) to analyze and predict long-term governance results for the entire pipe network system. The training and reasoning of these models can be implemented through Python's deep learning libraries (such as TensorFlow or PyTorch) and integrated with the GIS system through ArcGIS's Python API.
[0067] The analysis results will be used to generate a sewage treatment rectification plan. For example, if the model predicts that a certain area may exceed the water quality standard in the next few months, the system will generate a plan with specific rectification measures, such as increasing the treatment capacity of a treatment plant, replacing a specific pipe section, or adjusting operating parameters. This plan generation process can be implemented using a rule engine (such as Drools), combining the analysis results with predefined decision rules to automatically generate reasonable rectification suggestions. This analysis method can provide a global view of the sewage system, identify potential problems and take preventive measures in advance. For example, the system may find that the pipe network load in a certain area is increasing year by year, and predict that it may reach the capacity limit within two years. Based on this prediction, the management department can plan the pipe network expansion project in advance to avoid possible overflow problems in the future. In addition, by integrating multi-source data and advanced analytical techniques, the system can discover complex patterns and associations that are difficult to detect with traditional methods, providing more accurate and effective decision support for sewage treatment.
[0068] The implementation principle of the application service layer is based on the microservice architecture, mainly using the Spring Cloud framework. This layer is responsible for receiving the sewage treatment rectification plans output by the data analysis layer and distributing these plans to the relevant sewage pipe network managers. First, the system will store the rectification plans in a distributed database (such as MongoDB) to ensure high availability and scalability of data. Then, notifications are sent to each microservice through a message queue system (such as RabbitMQ). These microservices include user management services, notification services, and plan management services. The user management service is responsible for maintaining the identity and permission information of managers to ensure that only authorized personnel can access the relevant rectification plans. The notification service is responsible for sending summaries and links of the rectification plans to managers through different channels (such as email, SMS, in-app push, etc.) according to preset rules. The plan management service is responsible for displaying the detailed content of the rectification plan, tracking progress, and updating it.
[0069] In terms of implementation, Spring Boot can be used to quickly build various microservices, Spring CloudNetflix's Eureka can be used for service registration and discovery, and Zuul can be used as an API gateway. For data transmission with high security requirements, Spring Security combined with JWT (JSON Web Token) can be used to implement authentication and authorization. In order to improve the availability and performance of the system, Redis can be used as a cache layer to store commonly used data such as user information and recent rectification plans. In addition, a workflow engine (such as Activiti) can be implemented to manage the execution process of the rectification plan, so that the system can automatically track the various stages of the plan and remind relevant personnel to take action when necessary. The microservice architecture improves the flexibility and scalability of the system, allowing the system to easily cope with sewage networks of different sizes and different numbers of users. Secondly, through the multi-channel notification mechanism, it ensures that important information can be delivered to relevant managers in a timely manner, improving the response speed. Thirdly, the implementation of permission management and security measures ensures the security of sensitive information and avoids the risk of data leakage. Finally, the introduction of workflow management makes the entire sewage treatment process more standardized and controllable, and managers can clearly understand the execution status of each rectification plan and discover and solve problems in a timely manner. This intelligent management method greatly improves the efficiency of sewage treatment, reduces human errors, and provides reliable data support for management decisions.
[0070] In one embodiment, referring to Figure 2, the content displayed on the front-end display layer is a comprehensive data panel of sewage performance operation informationization, which focuses on displaying the key indicators and operation status of the sewage treatment system. The data displayed on the panel, such as the number of sewage treatment plants, design capacity, design length, built length and completion rate of the pipeline network, come from the low-frequency storage chain of the data storage layer, reflecting the infrastructure of the sewage treatment system. The real-time data, cumulative data and compliance rate of sewage treatment volume and COD reduction displayed on the panel come from the real-time monitoring of the data collection layer, and are presented after being processed by the data analysis layer. The map in the center of the panel shows the sewage treatment effects in different regions, which are distinguished by color depth, and mainly uses the visualization results generated by the GIS system.
[0071] The rectification score ranking of each region shown on the panel is the result generated based on the sewage treatment rectification plan of the data analysis layer, and shows the trend of evaluation scores in each quarter, reflecting the overall operation effect of the sewage treatment system. The two tables of performance evaluation and rectification status list the specific evaluation indicators and rectification progress of each region in detail, which are the results processed by the application service layer.
[0072] The front-end display layer specifically uses the component system of Vue.js to decompose the panel into multiple independent components, such as map components, chart components, data table components, etc., for easy management and reuse. Using the responsive characteristics of Vue.js, the back-end data is bound to the front-end display elements to ensure that the interface is automatically updated when the data changes. Use WebSocket or polling technology to maintain connection with the back-end to obtain the latest data such as processing volume and COD reduction in real time. Use a dedicated map library (such as Leaflet or MapboxGL) combined with GIS data to render the map, and dynamically update the regional color according to the data. Use chart libraries such as ECharts or D3.js to draw a bar chart of the evaluation score. Use a dedicated table component library to implement a sortable and filterable data table. In summary, the front-end display layer effectively converts complex back-end data into an intuitive and easy-to-understand visual interface, allowing managers to quickly grasp the operating status of the sewage treatment system and make timely decisions and adjustments. It reflects the processing results of the data analysis layer and the application service layer, and is an important output port of the entire sewage treatment operation cloud platform.
[0073] In one embodiment, the data collection layer includes a data collection module and a data distribution module. The data collection module is used to obtain the user water use data uploaded by all user clients and the sewage pipe network data collected by all IoT devices. The user water use data includes the client location information of the user client, and the sewage pipe network data includes the device location information of the IoT device. The data distribution module is used to receive the user water use data and the sewage pipe network data, and obtain the node location information of all edge computing nodes, and calculate the target distance between the user client or the IoT device and any target edge computing node in combination with the client location information, the device location information and the node location information. If the target distance is less than or equal to the preset distance threshold, the corresponding user water use data or sewage pipe network data is distributed to the target edge computing node.
[0074] In the above implementation, specifically, the data acquisition module is the data entry of the entire system, which is responsible for acquiring data from two main sources: user clients and IoT devices. For user clients, water use data, such as water consumption, water use time, water use type, etc., can be actively entered or automatically recorded by users through mobile applications or web interfaces. These data are transmitted to the server in JSON or XML format via HTTP or HTTPS protocol. Client location information may be obtained through GPS, Wi-Fi positioning or IP address positioning, accurate to longitude and latitude coordinates. For IoT devices, such as various sensors installed in sewage pipe networks, including flow meters, water quality detectors, pressure sensors, etc., these devices transmit the collected data to the cloud server regularly through low-power wide area networks (LPWAN) such as LoRaWAN or NB-IoT, or through cellular networks such as 4G / 5G networks. The device location information may be a pre-configured fixed coordinate, or obtained in real time through a built-in GPS module.
[0075] The data acquisition module needs to handle a large number of concurrent connections and may adopt a distributed architecture, using message queues such as Kafka or RabbitMQ to buffer and process inbound data streams. In order to ensure the integrity and security of the data, the module also needs to implement mechanisms such as data verification, encrypted transmission, and identity authentication. In actual applications, for example, a user records a water consumption of 100 liters through a mobile phone APP, the time is 10:00 on July 21, 2023, the water type is domestic water, and the location is 39.9042°N, 116.4074°E. This data will be packaged into JSON format: {"user_id": "123456", "water_usage": 100, "time": "2023-07-21T10:00:00", "type": "domestic", "latitude": 39.9042, "longitude": 116.4074}, and then sent to the server through an HTTPS POST request. At the same time, a flow meter installed in a sewage pipe network may send data every 5 minutes, including information such as flow, pH value, turbidity, as well as device ID and location information. This data collection method can achieve near real-time data collection, providing timely and accurate basic data support for subsequent data analysis and decision-making.
[0076] As a bridge between the data collection layer and the edge computing layer, the data distribution module needs to efficiently distribute the collected data to the most suitable edge computing node. First, the module needs to maintain a dynamic registry of edge computing nodes, which contains each node's unique identifier, location information (latitude and longitude), current load status, etc. This registry is stored in a distributed database such as Cassandra or HBase to support high-concurrency reading and writing. When receiving user water usage data or sewage network data, the module first parses the location information in the data. Then, it traverses the edge computing node registry and calculates the distance to the data source for each node. The distance calculation can use the Haversine formula, which takes into account the curvature of the earth and can accurately calculate the distance between two longitude and latitude coordinates. The Haversine formula is as follows:
[0077]
[0078] Among them, r is the radius of the earth (about 6371km), φ1 and λ1 are the latitude and longitude of the first point, and φ2 and λ2 are the latitude and longitude of the second point.
[0079] The calculated distance is compared with a preset distance threshold, which can be set according to the specific needs of the system, such as 5 kilometers. If the distance is less than or equal to the threshold, the data will be distributed to the edge node. In actual implementation, multiple factors will be considered to select the best edge node, including not only the distance, but also the current load of the node, network latency, etc. Data distribution can be achieved through a message queue system such as Apache Kafka, with each edge node subscribing to a specific topic. To optimize performance, a geospatial index such as Geohash can be used to speed up the search for neighboring nodes. For example, suppose there is a water use data location (39.9042°N, 116.4074°E), and there are three edge nodes A (39.9000°N, 116.4000°E), B (39.9500°N, 116.4500°E) and C (40.0000°N, 116.5000°E) in the system. After calculating the distance using the Haversine formula, we get: Node A is 0.92km away, Node B is 5.76km away, and Node C is 12.38km away. If the distance threshold is set to 5km, then this data will only be distributed to Node A. This distribution mechanism ensures that data is sent to the edge node closest to the geographic location for processing, thereby reducing network latency and improving data processing efficiency. It can also achieve load balancing and prevent some nodes from being overloaded while other nodes are idle.
[0080] In one embodiment, the data storage layer also includes a first storage module, a storage data processing module and a second storage module. The first storage module is configured with a high-frequency storage chain, and the second storage module is configured with a low-frequency storage chain. The first storage module is used to perform preliminary screening of user water use data and sewage network data according to a preset environmental contribution proof mechanism, and upload the preliminary screened user water use data and sewage network data to the high-frequency storage chain for temporary storage.
[0081] The storage data processing module is used to perform quality scoring on the high-frequency storage chain data temporarily stored in the high-frequency storage chain according to the preset data quality assessment smart contract, obtain the data quality score of each high-frequency storage chain data, perform secondary screening on the high-frequency storage chain data based on the data quality score, and perform data aggregation and homomorphic encryption processing on the high-frequency storage chain data after secondary screening to obtain the comprehensive data of the sewage pipe network. The second storage module is used to permanently store the comprehensive data of the sewage pipe network in the low-frequency storage chain.
[0082] In the above implementation, specifically, the first storage module is configured with a high-frequency storage chain, which is mainly responsible for preliminary screening and temporary storage of user water usage data and sewage network data. High-frequency storage chains usually use distributed ledger technology, such as private chains or alliance chains based on blockchain platforms such as Hyperledger Fabric or Ethereum. This storage method can ensure the immutability and traceability of data. Proof of Environmental Contribution (PoEC) is a custom consensus algorithm for preliminary screening of data. This mechanism may consider the following aspects: the timeliness of data (such as data within the last 24 hours), the integrity of the data (whether it contains all necessary fields), the consistency of the data (whether it is consistent with the trend of historical data), and the reputation score of the data provider (based on the quality of historical data).
[0083] When implementing it, you can design a scoring system, for example:
[0084] 1. Timeliness score = 100*(1-(current time-data time) / 24 hours)
[0085] 2. Completeness score = (actual number of fields / required number of fields)*100
[0086] 3.Consistency score = 100-|(current value-historical average) / historical standard deviation|
[0087] 4. Reputation score = the historical average score of the data provider (0-100)
[0088] Total score = (timeliness score*0.3+completeness score*0.2+consistency score*0.3+reputation score*0.2)
[0089] If the total score exceeds the preset threshold (such as 80 points), the data passes the preliminary screening. The data that passes the screening will be packaged into blocks and added to the high-frequency storage chain through a consensus algorithm (such as the Practical Byzantine Fault Tolerance algorithm PBFT). Each block may contain multiple data records, as well as metadata such as timestamps and hash values of the previous block. This preliminary screening and temporary storage mechanism can effectively filter out obviously unqualified data, reduce the amount of data to be processed later, and ensure the real-time and credibility of the data. The use of the high-frequency storage chain provides a safe and transparent temporary storage environment for the data, laying the foundation for subsequent in-depth analysis and processing.
[0090] The storage data processing module is the core component of the data storage layer, responsible for deep processing and screening of data in the high-frequency storage chain. Its core function is to score the data quality through the preset data quality assessment smart contract, then perform secondary screening based on the scoring results, and finally aggregate and encrypt the screened data. The data quality assessment smart contract is an automatically executed program deployed on the blockchain, which may include the following main evaluation dimensions:
[0091] 1. Data accuracy: Evaluated by comparing with adjacent sensor data or analyzing historical data trends.
[0092] 2. Data integrity: Check for missing values or outliers.
[0093] 3. Data consistency: Check whether the logical relationship between different data sources is reasonable.
[0094] 4. Data timeliness: Evaluate the interval between the time data is generated and the time it is processed.
[0095] 5. Data availability: Assess whether the data conforms to the predetermined format and standards.
[0096] Different weights can be set for each dimension, and a comprehensive score of 0-100 is finally obtained. In specific implementation, for accuracy, the deviation of the data from the average value of the five adjacent sensors can be calculated. If the deviation is within the allowable range (such as ±10%), full marks are obtained, otherwise points are deducted according to the degree of deviation. The integrity check will traverse all necessary fields, and 20 points will be deducted for missing a field. The consistency check may compare whether the water volume data and water quality data are logically consistent, for example, whether the pollutant concentration is diluted accordingly when the water volume suddenly increases. Timeliness can be scored based on the difference between the data generation time and the processing time. For example, if the difference is less than 1 hour, full marks are obtained, and 10 points are deducted for each additional hour. The availability check verifies whether the data format, unit, etc. meet the predetermined standards. Based on the scores calculated by the above process, the system will perform secondary screening, for example, only retaining data with a score of more than 90 points. The screened data will enter the aggregation stage, at which time the system will group and count according to time, geographic location or other dimensions to obtain higher-level comprehensive data. Finally, the aggregated data is homomorphically encrypted. Homomorphic encryption is a special type of encryption that allows specific algebraic operations to be performed directly on ciphertext without first decrypting it. This allows necessary data analysis while protecting data privacy. Common homomorphic encryption algorithms include the Paillier encryption system or the BGV scheme. This processing method can improve the availability and security of data while ensuring data quality. The automated evaluation and screening process achieved through smart contracts greatly reduces human intervention and improves efficiency and objectivity. The use of homomorphic encryption provides security for subsequent data analysis and sharing, allowing limited but meaningful data operations even in an encrypted state.
[0097] The second storage module is equipped with a low-frequency storage chain, which is mainly responsible for permanently storing the processed and encrypted comprehensive data of the sewage network. Low-frequency storage chains usually adopt a more stable and scalable blockchain architecture, such as an enterprise-level blockchain based on Hyperledger Fabric or an optimized Ethereum private chain. Compared with high-frequency storage chains, low-frequency storage chains may have a longer block time, but each block can contain more data and adopt a stronger consensus mechanism to ensure the long-term reliability of data. Low-frequency storage chains use Merkle Tree or more advanced data structures such as IAVL+ trees to organize block data to improve data verification and retrieval efficiency. Low-frequency storage chains use consensus algorithms such as proof of work (PoW), proof of stake (PoS), or practical Byzantine fault tolerance (PBFT) that is more suitable for enterprise applications. For example, when using DPoS (delegated proof of stake), nodes with good reputation can be selected as validators to determine which nodes have the right to generate new blocks through voting.
[0098] In specific implementation, when the comprehensive data of the sewage pipe network is transmitted from the storage data processing module to the second storage module, the system will perform the following steps: first, verify the integrity of the data and the legitimacy of the source, specifically using digital signature technology; classify the data according to data type, timestamp, geographic location and other information, and establish a corresponding index structure for future connection and fast retrieval; compress large data sets, and may use erasure coding technology to enhance the fault tolerance of the data; package the processed data into blocks, each block may contain comprehensive data for multiple hours or a whole day; reach a consensus in the network through the selected consensus mechanism to confirm the validity of the new block; link the newly confirmed block with the existing block to form an unalterable data chain; replicate the storage block data on multiple nodes to ensure high availability and disaster resistance of the data.
[0099] For example, suppose there is a set of aggregated and encrypted sewage network data, which contains the sewage treatment situation of a certain area for 24 hours:
[0100] {
[0101] "region_id": "R001",
[0102] "date": "2023-07-21",
[0103] "hourly_data": [
[0104] {"hour": 0, "avg_flow": "Enc(230.5)", "avg_pH": "Enc(7.1)", ...},
[0105] {"hour": 1, "avg_flow": "Enc(210.2)", "avg_pH": "Enc(7.2)", ...}, ...
[0106] {"hour": 23, "avg_flow": "Enc(245.3)", "avg_pH": "Enc(7.0)", ...}
[0107] ],
[0108] "daily_summary": {
[0109] "total_flow": "Enc(5780.6)",
[0110] "avg_pH": "Enc(7.15)",
[0111] "pollution_reduction": "Enc(85.3%)"
[0112] }
[0113] }
[0114] This set of data will be compressed and then packaged into a new block together with the data from other regions at the same time. The block header may contain the following information:
[0115] {
[0116] "block_number": 1234567,
[0117] "timestamp": "2023-07-21T00:00:00Z",
[0118] "previous_block_hash": "0x1a2b3c...",
[0119] "merkle_root": "0x4d5e6f...",
[0120] "nonce": 9876543210
[0121] }
[0122] After confirmation through the consensus process, the new block will be added to the low-frequency storage chain and become part of the permanent record. This permanent storage mechanism ensures that important comprehensive data of the sewage network is preserved in a long-term, secure and traceable manner. It provides a reliable data foundation for future data analysis, trend forecasting and policy making, while also meeting data audit and compliance requirements. By combining blockchain technology and advanced encryption methods, this storage system not only ensures the integrity and immutability of the data, but also maintains the privacy of the data, providing strong data support for the long-term operation and optimization of the entire sewage treatment system.
[0123] The data analysis layer first needs to extract the comprehensive data of the sewage network from the low-frequency storage chain. Since the data on the low-frequency storage chain is processed by homomorphic encryption, the corresponding decryption algorithm is needed to restore the original data. Specifically, the decryption algorithm of the Paillier encryption system may be used. Assuming that there is an encrypted data E(m), the decryption process can be expressed as: m=D(E(m), sk), where D is the decryption function and sk is the private key. In actual operation, it may be necessary to use a distributed key management system to securely store and use private keys to prevent single point failures and improve the security of the system. The data extraction process may use the index structure of the blockchain to quickly locate the required data blocks. For example, if you need to extract data for a specific time period and geographic area, you can use the timestamp in the block header and the geographic identifier in the data to filter the relevant blocks.
[0124] Next, the data analysis layer inputs the extracted and decrypted sewage network comprehensive data into the GIS system. Geographic Information System (GIS) is a powerful tool that can process, analyze and visualize geospatial data. The process of inputting sewage network data into the GIS system involves steps such as data format conversion, coordinate system matching and data import. First, the sewage network comprehensive data needs to be converted into a format that the GIS system can recognize, such as shapefile or GeoJSON. This process involves the reorganization of the data structure and the mapping of attribute fields. For example, the JSON format data mentioned above needs to be converted into GeoJSON format. Secondly, it is necessary to ensure that the coordinate system of the data is consistent with the coordinate system used by the GIS system, such as converting from WGS84 to the projection coordinate system used locally. The data import process may use the API or command line tool provided by the GIS system. For example, the PyQGIS API of the open source GIS software QGIS can be used to programmatically automate the import.
[0125] Next, the GIS system needs to combine the comprehensive data of the sewage network, the network deployment data, and the environmental spatial data to build a multi-dimensional dynamic GIS model. Building such a multi-dimensional dynamic GIS model enables us to analyze the operation status of the sewage network from multiple perspectives and observe its changes over time. This provides a rich data foundation for subsequent analysis and prediction. Finally, the data analysis layer analyzes and predicts the multi-dimensional dynamic GIS model through the sewage treatment rectification analysis model, and generates a sewage treatment rectification plan for the sewage network based on the analysis and prediction results. This process involves multiple complex data analysis and machine learning techniques, including but not limited to time series analysis, spatial statistical analysis, network flow analysis, and predictive modeling. This method based on a multi-dimensional dynamic GIS model and a complex analysis model can comprehensively evaluate the current status of the sewage network, accurately predict future problems, and provide targeted solutions. It not only takes into account the physical characteristics and operation data of the network, but also incorporates environmental factors and socio-economic factors, thereby producing a more comprehensive and feasible rectification plan. The effect of this approach is significant: it can help decision makers better understand the complexity of the system, optimize resource allocation, formulate long-term sustainable development strategies, and ultimately achieve efficient, economical, and environmentally friendly operation of the sewage treatment system.
[0126] Reference Figure 3 , Figure 3 FIG. 1 is a flow chart of a sewage treatment operation method based on the Internet of Things and GIS in one embodiment. It should be understood that although Figure 3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 3 As shown, the sewage treatment operation method based on the Internet of Things and GIS disclosed in the present invention specifically includes the following steps:
[0127] S101. Obtain the user water consumption data uploaded by all user clients and the sewage pipe network data collected by all IoT devices, and distribute all user water consumption data and all sewage pipe network data to each edge computing node.
[0128] Among them, the user client may be a smartphone application or a web application, through which users can enter or upload their water use information, such as water consumption, water use time, water use type, etc. These data are usually transmitted to the central server in a structured format (such as JSON or XML) through the encrypted HTTPS protocol. At the same time, IoT devices distributed throughout the sewage network, such as flow meters, water quality sensors, pressure sensors, etc., will continuously collect network operation data. These devices may use low-power wide area network (LPWAN) technologies such as LoRaWAN or NB-IoT for data transmission to ensure good coverage and low power operation in complex urban environments. The frequency of data collection may vary depending on the type of device and monitoring requirements. For example, flow may be collected every 5 minutes, while water quality data may be collected every hour. These raw data are first transmitted to the nearest gateway device and then transmitted to the central data center through a secure VPN channel. In the data center, a load balancer receives all incoming data streams and distributes the data to multiple edge computing nodes according to predefined rules.
[0129] The distribution algorithm may consider multiple factors, such as geographic location, data type, current node load, etc. For example, a consistent hashing algorithm can be used to ensure that related data is always sent to the same node while achieving load balancing. Assuming that we have N edge nodes, the target node of each data packet can be determined by the formula node_id=hash(data_key)%N. This distribution method can fully utilize the advantages of distributed systems and improve the parallelism and efficiency of data processing. The effect of implementing this step is that the massive data originally scattered among many user terminals and IoT devices are collected and distributed to appropriate computing resources in an orderly manner, laying the foundation for subsequent data processing and analysis. This method can not only handle large-scale real-time data streams, but also dynamically adjust resource allocation according to the system load to ensure efficient operation and rapid response of the entire system.
[0130] S102. Preprocess all user water consumption data and all sewage network data through edge computing nodes.
[0131] Among them, the preprocessing process usually includes multiple sub-steps such as data cleaning, standardization, deduplication, anomaly detection and preliminary feature extraction. First of all, data cleaning aims to deal with missing values, outliers and inconsistent formats. For example, for missing traffic data, the average value of the previous and next time points or more complex interpolation algorithms (such as linear interpolation, spline interpolation) may be used to fill in. For outliers, statistical methods such as Z-score or IQR (interquartile range) method can be used for detection and processing. Assuming that the Z-score of a traffic data point is z, it can be calculated by the formula z=(x-μ) / σ, where x is the value of the data point, μ is the mean of the data set, and σ is the standard deviation. If |z|>3, the point may be marked as an anomaly and further verified or adjusted.
[0132] Data normalization is an important step to ensure that data from different sources and types are comparable. For example, all numerical features can be adjusted to the interval [0,1] using Min-Max scaling: x_normalized=(x-x_min) / (x_max-x_min). For categorical data, such as pipe material type, one-hot encoding may be required to convert to numerical representation. The deduplication process ensures that there are no duplicate records in the dataset, which is crucial to avoid bias in subsequent analysis. Duplicate records can be efficiently identified and removed using hash tables or sorting algorithms. Anomaly detection is not just about identifying individual outliers, but also about identifying abnormal patterns. For example, time series analysis methods such as ARIMA models are used to detect abnormal fluctuations in water consumption, or clustering algorithms such as DBSCAN are used to identify abnormal points in space. Preliminary feature extraction aims to extract meaningful features from the raw data to facilitate subsequent in-depth analysis. For example, for time series data, statistical features such as moving average, standard deviation, peak value, etc. can be calculated; for spatial data, spatial features such as distance and density can be calculated. All these preprocessing steps are usually performed in parallel on edge nodes to fully utilize the computing power of distributed systems. Each edge node may be equipped with a high-performance CPU and GPU, running optimized data processing libraries such as NumPy, Pandas, or custom C++ programs. Necessary data exchange and synchronization may also be performed between nodes to ensure global consistency.
[0133] S103. Temporarily store the user water use data and sewage network data pre-processed by the edge computing node in the high-frequency storage chain, evaluate, screen and aggregate all data in the high-frequency storage chain to obtain the comprehensive data of the sewage network, and permanently store the comprehensive data of the sewage network in the low-frequency storage chain.
[0134] Among them, high-frequency storage chains usually use a higher-performance consensus algorithm. Each preprocessed data point is encapsulated into a transaction, which contains information such as data value, timestamp, source device ID, etc. These transactions are packaged into blocks and agreed upon between nodes in the network through a consensus algorithm. The block generation interval of the high-frequency chain may be very short, such as one block per second, to meet the writing needs of a large amount of real-time data. The storage of data on the high-frequency chain uses lightweight encryption methods, such as the symmetric encryption algorithm AES, to balance security and performance. Next, a series of predefined rules are applied to filter the data, and then the sliding window technology is used to aggregate the time series data, such as calculating the average flow rate per hour, the maximum water quality parameter value, etc. For spatial data, spatial clustering algorithms such as DBSCAN can be used to identify hot spots. Data aggregation may also involve multi-dimensional analysis, such as combining water use data and sewage data to calculate water balance indicators. The aggregated data forms comprehensive sewage network data, which contains higher-level information such as trends, patterns, and key performance indicators. Finally, these comprehensive data are permanently stored in the low-frequency storage chain.
[0135] Low-frequency chains usually use more secure but relatively slower consensus algorithms, such as proof of work (PoW) or proof of stake (PoS). Data may be more strongly encrypted before being written to the low-frequency chain, such as homomorphic encryption, to support data analysis in an encrypted state. The block generation interval of the low-frequency chain may be longer, such as one block per hour or day. Each block may contain all the comprehensive data within a time period, as well as an index pointing to the corresponding original data of the high-frequency chain. The effect of this dual-chain structure is significant: it can meet the rapid writing needs of real-time data while ensuring the long-term secure storage of important comprehensive data. The high-frequency chain provides immediate availability and transparency of data, while the low-frequency chain ensures the integrity and immutability of the data. This approach not only improves the performance and scalability of the entire system, but also provides a reliable data foundation for subsequent data analysis and decision-making.
[0136] S104. Extract the comprehensive data of the sewage pipe network in the low-frequency storage chain, and input the comprehensive data of the sewage pipe network into the GIS system.
[0137] Among them, for data extraction from low-frequency storage chains, the system needs to implement an efficient blockchain query mechanism. This involves using a Merkle tree structure to quickly locate and verify specific data blocks. For example, if data for a specific time period and geographic area needs to be extracted, the query process may first filter the relevant blocks by the timestamp in the block header, and then use the Merkle tree's path verification to confirm the integrity of the data. Assuming that a Merkle tree has n leaf nodes, the complexity of verifying a data block is O(log n), which greatly improves the query efficiency of large-scale data sets. Secondly, since the data on the low-frequency chain may use advanced encryption technology such as homomorphic encryption, the decryption process requires special attention. Homomorphic encryption allows specific computing operations to be performed in the encryption domain, but the final result still needs to be decrypted. The final decryption process may require the use of a distributed key management system to enhance security. The decrypted data needs to be formatted to meet the input requirements of the GIS system. Specifically, the blockchain-specific data structure can be converted into a standard geographic data format, such as GeoJSON or Shapefile.
[0138] Before the data is entered into the GIS system, coordinate system conversion and spatial reference alignment are also required. For example, from a global coordinate system such as WGS84 to a local projection coordinate system. The conversion process can use the following projection formula: x'=R*(λ-λ0), y'=R*ln[tan(π / 4+φ / 2)]. Where R is the radius of the earth, λ and φ are longitude and latitude, and λ0 is the central meridian. This ensures that the data extracted from the blockchain can be correctly located and represented in the GIS system. Data entry into the GIS system may be done in batch import or real-time streaming, depending on the amount of data and real-time requirements. For large amounts of historical data, batch import tools such as ogr2ogr may be used, while for data that needs to be updated in real time, dynamic import may be performed using the API provided by the GIS system. The effect of the entire process is to convert the encrypted data stored in distributed storage into a spatial dataset that can be directly used by the GIS system. This not only ensures the security and integrity of the data, but also lays the foundation for subsequent spatial analysis and visualization. In this way, the comprehensive data of the sewage network is effectively integrated into the spatial information system, enabling managers to fully understand and analyze the operation of the sewage network from a geospatial perspective.
[0139] S105. Use the GIS system to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental spatial data to build a multi-dimensional dynamic GIS model, analyze and predict the multi-dimensional dynamic GIS model through the sewage treatment rectification analysis model, and generate a sewage treatment rectification plan for the sewage pipe network based on the analysis and prediction results.
[0140] S106. Distribute all sewage treatment rectification plans to the management client held by the sewage network manager.
[0141] Among them, the distribution of rectification plans requires a secure and reliable transmission mechanism. Specifically, end-to-end encryption technology, such as TLS1.3 protocol, can be used to ensure that data is not eavesdropped or tampered with during transmission. The encryption process can use asymmetric encryption algorithms such as RSA or elliptic curve encryption (ECC) to exchange session keys, and then use symmetric encryption algorithms such as AES-256-GCM for efficient data encryption. In order to support the differentiated access needs of managers at different levels, the system may implement a role-based access control (RBAC) mechanism. RBAC can define multiple roles, such as system administrators, regional supervisors, field operators, etc., each with a different set of permissions. This fine-grained permission control ensures the security and confidentiality of information, while also facilitating collaboration and division of responsibilities.
[0142] Responsive design principles will be adopted in client design to adapt to different device types, such as desktops, tablets, and smartphones. The user interface may use modern UI frameworks such as Material Design to provide intuitive navigation and interactive experience. For example, the rectification plan may be presented in the form of a timeline, allowing users to browse tasks in different time periods through gestures. For complex spatial data, lightweight GIS components such as Leaflet.js or MapboxGL may be integrated to provide interactive map displays. In order to ensure the real-time and consistency of information, the system may adopt push notification mechanisms and data synchronization strategies. For example, the WebSocket protocol is used to establish a persistent connection between the server and the client to achieve real-time updates. Data synchronization may adopt an incremental synchronization strategy to transmit only the changed parts to reduce the amount of data transmission and improve synchronization efficiency. The synchronization algorithm may be based on version vector or operational transformation technology to resolve concurrent editing conflicts.
[0143] Considering the offline working requirements of mobile devices, the client may implement local storage and offline synchronization functions. This can be achieved by using browser database technologies such as IndexedDB or WebSQL. The synchronization of offline data can use a distributed version control idea similar to Git, allowing users to make local modifications while offline, and then synchronize and resolve conflicts with the server when reconnected. In order to improve user acceptance and usage efficiency, the system may integrate intelligent assistant functions. This may include a query interface based on natural language processing (NLP), allowing users to use natural language to retrieve and operate rectification plans. For example, a user can enter "show high priority tasks for the next month", and the system will automatically parse this request and display relevant information. In addition, the system may use machine learning algorithms to analyze user usage patterns and provide personalized suggestions and operation prompts. In terms of system integration, the management client may need to connect with other enterprise systems, such as project management systems, asset management systems, financial systems, etc. This may involve the use of enterprise service buses (ESBs) or API gateways to implement data exchange and function calls between systems. The integration may adopt modern interface technologies such as REST APIs or GraphQL to ensure flexibility and scalability.
[0144] In one embodiment, obtaining user water consumption data uploaded by all user clients and sewage pipe network data collected by all IoT devices, and distributing all user water consumption data and all sewage pipe network data to each edge computing node includes the following steps:
[0145] Obtain the user water usage data uploaded by all user clients and the sewage pipe network data collected by all IoT devices. The user water usage data contains the client location information of the user clients, and the sewage pipe network data contains the device location information of the IoT devices.
[0146] Obtain node location information and current load of all edge computing nodes;
[0147] The target distance between the user client or IoT device and any target edge computing node is calculated by combining the client location information, device location information and node location information. If the target distance is less than or equal to the preset distance threshold, the corresponding user water usage data or sewage pipe network data is distributed to the target edge computing node;
[0148] If the distance between the user client or IoT device and all edge computing nodes is greater than the distance threshold, the corresponding user water use data or sewage network data will be distributed to the idle edge computing nodes based on the current load and in accordance with the dynamic load balancing algorithm.
[0149] In this embodiment, the user client can be an application installed on a smartphone or a home smart water meter, which will upload water consumption, water use time, water pressure and other data regularly or in real time. These data are usually transmitted in a structured format such as JSON or XML through the HTTPS protocol to ensure the security of data transmission. Each piece of data will contain a timestamp and the location information of the client, which may be GPS coordinates (longitude and latitude) or more precise geocoding (such as street address). IoT devices may include sensors installed at key nodes of the sewage network, such as flow meters, water quality analyzers, pressure sensors, etc. These devices usually transmit data through low-power wide area network (LPWAN) technologies such as LoRaWAN or NB-IoT to ensure stable communication in complex underground environments. The data of IoT devices also contains device ID, timestamp, measurement value and device location information. The location information may be pre-configured fixed coordinates or obtained in real time through a GPS module. The frequency of data collection may vary depending on the type of device and monitoring requirements, ranging from once a minute to once an hour. In order to handle possible network interruptions, devices usually have local storage capabilities and can upload data in batches after the network is restored. The entire data collection process needs to consider the integrity, accuracy and timeliness of the data. For example, the system may use data verification algorithms such as CRC (cyclic redundancy check) to ensure that the data has not been tampered with or damaged during transmission. For critical data, redundant transmission or request retransmission mechanisms may be used to ensure data reliability. In addition, the system needs to be able to handle a large number of concurrent data upload requests, which may require the use of load balancers and distributed data receiving servers to ensure the stability and scalability of the system. After the data is received, the system will perform preliminary cleaning and verification of the data, such as checking the data format, removing obvious outliers, and supplementing missing fields. This process may use some statistical methods, such as the z-score rule to identify outliers, or interpolation algorithms such as linear interpolation to estimate missing data. After this step of processing, the data will be temporarily stored in a distributed cache system, such as a Redis cluster, for subsequent rapid processing and distribution. This comprehensive and meticulous data collection and preprocessing process provides a solid foundation for subsequent data analysis and decision-making, ensuring that the entire sewage pipe network management system can operate based on high-quality, real-time data.
[0150] Next, the system needs to obtain the node location information and current load of all edge computing nodes. Edge computing nodes are usually distributed in various areas of the city and can be installed in communication base stations, substations or dedicated edge data centers. Each node has a fixed geographical location, usually expressed in latitude and longitude coordinates. This location information is configured in the system when the node is deployed and stored in a central database. Database systems that support geographic information such as PostGIS can be used. The current load of the node is a dynamically changing indicator that needs to be monitored and updated in real time. The load can include multiple aspects: CPU usage, memory usage, network bandwidth usage, storage space usage, etc. Each edge node runs a monitoring agent that collects these indicators regularly (every second or every few seconds) and reports them to the central management system. The transmission of monitoring data can use lightweight protocols such as MQTT to reduce network overhead.
[0151] In order to handle concurrent reports from a large number of nodes, the central management system can use an architecture similar to a time series database (such as InfluxDB or Prometheus) to efficiently store and query these rapidly changing indicator data. Load data is usually normalized, such as expressing CPU usage as a percentage value from 0 to 100. The system also calculates a comprehensive load indicator, such as using a weighted average method: Comprehensive load = 0.4*CPU usage + 0.3*memory usage + 0.2*network usage + 0.1*storage usage. The weights in this formula can be adjusted according to the actual application scenario. In order to cope with network fluctuations or node failures, the system will set up a heartbeat mechanism. If a node does not report the load within a predetermined time (such as 30 seconds), it will be marked as a possible failure, and thus temporarily excluded from subsequent task allocation. In addition, the system will maintain a load history record for analyzing load trends and predicting future load conditions. This can use time series analysis techniques such as moving average or exponential smoothing. For example, the simple moving average method is used to calculate the average load of the past 5 minutes: average load = (L1+L2+L3+L4+L5) / 5, where L1 to L5 represent the load values of the last 5 minutes. The purpose of obtaining this information is to perform intelligent task allocation and load balancing in the future, to ensure that system resources are used efficiently, and to avoid situations where some nodes are overloaded while other nodes are idle. This real-time, dynamic resource monitoring mechanism provides an important guarantee for the efficient operation of the entire system.
[0152] Then, the system needs to combine the client location information, device location information, and node location information to calculate the target distance between the user client or IoT device and any target edge computing node, and determine the data distribution strategy based on the distance threshold. This process involves a lot of geospatial calculations and decision logic. First, the system needs to select an appropriate distance calculation method. Considering the curvature of the earth's surface, the Haversine formula is usually used to calculate the great circle distance between two points. The system calculates the distance between each data source (user client or IoT device) and each edge node, which can involve a lot of computing operations. To improve efficiency, the system uses spatial indexing techniques, such as R-tree or quadtree, to quickly filter out nodes that can be within the distance threshold, and then only perform accurate distance calculations on these candidate nodes. The setting of the distance threshold needs to consider multiple factors, such as network latency, data transmission cost, and edge node processing power. After calculating the distance, the system compares the distance with the threshold. If the distance is less than or equal to the threshold, the system distributes the corresponding data to the edge node. This process can involve data serialization (such as using ProtocolBuffers) and network transmission (such as using gRPC). To ensure the integrity and security of the data, TLS encryption and digital signatures are used during the transmission process. If there are multiple nodes that meet the distance requirements, the system will further consider the current load of the node and select the node with lower load for distribution. This intelligent distribution strategy based on distance and load can significantly reduce data transmission delays and improve system response speed. It can also better utilize edge computing resources and reduce the pressure on central servers. In addition, this method can also realize local processing of data, which is particularly important for applications that require fast response (such as real-time sewage monitoring and alarm). The effect of the whole process is to achieve the decentralization and localization of data processing, improving the overall efficiency and reliability of the system.
[0153] If the distance between the user client or IoT device and all edge computing nodes is greater than the distance threshold, the system will distribute the corresponding user water consumption data or sewage network data to the idle edge computing nodes based on the current load and in accordance with the dynamic load balancing algorithm. This situation can occur in the edge areas of the system coverage, or when the edge nodes in some areas are temporarily unavailable. The core goal of the dynamic load balancing algorithm is to maximize resource utilization while ensuring system stability. A commonly used method is an improved version of the "minimum number of connections" algorithm, which not only considers the number of connections of the node, but also the processing power and current load status of the node. The algorithm calculates a comprehensive score for each node: score = w1*(1-current load rate)+w2*(maximum number of connections-current number of connections) / maximum number of connections+w3*processing power index, where w1, w2, and w3 are weight factors that can be adjusted according to the specific needs of the system. For example, if the system pays more attention to load balancing, the value of w1 can be increased; if it pays more attention to the processing power of the node, the value of w3 can be increased. The processing power index can be calculated based on factors such as the node's CPU speed, memory size, and network bandwidth. The algorithm selects the node with the highest score as the target of data distribution. In order to avoid all data being concentrated on a few high-performance nodes, the system introduces some randomness, such as using the softmax function to convert the score into a selection probability: P(i)=exp(Si) / Σjexp(Sj), where Si is the score of node i. The roulette wheel selection method is then used to determine the final target node. This method ensures that high-scoring nodes are more likely to be selected, while giving other nodes a certain chance, which helps long-term load balancing.
[0154] In actual operation, the system will recalculate the scores of all nodes regularly (such as every minute) and update the selection probability to adapt to the changing network conditions. In order to improve the robustness of the algorithm, the system can also set up some protection mechanisms. For example, if the load of a node exceeds a preset threshold (such as 90%), even if its comprehensive score is high, it will be temporarily removed from the candidate list until the load is reduced to a safe level. In addition, the system will implement a feedback mechanism to regularly evaluate the effect of load balancing and automatically adjust the algorithm parameters. For example, the standard deviation of the load between nodes can be calculated. If the standard deviation is too large, it means that the load distribution is uneven, and the system will automatically increase the weight of the load factor. The effect of this dynamic load balancing strategy is significant: it can effectively prevent individual nodes from being overloaded and improve the stability and reliability of the entire system; at the same time, it can also improve resource utilization, reduce resource waste, and thus reduce operating costs. For users, this means that even in the case of unsatisfactory network conditions, they can still obtain relatively stable and fast service responses. In the long run, this strategy can also provide a basis for the horizontal expansion of the system, making it easier and smoother to add or remove nodes.
[0155] In one implementation, the user water use data and sewage pipe network data are preliminarily screened according to a preset environmental contribution certification mechanism, and the preliminarily screened user water use data and sewage pipe network data are uploaded to the high-frequency storage chain for temporary storage;
[0156] According to the preset data quality assessment smart contract, the high-frequency storage chain data temporarily stored in the high-frequency storage chain is scored for quality, and the data quality score of each high-frequency storage chain data is obtained;
[0157] The high-frequency storage chain data is secondary screened based on the data quality score, and the comprehensive data of the sewage pipe network is obtained by performing data aggregation and homomorphic encryption on the high-frequency storage chain data after secondary screening;
[0158] Permanently store comprehensive sewage network data in a low-frequency storage chain.
[0159] In this embodiment, first, the system will preliminarily screen the user water use data and sewage network data according to the preset environmental contribution proof mechanism, and upload the preliminarily screened data to the high-frequency storage chain for temporary storage. This environmental contribution proof mechanism is an innovative incentive and verification method that aims to encourage users and devices to provide high-quality and authentic data while screening out possible false or low-quality data. Specifically, this mechanism may evaluate the credibility and value of data based on multiple factors. For user water use data, the system may consider the user's historical data provision records, the rationality of water use patterns, the temporal continuity of data, etc. For example, the system may use statistical methods to detect outliers. If a user's water consumption suddenly exceeds the historical average by three standard deviations, this data may be marked as requiring further verification. At the same time, the system will also consider the user's credit score, which may be based on factors such as the accuracy of the data provided by the user in the past and the frequency of participation in water-saving activities. For sewage network data, the system will check the integrity of the data, the working status of the sensor, whether the timestamp of the data is continuous, etc. Only data that reaches a certain threshold (such as 70 points) will be considered to have passed the preliminary screening. The data that passes the preliminary screening will be packaged and added to the high-frequency storage chain.
[0160] A high-frequency storage chain is a specially designed blockchain structure that is specifically designed to handle high-frequency, large-capacity data storage needs. It may use a data structure similar to a DAG (directed acyclic graph) instead of a traditional linear blockchain to improve concurrent processing capabilities. When data is uploaded to the chain, it is given a unique identifier and metadata such as timestamps and source information are attached. In order to improve storage and retrieval efficiency, data may be organized into a Merkle tree structure, where only the root hash is recorded on the main chain, while specific data is stored in a distributed storage system (such as IPFS). This design not only ensures the integrity and traceability of the data, but also greatly improves the throughput of the system.
[0161] Next, the system will perform a quality score on the high-frequency storage chain data temporarily stored in the high-frequency storage chain according to the preset data quality assessment smart contract, and obtain the data quality score of each high-frequency storage chain data. This process involves complex data analysis and smart contract execution. The data quality assessment smart contract is a pre-written, automatically executed code that runs on the blockchain network. Its main purpose is to conduct a more in-depth and comprehensive quality assessment of the data that has passed the initial screening. This smart contract may consider multiple dimensions to evaluate data quality, including but not limited to: data accuracy, completeness, consistency, timeliness, interpretability, etc. For each dimension, the contract will have specific evaluation criteria and calculation methods. For example, for data accuracy, the contract may check whether the data is within a reasonable range and whether it is consistent with the trend of historical data. It may use the standard deviation or interquartile range to identify outliers. For completeness, the contract will check whether there are missing values, and if so, it will evaluate the impact of these missing values on the overall data quality. The consistency check may involve comparing data from different sensors or users in the same time period to see if there are obvious contradictions. The timeliness assessment takes into account the delay between the generation time of the data and the time of uploading the data. The shorter the delay, the higher the score. Interpretability may involve checking whether the data is in a standardized format and whether the units are consistent.
[0162] The execution of smart contracts is automatic. Once triggered (either periodically or immediately when new data is uploaded to the chain), it will score each piece of data on the chain. The scoring results are attached to the original data as metadata and become part of the data record. An important feature of this process is its transparency and immutability - all scoring rules and results are recorded on the blockchain, and anyone can verify the fairness of the scoring process. In addition, in order to handle large amounts of data, the system may adopt sharding technology to distribute the data to multiple parallel smart contracts for processing, and then aggregate the results. The effects of this data quality assessment mechanism are multifaceted: it provides an objective and unified data quality metric that helps data users quickly identify high-quality data; it can motivate data providers to continuously improve data quality; it provides a reliable quality reference for subsequent data processing and decision-making; in the long run, it will significantly improve the data quality of the entire system and provide more reliable data support for water resource management decisions. Then, the system will perform secondary screening on the high-frequency storage chain data based on the data quality score, and perform data aggregation and homomorphic encryption on the high-frequency storage chain data after secondary screening, and finally obtain comprehensive data of the sewage network.
[0163] The purpose of data aggregation is to transform large amounts of raw data into a more compact and information-dense form. This process may involve a variety of statistical and data mining techniques. For example, for time series data, the system may calculate the average, maximum, minimum, and standard deviation for each hour or day. For spatial data, geographic aggregation may be performed to calculate comprehensive indicators for each region. The system may also use dimensionality reduction techniques such as principal component analysis (PCA) to extract the most important features in the data. During the aggregation process, the system pays special attention to preserving the temporal and spatial distribution characteristics of the data to ensure that the aggregated data still reflects the key patterns and trends of the original data. The aggregated data will greatly reduce the burden of storage and processing while retaining key information. After the data aggregation is completed, the system will perform homomorphic encryption on the aggregated data. Homomorphic encryption is a special encryption technology that allows calculations to be performed directly on encrypted data without first decrypting it. This is very important for protecting sensitive water resources data because it allows data analysis while protecting privacy. The system may use partially homomorphic encryption schemes such as the Paillier encryption system. A key feature of the Paillier system is that it supports additive homomorphism, that is, E(m1) * E(m2) = E(m1 + m2), where E() represents the encryption function and m1 and m2 are plaintext messages. This means that summation operations can be performed in the encrypted domain, which is very useful for calculating statistical indicators such as averages and totals. These encrypted data can be securely stored and transmitted, and the original information cannot be read even if intercepted. It should be noted that homomorphic encryption usually significantly increases the size of the data, so the system needs to find a balance between security and storage efficiency. Possible solutions include using compression algorithms or encrypting only the most sensitive data fields. Through this series of processing, the original high-frequency, large-volume water resource data is transformed into high-quality, information-intensive and secure sewage network comprehensive data. This comprehensive data not only retains the key characteristics of the original data, but also greatly reduces the burden of storage and processing, while ensuring the security of the data through encryption. This lays a solid foundation for subsequent data analysis, decision support and privacy-preserving data sharing, greatly improving the efficiency and security of the entire water resource management system.
[0164] Finally, the system permanently stores the comprehensive data of the sewage pipe network in a low-frequency storage chain. A low-frequency storage chain is a blockchain system specially designed to store important, fully processed and verified data. Unlike high-frequency storage chains, the design of low-frequency storage chains focuses on long-term preservation, integrity assurance and efficient retrieval of data, rather than high-frequency data writing. The block structure of a low-frequency storage chain may be more complex than that of a traditional blockchain. In addition to the data itself, each block may also contain metadata, data summaries, access control information, etc. For example, the structure of a block may be as follows: block header (containing the hash, timestamp, Merkle tree root, etc. of the previous block), data body (containing the actual comprehensive data of the sewage pipe network), metadata (such as data source, processing method, quality score, etc.), access control list (defining who has the right to view or use this data).
[0165] Before the data is written to the low-frequency storage chain, the system will perform a series of preparations. First, the data will be formatted to ensure that it conforms to the predefined data model. This may involve serialization of the data, such as using a binary serialization format such as ProtocolBuffers or Avro to improve storage efficiency and parsing speed. The system then calculates the encrypted hash value of the data (such as using the SHA-256 algorithm), which will be used as the basis for verifying the integrity of the data. If the amount of data is large, a Merkle tree structure may be used, where only the Merkle root is stored in the block, while the specific data shards are stored in a distributed file system (such as IPFS). This approach can greatly reduce the storage pressure of the blockchain itself while still ensuring the integrity of the data.
[0166] During the data writing process, the system uses a consensus algorithm to ensure that all nodes in the network agree on the newly added data. Considering the characteristics of the low-frequency storage chain, an improved Proof of Stake algorithm may be used here. For example, a weight system based on data quality and contribution can be introduced to allow nodes that provide high-quality data for a long time to have a higher chance of being selected as block generators. This can not only improve consensus efficiency, but also further motivate participants to provide high-quality data. Once a new block is confirmed and added to the chain, the system broadcasts this information and all nodes update their local copies. In order to ensure the long-term availability of data, the low-frequency storage chain may implement a special data replication and recovery mechanism. For example, using the Erasure Coding technology, the data is divided into multiple fragments and stored in a distributed manner among different nodes in the network. In this way, even if some nodes are offline or the data is damaged, the complete data can still be recovered from other nodes. Specifically, if the (10,4) erasure coding scheme is used, the original data will be divided into 10 copies, and as long as any 4 of them exist, the complete data can be reconstructed. This greatly improves the fault tolerance and availability of the data. In addition, low-frequency storage chains may also implement regular data integrity checks. The system may perform a full scan of the stored data on a regular basis (e.g., monthly) to verify its integrity and accessibility. If any problems are found, the system automatically triggers the data repair process to rebuild the damaged data from other copies.
[0167] To cope with long-term technological evolution, the low-frequency storage chain may include a data migration mechanism. Over time, if more advanced storage or encryption technologies emerge, the system can smoothly migrate data to a new format or structure without losing any information or compromising the integrity of the data. This may involve complex version control and compatibility management. In terms of data access, the low-frequency storage chain will implement fine-grained access control. Each piece of data will have a corresponding access control list that defines the access rights of different roles (such as administrators, researchers, the public, etc.). The system may use attribute-based encryption (ABE) to achieve this function. Using ABE, data can be encrypted based on user attributes (such as position, department, security level, etc.), and only users with corresponding attributes can decrypt and access the data. This approach not only ensures data security, but also provides flexible access control. To improve the efficiency of data retrieval, the low-frequency storage chain may implement an advanced indexing mechanism. For example, an inverted index can be used to support fast full-text search, or a spatial index (such as R-tree) can be used to support efficient geospatial queries. These index structures may be stored off-chain, but their root hashes are recorded on-chain to ensure integrity. In addition, the system may implement an intelligent caching mechanism that caches part of the data in fast storage media based on the access frequency and importance of the data to increase the access speed of frequently used data. An important feature of the low-frequency storage chain is that it supports the long-term evolution of data.
[0168] Over time, the stored data may need to be supplemented or corrected. The system handles these updates through special transaction types to ensure that all changes are accurately recorded and traceable. For example, if an error is found in a piece of historical data, the system will not modify the original data directly, but add a new transaction to mark the error and provide the correct value. In this way, all data change histories are fully preserved, which is helpful for subsequent audits and analysis. In terms of performance optimization, low-frequency storage chains may use sharding technology to improve scalability. Data may be allocated to different shards based on geographic location, time period, or other dimensions. Each shard can process transactions and store data independently, greatly improving the throughput of the entire system. At the same time, the system will implement a cross-shard consistency protocol to ensure that a complete and consistent view of the data can be obtained when needed.
[0169] In one embodiment, a GIS spatial analysis module in a GIS system is used to extract spatial features of pipe network data in pipe network deployment data and environmental spatial data, and temporal features of pipe network data are extracted from comprehensive sewage pipe network data;
[0170] Combine the spatial characteristics of the pipe network data and the temporal characteristics of the pipe network data to determine multiple comprehensive environmental impact indicators of the sewage pipe network;
[0171] The principal component analysis method is used to conduct a dimensionality reduction comprehensive analysis of the sewage network comprehensive data, network deployment data and environmental space data to generate a single environmental impact index for the sewage network;
[0172] Construct a multi-dimensional dynamic GIS model of the sewage network based on multiple comprehensive environmental impact indicators and environmental impact indexes;
[0173] The sewage treatment rectification analysis model is used to analyze and predict the multi-dimensional dynamic GIS model, and a sewage treatment rectification plan for the sewage network is generated based on the analysis and prediction results.
[0174] In this embodiment, first, multiple comprehensive environmental impact indicators of the sewage pipe network are determined by combining the spatial characteristics and time series characteristics of the pipe network data. Specifically, it is necessary to consider the spatial distribution characteristics of the pipe network, such as pipe network density, coverage, topological structure, etc., as well as time series characteristics, such as flow changes, water quality fluctuations, and pipe network aging. By analyzing these characteristics, a series of comprehensive indicators reflecting the environmental impact of the sewage pipe network can be constructed. For example, a "pipe network load index" can be defined, which comprehensively considers the spatial distribution and time variation characteristics of the pipe network: NLI=Σ(Li*Fi*Ti) / A, where Li represents the length of the i-th pipe network, Fi represents the average flow of the pipe network, Ti represents the use time of the pipe network, and A represents the area of the entire area. This index reflects the load of the pipe network per unit area, taking into account the spatial distribution and use intensity of the pipe network. Another possible indicator is the "water quality impact index": WQI=Σ(Ci*Vi) / V, where Ci represents the concentration of a certain pollutant, Vi represents the volume of the water body where the pollutant is located, and V represents the total water body volume. This index reflects the degree of impact of sewage on the surrounding water body. In addition, multiple indicators such as "pipeline aging index" and "leakage risk index" can be defined, which together constitute a multi-dimensional environmental impact assessment system. Through this method, complex spatial and temporal data can be converted into a series of quantifiable and comparable comprehensive indicators, providing a basis for subsequent analysis and decision-making.
[0175] Next, the principal component analysis method is used to perform dimensionality reduction and comprehensive analysis on the comprehensive data of sewage pipe network, pipe network deployment data and environmental space data to generate a single environmental impact index of the sewage pipe network. Principal component analysis (PCA) is a commonly used dimensionality reduction technique that can convert multiple potentially related variables into a small number of unrelated variables (i.e., principal components). In this step, the multiple environmental impact comprehensive indicators obtained in the previous step, as well as the pipe network deployment data (such as pipe network materials, pipe diameters, burial depths, etc.) and environmental space data (such as topography, soil type, groundwater level, etc.) are first combined into a high-dimensional data matrix X. Assuming that there are n samples (for example, n regions or n time points), each sample has p features, then X is an n×p matrix. Next, X is centered and standardized to obtain the matrix Z. Then the covariance matrix C=(1 / n)Z^TZ of Z is calculated. Perform eigenvalue decomposition on C to obtain eigenvalues λ1≥λ2≥...≥λp and corresponding eigenvectors u1,u2,...,up. Select the first k eigenvalues so that their sum accounts for more than a preset threshold (e.g. 85%) of the total eigenvalue sum. These k eigenvectors form the basis of the principal component space. Finally, the original data can be projected into this k-dimensional space to obtain the reduced-dimensional data Y=ZU, where U is a p×k matrix consisting of the first k eigenvectors. Each column of Y is a principal component, and the weighted sum of these k principal components can be used as a single environmental impact index: EII=Σ(wi*PCi), where wi is the weight of the i-th principal component (which can be expressed as the proportion of variance explained by the principal component), and PCi is the value of the i-th principal component.
[0176] This environmental impact index integrates most of the information in the original data and can be used as a single indicator to assess the environmental impact of the sewage network. The effect of implementing this step is to significantly reduce the dimensionality of the data while retaining the main information in the original data. This not only simplifies the subsequent analysis process, but also reveals the main change patterns and potential correlation structures in the data. Through principal component analysis, it is possible to identify which factors are most important for the environmental impact of the sewage network, which is of great significance for the formulation of targeted management strategies. In addition, the resulting single environmental impact index provides an intuitive comparison standard for the environmental impact of sewage networks in different regions or at different times.
[0177] Based on multiple comprehensive environmental impact indicators and environmental impact indexes, a multi-dimensional dynamic GIS model of the sewage network can be constructed. This model organically combines spatial information, temporal information and attribute information, providing a comprehensive sewage network environmental impact visualization and analysis cloud platform. Specifically, it is necessary to first establish a basic GIS database, which contains information such as the spatial layout of the sewage network, network attributes (such as pipe diameter, material, age, etc.), and spatial distribution of environmental factors (such as topography, soil, hydrology, etc.). Then, the comprehensive environmental impact indicators and single environmental impact index obtained in the previous steps are associated with this basic GIS data. For example, a corresponding indicator value can be assigned to each pipe network segment, or an indicator value can be assigned to each grid unit in a gridded space. In order to reflect the dynamic characteristics, the time dimension needs to be introduced into the model. For example, the indicator values at different time points can be stored, allowing users to view and analyze data at different time scales. In terms of technical implementation, spatiotemporal databases (such as PostgreSQL with PostGIS and TimescaleDB extensions) can be used to store and manage these multidimensional data.
[0178] In order to achieve efficient data retrieval and analysis, a multidimensional index structure can be established, such as a combination of R-tree (for spatial index) and B-tree (for temporal index). In terms of visualization, WebGL technology can be used to achieve real-time rendering of large-scale three-dimensional geographic data. For example, color coding can be used to represent the environmental impact index of different regions, height can be used to represent a specific environmental impact index, and animation effects can be used to show the changes of these indicators over time. In addition, interactive query and analysis functions can be implemented, such as allowing users to select specific areas or time periods for detailed analysis, or to perform spatial and temporal trend analysis. For example, users can select an area of interest by drawing a polygon, and then the system will automatically calculate the average environmental impact index in the area and generate a time series graph to show the trend of the index over time. By implementing this step, a powerful, intuitive and easy-to-use multidimensional dynamic GIS model is obtained. This model can not only intuitively display the environmental impact of the sewage network, but also support complex spatial analysis and time series analysis. It provides decision makers with a tool to fully understand the environmental impact of the sewage network, which helps to identify problem areas, predict future trends, and evaluate the effectiveness of management measures. This visual and interactive analysis method greatly improves the interpretability and usability of data, making complex environmental impact assessment results understandable and usable by a wider user group.
[0179] Finally, the multi-dimensional dynamic GIS model is analyzed and predicted through the sewage treatment rectification analysis model, and the sewage treatment rectification plan of the sewage network is generated based on the analysis and prediction results. This step involves complex data analysis, predictive modeling and decision optimization processes. First, a sewage treatment rectification analysis model needs to be built. This model can predict the potential effects of different rectification measures on environmental impact indicators based on machine learning algorithms.
[0180] Based on these prediction results, a multi-objective optimization problem can be constructed, and the objective function includes factors such as the degree of improvement of environmental impact, cost, and time. The optimization problem can be expressed as: minF(A)=w1*E(A)+w2*C(A)+w3*T(A), where E(A), C(A) and T(A) represent the environmental impact, cost and time brought by the rectification action A, respectively, and w1, w2 and w3 are the corresponding weights. By solving this optimization problem, an optimal rectification plan that balances environmental benefits and economic costs can be obtained. In actual operation, genetic algorithms or particle swarm optimization algorithms can be used to solve this complex optimization problem. After obtaining the optimal plan, it needs to be converted into a specific sewage treatment rectification plan. This plan should include detailed information such as specific rectification measures, implementation schedule, and required resources. For example, the plan may include specific action items such as "replace 500 meters of old pipe network in area A within the next 6 months, with a budget of X million yuan." In order to ensure the feasibility of the plan, various practical constraints such as budget constraints, technical feasibility, and regulatory requirements need to be considered. The resulting rectification plan should be a comprehensive, detailed, and actionable document that provides clear guidance for the transformation and management of the sewage network. The implementation effect of this step is to transform the complex data analysis results into a specific action plan. In this way, not only can the potential effects of different rectification measures be predicted, but also the optimal rectification strategy can be found to balance environmental benefits and economic costs. This data- and model-based decision-making method greatly improves the scientific nature and effectiveness of the rectification plan, and helps to achieve the sustainable management of the sewage network and environmental protection goals. At the same time, the flexibility of this method also allows decision makers to quickly adjust and formulate rectification plans according to different scenarios and priorities, improving the responsiveness and adaptability of decision-making.
[0181] In one embodiment, the pipeline network deployment data and environmental spatial data are combined and a Monte Carlo simulation method is used to generate a variety of different future sewage environment change scenarios;
[0182] After importing a variety of future sewage environment change scenarios into the multi-dimensional dynamic GIS model, the LSTM network model in the sewage treatment rectification analysis model is used to make short-term predictions on the multi-dimensional dynamic GIS model to obtain the short-term sewage change prediction results.
[0183] The GNN network model in the sewage treatment rectification analysis model is used to make long-term predictions on the multi-dimensional dynamic GIS model, and the long-term change prediction results of sewage are obtained;
[0184] Construct a rectification plan decision model based on Markov decision process;
[0185] The short-term sewage change prediction results and the long-term sewage change prediction results are input into the rectification plan decision model to generate a sewage treatment rectification plan for the sewage network.
[0186] In this embodiment, first, the Monte Carlo simulation method is used to generate a variety of different future sewage environment change scenarios in combination with the pipeline network deployment data and environmental spatial data. The core of this step is to explore the future situation through random simulation, so as to provide diversified inputs for subsequent predictions and decisions. In specific implementation, it is necessary to first determine the key influencing factors, such as population growth rate, industrial development speed, climate change trend, etc. For each factor, a probability distribution model is established. For example, the population growth rate follows a normal distribution N(μ, σ^2), where μ is the historical average growth rate and σ is the historical fluctuation. Next, a random number generator is used to sample from these distributions to generate a large number (usually thousands to tens of thousands) of scenarios. Each scenario is a set of specific parameter combinations that represents a future development path. For example, a specific scenario contains the following parameters: an annual population growth rate of 2.1%, an annual industrial water use increase of 3.5%, an average annual rainfall decrease of 1.2%, etc. Then, these parameters are input into the sewage system model, which can be a hydrological model based on physical principles or a statistical learning model. The model output is the state of the sewage system under this scenario, such as pipeline network load, water quality indicators, etc. By repeating this process thousands of times, a set of future scenarios was obtained. The advantage of this method is that it can capture the uncertainty and complexity of the future. And it can obtain a comprehensive, probability-based prediction set of future sewage environmental changes. This prediction set not only includes the most likely situation, but also includes various extreme situations and rare events, making subsequent analysis and decision-making more comprehensive and robust. At the same time, this probability-based method also helps to quantify and manage future uncertainties, providing a basis for risk assessment and emergency plan formulation.
[0187] Next, after importing a variety of future sewage environment change scenarios into the multi-dimensional dynamic GIS model, the LSTM (Long Short-Term Memory) network model in the sewage treatment rectification analysis model is used to perform short-term predictions on the multi-dimensional dynamic GIS model to obtain the short-term sewage change prediction results. LSTM is a special recurrent neural network (RNN) that is particularly suitable for processing and predicting time series data. In this step, the spatiotemporal data in the GIS model must first be converted into sequence data that can be processed by LSTM. For example, the state of each spatial unit (such as a grid or pipe section) at different time points can be represented as a time series. Specifically, assuming that there are n spatial units, each unit has m features (such as water quality indicators, flow, etc.), and the observation time length is T, then the input data shape will be (n, T, m). The architecture of the LSTM model usually includes an input layer, one or more LSTM layers, and an output layer. Each LSTM unit contains three "gate" structures: input gate, forget gate, and output gate, which enables LSTM to selectively remember or forget information, thereby capturing long-term dependencies.
[0188] The mathematical expression of the model can be simplified as: h_t = LSTM(x_t, h_{t-1}, c_{t-1}), where x_t is the input at time t, h_t is the hidden state at time t, and c_t is the unit state at time t. During the training process, the model is trained using historical data and the short-term future scenarios generated in the previous steps. The training goal is to minimize the error between the predicted value and the actual value, and the mean square error (MSE) is usually used as the loss function. After the training is completed, the model can be used to predict the state of the sewage system in the short term in the future (such as the next few days to weeks). When predicting, the current state and the predicted future scenario parameters are input, and the model will output the predicted future state. In order to improve the robustness of the prediction, an ensemble learning method can be used to train multiple LSTM models, and then take the average or weighted average of their prediction results as the final prediction result. The implementation effect of this method is to obtain a prediction model that can accurately capture short-term dynamic changes. In this way, the state prediction of each spatial unit in the short term in the future can be obtained, including water quality indicators, flow, treatment load, etc.
[0189] Then, the multi-dimensional dynamic GIS model is predicted long-term through the GNN (Graph Neural Network) network model in the sewage treatment rectification analysis model to obtain the long-term change prediction results of sewage. GNN is a neural network model specially designed to process graph structure data, which is very suitable for modeling systems with complex spatial relationships such as sewage pipe networks. In this step, the sewage pipe network needs to be represented as a graph structure. Specifically, the nodes in the pipe network (such as pumping stations, treatment plants, key intersections, etc.) can be represented as vertices of the graph, and the pipes connecting the nodes can be represented as edges of the graph. Each vertex and edge can have multiple attributes, such as the treatment index and water quality index of the node, the flow rate and pipe diameter of the edge, etc. Formally, the graph can be represented as G=(V,E), where V is the vertex set and E is the edge set. Each vertex v∈V has a feature vector x_v, and each edge e∈E has a feature vector x_e. The core idea of the GNN model is to update the representation of the node through a message passing mechanism. At each layer, each node collects information about its neighboring nodes and then updates its own state. This process can be expressed as: h_v^(k+1)=UPDATE(h_v^k,AGGREGATE({h_u^k:u∈N(v)})), where h_v^k is the representation of node v at the kth layer, N(v) is the set of v’s neighbors, AGGREGATE is an aggregation function (such as sum or average), and UPDATE is an update function (usually a neural network).
[0190] By stacking multiple layers of such operations, the model can capture a wider range of spatial dependencies. During the training process, the model is trained using historical data and long-term future scenarios generated in the previous steps. The training goal is to minimize the error between the predicted value and the actual value, and the mean square error (MSE) or other suitable loss functions can be used. After the training is completed, the model can be used to predict the state of the sewage system in the long term in the future (such as the next few months to years). When predicting, the current state and the predicted future scenario parameters are input, and the model will output the predicted future state. In order to deal with the problem of cumulative error in long-term prediction, a multi-step prediction strategy can be adopted, that is, the recent state is predicted first, and then this prediction result is used as input to predict the more distant future. The implementation effect of this method is to obtain a prediction model that can accurately capture the long-term spatial-temporal dynamic changes. The advantage of the GNN model is that it can naturally model complex spatial relationships. For example, the model will learn how the development of the upstream area affects the long-term load changes of the entire pipe network system. In this way, the state prediction of the entire pipe network system in the long term in the future can be obtained, including the load, water quality, maintenance requirements, etc. of each node and pipe section. These long-term prediction results provide important support for the formulation of long-term development strategies and the planning of infrastructure investment.
[0191] Next, a rectification plan decision model is constructed based on the Markov Decision Process (MDP). MDP is a mathematical framework used to model the process of how the system state changes over time in the case of decision making, and is particularly suitable for dealing with uncertainty and long-term planning problems. In this step, the five basic elements of MDP need to be defined first: state space S, action space A, transfer function P, reward function R, and discount factor γ. The state space S includes all states of the sewage pipe network system, which can be represented by the output of the GIS model and the prediction model in the previous step, such as the water quality indicators of each node, the load of the pipe network, etc. The action space A includes all rectification actions, such as replacing pipes, increasing treatment capacity, and implementing rainwater and sewage separation. The transfer function P(s'|s,a) represents the probability that the system transfers to state s' after taking action a in state s. This probability can be obtained from the previous short-term and long-term prediction models. The reward function R(s,a) defines the immediate reward for taking action a in state s, which can be defined based on factors such as the degree of environmental improvement and cost. For example, R(s,a)=w1*ΔE-w2*C(a), where ΔE is the degree of environmental improvement, C(a) is the cost of action a, and w1 and w2 are weights. The discount factor γ∈[0,1] is used to balance immediate rewards and long-term benefits. With these definitions, the goal is to find an optimal policy π* that maximizes the expected cumulative reward obtained by following the policy starting from any starting state s. Mathematically expressed as: V*(s)=max_πE[Σ_tγ^tR(s_t,π(s_t))|s_0=s].
[0192] To solve this optimization problem, dynamic programming methods such as value iteration or policy iteration can be used. However, considering the complexity of the problem and the huge dimensionality of the state space, it is computationally infeasible to use these methods directly. Therefore, methods from reinforcement learning such as Q-learning or policy gradient methods can be used. For example, when using a deep Q-network (DQN), a neural network can be used to approximate the action-value function Q(s,a). The input of the network is the state s, and the output is the Q value of each action a. During training, experience replay and target networks are used to improve the learning process. After training, the action with the highest Q value is selected as the optimal action given the state s. The effect of this method is to obtain a decision model that can give the optimal corrective action under different states. The model is able to balance short-term and long-term benefits while taking into account the costs and environmental benefits of the corrective actions. For example, the model will recommend that pipeline network updates be prioritized in some areas, while rectification can be temporarily postponed in other areas. The advantage of this MDP-based decision model is that it can handle the dynamics and uncertainty of the system and provide a long-term optimal rectification strategy, rather than just a short-sighted immediate decision.
[0193] Finally, the short-term and long-term sewage change prediction results are input into the rectification plan decision model to generate a sewage treatment rectification plan for the sewage network. This step integrates all the previous analysis and model results to form a specific and actionable rectification plan. In the specific implementation, it is first necessary to convert the short-term and long-term prediction results into state representations in the MDP model. For example, the predicted water quality indicators, flow, processing load, etc. can be mapped to a discrete state space. For short-term predictions, more attention will be paid to real-time state transfer and rewards; for long-term predictions, more consideration will be given to the long-term evolution trend of the state. Next, the trained MDP decision model is used to generate a rectification plan.
[0194] This process can be described as a series of decision steps: 1) observe the current state s_t; 2) use the decision model to select the optimal actiona_t=argmax_aQ(s_t,a); 3) execute actiona_t, observe the new state s_{t+1} and reward r_t; 4) update the state, t=t+1, and then return to step 1). Repeat this process until the planned time horizon is reached or the system state reaches the predetermined goal. In this process, it is necessary to balance the short-term and long-term forecast results. For example, a weighted objective function can be used to balance the objective function values based on short-term and long-term forecasts. This ensures that the rectification plan can both respond to short-term challenges and prepare for long-term development. The generated rectification plan should include the following key parts: 1) a specific list of rectification measures, such as which pipe sections need to be replaced and which treatment facilities need to be upgraded; 2) an implementation schedule, detailing the start and end time of each measure; 3) a resource allocation plan, including the allocation of funds, manpower and equipment; 4) an expected effect evaluation, including quantitative predictions of water quality improvement, system efficiency improvement, etc.; 5) a risk assessment and emergency plan to deal with emergencies.
[0195] In practice, this process needs to be run multiple times with different parameter settings and initial conditions to generate multiple rectification plan options. These options can then be evaluated and compared through Multi-Criteria Decision Analysis (MCDA) to select the best or most appropriate option. MCDA can consider multiple criteria, such as cost-effectiveness ratio, environmental impact, social acceptance, etc. For example, a weighted sum model can be used to calculate the comprehensive score of each option: Score = Σ_iw_i*c_i, where w_i is the weight of the i-th criterion and c_i is the score of the option on the i-th criterion. The weights can be determined through expert consultation or Analytic Hierarchy Process (AHP). The result of this approach is a comprehensive, balanced, and actionable sewage network rectification plan. This plan not only takes into account the dynamics and uncertainties of the system, but also balances short-term needs and long-term goals. For example, the plan would recommend replacing certain severely aged pipe sections in the first year as a priority, while launching a five-year treatment plant upgrade project. The plan also includes phased goals, such as reducing overflow events in a certain area by 50% within three years and achieving specific water quality standards for the entire city within ten years. The advantages of this rectification plan that integrates multiple advanced models and methods are: 1) it can adapt to complex dynamic systems and take into account various future scenarios; 2) it provides a long-term strategic framework while also providing specific short-term action guidelines; 3) it is data- and model-driven, which improves the scientificity and reliability of decision-making; 4) it considers multiple goals and constraints and can strike a balance between environmental benefits, economic costs and social impacts. The rectification plan generated by this method can not only guide the implementation of specific projects, but also provide strong support for policy making, budget allocation and public communication. For example, a phased investment strategy can be formulated based on this plan, or it can be used to illustrate the necessity and benefits of certain seemingly costly measures in the long run. At the same time, this plan is dynamic and can be adjusted and optimized based on actual implementation and new data input to ensure that the rectification process can continue to adapt to changing circumstances and needs.
[0196] The present invention also discloses a sewage treatment operation device based on the Internet of Things and GIS, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the sewage treatment operation method based on the Internet of Things and GIS as described in any one of the above embodiments is implemented.
[0197] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.
[0198] Among them, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the computer device, and the memory can also be a combination of an internal storage unit and an external storage device of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.
[0199] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.
[0200] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.
Claims
1. A sewage treatment operation cloud platform based on the Internet of Things and GIS, characterized by: The cloud platform includes a data collection layer, a data storage layer, a data analysis layer and an application service layer. The data collection layer is deployed with multiple edge computing nodes, and the multiple edge computing nodes are connected through a hybrid network architecture communication; the data storage layer is configured with a high-frequency storage chain and a low-frequency storage chain, and the data storage layer is also configured with a cross-chain communication module for data interaction between the high-frequency storage chain and the low-frequency storage chain; the data analysis layer is deployed with a GIS system and a sewage treatment rectification analysis model, and the GIS system pre-stores the sewage pipe network deployment data and the environmental space data of the coverage area of the sewage pipe network; The data collection layer is used to obtain the user water use data uploaded by all user clients and the sewage pipe network data collected by all IoT devices, and distribute all the user water use data and all the sewage pipe network data to each edge computing node. The edge computing node is used to pre-process all the user water use data and all the sewage pipe network data. All the IoT devices are pre-deployed at each sewage treatment process node of the sewage pipe network. All the user clients are held by all water users within the coverage of the sewage pipe network. The data storage layer is used to temporarily store the user water use data and the sewage pipe network data preprocessed by the edge computing node in the high-frequency storage chain, and to evaluate, screen and aggregate all the data in the high-frequency storage chain to obtain the comprehensive data of the sewage pipe network, and to permanently store the comprehensive data of the sewage pipe network in the low-frequency storage chain; The data analysis layer is used to extract the comprehensive data of the sewage pipe network in the low-frequency storage chain, and input the comprehensive data of the sewage pipe network into the GIS system, and use the GIS system to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental space data to build a multi-dimensional dynamic GIS model, analyze and predict the multi-dimensional dynamic GIS model through the sewage treatment rectification analysis model, and generate a sewage treatment rectification plan for the sewage pipe network according to the analysis and prediction results; The application service layer is used to obtain all the sewage treatment and rectification plans output by the data analysis layer, and distribute all the sewage treatment and rectification plans to the management client held by the sewage pipe network manager.
2. The sewage treatment operation cloud platform based on Internet of Things and GIS according to claim 1 is characterized in that: The data collection layer includes a data collection module and a data distribution module. The data collection module is used to obtain user water usage data uploaded by all user clients and sewage pipe network data collected by all IoT devices. The user water usage data contains the client location information of the user clients, and the sewage pipe network data contains the device location information of the IoT devices. The data distribution module is used to receive the user water consumption data and the sewage network data, and obtain the node location information of all the edge computing nodes, and calculate the target distance between the user client or the Internet of Things device and any target edge computing node in combination with the client location information, the device location information and the node location information. If the target distance is less than or equal to a preset distance threshold, the corresponding user water consumption data or the sewage network data is distributed to the target edge computing node.
3. The sewage treatment operation cloud platform based on Internet of Things and GIS according to claim 1 is characterized in that: The data storage layer further includes a first storage module, a storage data processing module, and a second storage module. The first storage module is configured with a high-frequency storage chain, and the second storage module is configured with a low-frequency storage chain. The first storage module is used to preliminarily screen the user water use data and the sewage pipe network data according to a preset environmental contribution certification mechanism, and upload the preliminarily screened user water use data and the sewage pipe network data to the high-frequency storage chain for temporary storage. The storage data processing module is used to perform a quality score on the high-frequency storage chain data temporarily stored in the high-frequency storage chain according to a preset data quality assessment smart contract, obtain a data quality score for each piece of the high-frequency storage chain data, perform a secondary screening on the high-frequency storage chain data based on the data quality score, and perform data aggregation and homomorphic encryption processing on the high-frequency storage chain data after the secondary screening to obtain the comprehensive data of the sewage pipe network; The second storage module is used to permanently store the comprehensive data of the sewage network in the low-frequency storage chain.
4. The sewage treatment operation cloud platform based on Internet of Things and GIS according to claim 1 is characterized in that: The hybrid network architecture includes 5G network, LoRaWAN network and satellite communication network.
5. A sewage treatment operation method based on the Internet of Things and GIS, applied to the sewage treatment operation cloud platform based on the Internet of Things and GIS according to any one of claims 1 to 4, characterized in that: The method comprises the following steps: Obtain user water usage data uploaded by all user clients and sewage pipe network data collected by all IoT devices, and distribute all the user water usage data and all the sewage pipe network data to each edge computing node; Preprocessing all the user water use data and all the sewage pipe network data through the edge computing node; Temporarily storing the user water consumption data and the sewage pipe network data preprocessed by the edge computing node in a high-frequency storage chain, and evaluating, screening and aggregating all the data in the high-frequency storage chain to obtain comprehensive sewage pipe network data, and permanently storing the comprehensive sewage pipe network data in a low-frequency storage chain; Extracting the comprehensive data of the sewage pipe network in the low-frequency storage chain, and inputting the comprehensive data of the sewage pipe network into the GIS system; The GIS system is used to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental space data to build a multi-dimensional dynamic GIS model, and the multi-dimensional dynamic GIS model is analyzed and predicted by the sewage treatment rectification analysis model, and a sewage treatment rectification plan of the sewage pipe network is generated according to the analysis and prediction results; All the sewage treatment rectification plans are distributed to the management client held by the sewage network manager.
6. The sewage treatment operation method based on the Internet of Things and GIS according to claim 5 is characterized in that: The step of obtaining the user water usage data uploaded by all user clients and the sewage pipe network data collected by all IoT devices, and distributing all the user water usage data and all the sewage pipe network data to each edge computing node includes the following steps: Acquire user water usage data uploaded by all user clients and sewage pipe network data collected by all IoT devices, wherein the user water usage data includes client location information of the user clients, and the sewage pipe network data includes device location information of the IoT devices; Obtaining node location information and current load of all edge computing nodes; Calculate the target distance between the user client or the IoT device and any target edge computing node in combination with the client location information, the device location information and the node location information; if the target distance is less than or equal to a preset distance threshold, distribute the corresponding user water usage data or the sewage pipe network data to the target edge computing node; If the distance between the user client or the IoT device and all edge computing nodes is greater than the distance threshold, the corresponding user water consumption data or sewage network data will be distributed to the idle edge computing nodes based on the current load and in accordance with the dynamic load balancing algorithm.
7. The sewage treatment operation method based on the Internet of Things and GIS according to claim 5 is characterized in that: The step of temporarily storing the user water use data and the sewage pipe network data preprocessed by the edge computing node in a high-frequency storage chain, evaluating, screening and aggregating all the data in the high-frequency storage chain to obtain comprehensive sewage pipe network data, and permanently storing the comprehensive sewage pipe network data in a low-frequency storage chain includes the following steps: Preliminarily screening the user water use data and the sewage pipe network data according to a preset environmental contribution certification mechanism, and uploading the user water use data and the sewage pipe network data after the preliminary screening to the high-frequency storage chain for temporary storage; Performing a quality score on the high-frequency storage chain data temporarily stored in the high-frequency storage chain according to a preset data quality assessment smart contract to obtain a data quality score for each piece of the high-frequency storage chain data; Performing secondary screening on the high-frequency storage chain data based on the data quality score, and performing data aggregation and homomorphic encryption processing on the high-frequency storage chain data after the secondary screening to obtain the comprehensive data of the sewage pipe network; The comprehensive data of the sewage pipe network is permanently stored in the low-frequency storage chain.
8. The sewage treatment operation method based on the Internet of Things and GIS according to claim 5 is characterized in that: The method of using the GIS system to combine the comprehensive data of the sewage pipe network, the pipe network deployment data and the environmental space data to build a multi-dimensional dynamic GIS model, analyzing and predicting the multi-dimensional dynamic GIS model through a sewage treatment rectification analysis model, and generating a sewage treatment rectification plan for the sewage pipe network according to the analysis and prediction results includes the following steps: Utilizing the GIS spatial analysis module in the GIS system to extract the spatial features of the pipe network data in the pipe network deployment data and the environmental spatial data, and extracting the temporal features of the pipe network data from the sewage pipe network comprehensive data; Determine multiple comprehensive environmental impact indicators of the sewage pipe network by combining the spatial characteristics of the pipe network data and the temporal characteristics of the pipe network data; Performing a dimensionality reduction comprehensive analysis on the sewage pipe network comprehensive data, the pipe network deployment data and the environmental space data by principal component analysis to generate a single environmental impact index of the sewage pipe network; Constructing a multi-dimensional dynamic GIS model of the sewage pipe network based on the plurality of comprehensive environmental impact indicators and the environmental impact index; The multi-dimensional dynamic GIS model is analyzed and predicted through the sewage treatment rectification analysis model, and a sewage treatment rectification plan for the sewage network is generated according to the analysis and prediction results.
9. The sewage treatment operation method based on the Internet of Things and GIS according to claim 8 is characterized in that: The method of analyzing and predicting the multi-dimensional dynamic GIS model through the sewage treatment rectification analysis model and generating a sewage treatment rectification plan for the sewage pipe network according to the analysis and prediction results includes the following steps: Combining the pipe network deployment data and the environmental space data and using a Monte Carlo simulation method to generate a variety of different future sewage environment change scenarios; After importing a variety of future sewage environment change scenarios into the multi-dimensional dynamic GIS model, the multi-dimensional dynamic GIS model is subjected to short-term prediction through the LSTM network model in the sewage treatment rectification analysis model to obtain the sewage short-term change prediction result; The multi-dimensional dynamic GIS model is used to make long-term predictions through the GNN network model in the sewage treatment rectification analysis model to obtain the long-term change prediction results of sewage; Construct a rectification plan decision model based on Markov decision process; The short-term sewage change prediction results and the long-term sewage change prediction results are input into the rectification plan decision model to generate a sewage treatment rectification plan for the sewage network.
10. A sewage treatment operation device based on the Internet of Things and GIS, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the sewage treatment operation method based on the Internet of Things and GIS as described in any one of claims 5 to 9.
Citation Information
Patent Citations
Water conservancy data operation system and method based on block chain and fog operation
CN113239325A
Flue gas treatment intelligent operation optimization control system and method
CN114307569A