Intelligent sewage operation management system based on Kubernetes

Abstract

The invention discloses an intelligent sewage operation management system based on Kubernetes, relates to the technical field of sewage treatment and management, and can effectively improve the intelligent and efficient management level of sewage treatment. The intelligent sewage operation management system based on Kubernetes comprises a data acquisition module, a big data storage and management module, an intelligent prediction and resource scheduling module, a fault detection and early warning module, a visual display and user interface module and a system integration and elastic expansion module, and through integration of Kubernetes, an Internet of Things sensor, edge computing and big data processing technologies, the intelligent sewage operation management system based on Kubernetes is realized. By combining a machine learning algorithm and a computer vision technology, automatic monitoring, intelligent prediction, resource scheduling optimization, fault detection and early warning of the sewage treatment system and the elastic expansion capability of the system are realized, so that the efficiency, safety and intelligent level of sewage treatment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sewage treatment and management, and in particular to a Kubernetes-based intelligent sewage operation and management system. Background Art

[0002] With the acceleration of urbanization and increasingly stringent environmental protection requirements, sewage treatment has become an important part of modern urban and rural infrastructure. However, existing technologies mainly use fixed-parameter monitoring equipment and traditional control algorithms, which lack intelligent analysis and processing of real-time data, and are difficult to deal with emergencies, equipment failures, and fluctuations in processing demand, resulting in low resource utilization and limited operating efficiency. In other words, traditional sewage treatment systems still face many challenges in operation and management, which are specifically manifested in the following aspects: Data silos and lack of real-time performance: Data collection in traditional sewage treatment systems is usually completed by multiple independent subsystems, such as water quality monitoring, flow measurement, equipment status monitoring, etc. However, these subsystems often lack an effective data sharing mechanism, resulting in serious data silos and difficulty in achieving global real-time monitoring. In addition, the frequency of data collection and transmission is limited, making it difficult to reflect the operating status of the sewage treatment system in a timely manner, thus affecting the decision-making efficiency of managers. Dependence on manual monitoring and delayed response to faults: In traditional sewage treatment facilities, the status monitoring of pipe networks and equipment mainly relies on manual inspections, which is labor-intensive and inefficient. At the same time, manual inspections can often only detect faults that have already occurred, and lack the ability to predict potential risks in advance. After a fault occurs, the lack of an automated early warning and response mechanism often leads to the expansion of the problem and even causes serious environmental pollution incidents. Poor system scalability and waste of resources: With the increasing demand for sewage treatment, the traditional system has significant deficiencies in scalability; fixed hardware and software resource configurations make it difficult to flexibly respond to fluctuations in sewage treatment volume, resulting in low resource utilization and even system overload during peak periods; in addition, the complexity and cost of system maintenance and upgrades also restrict the long-term stable operation of sewage treatment equipment; Insufficient data analysis and decision support: Traditional sewage treatment systems usually lack effective means of analyzing large amounts of historical data, making it difficult to provide valuable decision support through data mining and predictive models. This makes it impossible for managers to make timely and accurate response strategies when faced with complex changes in sewage treatment demand, resulting in improper resource scheduling and low treatment efficiency. In addition, although some existing sewage treatment systems have adopted the Internet of Things (IoT) technology and monitored water quality parameters and equipment status through sensors, most systems are limited to data collection and simple feedback, and lack effective prediction and scheduling mechanisms. With the increase in sewage treatment demand and environmental complexity, such systems are prone to data lag, monitoring blind spots, and the inability to dynamically adjust treatment plans, especially when processing large amounts of data, there are problems with data transmission delays and insufficient processing capacity. Moreover, fault detection in traditional systems often relies on equipment alarms and regular maintenance, and potential equipment failures cannot be discovered in a timely manner, which in turn affects the stability and safety of the entire sewage treatment process. At the same time, with the continuous increase in sewage treatment demand, traditional systems have significant deficiencies in scalability, and fixed hardware and software resource configurations make it difficult to flexibly respond to fluctuations in sewage treatment volume, resulting in low resource utilization and even system overload during peak periods. The complexity and cost of system maintenance and upgrades also restrict the long-term stable operation of sewage treatment facilities. Summary of the invention

[0003] The purpose of the present invention is to provide a Kubernetes-based intelligent sewage operation and management system to solve the technical problems of poor real-time performance, insufficient intelligent prediction and scheduling capabilities, delayed fault detection, and poor system scalability in existing sewage treatment systems. It can realize real-time monitoring, data analysis, fault prediction and resource scheduling of sewage treatment equipment, thereby effectively improving the intelligent and efficient management level of sewage treatment.

[0004] In order to achieve the above object, the present invention adopts the following technical scheme: A Kubernetes-based intelligent sewage operation and management system, comprising: a data acquisition module: the data acquisition module comprises: an Internet of Things sensor, installed near a sewage pool, an inlet and outlet pipe network and a treatment device, for real-time collection of water quality parameters, flow and equipment status parameters, the Internet of Things sensor comprising a pH sensor, a dissolved oxygen sensor, a turbidity sensor, a temperature sensor, and a flow meter; a camera, installed inside and outside a sewage treatment site, for capturing real-time video and image data of equipment operation status, pipe network conditions and water body conditions; an edge computing device, connected to the Internet of Things sensor and the camera, for preliminary processing of sensor data and video image data; A big data storage and management module, which includes: a data storage unit, which is divided into structured data storage and unstructured data storage; a data processing unit, which uses the Apache Spark processing engine to perform real-time stream processing and batch processing analysis on massive data, and supports complex statistical analysis and trend prediction; an ETL service container is used to extract, transform and load data, and the processed data is stored in a distributed database or data lake to ensure data consistency and availability; a Kubernetes cluster, which is used to manage the containerized service of the entire data storage system, and the Kubernetes cluster can provide automated resource scheduling, load balancing and fault recovery functions to ensure elastic expansion and high reliability of data storage; Intelligent prediction and resource scheduling module, which includes: an intelligent prediction unit, which is deployed in the Kubernetes cluster and is responsible for predicting future water quality changes, sewage treatment needs and equipment failure risks; a resource scheduling engine, which automatically generates a resource scheduling plan based on the prediction results, optimizes resource allocation in the sewage treatment process, and monitors the system execution in real time, dynamically adjusts the scheduling strategy, and ensures the efficient operation of the sewage treatment process; A fault detection and early warning module, which includes: a fault detection unit, a service container running in the Kubernetes cluster, which uses the YOLO deep learning model to process video and image data, and automatically identifies equipment failures, pipe cracks, and abnormalities in pollutant detection in the sewage treatment system; an early warning unit, which is integrated in the Kubernetes platform, processes alarm data and triggers early warnings; Visual display and user interface module, the visual display and user interface module includes: intelligent prediction and scheduling visualization interface, which displays the intelligent prediction results and resource scheduling in the sewage treatment process, including future water quality change trends, sewage treatment demand prediction, and equipment failure risk assessment; fault detection and early warning visualization interface, which integrates the real-time detection results of the fault detection module. The abnormal recognition data based on the YOLO model will be intuitively displayed in the form of images and videos, such as equipment failure, pipeline damage, and pollutant exceeding the standard. Combined with the early warning unit, the user can view the alarm history and the specific conditions for the early warning trigger in the visualization interface, and can quickly respond and confirm the authenticity of the alarm through the interface; GIS geographic information system and BIM three-dimensional model display, GIS service is used to display the geographical location, pipe network layout, equipment distribution and real-time data of key nodes of the entire sewage treatment system. Combined with the results of the intelligent prediction unit and the fault detection unit, the user can view the sewage treatment demand forecast, equipment health status and potential fault location of a specific area in the GIS interface. The BIM three-dimensional model container provides a three-dimensional display of the treatment facilities. The user can deeply understand the real-time status, scheduling results and fault conditions of each device through the three-dimensional interactive interface; System integration and elastic expansion module, which includes: Kubernetes cluster management module, which is responsible for the deployment, expansion and monitoring of all containerized services in the system, and Kubernetes provides automated operation and maintenance functions; service grid, which is used to manage the communication between microservices, provide traffic management, fault injection, monitoring and security control functions, and ensure efficient collaboration and security between services in the system; cross-regional multi-cluster management platform, which supports cross-regional multi-cluster deployment, realizes global management and scheduling through the central console, and ensures efficient operation of the system and optimal configuration of resources; The data generated by the data acquisition module are stored in structured and unstructured data storage, respectively. The Kubernetes cluster monitors and manages the data storage system to ensure efficient storage and access of data when the load fluctuates. The prediction results of the intelligent prediction unit are transmitted to the resource scheduling engine to generate and execute the resource scheduling plan, and the actual scheduling effect is fed back to the prediction model to continuously and dynamically optimize the scheduling strategy. The data preprocessed by the edge computing device is transmitted to the fault detection unit, which analyzes the image data through the YOLO model to identify abnormal conditions of equipment failure or pipeline damage, and processes the alarm data through the early warning unit to trigger an early warning. The water quality change trend, sewage treatment demand and equipment failure risk generated by the intelligent prediction unit are visualized The interface is presented intuitively and provides an optimization basis for the resource scheduling module. The scheduling execution effect is also fed back in real time in the interface. The fault detection unit uses a deep learning model to detect abnormal equipment conditions, and the early warning module triggers an alarm based on the detection results. This information is synchronized to the visualization interface for users to view and manage in real time. Finally, users monitor the system operation through the visualization platform and dynamically adjust the scheduling strategy to form a closed-loop optimization management. The Kubernetes cluster is responsible for the containerized deployment and operation of all services in the system, providing elastic expansion and high availability. The service grid ensures efficient communication and collaboration between modules, optimizes system performance, and the cross-regional multi-cluster management platform manages clusters distributed in different regions through a unified interface to achieve optimal scheduling and unified management of global resources.

[0005] In actual application, the IoT sensor formulates different data collection frequencies according to the production cycle of the pollution discharge unit as follows: if the production cycle is within 8 hours, sampling is performed once every hour; if the production cycle is greater than 8 hours, sampling is performed once every two hours.

[0006] The edge computing device first performs denoising and then compression processing on the video and image collected by the camera; Specifically, the denoising process uses the improved BM3D algorithm based on deep learning. This algorithm combines similar local structures in videos and images to form a three-dimensional data set, and then performs filtering processing in this three-dimensional data set to effectively remove noise. In terms of compression, the new generation of video coding standard H.265 / HEVC is used, which has higher compression efficiency and can greatly reduce the amount of data at the same video quality, thereby significantly reducing transmission bandwidth requirements and storage costs. The BM3D algorithm includes the following steps: data segmentation: segmenting the collected real-time data into small local data sets; 3D transformation: performing 3D transformation on these small blocks to extract similar local structures; denoising: applying filtering algorithms to the 3D data set to remove noise; During data compression processing, the H.265 compression algorithm is executed on the edge computing device, which specifically includes the following steps: frame segmentation: divide the video frame into small rectangular blocks, which are the basic units of compression; predictive coding: encode the differences between adjacent frames through intra-frame and inter-frame prediction. Inter-frame prediction uses the similarity between the previous and next frames and only encodes the changed parts, thereby reducing redundant data; transformation and quantization: transform and quantize the predicted residual, transform the data in the spatial domain to the frequency domain, and then quantize the high-frequency part of the data to further reduce the amount of data; entropy coding: entropy encode the quantized coefficients to effectively compress the data.

[0007] Furthermore, for data storage, a distributed database cluster is used to store structured data such as water quality parameters, flow rate, and equipment status, and a distributed file system HDFS is used to store unstructured data such as videos, images, and log files.

[0008] Furthermore, for data processing, the Apache Spark engine is used to perform stream processing on the received real-time data to analyze the changing trends of pH value and turbidity; wherein, the Apache Spark engine uses the following steps: data access: real-time data such as pH value and turbidity are accessed to the Apache Spark Streaming module through the Kafka message queue; data processing: cleaning, filtering, conversion and other operations are performed on the incoming real-time data; combined with historical data, statistical analysis and data aggregation batch analysis are performed.

[0009] Furthermore, the intelligent prediction unit uses a long short-term memory network to train based on historical water quality data and predict future water quality trends; uses an Adaptive Random Forest model to predict changing sewage treatment needs; uses a decision tree to train based on historical equipment operation data and predict the risk of equipment failure; Among them, the LSTM model includes the following steps: data preparation: organize historical data into time series data, including past pH and turbidity records; model training: use the training data set to train the LSTM model to enable it to learn the time series features in the data; trend prediction: input the latest real-time data to let the model predict future water quality changes; at the same time, based on the prediction results, the resource scheduling engine automatically generates a resource scheduling plan.

[0010] Furthermore, the early warning unit: in order to prevent false alarms caused by individual dirty data, formulates the following early warning triggering mechanism: the same type of alarm data will trigger an alarm when there are more than two abnormal data in a production cycle.

[0011] Furthermore, during fault detection and early warning, the YOLO model is used to analyze the real-time video images collected by the edge computing device to detect the operating status of the sewage treatment equipment; the YOLO model includes the following steps: image input: the processed image data is passed into the YOLO model; target detection: the model analyzes the image data in real time to identify equipment operation abnormalities, pipe cracks and other problems; alarm generation: once a fault is detected, the system will generate alarm data.

[0012] Compared with the prior art, the Kubernetes-based smart sewage operation and management system described in the present invention has the following advantages: The Kubernetes-based smart sewage operation and management system provided by the present invention improves the real-time monitoring, data processing real-time and efficiency, intelligent fault detection and early warning, elastic expansion and resource optimization management capabilities of the sewage treatment system by integrating Kubernetes, Internet of Things sensors, edge computing, big data processing technology, computer vision technology (YOLO model) and intelligent prediction and decision-making, and combining machine learning algorithms (long short-term memory network LSTM, Adaptive Random Forest); compared with the existing technology, the Kubernetes-based smart sewage operation and management system provided by the present invention can realize real-time integration and processing of multi-source data, automatically identify and warn equipment failures, and can intelligently predict and schedule and optimize sewage treatment processes; in addition, the system adopts a containerized architecture, has high integration and easy maintainability, and supports unified management across regions, which improves the safety, efficiency and intelligence level of the sewage treatment system as a whole. In other words, the Kubernetes-based smart sewage operation and management system provided by the present invention can achieve full-process automation and intelligent management from data collection, storage, processing, to intelligent analysis and prediction, and then to resource scheduling and fault warning through the collaborative work between modules; among them, the application of Kubernetes technology has significantly improved the system's elastic scalability, resource utilization and operation and maintenance efficiency, enabling the system to adapt to the dynamic changes in sewage treatment needs and ensure an efficient and stable sewage treatment process; at the same time, the integration of computer vision and big data technology gives the system unprecedented intelligent capabilities in fault detection, warning and decision support, significantly improving the safety and management efficiency of sewage treatment. DETAILED DESCRIPTION

[0013] The embodiment of the present invention provides a Kubernetes-based intelligent sewage operation and management system, including: a data acquisition module: the data acquisition module includes: an Internet of Things sensor installed near a sewage pool, an inlet and outlet pipe network, and a treatment device, for real-time collection of water quality parameters, flow rates, and equipment status parameters; a camera installed inside and outside a sewage treatment site, for capturing real-time video and image data of equipment operation status, pipe network conditions, and water body conditions; an edge computing device connected to the Internet of Things sensor and the camera, for preliminary processing of sensor data and video image data; Big data storage and management module, which includes: data storage unit, which is divided into structured data storage and unstructured data storage; data processing unit, which uses Apache Spark processing engine to perform real-time stream processing and batch processing analysis on massive data, and supports complex statistical analysis and trend prediction; uses ETL service container to extract, transform and load data, and stores the processed data in distributed database or data lake to ensure data consistency and availability; Kubernetes cluster, which is used to manage the containerized service of the entire data storage system, and Kubernetes cluster can provide automated resource scheduling, load balancing and fault recovery functions to ensure elastic expansion and high reliability of data storage; Intelligent prediction and resource scheduling module, which includes: intelligent prediction unit, deployed in Kubernetes cluster, responsible for predicting future water quality changes, sewage treatment needs and equipment failure risks; resource scheduling engine, which automatically generates resource scheduling plans based on prediction results, optimizes resource allocation in the sewage treatment process (for example: agent delivery, equipment start and stop, pipe network pressure regulation), and monitors system execution in real time, dynamically adjusts scheduling strategies, and ensures efficient operation of the sewage treatment process; Fault detection and early warning module,The fault detection and early warning module includes: fault detection unit, a service container running in the Kubernetes cluster,,using the YOLO deep learning model to process video and image data,,automatically identifying equipment failures, pipe cracks, and abnormalities in pollutant detection in the sewage treatment system;,early warning unit, integrated in the Kubernetes platform,,processes alarm data, and triggers early warnings; Visualization display and user interface module. The visualization display and user interface module includes: intelligent prediction and scheduling visualization interface, which displays the intelligent prediction results and resource scheduling in the sewage treatment process, including future water quality change trends, sewage treatment demand prediction, and equipment failure risk assessment; fault detection and early warning visualization interface, which integrates the real-time detection results of the fault detection module. The abnormal identification data based on the YOLO model will be intuitively displayed in the form of images and videos, such as equipment failure, pipeline damage, and pollutant exceeding the standard. Combined with the early warning unit, users can view the alarm history and specific conditions for early warning triggering in the visualization interface, and can quickly respond and confirm the authenticity of the alarm through the interface; GIS geographic information system and BIM three-dimensional model display. GIS services are used to display the geographical location, pipe network layout, equipment distribution, and real-time data of key nodes of the entire sewage treatment system. Combined with the results of the intelligent prediction unit and the fault detection unit, users can view the sewage treatment demand forecast, equipment health status, and potential fault locations of specific areas in the GIS interface. The BIM three-dimensional model container provides a three-dimensional display of the treatment facilities. Users can gain an in-depth understanding of the real-time status, scheduling results, and fault conditions of each device through the three-dimensional interactive interface; System integration and elastic expansion module, which includes: Kubernetes cluster management module, which is responsible for the deployment, expansion and monitoring of all containerized services in the system, and Kubernetes provides automated operation and maintenance functions; service grid, which is used to manage the communication between microservices, provide traffic management, fault injection, monitoring and security control functions, and ensure efficient collaboration and security between services in the system; cross-regional multi-cluster management platform, which supports cross-regional multi-cluster deployment, realizes global management and scheduling through the central console, and ensures efficient operation of the system and optimal configuration of resources; The data generated by the data acquisition module are stored in structured and unstructured data storage respectively. The Kubernetes cluster monitors and manages the data storage system to ensure efficient storage and access of data when the load fluctuates. The prediction results of the intelligent prediction unit are passed to the resource scheduling engine to generate and execute the resource scheduling plan, and the actual scheduling effect is fed back to the prediction model to continuously and dynamically optimize the scheduling strategy. The data pre-processed by the edge computing device is transmitted to the fault detection unit, which analyzes the image data through the YOLO model to identify abnormal conditions of equipment failure or pipeline damage, and processes the alarm data through the early warning unit to trigger an early warning. The water quality change trend, sewage treatment demand and equipment failure risk generated by the intelligent prediction unit are directly displayed through the visual interface. The system is presented visually and provides an optimization basis for the resource scheduling module. The scheduling execution effect is also fed back in real time in the interface. The fault detection unit uses a deep learning model to detect abnormal equipment conditions, and the early warning module triggers an alarm based on the detection results. This information is synchronized to the visualization interface for users to view and manage in real time. Finally, users monitor the system operation through the visualization platform and dynamically adjust the scheduling strategy to form a closed-loop optimization management. The Kubernetes cluster is responsible for the containerized deployment and operation of all services in the system, providing elastic expansion and high availability. The service grid ensures efficient communication and collaboration between modules, optimizes system performance, and the cross-regional multi-cluster management platform manages clusters distributed in different regions through a unified interface to achieve optimal scheduling and unified management of global resources.

[0014] Compared with the prior art, the Kubernetes-based smart sewage operation and management system described in the embodiment of the present invention has the following advantages: In the Kubernetes-based smart sewage operation and management system provided in the embodiment of the present invention, by integrating Kubernetes, Internet of Things sensors, edge computing, big data processing technology, computer vision technology (YOLO model) and intelligent prediction and decision-making, and combining machine learning algorithms (long short-term memory network LSTM, Adaptive Random Forest), the real-time monitoring, data processing real-time and efficiency, intelligent fault detection and warning, elastic expansion and resource optimization management capabilities of the sewage treatment system are improved; compared with the existing technology, the Kubernetes-based smart sewage operation and management system provided in the embodiment of the present invention can realize real-time integration and processing of multi-source data, automatically identify and warn equipment failures, and can intelligently predict and schedule and optimize sewage treatment processes; in addition, the system adopts a containerized architecture, has high integration and easy maintainability, and supports unified management across regions, which improves the safety, efficiency and intelligence level of the sewage treatment system as a whole. In other words, the Kubernetes-based smart sewage operation and management system provided in the embodiment of the present invention can achieve full-process automation and intelligent management from data collection, storage, processing, to intelligent analysis and prediction, and then to resource scheduling and fault warning through the collaborative work between modules; among them, the application of Kubernetes technology has significantly improved the system's elastic scalability, resource utilization and operation and maintenance efficiency, enabling the system to adapt to the dynamic changes in sewage treatment needs and ensure an efficient and stable sewage treatment process; at the same time, the integration of computer vision and big data technology gives the system unprecedented intelligent capabilities in fault detection, warning and decision support, significantly improving the safety and management efficiency of sewage treatment.

[0015] In actual application, the IoT sensors in the above data acquisition module formulate different data collection frequencies according to the production cycle of the pollution discharge unit as follows: if the production cycle is within 8 hours, sampling is performed once every hour; if the production cycle is greater than 8 hours, sampling is performed once every two hours.

[0016] Furthermore, the edge computing device in the above-mentioned data acquisition module is connected to the IoT sensor and camera for preliminary processing of sensor data and video image data; the video and image collected by the camera are first denoised and then compressed; specifically, the denoising link adopts the improved BM3D (Block-Matching and 3D filtering) algorithm based on deep learning, which combines similar local structures in the video and image to form a three-dimensional data set, and then performs filtering in this three-dimensional data set to effectively remove noise; in terms of compression, the new generation of video coding standard H.265 / HEVC (High Efficiency Video Coding) is used. Compared with the previous generation H.264, H.265 provides higher compression efficiency and can greatly reduce the amount of data under the same video quality, thereby significantly reducing the transmission bandwidth requirements and storage costs; The BM3D algorithm steps are as follows: (1) Data segmentation: divide the collected real-time data into blocks to form small local data sets; (2) Three-dimensional transformation: perform three-dimensional transformation on these small blocks to extract similar local structures; (3) Denoising: apply filtering algorithms to the three-dimensional data set to remove noise; Data compression: The H.265 compression algorithm is executed on the edge computing device. The specific steps are as follows: (1) Frame segmentation: divide the video frame into small rectangular blocks (macroblocks), which are the basic units of compression; (2) Prediction coding: encode the differences between adjacent frames through intra-frame and inter-frame prediction. Inter-frame prediction uses the similarity between the previous and next frames and only encodes the changed parts, thereby reducing redundant data; (3) Transformation and quantization: transform and quantize the predicted residual, transform the data in the spatial domain to the frequency domain, and then quantize the high-frequency part of the data to further reduce the amount of data; (4) Entropy coding: entropy coding (CABAC, Context-adaptive binaryarithmetic coding) is performed on the quantized coefficients to effectively compress the data.

[0017] In other words, the Kubernetes-based smart sewage operation and management system provided by the embodiment of the present invention has enhanced real-time data collection and processing capabilities. Through the collaborative work of IoT sensors and edge computing devices, it significantly improves the data collection efficiency and real-time performance of the sewage treatment system; the edge computing devices perform local denoising, compression and preliminary processing of video, image and water quality data, reducing transmission delays and bandwidth requirements, and ensuring that data can be processed and fed back in a timely and efficient manner.

[0018] Furthermore, the data storage in the above-mentioned big data storage and management module uses a distributed database cluster to store structured data such as water quality parameters, flow, and equipment status, and uses a distributed file system HDFS to store unstructured data such as videos, images, and log files.

[0019] Furthermore, the data processing in the above-mentioned big data storage and management uses the Apache Spark processing engine to perform real-time stream processing on massive data and analyze the changing trends of pH value and turbidity. The use of the Apache Spark engine includes the following steps: (1) Data access: real-time data such as pH value and turbidity are accessed to the Apache Spark Streaming module through the Kafka message queue; (2) Data processing: cleaning, filtering, conversion and other operations are performed on the incoming real-time data; combined with historical data, statistical analysis and data aggregation batch processing analysis are performed.

[0020] Furthermore, in the above-mentioned intelligent prediction and resource scheduling module, the long short-term memory network (LSTM) is used for training based on historical water quality data and predicting the trend of future water quality changes; the Adaptive Random Forest (aRF) model can adapt to data changes in real time and process a large amount of data streams, which is suitable for predicting the changing needs of sewage treatment; the decision tree is used for training based on the historical operation data of the equipment (such as sensor data, log files, etc.) and predicting the risk of equipment failure; The LSTM model includes the following steps: (1) Data preparation: Organize historical data into time series data, including past pH and turbidity records; (2) Model training: Use the training data set to train the LSTM model so that it can learn the time series features in the data; (3) Trend prediction: Input the latest real-time data to let the model predict future water quality changes; At the same time, based on the prediction results, the resource scheduling engine automatically generates a resource scheduling plan.

[0021] In other words, the Kubernetes-based smart sewage operation and management system provided by the embodiment of the present invention has improved intelligent prediction and optimization scheduling capabilities. It adopts deep learning algorithms such as long short-term memory network (LSTM) and Adaptive Random Forest (aRF), which can predict sewage treatment needs, water quality changes and equipment failure risks in real time, and accurately provide intelligent resource scheduling suggestions for the system; therefore, it effectively optimizes resource allocation such as drug delivery and equipment start and stop, greatly improves the efficiency and stability of sewage treatment, and reduces resource waste.

[0022] Furthermore, in the above fault detection and early warning module, in order to prevent false alarms caused by individual dirty data, the following early warning trigger mechanism is formulated: the same type of alarm data will trigger an alarm when there are more than 2 abnormal data in a production cycle; During fault detection and early warning, the YOLO model is used to analyze the real-time video images collected by the edge computing device to detect the operating status of the sewage treatment equipment. The YOLO model includes the following steps: (1) Image input: the processed image data is passed into the YOLO model; (2) Target detection: the model analyzes the image data in real time to identify problems such as equipment operation abnormalities and pipeline cracks; (3) Alarm generation: once a fault is detected, the system will generate alarm data.

[0023] In other words, the Kubernetes-based smart sewage operation and management system provided in the embodiment of the present invention has an automated fault detection and early warning mechanism. Through the application of the YOLO deep learning model, the system can automatically analyze real-time video and image data, and accurately identify equipment failures, pipe cracks and other abnormal conditions. Combined with the intelligent early warning module, the system can issue an early warning in time before a fault occurs, thereby avoiding the expansion of the accident and reducing system downtime and maintenance costs.

[0024] In addition, the Kubernetes-based smart sewage operation and management system provided in the embodiment of the present invention has improved visual monitoring and user experience. It combines the GIS geographic information system and the BIM three-dimensional model to provide an intuitive visual display interface, allowing users to monitor water quality changes, equipment status, fault conditions, etc. in real time, intuitively analyze the system operation status and effectively manage it; therefore, it effectively improves the user's decision-making efficiency and facilitates timely adjustment of strategies.

[0025] In addition, the Kubernetes-based smart sewage operation and management system provided by the embodiment of the present invention has enhanced system scalability and elasticity. It adopts Kubernetes cluster management and service grid technology to achieve elastic expansion and high-availability management of the sewage treatment system; regardless of load fluctuations or changes in processing requirements, the system can dynamically adjust resource allocation and ensure stable and efficient operation; compared with traditional systems, it has stronger scalability and adaptability.

[0026] Therefore, in the Kubernetes-based smart sewage operation and management system provided by the embodiment of the present invention, the collaborative work of various modules realizes the full process automation and intelligent management from data collection, storage, processing, to intelligent analysis, prediction, and then to resource scheduling and fault warning; the system integrates Internet of Things sensors and edge computing devices to ensure efficient data collection and preprocessing; uses Apache Spark big data processing technology to ensure high availability of data storage and processing; the intelligent prediction and resource scheduling module is based on LSTM and Adaptive Random Forest models to analyze water quality changes and equipment operating status in real time and optimize resource allocation; the fault detection and warning module performs real-time image analysis through the YOLO deep learning model to detect equipment failures and provide timely warnings; the visualization display module intuitively presents the prediction results, scheduling execution effects and fault information through GIS, BIM three-dimensional models, etc., to help users monitor and optimize system operation in real time; Kubernetes technology improves the elastic scalability and resource utilization of the system.

[0027] In summary, the Kubernetes-based smart sewage operation and management system provided by the embodiment of the present invention has the following advantages: 1. Enhanced real-time data collection and processing capabilities: Through the collaborative work of IoT sensors and edge computing devices, the data collection efficiency and real-time performance of the sewage treatment system have been significantly improved; edge computing devices perform local denoising, compression and preliminary processing of video, image and water quality data, reducing transmission delays and bandwidth requirements, ensuring that data can be processed and fed back in a timely and efficient manner; 2. Improved intelligent prediction and optimization scheduling capabilities: This invention uses deep learning algorithms such as long short-term memory network (LSTM) and Adaptive Random Forest (aRF), which can predict sewage treatment demand, water quality changes and equipment failure risks in real time, and accurately provide intelligent resource scheduling suggestions for the system; this effectively optimizes resource allocation such as drug delivery and equipment start-up and shutdown, greatly improving the efficiency and stability of sewage treatment and reducing resource waste; 3. Automated fault detection and early warning mechanism: Through the application of the YOLO deep learning model, the system can automatically analyze real-time video and image data to accurately identify equipment failures, pipeline cracks and other abnormal conditions; combined with the intelligent early warning module, the system can issue early warnings before a failure occurs, avoiding the expansion of accidents and reducing system downtime and maintenance costs; 4. Enhanced system scalability and elasticity: By adopting Kubernetes cluster management and service grid technology, the present invention realizes elastic expansion and high-availability management of the sewage treatment system; regardless of load fluctuations or changes in processing requirements, the system can dynamically adjust resource allocation and ensure stable and efficient operation; compared with traditional systems, the present invention has stronger scalability and adaptability; 5. Visual monitoring and improved user experience: Combining GIS geographic information system and BIM three-dimensional model, the present invention provides an intuitive visual display interface, which enables users to monitor water quality changes, equipment status, fault conditions, etc. in real time, intuitively analyze the system operation status and effectively manage it. This improves the user's decision-making efficiency and facilitates timely adjustment of strategies. Specific embodiments

[0028] Data collection: The system installs a variety of sensors and high-definition cameras at key nodes of the water pipe network. These devices collect water quality parameters, flow and equipment operating status in real time. The data is pre-processed by edge computing devices and uploaded to the big data storage and management module managed by the Kubernetes cluster. Specifically, pH sensors and turbidity sensors are installed in the sewage treatment pool to monitor the pH value and turbidity of the sewage in real time. In addition, the data collection frequency is set according to the production cycle. Assuming that the production cycle is less than 8 hours, data is collected once an hour. At the same time, the sensors are connected to the edge computing devices to transmit the collected pH and turbidity data in real time. The BM3D algorithm based on deep learning is used to denoise the data collected by the sensors to eliminate environmental interference factors and improve the accuracy of the data. Among them, the BM3D algorithm steps are as follows: (1) Data segmentation: divide the collected real-time data into blocks to form small local data sets; (2) Three-dimensional transformation: perform three-dimensional transformation on these small blocks to extract similar local structures; (3) Denoising: apply filtering algorithms to the three-dimensional data set to remove noise; Data compression: The H.265 compression algorithm is executed on the edge computing device. The specific steps are as follows: (1) Frame segmentation: divide the video frame into small rectangular blocks (macroblocks), which are the basic units of compression; (2) Prediction coding: encode the differences between adjacent frames through intra-frame and inter-frame prediction. Inter-frame prediction uses the similarity between the previous and next frames and only encodes the changed parts, thereby reducing redundant data; (3) Transformation and quantization: transform and quantize the predicted residual, transform the data in the spatial domain to the frequency domain, and then quantize the high-frequency part of the data to further reduce the amount of data; (4) Entropy coding: entropy coding (CABAC, Context-adaptive binaryarithmetic coding) is performed on the quantized coefficients to effectively compress the data; Big data storage and management: The edge computing device transmits the processed pH and turbidity data to the big data storage and management platform through the wireless communication module; the big data platform uses a distributed database to store structured data (pH, turbidity) and unstructured data (images, videos); the Apache Spark engine is used to stream the received real-time data and analyze the changing trends of pH and turbidity; the Apache Spark engine is used in the following steps: (1) Data access: real-time data such as pH and turbidity are accessed to the Apache Spark Streaming module through the Kafka message queue; (2) Data processing: cleaning, filtering, conversion and other operations are performed on the incoming real-time data; combined with historical data, statistical analysis and data aggregation batch analysis are performed; Intelligent prediction and scheduling: Based on the collected historical data and real-time data, the system predicts future water quality changes and water demand through machine learning models. The elastic expansion mechanism under Kubernetes management ensures that the training and inference services of the machine learning model can be automatically expanded as needed. The prediction results are used to optimize water resource scheduling and prevent possible equipment failures in advance. Specifically, the LSTM (Long Short-Term Memory Network) model is used to train historical water quality data to predict future pH and turbidity trends. The LSTM model steps are as follows: (1) Data preparation: organize historical data into time series data, including past pH and turbidity records; (2) Model training: use the training data set to train the LSTM model so that it learns the time series features in the data; (3) Trend prediction: input the latest real-time data to let the model predict future water quality changes; at the same time, based on the prediction results, the resource scheduling engine automatically generates a resource scheduling plan; for example, if the predicted pH value drops to a certain critical value, the system will automatically adjust the amount of reagents to neutralize the acidity of the sewage; the scheduling plan is transmitted to the specific treatment equipment through the edge computing device to achieve real-time adjustment; Fault detection and early warning: The YOLO model is used to analyze real-time video images collected by edge computing devices to detect the operating status of sewage treatment equipment. The YOLO model has the following steps: (1) Image input: The processed image data is passed into the YOLO model; (2) Target detection: The model analyzes image data in real time to identify problems such as equipment operation anomalies and pipe cracks; (3) Alarm generation: Once a fault is detected, the system will generate alarm data. Specifically, the early warning trigger mechanism set by the system will trigger an alarm when the same type of alarm data appears more than twice in a production cycle, ensuring that false alarms are effectively avoided. Visual display: The processed data and fault detection results are displayed through a visual interface, including real-time water quality monitoring data, scheduling execution status and alarm information; users can view water quality change trends and equipment status through the interface, and adjust operation strategies based on system feedback; Elastic expansion of the system: When the amount of data increases suddenly, the system will monitor the load of each module in real time through Kubernetes cluster management, and automatically expand the container resources of the corresponding module to maintain normal operation; when the pressure of edge computing devices is detected to increase, the system will automatically deploy additional edge computing containers and dynamically expand the container resources based on the load of data traffic; the increase in data inflow will lead to increased storage and processing requirements, and the system will increase the distributed database cluster nodes through the Kubernetes cluster; at the same time, the number of Apache Spark processing engine containers will be expanded to increase processing capacity, ensuring that big data can be processed in a timely manner for stream processing and batch processing analysis, and support subsequent intelligent prediction and scheduling; due to the substantial increase in data volume, the system prediction module needs to process more historical data and real-time data to make more detailed water quality change predictions and resource scheduling plans; the system elastically expands the resources of the intelligent prediction module, dynamically increases the computing resources for LSTM model training and Adaptive RandomForest (aRF) model calculations, so that the prediction model can process large-scale data and output optimized sewage treatment scheduling plans; the system will expand the computing container of the YOLO model according to the increase in real-time video and image data to improve the real-time detection capability of equipment failures and pipe network anomalies, ensure timely triggering of warnings, and prevent the expansion of failures.

[0029] In the smart sewage operation and management system provided by the embodiment of the present invention, the problem of lag in real-time monitoring and data processing is addressed: by introducing IoT sensors and edge computing devices, the problem of data transmission and processing delay in the existing system is solved, ensuring that key parameters in the sewage treatment process can be collected, analyzed and processed in real time; in addition, a suitable data collection frequency is formulated according to the production cycle of the sewage discharge unit; To address the shortcomings of intelligent prediction and resource scheduling: through deep learning and machine learning algorithms, accurate prediction of water quality changes, sewage treatment needs and equipment status can be achieved, and resource scheduling plans can be optimized accordingly, thereby improving the operating efficiency and treatment effect of the system; To address the problem of untimely fault detection: the present invention uses the computer vision technology YOLO model to analyze real-time video and image data, automatically identifies abnormal situations such as equipment failure and pipeline damage, and issues alarms in a timely manner through an intelligent early warning mechanism to ensure the safety and stability of the sewage treatment process. In order to prevent false alarms caused by individual dirty data, an effective early warning trigger mechanism is formulated; To address the problem of poor system scalability and integration: Through Kubernetes cluster management and service mesh technology, elastic expansion and unified management of system modules are achieved, ensuring that the system can efficiently allocate resources and dynamically expand when dealing with changes in demand, thereby improving the flexibility and maintainability of the system.

[0030] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A smart sewage operation and management system based on Kubernetes, characterized in that: include: A data acquisition module, the data acquisition module includes: an Internet of Things sensor installed near the sewage pool, the import and export pipe network and the treatment equipment, which collects data on water quality parameters, flow and equipment status in real time, and the Internet of Things sensor includes a pH sensor, a dissolved oxygen sensor, a turbidity sensor, a temperature sensor, and a flow meter; a camera installed inside and outside the sewage treatment site, which is used to capture real-time video and image data of equipment operation status, pipe network conditions and water body conditions; an edge computing device connected to the Internet of Things sensor and the camera, which is used to preliminarily process sensor data and video image data; A big data storage and management module, which includes: a data storage unit, which is divided into structured data storage and unstructured data storage; a data processing unit, which uses the Apache Spark processing engine to perform real-time stream processing and batch processing analysis on massive data, and supports complex statistical analysis and trend prediction; an ETL service container is used to extract, transform and load data, and the processed data is stored in a distributed database or data lake to ensure data consistency and availability; a Kubernetes cluster, which is used to manage the containerized service of the entire data storage system, and the Kubernetes cluster can provide automated resource scheduling, load balancing and fault recovery functions to ensure elastic expansion and high reliability of data storage; Intelligent prediction and resource scheduling module, which includes: an intelligent prediction unit, which is deployed in the Kubernetes cluster and is responsible for predicting future water quality changes, sewage treatment needs and equipment failure risks; a resource scheduling engine, which automatically generates a resource scheduling plan based on the prediction results, optimizes resource allocation in the sewage treatment process, and monitors the system execution in real time, dynamically adjusts the scheduling strategy, and ensures the efficient operation of the sewage treatment process; A fault detection and early warning module, which includes: a fault detection unit, a service container running in the Kubernetes cluster, which uses the YOLO deep learning model to process video and image data, and automatically identifies equipment failures, pipe cracks, and abnormalities in pollutant detection in the sewage treatment system; an early warning unit, which is integrated in the Kubernetes platform, processes alarm data and triggers early warnings; Visual display and user interface module, the visual display and user interface module includes: intelligent prediction and scheduling visualization interface, which displays the intelligent prediction results and resource scheduling in the sewage treatment process, including future water quality change trends, sewage treatment demand prediction, and equipment failure risk assessment; fault detection and early warning visualization interface, which integrates the real-time detection results of the fault detection module. The abnormal recognition data based on the YOLO model will be intuitively displayed in the form of images and videos, such as equipment failure, pipeline damage, and pollutant exceeding the standard. Combined with the early warning unit, the user can view the alarm history and the specific conditions for the early warning trigger in the visualization interface, and can quickly respond and confirm the authenticity of the alarm through the interface; GIS geographic information system and BIM three-dimensional model display, GIS service is used to display the geographical location, pipe network layout, equipment distribution and real-time data of key nodes of the entire sewage treatment system. Combined with the results of the intelligent prediction unit and the fault detection unit, the user can view the sewage treatment demand forecast, equipment health status and potential fault location of a specific area in the GIS interface. The BIM three-dimensional model container provides a three-dimensional display of the treatment facilities. The user can deeply understand the real-time status, scheduling results and fault conditions of each device through the three-dimensional interactive interface; System integration and elastic expansion module, which includes: Kubernetes cluster management module, which is responsible for the deployment, expansion and monitoring of all containerized services in the system, and Kubernetes provides automated operation and maintenance functions; service grid, which is used to manage the communication between microservices, provide traffic management, fault injection, monitoring and security control functions, and ensure efficient collaboration and security between services in the system; cross-regional multi-cluster management platform, which supports cross-regional multi-cluster deployment, realizes global management and scheduling through the central console, and ensures efficient operation of the system and optimal configuration of resources; The data generated by the data acquisition module are stored in structured and unstructured data storage, respectively. The Kubernetes cluster monitors and manages the data storage system to ensure efficient storage and access of data when the load fluctuates. The prediction results of the intelligent prediction unit are transmitted to the resource scheduling engine to generate and execute the resource scheduling plan, and the actual scheduling effect is fed back to the prediction model to continuously and dynamically optimize the scheduling strategy. The data preprocessed by the edge computing device is transmitted to the fault detection unit, which analyzes the image data through the YOLO model to identify abnormal conditions of equipment failure or pipeline damage, and processes the alarm data through the early warning unit to trigger an early warning. The water quality change trend, sewage treatment demand and equipment failure risk generated by the intelligent prediction unit are visualized The interface is presented intuitively and provides an optimization basis for the resource scheduling module. The scheduling execution effect is also fed back in real time in the interface. The fault detection unit uses a deep learning model to detect abnormal equipment conditions, and the early warning module triggers an alarm based on the detection results. This information is synchronized to the visualization interface for users to view and manage in real time. Finally, users monitor the system operation through the visualization platform and dynamically adjust the scheduling strategy to form a closed-loop optimization management. The Kubernetes cluster is responsible for the containerized deployment and operation of all services in the system, providing elastic expansion and high availability. The service grid ensures efficient communication and collaboration between modules, optimizes system performance, and the cross-regional multi-cluster management platform manages clusters distributed in different regions through a unified interface to achieve optimal scheduling and unified management of global resources.

2. According to the Kubernetes-based intelligent sewage operation and management system according to claim 1, it is characterized in that: The IoT sensor formulates different data collection frequencies according to the production cycle of the pollution discharge unit as follows: if the production cycle is within 8 hours, sampling is performed once every hour; if the production cycle is greater than 8 hours, sampling is performed once every two hours.

3. According to the Kubernetes-based intelligent sewage operation and management system according to claim 1, it is characterized in that: The edge computing device first performs denoising and then compression processing on the video and image collected by the camera; Specifically, the denoising process uses the improved BM3D algorithm based on deep learning. This algorithm combines similar local structures in videos and images to form a three-dimensional data set, and then performs filtering processing in this three-dimensional data set to effectively remove noise. In terms of compression, the new generation of video coding standard H.265 / HEVC is used, which has higher compression efficiency and can greatly reduce the amount of data at the same video quality, thereby significantly reducing transmission bandwidth requirements and storage costs. BM3D algorithm The method includes the following steps: data segmentation: segmenting the collected real-time data into blocks to form small local data sets; three-dimensional transformation: performing three-dimensional transformation on these small blocks to extract similar local structures therein; Denoising: Apply filtering algorithms to the 3D data set to remove noise; During data compression processing, the H.265 compression algorithm is executed on the edge computing device. The following steps are included: frame segmentation: divide the video frame into small rectangular blocks, which are the basic units of compression; predictive coding: encode the difference between adjacent frames through intra-frame and inter-frame prediction. Inter-frame prediction uses the similarity of the previous and next frames to encode only the changed parts, thereby reducing redundant data; transformation and quantization: transform and quantize the predicted residual, transform the data in the spatial domain to the frequency domain, and then quantize the high-frequency part of the data to further reduce the amount of data; Entropy coding: Entropy coding is performed on the quantized coefficients to effectively compress the data.

4. According to the Kubernetes-based intelligent sewage operation and management system according to claim 1, it is characterized in that: For data storage, a distributed database cluster is used to store structured data such as water quality parameters, flow, and equipment status, and a distributed file system HDFS is used to store unstructured data such as videos, images, and log files.

5. According to the Kubernetes-based intelligent sewage operation and management system according to claim 1, it is characterized in that: Data processing uses the Apache Spark engine to perform stream processing on the received real-time data and analyze the changing trends of pH value and turbidity. The Apache Spark engine uses the following steps: Data access: Real-time data such as pH value and turbidity are accessed to the Apache Spark Streaming module through the Kafka message queue; Data processing: Cleaning, filtering, and conversion of the incoming real-time data; Combined with historical data, statistical analysis and data aggregation batch analysis are performed.

6. The Kubernetes-based intelligent sewage operation and management system according to claim 1 is characterized in that: The intelligent prediction unit uses a long short-term memory network to perform training based on historical water quality data and predict future water quality trends; and uses an Adaptive Random Forest model to predict changing sewage treatment needs; Use decision trees to train based on historical equipment operation data and predict the risk of equipment failure; Among them, the LSTM model includes the following steps: data preparation: organize historical data into time series data, including past pH and turbidity records; model training: use the training data set to train the LSTM model to enable it to learn the time series features in the data; trend prediction: input the latest real-time data to let the model predict future water quality changes; at the same time, based on the prediction results, the resource scheduling engine automatically generates a resource scheduling plan.

7. The Kubernetes-based intelligent sewage operation and management system according to claim 1 is characterized in that: The early warning unit: In order to prevent false alarms caused by individual dirty data, the following early warning trigger mechanism is formulated: the same type of alarm data will trigger an alarm when there are more than two abnormal data in a production cycle.

8. The Kubernetes-based intelligent sewage operation and management system according to claim 1 is characterized in that: During fault detection and early warning, the YOLO model is used to analyze the real-time video images collected by the edge computing device to detect the operating status of the sewage treatment equipment. The YOLO model includes the following steps: Image input: the processed image data is passed into the YOLO model; Target detection: the model analyzes the image data in real time to identify equipment operation abnormalities, pipe cracks and other problems; Alarm generation: once a fault is detected, the system will generate alarm data.