A smart water affair infrastructure health monitoring system based on multi-modal cognitive reasoning

CN122021917BActive Publication Date: 2026-09-22HARBIN INST OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202610191488.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-02-10
Publication Date
2026-09-22
Estimated Expiration
2046-02-10

AI Technical Summary

Technical Problem

[0006]为了克服现有智慧水务监测系统对数据利用浅层、缺乏长周期关联分析、模型可解释性差且严重依赖标注数据的缺陷,本发明提供了一种基于多模态认知推理的智慧水务基础设施健康监测系统,该系统能够提升诊断准确性、可解释性及长期隐患发现能力

Benefits of technology

[0029]1、通过将时序数据转换为可视化图谱并利用大型视觉语言模型的强大零样本/少样本推理能力,显著降低了对大量标注故障数据的依赖,实现了数据高效的高精度诊断。

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122021917B_ABST
    Figure CN122021917B_ABST
Patent Text Reader

Abstract

The application discloses a kind of based on multi-modal cognitive inference's wisdom water infrastructure health monitoring system, the system includes data acquisition and conversion module, multi-modal cognitive inference engine and report generation module, wherein: data acquisition and conversion module is used to collect the multi-source heterogeneous time series data of water infrastructure and is converted into dynamic visualization atlas;When multi-modal cognitive inference engine receives abnormal signal, trigger long cycle correlation analysis mechanism, retrieve and integrate the history and contemporaneous context data associated with the current abnormal point based on water field dynamic knowledge base, generate enhanced diagnostic atlas containing multi-dimensional comparative analysis view, and call large visual language model through guided prompt engineering to perform visual semantic analysis and reasoning on enhanced diagnostic atlas, complete abnormal diagnosis and root cause analysis;Report generation module is used to output structured natural language health assessment report.The system can improve diagnostic accuracy, explainability and long-term hidden danger discovery ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of smart water management and infrastructure health monitoring technology, and relates to a smart water management health monitoring system, specifically a smart water infrastructure health monitoring system that integrates the Internet of Things, visualization analysis and large-scale visual language models for multimodal cognitive reasoning. Background Technology

[0002] As a critical infrastructure, the safe and stable operation of urban water supply networks is of paramount importance. However, factors such as aging networks, increasing loads, and extreme weather events are exacerbating the risk of leaks and pipe bursts, resulting in significant water resource losses and economic costs. Currently, with the widespread application of Industrial Internet of Things (IIoT) technology in smart water management, sensors deployed in the network can continuously generate massive amounts of time-series monitoring data, such as pressure and flow rates.

[0003] Currently, mainstream health monitoring and anomaly diagnosis methods mainly rely on two types of technologies: First, methods based on statistical thresholds or rules. These methods are simple and direct, but have poor adaptability and cannot identify complex, subtle early fault patterns, resulting in high false alarm and false negative rates. Second, methods based on deep learning, such as using recurrent neural networks or temporal convolutional networks to model sensor sequences. While these methods can capture complex patterns, they have significant limitations: First, model training heavily relies on a large amount of historical fault data precisely labeled by experts. However, in real-world water management scenarios, such high-quality labeled data is scarce and costly to obtain, severely restricting the application and generalization capabilities of the models. Second, deep learning models are often considered "black boxes," with opaque internal decision-making logic and a lack of interpretability in diagnostic results. This makes it difficult for maintenance personnel to understand and trust the model's output, hindering its adoption in practical decision-making.

[0004] More importantly, existing methods mostly focus on reacting to instantaneous or short-term data anomalies, lacking effective analysis of long-term, trend-based correlations. For example, a slight, persistent drop in pressure may be an early sign of a hidden leak, but it is difficult to detect by analyzing a single point of data in isolation. Existing technologies fail to systematically integrate data from several months before and after the current anomaly point with historical data from the same period for comparative analysis, thus failing to reveal long-term hidden dangers caused by slow degradation of infrastructure performance or seasonal factors.

[0005] Therefore, there is an urgent need for a new smart water health monitoring technology solution that utilizes data more deeply, enables long-term correlation reasoning, and has a transparent and explainable diagnostic process. Summary of the Invention

[0006] To overcome the shortcomings of existing smart water monitoring systems, such as shallow data utilization, lack of long-term correlation analysis, poor model interpretability, and heavy reliance on labeled data, this invention provides a smart water infrastructure health monitoring system based on multimodal cognitive reasoning. This system can improve diagnostic accuracy, interpretability, and the ability to detect long-term hidden dangers.

[0007] The objective of this invention is achieved through the following technical solution:

[0008] A smart water infrastructure health monitoring system based on multimodal cognitive reasoning includes a data acquisition and conversion module, a multimodal cognitive reasoning engine, and a report generation module, wherein:

[0009] The data acquisition and conversion module is used to collect multi-source heterogeneous time-series data of water infrastructure through a deployed Internet of Things sensor network, and convert the time-series data into standardized dynamic visualization maps;

[0010] The IoT sensors include pressure sensors, flow meters, and water quality monitors;

[0011] The dynamic visualization map is at least one of a line graph, a heat map, or a spatiotemporal distribution map;

[0012] The multimodal cognitive reasoning engine is connected to the data acquisition and conversion module to receive dynamic visualization maps;

[0013] The multimodal cognitive reasoning engine integrates a large-scale visual language model and a dynamic knowledge base in the water sector. When an abnormal signal is received, a long-term correlation analysis mechanism is triggered. Based on the dynamic knowledge base in the water sector, historical and concurrent contextual data related to the current abnormal point are retrieved and integrated to generate an enhanced diagnostic map containing multi-dimensional comparative analysis views. The enhanced diagnostic map is then visually semantically parsed and reasoned through a guided prompting process by calling the large-scale visual language model to complete the abnormal diagnosis and root cause analysis.

[0014] The long-cycle correlation analysis mechanism takes the point when the anomaly is detected as the center, and extracts data of a preset time length to form a first analysis window. At the same time, it retrieves data from the historical database that are in the same season or working conditions as the current anomaly point to form a second analysis window. The data from the first and second analysis windows are converted into a visualization map and displayed side by side or overlaid to form a multi-dimensional comparative analysis view.

[0015] The guided prompting project includes system instruction prompts, context example prompts, and domain knowledge prompts. The system instruction prompts are used to define the roles and tasks of the large visual language model. The context example prompts are used to provide a few typical cases to guide the model to understand water failure modes. The domain knowledge prompts are used to extract entities, relationships, and rules related to the current anomaly from the dynamic knowledge base of the water domain and inject them into the model reasoning process.

[0016] The dynamic knowledge base in the water sector stores topological information of water infrastructure, physical parameters of equipment, historical failure case database, maintenance procedures, and causal reasoning rules based on expert experience.

[0017] The report generation module is connected to the multimodal cognitive reasoning engine to receive diagnostic reasoning results and output a structured natural language health assessment report containing visual evidence, logical reasoning chains, and uncertainty quantification.

[0018] The health assessment report includes: a description of the abnormal event, identification of associated visual evidence, the diagnosed fault type or pattern, the inferred underlying cause, recommended countermeasures or maintenance suggestions, and a diagnostic uncertainty score calculated based on model confidence or sufficiency of evidence.

[0019] A method for health monitoring of smart water infrastructure based on multimodal cognitive reasoning using the above system includes the following steps:

[0020] Step 1: Continuously collect multi-source heterogeneous time-series data of water infrastructure through an IoT sensor network;

[0021] Step 2: Convert the collected time-series data into standardized dynamic visualization maps in real time or at preset cycles according to the predefined template;

[0022] Step 3: Use the anomaly detection unit to perform preliminary screening of the dynamic visualization map or raw data to identify abnormal signals;

[0023] Step 4: When an abnormal signal is detected, the multimodal cognitive reasoning engine is triggered to perform long-term correlation analysis, retrieve and integrate relevant contextual data to generate a multidimensional comparative analysis view;

[0024] Step 5: Retrieve domain knowledge fragments related to the current abnormal scenario from the dynamic knowledge base of the water sector, use the domain knowledge fragments as part of the prompts, and combine them with the multi-dimensional comparative analysis view to form an enhanced diagnostic map, and input them into the large visual language model.

[0025] Step 6: Drive the large-scale visual language model through guided prompting engineering to perform visual semantic parsing and reasoning on the enhanced diagnostic map, and complete the anomaly diagnosis and root cause analysis. Among them, under the condition that the labeled fault data is scarce or zero, the anomaly diagnosis of zero samples or a few samples can be achieved through guided prompting engineering and contextual example prompts.

[0026] Step 7: Perform structured parsing and verification of the reasoning output of the large visual language model, extract key diagnostic elements, and verify their consistency with the rules in the knowledge base.

[0027] Step 8: Generate and output a structured natural language health assessment report.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] 1. By converting time-series data into visual graphs and leveraging the powerful zero-shot / few-shot reasoning capabilities of large-scale visual language models, the reliance on a large amount of labeled fault data is significantly reduced, achieving efficient and high-precision diagnosis based on data.

[0030] 2. By introducing a long-term correlation analysis mechanism and a domain knowledge base, it is possible to discover slow degradation trends and potential seasonal correlations that are difficult to detect in short-term analysis, thereby improving the depth and foresight of monitoring.

[0031] 3. The entire diagnostic process is based on visualized evidence and a traceable logical chain, outputting highly readable natural language reports, which greatly enhances the transparency and interpretability of the system, helps to build trust between humans and machines, and promotes the practical adoption of intelligent diagnostic results.

[0032] 4. A general "data-graph-knowledge-reasoning" framework has been constructed, which is not only applicable to water pipe networks, but its methodology can also be extended to other similar critical infrastructure health monitoring fields. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the overall architecture of a smart water infrastructure health monitoring system. 101-Data acquisition and conversion module, 102-Multimodal cognitive reasoning engine, 103-Report generation module, 104-Internet of Things sensor network, 105-Dynamic knowledge base for water affairs, 106-Large-scale visual language model.

[0034] Figure 2This is a flowchart illustrating the long-term correlation analysis and diagnostic reasoning process of a multimodal cognitive reasoning engine. 201 - Original time-series data flow, 202 - Anomaly detection trigger point, 203 - Current anomaly period map, 204 - Long-term historical data retrieval, 205 - Historical comparison map, 206 - Enhanced diagnostic map, 207 - Guided prompt construction, 208 - Visual language model reasoning, 209 - Structured diagnostic output. Detailed Implementation

[0035] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0036] This invention provides a smart water infrastructure health monitoring system based on multimodal cognitive reasoning. This system elevates data monitoring to cognitive insight by simulating the cognitive process of domain experts. It mainly comprises three core modules: a data acquisition and transformation module, a multimodal cognitive reasoning engine, and a report generation module.

[0037] The data acquisition and conversion module is responsible for continuously capturing multi-source, heterogeneous, and high-dimensional time-series data from the water supply network through widely deployed IoT devices such as pressure sensors, flow meters, and water quality monitors. The core function of this module is to convert these abstract numerical sequences into standardized, dynamic visualization maps according to predefined, semantically rich templates (such as time-series curves, multi-sensor data heatmaps, and spatial pressure distribution maps of the pipeline network). This conversion transforms the difficult-to-understand digital stream into a "diagnostic image" containing rich visual semantics such as trends, comparisons, and spatial relationships, laying the foundation for subsequent visual understanding.

[0038] The multimodal cognitive reasoning engine is the core of this invention. It is not simply a combination of algorithms, but a collaborative reasoning system integrating a large-scale Visual Language Model (VLM) and a dynamic knowledge base in the water sector. When the system initially detects an abnormal signal through basic thresholds or a lightweight model, the multimodal cognitive reasoning engine automatically triggers its unique long-term correlation analysis mechanism. This mechanism focuses on the point of anomaly, intelligently retrieving time-series data from weeks or even months before and after the current time point (providing recent evolution context), while simultaneously retrieving data from the same period last year or under similar operating conditions from the historical database (providing historical comparison benchmarks). This data is synchronously converted into visual maps and integrated into an "enhanced diagnostic map" through parallel and overlay methods, thus placing isolated anomalies within a long-term, multi-dimensional comparative analysis perspective. Subsequently, the guided prompting process begins. The system constructs a structured prompt, which includes system instructions defining the VLM as a "water diagnostic expert," a small number of typical fault map examples for few-shot learning, and domain knowledge (such as pipeline topology, equipment parameters, and historical case rules) extracted in real-time from the dynamic knowledge base relevant to the current scenario. This enhanced diagnostic atlas, along with structured cues, is input into a large-scale visual language model. Leveraging its powerful visual-language cross-modal understanding and reasoning capabilities, the VLM "reads" the visual patterns in the atlas. Combining this with injected domain knowledge, it reasons like a human expert, completing a full-chain diagnostic analysis from "What abnormal patterns are shown in the graph?" and "How do these patterns differ from historical patterns?" to "What are the possible root causes of these patterns?"

[0039] The report generation module receives structured output from the multimodal cognitive reasoning engine. Instead of simply listing results, it generates a comprehensive natural language health assessment report. This report explicitly cites atlas regions as evidence (such as "e.g.,..."). Figure 1 (As shown by the red curve, the pressure has shown a continuous downward trend over the past two weeks). This clearly illustrates the logical reasoning chain from evidence to conclusion, provides specific fault type judgments and root cause hypotheses, and offers maintenance recommendations. Furthermore, the report can also add an uncertainty quantification score based on the confidence level of the model output or the sufficiency of the evidence, for maintenance personnel to reference.

[0040] This invention also provides a method for health monitoring of smart water infrastructure based on multimodal cognitive reasoning using the above-mentioned system. The method collects multi-source heterogeneous time-series data such as pressure, flow, and water quality through the Internet of Things and converts it into a semantic dynamic visualization map. It constructs a multimodal cognitive reasoning engine that integrates generalized reasoning from a large-scale visual language model with a water domain knowledge base. When an anomaly occurs, it triggers long-term correlation analysis, integrating data from several months before and after the anomaly point, as well as historical data from the same period, to form a multi-dimensional comparative view. With guided prompts and the injection of domain knowledge, it collaboratively completes the entire chain of diagnosis, from anomaly perception and pattern recognition to root cause inference, outputting a structured natural language health assessment report with clear visual evidence and logical chains, quantifying diagnostic uncertainty. The specific steps are as follows:

[0041] Step 1: Continuously collect multi-source heterogeneous time-series data of water infrastructure through an IoT sensor network;

[0042] Step 2: Convert the collected time-series data into standardized dynamic visualization maps in real time or at preset cycles;

[0043] Step 3: Use the anomaly detection unit to perform preliminary screening of the dynamic visualization map or raw data to identify abnormal signals;

[0044] Step 4: When an abnormal signal is detected, the multimodal cognitive reasoning engine is triggered to perform long-term correlation analysis, retrieve and integrate relevant contextual data to generate a multidimensional comparative analysis view;

[0045] Step 5: Retrieve domain knowledge fragments related to the current abnormal scenario from the dynamic knowledge base of the water sector, use the domain knowledge fragments as part of the prompts, and combine them with the multi-dimensional comparative analysis view to form an enhanced diagnostic map, and input them into the large visual language model.

[0046] Step 6: Drive the large-scale visual language model through guided prompting engineering to perform visual semantic parsing and reasoning on the enhanced diagnostic map, and complete the anomaly diagnosis and root cause analysis. Among them, under the condition that the labeled fault data is scarce or zero, the anomaly diagnosis of zero samples or a few samples can be achieved through guided prompting engineering and contextual example prompts.

[0047] Step 7: Perform structured parsing and verification of the reasoning output of the large visual language model, extract key diagnostic elements, and verify their consistency with the rules in the knowledge base.

[0048] Step 8: Generate and output a structured natural language health assessment report.

[0049] Example 1:

[0050] refer to Figure 1This embodiment provides a complete smart water infrastructure health monitoring system based on multimodal cognitive reasoning. The system hardware includes various Internet of Things sensors 104 deployed at key nodes of the water supply network (such as pumping stations, water transmission trunk lines, and zoned metering areas), including pressure transmitters, electromagnetic flow meters, turbidity meters, and residual chlorine analyzers. These sensors transmit the collected time-series data to the central server or cloud platform in real time via wired or wireless networks.

[0051] The data acquisition and conversion module 101 runs on the server and receives the raw time-series data stream 201. This module incorporates various data preprocessing algorithms, such as missing value imputation, noise filtering, and data standardization. Its core function is graph conversion. For example, for pressure data from a single monitoring point, the module can generate a "pressure change curve over the past 24 hours"; for multiple pressure sensors on the same pipe section, it can generate a "pressure distribution profile along the pipeline"; and for a zoned metering area, it can plot the difference between inlet and outlet flow rates as a "minimum flow trend chart at night." These graph templates are predefined, ensuring output standardization and semantic consistency.

[0052] The multimodal cognitive reasoning engine 102 is the intelligent hub of the system. The dynamic knowledge base 105 in the water sector stores structured knowledge, including: the topology of the pipeline network GIS, pipe material and diameter information, pump characteristic curves, historical records of pipe bursts and leaks (including time, location, phenomenon, and ultimate cause), and expert experience rules (such as "if the pressure at a certain point continues to drop slowly while the pressure at adjacent points remains unchanged, then there is a suspected hidden leak upstream of that point"). When the anomaly detection unit (which can be a simple threshold comparison or a lightweight unsupervised model) identifies an anomaly detection trigger point 202 in the data stream (such as a sudden and sharp drop in pressure at a certain point exceeding a threshold), the multimodal cognitive reasoning engine 102 is activated.

[0053] The multimodal cognitive reasoning engine 102 first performs long-term correlation analysis. For example... Figure 2 As shown, it not only extracts data near the time of the anomaly to generate the current anomaly period map 203, but also initiates two parallel searches: one is to search for data from the 30 days before and 7 days after the point (the time window is configurable) to observe the precursors and aftereffects of the anomaly; the other is to search for data from the same month last year with similar weather conditions. These data are converted into the same type of map, such as generating a "stress curve for 37 days before and after this anomaly" and a "stress curve for 37 days in the same period last year". Subsequently, image processing technology is used to overlay these two curves on the same coordinate system to form an enhanced diagnostic map 206, in which different colors are used to distinguish the current curve from the historical curve, making the differences immediately apparent.

[0054] Next, the project module will display a guided build prompt 207. A specific example of this prompt is as follows:

[0055] [System Command] You are a senior water supply network diagnostic expert. Please analyze the pressure curve comparison chart below to complete the diagnostic task.

[0056] [Contextual Example] (You can insert 1-2 examples of corresponding text and images here. For example, if the image shows "periodic fluctuations in the pressure curve", the corresponding description is "This pattern may be related to the start and stop of the water pump or the user's water usage patterns.")

[0057] [Domain Knowledge] The current monitoring point is located on a cast iron main pipeline with a diameter of DN300, and there have been no maintenance records in the past three years. The blue curve in the graph represents the data for the current monitoring period, and the red curve represents the historical data for the same period.

[0058] [Analysis Task] 1. Describe the overall trend of the blue curve near the marked anomaly point and over a long period. 2. Compare the blue and red curves and point out significant differences. 3. Based on the trend and differences, infer the most likely anomaly type (e.g., pipe burst, leak, equipment failure, data anomaly, etc.) and potential root causes. 4. Provide recommendations for the next steps in operation and maintenance.

[0059] The enhanced diagnostic atlas 206 and the constructed hints are fed into the large visual language model 106 for visual language model inference 208. The VLM outputs expert-like analysis text. The report generation module 103 performs structured parsing of this text, extracts key entities and conclusions, fills them into a standard report template, and outputs the final health assessment report 209. The health assessment report may include: "Diagnostic conclusion: High probability of slow leakage. Main basis: The current pressure curve (blue) has shown a continuous slow decreasing trend of approximately 0.01 MPa / day over the past two weeks, while the historical curve (red) has remained stable during this period. This trend is more pronounced during low water usage periods at night. Combined with the knowledge that this pipe section is an old cast iron pipe, it is inferred that the leakage may be caused by pinholes formed by pipe wall corrosion. Recommendation: Arrange for leak detection or deploy area leak noise monitoring instruments for location. Diagnostic confidence level: 85%."

[0060] Example 2:

[0061] As an extension and variation of Example 1, this example focuses on the proactive health assessment mode of the system in the absence of clear instantaneous anomaly triggers. In this mode, the system can automatically perform long-term correlation analysis at fixed intervals (e.g., monthly). The data acquisition and conversion module 101 generates a comparison chart of the "daily average pressure curve of this month" and the "average curve of the past 12 months" for each monitoring point, or a bar chart comparing the "minimum nighttime flow of this month" and the "value of the same period last year".

[0062] The multimodal cognitive inference engine 102 receives these periodic graphs and guides the VLM to perform trend-based health scans with prompts. The prompt task can be set as: "Identify any noteworthy performance degradation trends or seasonal anomalies in the graphs." For example, the VLM might identify from a comparative graph that the minimum nighttime flow in a certain area is slowly increasing year by year, while the pressure is slightly decreasing, thus prompting a prompt: "There may be a risk of a slow increase in background leakage in this area; a leakage level assessment is recommended." This approach enables the system to shift from passive alarms to proactive preventative health status assessments, identifying potential problems in advance.

Claims

1. A smart water infrastructure health monitoring system based on multimodal cognitive reasoning, characterized in that... The system includes a data acquisition and conversion module, a multimodal cognitive reasoning engine, and a report generation module, wherein: The data acquisition and conversion module is used to collect multi-source heterogeneous time-series data of water infrastructure through a deployed Internet of Things sensor network, and convert the time-series data into standardized dynamic visualization maps; The multimodal cognitive reasoning engine is connected to the data acquisition and conversion module to receive dynamic visualization maps; The multimodal cognitive reasoning engine integrates a large-scale visual language model and a dynamic knowledge base in the water sector. When an abnormal signal is received, a long-term correlation analysis mechanism is triggered. Based on the dynamic knowledge base in the water sector, historical and concurrent contextual data related to the current abnormal point are retrieved and integrated to generate an enhanced diagnostic map containing multi-dimensional comparative analysis views. The enhanced diagnostic map is then visually semantically parsed and reasoned through a guided prompting process by calling the large-scale visual language model to complete the abnormal diagnosis and root cause analysis. The long-cycle correlation analysis mechanism takes the point when the anomaly is detected as the center, and extracts data for a preset time length to form a first analysis window. At the same time, it retrieves data from the historical database that are in the same season or under the same operating conditions as the current anomaly point to form a second analysis window. The data from the first and second analysis windows are converted into a visualization map and displayed side by side or overlaid to form a multi-dimensional comparative analysis view. Domain knowledge fragments related to the current anomaly scenario are retrieved from the dynamic knowledge base of the water industry, and these domain knowledge fragments are used as part of the prompts, which together with the multi-dimensional comparative analysis view constitute an enhanced diagnostic map. The report generation module is connected to the multimodal cognitive reasoning engine to receive diagnostic reasoning results and output a structured natural language health assessment report that includes visual evidence, logical reasoning chains, and uncertainty quantification.

2. The smart water infrastructure health monitoring system based on multimodal cognitive reasoning according to claim 1, characterized in that... The IoT sensors include pressure sensors, flow meters, and water quality monitors.

3. The smart water infrastructure health monitoring system based on multimodal cognitive reasoning according to claim 1, characterized in that... The dynamic visualization map is at least one of a curve graph, a heat map, or a spatiotemporal distribution map.

4. The smart water infrastructure health monitoring system based on multimodal cognitive reasoning according to claim 1, characterized in that... The guided prompting engineering includes system instruction prompts, contextual example prompts, and domain knowledge prompts, wherein: the system instruction prompts are used to define the roles and tasks of the large visual language model; the contextual example prompts are used to provide a few typical cases to guide the model to understand water failure modes; and the domain knowledge prompts are used to extract entities, relationships, and rules related to the current anomaly from the dynamic knowledge base of the water domain and inject them into the model reasoning process.

5. The smart water infrastructure health monitoring system based on multimodal cognitive reasoning according to claim 1, characterized in that... The dynamic knowledge base for the water sector stores topological information of water infrastructure, physical parameters of equipment, a database of historical failure cases, maintenance procedures, and causal reasoning rules based on expert experience.

6. The smart water infrastructure health monitoring system based on multimodal cognitive reasoning according to claim 1, characterized in that... The health assessment report includes: a description of the abnormal event, identification of associated visual evidence, the diagnosed fault type or pattern, the inferred underlying cause, recommended countermeasures or maintenance suggestions, and a diagnostic uncertainty score calculated based on model confidence or sufficiency of evidence.

7. A method for health monitoring of smart water infrastructure based on multimodal cognitive reasoning using the system described in any one of claims 1-6, characterized in that... The method includes the following steps: Step 1: Continuously collect multi-source heterogeneous time-series data of water infrastructure through an IoT sensor network; Step 2: Convert the collected time-series data into standardized dynamic visualization maps in real time or at preset cycles according to the predefined template; Step 3: Use the anomaly detection unit to perform preliminary screening of the dynamic visualization map or raw data to identify abnormal signals; Step 4: When an abnormal signal is detected, the multimodal cognitive reasoning engine is triggered to perform long-term correlation analysis, retrieve and integrate relevant contextual data to generate a multidimensional comparative analysis view; Step 5: Retrieve domain knowledge fragments related to the current abnormal scenario from the dynamic knowledge base of the water sector, use the domain knowledge fragments as part of the prompts, and combine them with the multi-dimensional comparative analysis view to form an enhanced diagnostic map, and input them into the large visual language model. Step 6: Drive the large-scale visual language model through guided prompting engineering to perform visual semantic parsing and reasoning on the enhanced diagnostic map, and complete the anomaly diagnosis and root cause analysis; Step 7: Perform structured parsing and verification of the reasoning output of the large visual language model, extract key diagnostic elements, and verify their consistency with the rules in the knowledge base. Step 8: Generate and output a structured natural language health assessment report.

8. The smart water infrastructure health monitoring method based on multimodal cognitive reasoning according to claim 7, characterized in that... In step 6, under the condition that the labeled fault data is scarce or zero, the abnormal diagnosis of zero or a few samples can be achieved through guided prompts and contextual example prompts.

Citation Information

Patent Citations

  • Smart park knowledge question-answering method and device based on large language model

    CN120523902A

  • Operation and maintenance data exception positioning method and computing equipment

    CN120596301A