Comprehensive pipe gallery operation and maintenance system and method based on AI large model

Abstract

The invention discloses a comprehensive pipe gallery operation and maintenance system and method based on an AI large model, and relates to the technical field of public safety operation and maintenance and smart city management. The method comprises the following steps: acquiring continuous time sequence data generated by a distributed environment sensor in a pipe gallery and a monitoring image sequence of a corresponding time period; calculating a change feature vector of the continuous time series data in a preset sliding time window to form a trend vector, and mapping the trend vector into a textualized trend abstract; inputting the textualized trend abstract into a pre-trained language model, and outputting a first anomaly inference result with confidence; performing differential calculation on the monitoring image sequence of the same time window to obtain a differential image sequence, and inputting the obtained differential image sequence into a pre-trained visual semantization model to generate visual change description; according to the method, the core problem that the slowly-changing or distributed small-amplitude abnormity is difficult to find in time due to the lack of cross-time and cross-space fusion in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of public security operation and maintenance and smart city management technology. More specifically, the present invention relates to an integrated pipeline corridor operation and maintenance system and method based on an AI large model. Background Art

[0002] Integrated utility corridors typically house numerous pipelines in a closed underground environment. To ensure the continuous and safe operation of urban water, gas, communications, and electricity, continuous monitoring of environmental and structural parameters such as temperature, humidity, and gas concentration is essential. Existing monitoring platforms mostly use a point-based threshold alarm model: the system compares each sensor's real-time reading against a preset safety threshold and triggers an alarm if the instantaneous data exceeds the limit. However, this approach assumes that anomalies are sudden jumps, ignoring the fact that hidden dangers can manifest slowly, incrementally, or through coordinated multi-point processes. In the event of a minor leak, localized moisture, or structural stress, multiple sensors will only exhibit a slight, gradually increasing offset. Similarly, temperature increases often occur in a gradient along the corridor, and single-point thresholds struggle to capture this gradual trend. Coupled with the high noise levels in the underground environment, sensor aging, and significant seasonal fluctuations, the system often misclassifies subtle changes as normal drift, leading to potential risks being misdiagnosed or obscured at an early stage.

[0003] The existing patent with publication number CN112766795A provides a method and system for intelligent information management of a pipe gallery based on cloud processing. The method includes: obtaining anomaly detection and attribute analysis data sets; training a deep neural network model for pipe gallery anomaly detection; training a deep neural network model for pipe gallery attribute analysis; obtaining smoke, temperature, vision, humidity and other information from the sensor group; using a deep network classifier to perform anomaly detection on the newly acquired image to be identified for visual information; using a deep network detector to perform attribute analysis on the newly acquired image to be processed for visual information; and modifying the attribute set of all sensor information and visual processing information based on the three-dimensional BIM model.

[0004] However, such a process still follows a single-moment, single-sensor paradigm. Images and numerical values ​​are not integrated across time and space. Slowly increasing small leaks or multi-node coordinated offsets can easily be regarded as noise and masked. Moreover, the algorithm only cares about visual anomalies, while other sensor curves still rely on traditional thresholds, resulting in a dull perception of gradual and compound risks.

[0005] The existing patent with publication number CN119152637A provides a method for monitoring and early warning of electrical fires in urban underground integrated pipe corridors, including: Step 1: Determine the indicators to be monitored and deploy multi-identification sensing nodes based on multi-identification wireless communication; the indicators to be monitored are collected through: gas sensors, temperature and humidity sensors, voltage acquisition devices, and near-infrared binocular cameras, and monitored and identified by the monitoring host. Step 2: Based on the current electrical distribution situation, an electrical fire monitoring and identification model is constructed; the monitored building is divided into test areas, and multiple corresponding test nodes are set in each area. The nodes are interconnected to form a cyclic monitoring structure; Step 3: Monitor abnormal data collection and edge-side data processing, and locate the fire point based on binocular vision.

[0006] However, it has no way to deal with hidden dangers other than fire, such as water seepage, corrosion, and gas leakage. Data discrimination is still centered on formula thresholds and geometric heuristics, and it also lacks deep coupling of historical sequences and multi-source information.

[0007] In summary, while existing technologies propose multi-sensor data fusion or machine learning anomaly detection, they focus on single-moment feature extraction or homologous data modeling, failing to achieve deep integration of historical sequences and spatial correlation information. Furthermore, they lack engineering solutions for dynamically identifying slowly changing patterns. This results in operations and maintenance personnel relying on experience and manual inspections to address system blind spots. Once a slowly changing risk crosses a threshold, it can develop into a major failure, resulting in unforeseen losses to urban public safety and operations and maintenance.

[0008] In view of this, the present invention proposes an integrated pipeline corridor operation and maintenance system and method based on an AI large model to solve the above problems. Summary of the Invention

[0009] In order to overcome the above-mentioned defects of the prior art and achieve the above-mentioned objectives, the present invention provides the following technical solutions: an integrated pipe gallery operation and maintenance method based on an AI large model, comprising:

[0010] Obtain the continuous time series data generated by the distributed environmental sensors in the tunnel and the monitoring image sequence of the corresponding time period;

[0011] Calculate the change feature vector of continuous time series data within a preset sliding time window to form a trend vector, and map the trend vector into a textual trend summary;

[0012] Input the textual trend summary into the pre-trained first language model and output the first anomaly inference result with confidence;

[0013] Perform differential calculation on the surveillance image sequence of the same time window to obtain a differential image sequence, and input the obtained differential image sequence into the pre-trained visual semantic model to generate a visual change description;

[0014] The first anomaly inference result and the visual change description are aligned and spliced ​​according to the timestamp, and then input into the pre-trained second language model to obtain a comprehensive risk hypothesis including cause, location, and risk level;

[0015] If the confidence level of the comprehensive risk hypothesis is higher than the preset confidence threshold, early warning information will be output.

[0016] Furthermore, the method of obtaining continuous time series data and monitoring image sequences includes:

[0017] Align and combine the continuously sampled temperature data series, humidity data series, and gas concentration data series according to timestamps to obtain continuous time series data;

[0018] The camera frame images are captured at a preset frame rate to obtain single-frame images; the single-frame images are spliced ​​in chronological order to obtain a monitoring image sequence.

[0019] Furthermore, the method of forming the trend vector includes:

[0020] Calculate the weighted average of continuous time series data within the preset sliding time window to obtain the mean sequence within the window;

[0021] Perform the first-order difference on the mean sequence within the window to obtain the difference sequence;

[0022] The pre-built slight threshold comparison table is queried based on the sensing type to obtain the slight threshold, the difference sequence is compared with the slight threshold element by element, and a Boolean over-threshold mark sequence is output;

[0023] The longest length and total number of consecutive segments with a value of 1 in the super-threshold marker sequence are counted to obtain the change feature vector;

[0024] The change feature vector is concatenated with the terminal value of the mean sequence within the window to generate a trend vector.

[0025] Furthermore, the method of mapping the trend vector into a textual trend summary includes:

[0026] Call the preset field mapping dictionary to map each dimension of the trend vector to the corresponding indicator-value key-value pair;

[0027] Call the pre-built natural language template library to fill each indicator-value key-value pair to generate a sentence fragment;

[0028] The sentence fragments are spliced ​​together in the order of the timestamps generated by each dimension of the trend vector in the data buffer to obtain a textual trend summary.

[0029] Furthermore, the training method of the first language model and the second language model includes:

[0030] Constructing a first data set, the first data set including a historical textual trend summary and corresponding pre-labeled anomaly categories and confidence levels, to obtain anomaly corpus;

[0031] Perform incremental adjustments on the general pre-trained language model based on the abnormal corpus, and output a first task model as the first language model;

[0032] Constructing a second data set, the second data set including the concatenated text of the historical first anomaly inference results and the visual change descriptions and their manually annotated causes, locations, and risk levels to obtain a fused corpus;

[0033] A multi-task learning approach is used to continue adjusting the first task model on the fused corpus, and the fused task model is output as the second language model.

[0034] Furthermore, the method for obtaining the differential image sequence includes:

[0035] The surveillance image sequence within the same time window is rearranged in ascending order of timestamps to obtain an ordered image sequence;

[0036] Pair adjacent frames of the ordered image sequence to obtain a set of frame pairs;

[0037] Calculate the absolute difference pixel by pixel for each pair of adjacent frames in the frame pair set to generate the corresponding single-frame difference image;

[0038] All single-frame differential images are stitched together in the order of timestamps to obtain a differential image sequence.

[0039] Furthermore, the training method of the visual semantic model includes:

[0040] S101: Collect historical data including differential image sequences and their corresponding pre-labeled visual change descriptions to construct a training dataset;

[0041] S102: Input the training dataset into the initialized semantic segmentation network to obtain a predicted visual change description;

[0042] S103: Calculating a loss function between the predicted visual change description and the pre-labeled visual change description to obtain a loss value;

[0043] S104: Perform back propagation on the semantic segmentation network according to the loss value and update the parameters to obtain an updated semantic segmentation network;

[0044] S105: Repeat steps S102 to S104 until the loss value meets the convergence condition, and obtain a converged visual semantic model.

[0045] Furthermore, the confidence threshold is preset by:

[0046] Calculate the confidence of historical comprehensive risk assumptions to obtain a confidence sample set;

[0047] Calculate the receiver operating characteristic curve according to the confidence sample set to obtain receiver operating characteristic curve data;

[0048] The confidence value with the largest Youden index was selected from the receiver operating characteristic curve data and set as the confidence threshold.

[0049] Furthermore, the early warning information includes at least the following elements:

[0050] The first is the cause of the risk, which uses a brief text to indicate the potential type of accident inferred by the second language model; the second is spatial positioning, which gives the coordinates of the specific section of the corridor, cabin or camera field of view; the third is the risk level, which uses a five-level system or a 0-1 floating point to indicate the urgency of the situation; the fourth is the confidence level, which indicates the reliability of the second language model's own inference; and the fifth is the response suggestion, which provides the on-duty personnel with operational instructions for the next inspection or linkage action.

[0051] The sensor acquisition module is used to obtain the continuous time series data generated by the distributed environmental sensors in the pipeline corridor and the monitoring image sequence of the corresponding time period;

[0052] The trend summary module is used to calculate the change feature vector of continuous time series data within a preset sliding time window, form a trend vector, and map the trend vector into a textual trend summary;

[0053] A model inference module is used to input the textual trend summary into the pre-trained first language model and output a first anomaly inference result with confidence;

[0054] The visual difference module is used to perform difference calculation on the surveillance image sequence of the same time window to obtain a difference image sequence, and then input the obtained difference image sequence into the pre-trained visual semantic model to generate a visual change description;

[0055] The fusion assessment module is used to align and concatenate the first anomaly inference result and the visual change description according to the timestamp, input the result into the pre-trained second language model, and obtain a comprehensive risk hypothesis including cause, location, and risk level;

[0056] The early warning output module is used to output early warning information if the confidence level of the comprehensive risk hypothesis is higher than a preset confidence threshold.

[0057] Compared with the existing technology, the technical effects and advantages of the integrated pipe gallery operation and maintenance system and method based on the AI ​​large model of the present invention are as follows:

[0058] The present invention first continuously and synchronously collects temperature, humidity, combustible gas concentration and monitoring images at the tunnel site, and organizes the numerical sequence and image sequence into a unified raw data stream through precise timing. The system calculates the weighted average, first-order difference and micro-threshold statistics of the numerical sequence within the sliding time window to obtain a trend vector describing the slow offset characteristics, and then generates a natural language summary with the help of dictionary mapping and templates and submits it to the language model, and outputs a preliminary abnormality judgment with confidence. At the same time, a frame difference operation is performed on the image sequence to obtain a differential image, which is input into the semantic segmentation network to generate a visual change description containing type, position, amplitude and trend. The language model receives the above two semantic information and gives the cause, location and risk level at one time. The system compares its confidence with the threshold generated by historical samples and pushes an early warning.

[0059] The present invention solves the core problem of the existing technology that lacks cross-time and cross-space fusion, resulting in the difficulty in timely detection of slowly changing or distributed small anomalies. It realizes the early identification of hidden dangers such as small leaks and gradual temperature increases, reduces the burden of manual inspections, improves the timeliness of early warnings and positioning accuracy, and provides clear semantic guidance and quantitative risk basis for subsequent disposal. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 Schematic diagram of an integrated pipe gallery operation and maintenance system based on an AI large model according to an embodiment of the present invention;

[0061] Figure 2 This is a flow chart of an integrated pipe gallery operation and maintenance method based on an AI large model according to an embodiment of the present invention;

[0062] Figure 3 This is a flow chart of a method for training a visual semantic model according to an embodiment of the present invention;

[0063] Figure 4 This is a diagram of the components of early warning information according to an embodiment of the present invention;

[0064] Figure 5 Schematic diagram of comparison between a differential sequence and a slight amplitude threshold according to an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present invention will be described in detail, clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. It should be noted that the specific embodiments described below are only used to better illustrate and describe the technical solutions of the present invention, and are intended to enable those skilled in the art to better understand and implement the present invention, and should not be construed as limiting the scope of protection of the present invention. Without departing from the spirit and essence of the present invention, those skilled in the art may modify, adjust or make equivalent replacements based on the contents disclosed in the present invention, and these should all be regarded as the scope of protection of the present invention.

[0066] Example 1

[0067] See also Figure 1 As shown, this embodiment discloses an integrated pipeline corridor operation and maintenance system based on an AI large model, including a sensor acquisition module, a trend summary module, a model reasoning module, a visual difference module, a fusion evaluation module and an early warning output module. Each module realizes data transmission through wired and / or wireless connections.

[0068] The sensor acquisition module is used to obtain the continuous time series data generated by the distributed environmental sensors in the pipeline corridor and the monitoring image sequence of the corresponding time period.

[0069] The continuously sampled temperature data sequence, humidity data sequence, and gas concentration data sequence are aligned and combined according to the timestamps to obtain continuous time series data.

[0070] The camera frame images are captured at a preset frame rate to obtain single-frame images; the single-frame images are spliced ​​in chronological order to obtain a monitoring image sequence.

[0071] To obtain such continuous and aligned data, temperature, humidity, and combustible gas probes are deployed every 15 to 25 meters throughout the tunnel. The physical coordinates of each probe are associated with the tunnel's BIM node. All probes record timestamps with millisecond accuracy via a local MCU or RTU. Temperature and humidity signals are sampled once per second, and gas signals are sampled every 5 seconds. Synchronized industrial cameras are fixed at the same location, capturing 1080p images at a frame per second and incorporating local camera timecode. Each acquisition unit synchronizes time with the edge gateway via PTP or IEEE-1588, ensuring cross-modal time error of no more than 50 milliseconds. The gateway pulls data from the sensor buffer at a period of v seconds, aligns timestamps, and generates continuous time series frames in JSON or Protobuf format. Image frames from the same interval are captured and sequentially stored in a solid-state ring buffer. Finally, the gateway packages the checksummed time series data frames with the corresponding image sequence and pushes them to the local inference server via MQTT reliable transmission or shared memory, with latency under 300 milliseconds. The data stream constructed in this way not only retains the original resolution of the four-way information of temperature, humidity, air and image, but also achieves strict synchronization in time and space, laying a reliable data foundation for subsequent sliding window slight change detection, textual trend generation, LLM semantic reasoning and visual cross-validation, and significantly improving the early warning capability of slowly evolving and distributed anomalies.

[0072] Continuous acquisition of temperature, humidity, combustible gas concentrations, and synchronized video images are crucial for long-term safety monitoring of utility corridors. Temperature sequences reveal the gradual heat buildup in cables or pipes due to increased loads. Humidity sequences capture localized tidal rises caused by minute water seepage over hours. Combustible gas sequences reflect low-concentration accumulations caused by slow leaks. Continuous images visualize imperceptible water stains, wall discoloration, or light smoke. Only when these heterogeneous data are collected and aligned in parallel on the same time scale can subsequent algorithms track growth rates over time and compare adjacent nodes in space, enabling early insight into slowly changing or distributed anomalies and fundamentally addressing the insensitivity of traditional threshold alarms to "small signals."

[0073] The trend summary module is used to calculate the change feature vector of continuous time series data within a preset sliding time window, form a trend vector, and map the trend vector into a textual trend summary.

[0074] Methods for forming trend vectors include:

[0075] The weighted average of continuous time series data is calculated within a preset sliding time window to obtain the mean sequence within the window.

[0076] Perform the first-order difference on the mean sequence within the window to obtain the difference sequence.

[0077] The slight threshold is obtained by looking up the table based on the sensor type, and the difference sequence is compared with the slight threshold element by element to output a Boolean over-threshold mark sequence.

[0078] The longest length of the continuous segment with a value of 1 and the total number of segments in the super-threshold marker sequence are counted to obtain the change feature vector.

[0079] The change feature vector is concatenated with the terminal value of the mean sequence within the window to generate a trend vector.

[0080] Methods for obtaining the mean sequence within the window include:

[0081] At the starting moment of the preset sliding time window, the system first reads the continuous time series data within the corresponding length range and sets the weight coefficient according to the order of data generation; the samples close to the end of the time window are given higher weights, and the samples close to the beginning of the time window are given lower weights.

[0082] Subsequently, the numerical value of each sampling point in the continuous time series data is multiplied by its corresponding weight and the sum is calculated. The weighted sum is then normalized by the sum of the weights to obtain the weighted average result covering the entire sliding time window, which is recorded as the in-window mean sequence. When the sliding time window advances forward with a fixed step size, the above weighted average process is synchronously updated to ensure that the in-window mean sequence maintains a continuous output on the time axis.

[0083] In a sliding time window, higher weights are assigned to samples near the end of the window, and lower weights are assigned to samples near the beginning of the window. The fundamental purpose is to make the mean sequence within the window more sensitive to the real changing trend at the most recent moment. The environmental parameters of the pipeline corridor show slight and gradual offsets in the slow-changing anomaly scenario, and these offsets are first reflected in the latest data segment of the time series. If the samples of the entire window are treated equally, the old samples will dilute the latest signal in the averaging operation, resulting in a lag in the mean and a delayed occurrence of anomalies. Through the incremental weight design, the calculation process automatically amplifies the contribution of data near the end, that is, the data closest to the current moment, while retaining the smoothing effect of the entire window. This makes the mean sequence within the window both noise-resistant and forward-looking: it can suppress the misjudgment caused by isolated spikes, and can promptly capture the subtle rise or fall of indicators, laying a foundation for the accuracy of subsequent differential operations to accurately depict slowly evolving trends.

[0084] Methods for obtaining difference sequences include:

[0085] After generating the mean sequence within the window, the system immediately subtracts the means of two adjacent moments and strictly retains the sign information, so that positive values ​​represent an increase in the indicator and negative values ​​represent a decrease in the indicator. The results are stored in chronological order to form a differential sequence. To avoid interference from numerical oscillations on subsequent judgments, a unified dimension conversion coefficient can be used to scale the amplitude of the differential sequence before storage to keep it at the same order of magnitude as the slight threshold.

[0086] Methods for obtaining Boolean super-threshold marker sequences include:

[0087] Based on the data source in the differential sequence, the system calls the corresponding sensor type to query the micro-threshold comparison table to obtain the micro-threshold; then each element of the differential sequence is compared with the matched micro-threshold element by element. When the absolute value of the differential value is greater than the micro-threshold, the output is a logical value "1", otherwise the output is a logical value "0", and the comparison result is written into the mark buffer in the original order, thus forming a Boolean super-threshold mark sequence; to ensure that the super-threshold mark sequence and the differential sequence correspond one-to-one in the index position, the system adopts an integer subscript synchronization mechanism during the writing process.

[0088] The method for constructing the slight threshold comparison table includes:

[0089] The slight threshold comparison table uses the sensor type as the primary key and stores one or more numerical thresholds set for slow offset detection for each environmental quantity, such as temperature, humidity, and combustible gas concentration.

[0090] The method for setting the numerical threshold includes:

[0091] First, during the equipment acceptance phase, the operation and maintenance department conducts long-term zero-load tests on various sensors to measure the noise distribution and instrument drift range under steady-state conditions. Second, at least one month of historical data is collected during the initial operation of the actual tunnel. The noise standard deviation for each time period is robustly estimated. Combined with the resolution and linear error curves provided by the sensor manufacturer, a 95% confidence interval is used to calculate the minimum effective difference that can distinguish "natural fluctuations" from "abnormal slight deviations." Finally, based on the early warning requirements for temperature rise, humidity increase, and gas leakage rate in industry standard documents, the thresholds are conservatively adjusted to a safer level and filled in the threshold fields. The resulting micro-threshold comparison table can be quickly retrieved after deployment on the edge gateway: when the differential sequence comes from a temperature sensor, the system automatically reads the temperature threshold; when it comes from a humidity sensor, the system reads the humidity threshold. This ensures that appropriate and clearly sourced critical standards are used for different physical quantities when determining slight deviations. Furthermore, as the seasons change or equipment ages, the micro-threshold comparison table can be recalculated and updated monthly to ensure that the thresholds remain consistent with actual operating conditions.

[0092] Methods for obtaining the change feature vector include:

[0093] The system starts scanning sequentially from the first bit of the Boolean super-threshold marker sequence, records the starting and ending positions, lengths, and total number of occurrences of continuous "1" segments, and at the end of the scan, selects the longest continuous "1" segment length and the total number of occurrences of all "1" segments; these two statistics are then combined in a fixed order into a set of two-element vectors, which are considered as changing feature vectors.

[0094] Combining these two statistics into a two-element vector in a fixed order means that when generating a change feature vector, the system always first writes the maximum length of a continuous "1" segment in the Boolean overthreshold marker sequence, followed by the total number of occurrences of all "1" segments in the sequence. The position and meaning of these two statistics remain unchanged within any time window. This prescribed order ensures that the first element of the vector directly reflects the maximum duration or depth of the slow-changing anomaly, allowing subsequent algorithms to quickly assess the persistence weight of potential risks during analysis. It also establishes a unified field alignment rule for vectors generated across all time windows, avoiding ambiguity introduced by element swapping during language model training or online inference, ensuring the traceability of feature meanings and the consistency of algorithm results.

[0095] First, a local segment of the trend vector is generated in the dimension of a single sensor, and then the local segments of all sensors are connected end to end within the entire time window to form the final trend vector. Specifically, the system performs the "mean-difference-threshold comparison-statistical feature" process for each physical quantity such as temperature, humidity, and combustible gas, and immediately merges the two quantitative features of "longest continuous 1 length" and "total number of times 1 segment" obtained with the end value of the window mean corresponding to the physical quantity in a fixed field order to generate a local ternary segment. After all physical quantities in this time window have generated their own local ternary segments, the system then splices these segments in sequence according to the preset arrangement of the sensor type, such as the order of temperature-humidity-gas, to obtain a complete trend vector containing nine scalars.

[0096] The trend vector compresses subtle fluctuations scattered across the timeline and sensor network into a complete, fixed-length, and semantically clear numerical signature within the current sliding window. It preserves the latest mean values ​​of each physical quantity while incorporating two dynamic indicators, the length and number of consecutive threshold-exceeding intervals, which are highly sensitive to slow evolution. This eliminates the need for single-point, instantaneous threshold judgments. Instead, the system first processes subtle increments in the original sequence into comparable and accumulative high-order features. These features are then stitched together in a fixed order across different sensors, aligning the change patterns of temperature, humidity, gas, and other dimensions within the same vector space. The subsequent textual trend summary generated by the language model is based on this vector encoding. The language model can quickly identify whether a physical quantity is experiencing long-term, sustained, above-threshold fluctuations or whether multiple quantities are simultaneously exhibiting slight upward trends. Based on this, it can deduce potential common-cause scenarios, such as slow leaks or progressive overloads. In other words, trend vectors condense information across time and space into feature structures that are easy for machines to understand and comparable to each other. They increase sensitivity to small gradual changes and lay a unified semantic foundation for multi-source fusion reasoning, allowing the system to identify risk signs and issue warnings at an early stage when hidden dangers are still within the threshold.

[0097] For example, let's set the sliding window to 10 minutes. A temperature sensor collects 600 data points within this window: 36.1, 36.2, 36.3, ..., 36.5, 36.6, 36.7, 36.8. The system assigns increasing weights to the samples within the window, with higher weights at the end and lower weights at the beginning. For example, a weight of 0.001 is assigned to the beginning and 0.002 to the end. The weighted average of these 600 temperature values ​​yields a final value of 36.5°C for the windowed mean sequence.

[0098] Perform the first-order difference on the mean sequence within the window to obtain the difference sequence:

[0099] [0.1,0.1,0.2,...,0.3,0.3].

[0100] Query the slight threshold comparison table based on the temperature sensor type and obtain the slight threshold as 0.3℃. Figure 5 As shown, the difference sequence is compared with a slight threshold to generate a Boolean over-threshold mark sequence:

[0101] [0,0,0,...,1,0,1,1,1].

[0102] Here, 1 indicates a difference greater than or equal to 0.3, and 0 indicates a difference less than 0.3. Most difference values ​​fall within the 0.1–0.2°C range and do not exceed the threshold. Only the last three consecutive differences reaching 0.3°C are marked as 1. The longest consecutive 1 in this sequence is 3, and the total number of such segments is 2, resulting in the change feature vector [3, 2]. Finally, the change feature vector is concatenated with the terminal value of the windowed mean sequence, 36.5°C, to generate the trend vector [3, 2, 36.5].

[0103] Methods for mapping trend vectors into textual trend summaries include:

[0104] Call the preset field mapping dictionary to map each dimension of the trend vector to the corresponding indicator-value key-value pair.

[0105] Call the pre-built natural language template library to fill each indicator-value key-value pair to generate a sentence fragment.

[0106] The sentence fragments are spliced ​​together in the order of the timestamps generated by each dimension of the trend vector in the data buffer to obtain a textual trend summary.

[0107] Methods for mapping each dimension of the trend vector to corresponding indicator-value key-value pairs include:

[0108] When parsing a trend vector, the system first queries the field mapping dictionary for the same length as the vector. Using array subscripts as search keys, the dictionary provides three pieces of metadata: "physical quantity name, measurement unit, and field meaning." The parser then reads the values ​​from the 0th, 1st, and last digits of the vector, simultaneously invoking the corresponding metadata. Using the "field meaning" as the indicator name, the value itself, along with the unit, is assembled into a string representation. This representation is then written to the key-value buffer in the form of "indicator name → numerical representation." This process ensures that statistics for different physical quantities, such as temperature, humidity, and gas, are accurately labeled at the semantic level, avoiding misalignment caused by changes in the order of vector dimensions.

[0109] The methods for constructing the field mapping dictionary include:

[0110] First, operations engineers determine the meaning and order of each dimension of the trend vector based on the actual sensor types deployed in the tunnel, such as temperature, humidity, and combustible gas. They then create a mapping table entry in the configuration management platform, following the fixed format of "physical quantity name - field meaning - unit of measurement." The vector subscript is then bound to the entry one by one, with subscript zero corresponding to the longest continuous temperature exceeding the threshold, subscript one corresponding to the number of temperature exceeding the threshold, and so on until the last digit. Upon completion, the entire table is exported as a JSON configuration file and distributed to the edge gateway and inference server. This file is then loaded into memory at system startup as a read-only dictionary. If sensor capacity is expanded or dimensionality is adjusted later, the corresponding entry needs to be modified and the trend vector order updated within the same configuration management platform. This update is then synchronized to each node via hot loading to ensure that the field meaning and vector dimension remain consistent.

[0111] The method of filling each indicator-value key-value pair into a generated sentence fragment includes:

[0112] Traverse the key-value cache area and call the template that matches the indicator name in the natural language template library for each record; the template body has reserved a "{value}" placeholder. When the program is running, it fills the placeholder with the numerical value in the key-value pair, and automatically completes the formatting of units and orders of magnitude, and finally outputs independent sentence fragments, such as "The average temperature rose to 36.4℃, and slightly exceeded the threshold a total of 3 times."

[0113] The method for constructing a natural language template library includes:

[0114] Based on each indicator listed in the field mapping dictionary, the operation and maintenance engineer extracts the ternary information of "physical quantity name, statistical dimension, and unit". Then, combined with the operation and maintenance expressions commonly used in the pipeline corridor operation procedures, they draft a sentence prototype that conforms to the technical word order, such as "The average value of {name} rises to {value}{unit}" or "{name} has slightly exceeded the limit for {count} times recently."

[0115] Subsequently, language technicians polished these prototypes for consistency, ensuring consistent terminology and variable placeholder writing, and limited all templates to single-sentence structures for subsequent splicing. Each template uses the indicator name as the search key and the sentence fragment containing the placeholder as the key value. It is finally archived in YAML format and pushed to the edge gateway and inference server. When the system starts, the YAML file is loaded as a resident memory object, and the corresponding sentence structure is instantly detected based on the indicator name using an O(1) hash index. The placeholders are then dynamically replaced during the inference phase to generate the complete text. To support version iteration, the template library maintains version numbers and verification through the configuration management system. Any new indicators or sentence modifications must go through the approval process and be hot-updated to each node to ensure the consistency and traceability of the corpus.

[0116] Methods for splicing sentence fragments in the order of timestamps generated in the data buffer by each dimension of the trend vector include:

[0117] Before generating the trend vector, the system generates a timestamp for each local ternary fragment record of each physical quantity. The textualization stage reads these timestamps and sorts the sentence fragments in chronological order. After the sorting is completed, the fragments are connected in sequence using semicolons or commas to form a smooth and coherent long sentence, which is output as a complete textual trend summary, ensuring that the descriptions of different physical quantities are consistent with the original data stream in terms of time logic, providing coherent and traceable corpus for subsequent language model reasoning.

[0118] Textual trend summaries translate rigid numerical vectors into short, contextual, and causal natural language sentences, enabling subsequent language models to directly understand the subtle changes occurring in various physical quantities and their persistence, much like reading an engineering log. Compared to confronting language models with raw sequences or high-dimensional vectors, text summaries explicitly semantically highlight key information such as "average temperature rise" and "three minor humidity violations," providing clear clues for language models to infer "multiple points rising in the same direction" and "single point sudden fluctuations." When multiple summaries of temperature, humidity, and air quality are stitched together into coherent sentences within the same time window, language models can capture cross-sensor and cross-space correlation patterns and, combining them with visual descriptions of changes, infer potential hazards such as "small water leaks" and "gradual heat sources." As a result, the system no longer relies on single-point alarms based on thresholds. Instead, leveraging linguistic insights into causal relationships, it can identify emerging risks and provide early warnings even before anomalies remain within thresholds and trigger traditional alerts. This fundamentally improves sensitivity and interpretability for slowly evolving and distributed anomalies.

[0119] The model inference module is used to input the textual trend summary into the pre-trained first language model and output the first anomaly inference result with confidence.

[0120] The training method of the first language model includes:

[0121] Constructing a first data set, the first data set including a historical textual trend summary and corresponding pre-labeled anomaly categories and confidence levels, to obtain anomaly corpus;

[0122] Incrementally adjust the general pre-trained language model based on the abnormal corpus and output a first task model as the first language model.

[0123] A method for performing incremental adjustment on a general pre-trained language model based on an abnormal corpus and outputting a first task model as a first language model includes:

[0124] The first language model is based on bidirectional pre-trained language models such as BERT / RoBERTa, and achieves abnormal category inference through single-task fine-tuning.

[0125] Before training begins, the anomaly corpus is randomly divided into a training set and a validation set. The textual trend summary and the corresponding anomaly category label are encoded using a tokenizer consistent with the general pre-trained language model, resulting in a set of input token sequences and supervision signals. The training set is then used as the iteration object. While keeping the original vocabulary unchanged, only the high-level Transformer Block and output head of the first language model are unfrozen. The cross-entropy loss function is used in conjunction with the Adam optimizer for parameter update. At the end of each epoch, the validation set is used to calculate the loss and F1 value. If the validation loss does not decrease significantly in two consecutive rounds, the early stopping strategy is triggered. After all iterations are completed, the best-performing weight snapshot on the validation set is derived and solidified as the first task model. Then, in the inference phase, a new textual trend summary is input to directly generate a first anomaly inference result with confidence from this first task model, laying the decision-making foundation for subsequent visual semantic fusion.

[0126] The first anomaly inference result is a preliminary diagnosis output by the first language model based solely on the textual trend summary and without integrating visual information. It consists of two parts: the most likely category label, selected from a set of pre-defined slowly varying or distributed anomaly categories, indicating the type of hazard most consistent with the data characteristics within the current window; and the numerical confidence level associated with that category label, reflecting the confidence of the first language model in its own judgment. The set of category labels is determined by the operator before the first language model is launched based on historical accidents and operational experience. These typically cover scenarios such as "normal," "slowly rising temperature," "slowly rising humidity," "slowly rising combustible gas," and "slowly rising multi-sources." "Slowly rising multi-sources" refers to situations where multiple physical quantities exhibit synchronized, slight deviations that cannot be explained by a single indicator. The confidence level ranges from zero to one. Values ​​closer to one indicate the first language model's confidence that the category represents the current state. Subsequent threshold checks use this confidence level to determine whether to proceed with the visual fusion and risk output process.

[0127] For example, when the text summary describes "the average temperature rose by 0.8 degrees Celsius within 30 minutes, with three consecutive minor exceeding limits, while the humidity and gas concentrations remained stable," the first language model would give a "temperature rising slowly" label and assign a confidence level of approximately 0.78. If the summary states that "the humidity rose slowly by four percentage points continuously, with five cumulative exceeding limits, and adjacent sensors also showing the same trend," the first language model would be more likely to output a "humidity rising slowly" or "multi-source coordinated rising slowly" label, with a confidence level of 0.71. If the summary only shows slight random fluctuations and none of them touch the threshold, the first language model would return a "normal" label with a confidence level as low as 0.21. Through this structured label-confidence result, the system can qualitatively and quantitatively characterize potential slow-changing hazards early before visual data is involved, laying a semantic and probabilistic foundation for subsequent multimodal fusion and final warning decisions.

[0128] In the inference phase of the first task model, the input is a templated textual trend summary. This summary explicitly describes, in natural language, key information such as the mean value of each physical quantity within the current time window, the length of consecutive cross-threshold occurrences, and the number of cross-threshold occurrences. After encoding by the tokenizer, the first language model maps this token sequence into a multidimensional embedding vector. This vector is then passed through multiple layers of Transformers to gradually capture inter-word dependencies and contextual semantics, ultimately generating a probability distribution over all predefined anomaly categories at the output. The first language model uses the category with the highest probability as the first anomaly inference result, and uses this highest probability value as the confidence level. Essentially, the relationship between input and output is a semantic mapping from "descriptive language" to "anomaly category and confidence level." The more the input text contains vocabulary and sentence structures strongly associated with a particular anomaly pattern, the more the attention weights learned during training by the first language model will tend to activate high-probability outputs for the corresponding category. Conversely, if the input only contains slight temperature fluctuations and lacks collaborative information such as humidity and gas, the first language model is more likely to judge it as "normal" or output a low confidence level. Specifically speaking of the impact of changes, when the numerical value in the text is modified from "increased by 0.5℃" to "increased by 2.5℃", or "continuously exceeded the limit once" is changed to "continuously exceeded the limit five times", such increases in key numerical values ​​and frequencies will significantly change the numerical entity words in the token sequence, generating new semantic shifts in the embedding space; the attention within the first language model is then redistributed, resulting in an increase in the output probability of the corresponding high-risk category and an increase in confidence. On the contrary, if "increase" is replaced by "decrease" or "stable", the activation of the first language model for the abnormal category will be suppressed, and the output will tend to "no abnormality" or low confidence. Therefore, the vocabulary, numerical values, and sentence details of the input text directly determine the semantic direction of the first language model embedding vector, which in turn affects the final abnormal category selection and confidence level, reflecting the one-to-one mapping and significant impact of input changes on the output results.

[0129] The visual difference module is used to perform difference calculation on the surveillance image sequence of the same time window to obtain a difference image sequence, and input the obtained difference image sequence into the pre-trained visual semantic model to generate a visual change description.

[0130] Methods for obtaining a differential image sequence include:

[0131] The surveillance image sequences within the same time window are rearranged in ascending order of timestamps to obtain an ordered image sequence.

[0132] Adjacent frames in the ordered image sequence are paired with each other to obtain a set of frame pairs.

[0133] The absolute difference is calculated pixel by pixel for each pair of adjacent frames in the frame pair set to generate the corresponding single-frame difference image.

[0134] All single-frame differential images are stitched together in the order of timestamps to obtain a differential image sequence.

[0135] For example, consider a ten-second time window where the camera captures eight frames at one frame per second. Due to network jitter, the timestamps that fall into the cache are 00:03, 00:01, 00:07, 00:06, 00:02, 00:05, 00:00, and 00:04 seconds. The system first reads these eight frames and reorders them by timestamp from smallest to largest, resulting in an ordered image sequence F0-F7. The corresponding timestamps are returned to the order of 00:00, 00:01, 00:02, 00:03, 00:04, 00:05, 00:06, and 00:07 seconds.

[0136] The program then groups adjacent frames into pairs in the order 〈F0, F1〉, 〈F1, F2〉, and so on to 〈F6, F7〉. It then aligns pixel coordinates within each frame pair and performs a |p1–p2| operation on the corresponding pixel grayscales to form single-frame differential images D0-D6. D3, due to the large-scale fluctuation in pixel grayscale caused by the passage of a human body between F3 and F4, appears as a brighter differential region than D0-D2.

[0137] Finally, the system splices D0-D6 in sequence according to the timestamp order of the frame pair starting frame and writes them into the differential image buffer in sequence, thus obtaining a differential image sequence corresponding to the ten-second time window; the differential image sequence not only retains the temporal integrity of once per second, but also quantifies all visual changes in consecutive frames in the form of absolute pixel differences, laying an accurate and coherent data foundation for the subsequent visual semantic model to extract spatial-temporal change features.

[0138] See also Figure 3 As shown in Figure 2, the training method of the visual semantic model includes:

[0139] S101: Collect historical data including differential image sequences and their corresponding manually annotated visual change descriptions to construct a training dataset.

[0140] S102: Input the training dataset into the initialized semantic segmentation network to obtain a predicted visual change description.

[0141] S103: Calculate the loss function between the predicted visual change description and the manually annotated visual change description to obtain a loss value.

[0142] S104: Perform back propagation on the semantic segmentation network according to the loss value and update the parameters to obtain an updated semantic segmentation network.

[0143] S105: Repeat steps S102 to S104 until the loss value meets the convergence condition, and obtain a converged visual semantic model.

[0144] The visual change description is a structured semantic information output by the visual semantic model after receiving the differential image sequence, as follows:

[0145] First, the description of visual changes is presented in the form of natural language short sentences or equivalent key-value pairs, mapping the visual anomalies detected in the inter-frame pixel difference into readable and parsable semantic entries; second, the description content includes at least the visual change type, spatial location, change intensity and amplitude, change trend and corresponding timestamp, where the change type is used to indicate the anomaly category, the spatial location locates the abnormal area with pixel box or tunnel segment coordinates, the change intensity and amplitude quantifies the scale of the anomaly by area, brightness or temperature, the change trend identifies the direction in which the anomaly strengthens, weakens or remains stable over time, and the timestamp records the corresponding sampling moment; third, all fields use fixed field names and unified units, so that the downstream second language model can directly cross-validate the sensor data trend at the semantic level, thereby improving the confidence of anomaly inference; finally, the description can be persistently stored after generation to support traceability analysis and continuous learning of the visual semantic model. Its structured properties ensure efficient data exchange and semantic consistency with other modules, forming a key bridge for multimodal spatiotemporal data fusion in the embodiment of the present invention.

[0146] The initialized semantic segmentation network can adopt the industry-wide DeepLabV3+ architecture and load the ResNet-50 backbone weights pre-trained on ImageNet. After initialization, its output layer is reset to two channels: foreground changes and background stability.

[0147] The loss function between the predicted and manually annotated visual change descriptions is a linear combination of weighted cross-entropy and Dice loss. The weighted cross-entropy emphasizes pixel-level classification accuracy, while the Dice loss emphasizes region overlap. The sum of the two is the loss function for a single forward pass. After multiple rounds of gradient backpropagation and continuous loss monitoring on the validation set, convergence is considered achieved when the validation loss decreases by less than 1% within three consecutive iterations. The resulting semantic segmentation network is the converged visual semanticization model.

[0148] For example, taking a differential image as input, the visual semantic model first generates a probability heat map on the deep feature map, then restores it to the original resolution through upsampling, and then applies a 0.5 threshold to the probability heat map to obtain a binary mask; the system calculates the connected domain based on the mask and outputs the structured text "Change type: water stain diffusion, location: S3-East section, roi: [212,180,368,396], area_m2: 0.25, trend: increasing, timestamp: 2025-06-28T09:42:00", which is the predicted visual change description.

[0149] The differential image sequence is pixel-level evidence that explicitly displays subtle changes in brightness, color or texture of the monitored image within the same time window through absolute pixel differences frame by frame. It retains the spatial distribution and evolution trajectory of the changes in chronological order. The visual semantic model inputs this sequence frame by frame during inference, extracts multi-scale features through the encoding-decoding network, and calculates a temporally consistent probability heat map. It then analyzes the connected domain, area, intensity and position, and finally generates a visual change description containing fields such as change type, spatial position, amplitude, trend and timestamp. Therefore, the two represent a mapping relationship from low-level pixel differences to high-level semantic labels: if the area of ​​brightness change in the differential image sequence increases or the differential region in consecutive frames continues to expand, the visual semantic model captures more significant feature responses in the convolutional and temporal features, and the output change type changes from "no significant change" to "water stain diffusion" or "smoke accumulation." The change amplitude field amplifies synchronously with the area or grayscale increment, and the trend field changes from "stable" to "increasing" with the coherence reinforcement in the temporal dimension. Conversely, if the input difference is merely isolated noise and lacks spatiotemporal coherence, the visual semantic model's attention struggles to form significant patterns, and its output type tends to be "background stable" with reduced amplitude and confidence. Therefore, the strength, morphology, and coherence of the input differential image sequence at the pixel and temporal levels directly determine the values ​​of elements such as category, position, amplitude, and trend in the visual change description, enabling sensitive semantic representation of slowly evolving or distributed visual hazards.

[0150] The fusion evaluation module is used to align and splice the first anomaly inference result and the visual change description according to the timestamp, input the pre-trained second language model, and obtain a comprehensive risk hypothesis including cause-location-risk level.

[0151] The training methods for the second language model include:

[0152] Constructing a second data set, the second data set including the concatenated text of the historical first anomaly inference results and the visual change descriptions and their manually annotated causes, locations, and risk levels to obtain a fused corpus;

[0153] A multi-task learning approach is used to continue adjusting the first task model on the fused corpus, and the fused task model is output as the second language model.

[0154] Methods for using multi-task learning to further adjust the first task model on the fused corpus include:

[0155] The second language model uses the same bidirectional model framework as the first language model, and is expanded into a comprehensive cause-location-risk level reasoning engine through multi-task learning, supporting cross-modal semantic fusion.

[0156] First, the fused corpus is randomly divided into a training subset and a validation subset, and the same tokenizer used in the first task model is used to encode the "first anomaly inference result and the visual change description spliced ​​text" into a token sequence.

[0157] Then, while keeping the vocabulary unchanged, only the top two TransformerBlock layers of the second language model and the newly added task output head are unfrozen: this path has three parallel channels, which output results for "cause classification", "location classification" and "risk level regression" respectively.

[0158] During training, the second language model forward-calculates three types of predictions for the same input text at one time; the system synchronously calculates three-way losses, where the cause and location use cross-entropy loss, and the risk level uses mean square error loss. The three are weighted and summed according to the empirical coefficient to obtain the overall loss and back-propagate to update the unfreezing parameters.

[0159] The empirical coefficient is determined using a data-driven calibration process, as follows:

[0160] First, before training begins, the original loss amplitudes of the three subtasks in the fusion corpus are counted. It is found that the cross entropy of cause classification and location classification is approximately between one and two, and the mean square error of risk level regression is usually less than 0.1. In order to eliminate the gradient skew caused by dimensional differences, the system first divides the three original losses by their average values ​​in the first round of iterations to make them of the same order of magnitude; then, a simple grid search is performed on this normalization basis, limiting the combination of weighted coefficients to the constraint that the sum of cause, location, and risk level is equal to one, and traversing all possible combinations with a step size of 0.1. Each time a set of coefficients is tried, a full training cycle is completed on the validation subset and the comprehensive indicators are recorded. Finally, the set of coefficients that can maximize the macro-average F1 value and the weighted mean square error score of the validation set and converge the fastest in the training process is selected and written into the configuration file. For example, after testing more than 50 combinations, the configurations of 0.45 for cause, 0.35 for location, and 0.2 for risk level performed best in most verification folds. They were then solidified as global empirical coefficients and saved along with the weights to ensure that the contributions of the three tasks to the overall loss are both balanced and meet the actual needs of accident judgment.

[0161] After each training round, the weighted loss and each single-task indicator are evaluated on the validation subset. If the weighted loss decreases by less than 1% in two consecutive rounds, early stopping is triggered to avoid overfitting.

[0162] At the end of the iteration, the weighted snapshot with the best comprehensive indicators in the verification phase is selected as the fusion task model and solidified. The fusion task model can directly receive the new spliced ​​text in the online reasoning, and jointly output the cause label, location label and risk level score, thereby completing the upgrade of the first task model to a multimodal decision engine, providing high-confidence multi-dimensional information support for the final comprehensive risk hypothesis.

[0163] The comprehensive risk hypothesis consists of three components: first, the cause, which is the potential fault type inferred by the second language model based on multi-source trends, such as "slow water leak," "continuous overload heating," or "slow rise of combustible gas." Second, the location, which is the second language model's spatial orientation to the area where the anomaly is most likely to occur. This can be represented by the tunnel segment number, cabin name, or ROI coordinates in the camera's field of view. Third, the risk level, which uses a 0-1 floating point or "five-level" number to reflect the urgency of the accident and potential losses. The second language model encapsulates these three pieces of information into structured short sentences or key-value pairs, which can be presented directly to on-duty personnel and can also be used by intelligent linkage logic.

[0164] For example, suppose the first anomaly inference result indicates "slow temperature rise with a confidence level of 0.72," and the visual change description shows "The thermal image of the wall temperature in the S3-East section shows a 0.3m² hot spot spreading and increasing." The second language model concatenates these two texts and inputs them. In the context, it recognizes that the multi-source, co-directional temperature rise aligns with the location of the hot spot, and outputs the cause as "progressive overload heating," the location as "S3-East section," and the risk level as 0.64. If the visual description changes to "spreading water stains on the wall" and the first anomaly inference result remains a slow temperature rise, the second language model reassesses the cause and may change the cause to "slow water leakage with localized thermal-humidity coupling," lowering the risk level to 0.48, as the explosion risk of water seepage alone is relatively low. If the concatenated text increases the confidence level of the slow temperature rise to 0.9 and the humidity rises simultaneously, the second language model will receive a stronger resonance signal in the hidden layer and output the cause as "cable overload with damp insulation," maintaining the location as "S3-East section," and increasing the risk level to 0.82.

[0165] As can be seen, the input to the second language model is a concatenation of two pieces of semantic information: the first anomaly inference result summarizes the numerical trend, and the visual change description summarizes the visual signs. The output is a set of three-dimensional decision vectors for the comprehensive risk hypothesis. The numerical magnitude, trend terms, spatial nouns, and confidence levels in the input text all form different weight distributions in the embedding space, driving the second language model's attention to generate corresponding probability fluctuations in the three channels of "cause," "location," and "risk level." When the input indicates consistency and continuity across multiple sources of data, and the magnitude increases, the cause is more likely to be classified as high-risk, and the risk level increases. When the trends conflict or the magnitude is small, the second language model weakens the urgency assessment and lowers the risk. Therefore, any input change, whether it is a numerical enhancement, a direction reversal, or a position replacement, will cause the cause category to switch, the location coordinates to update, and the risk level to increase or decrease at the output, enabling real-time, semantic, and quantitative response to slowly evolving and distributed hazards.

[0166] The early warning output module is used to output early warning information if the confidence level of the comprehensive risk hypothesis is higher than a preset confidence threshold.

[0167] The confidence level of the comprehensive risk hypothesis comes from the quantification of the reliability of the second language model's own judgment during reasoning after multi-task fine-tuning, as follows:

[0168] After receiving the concatenated text of the first anomaly inference result and the visual change description, the second language model simultaneously outputs the cause category distribution, location distribution, and a continuous value for the risk level. The soft-max layer of each of the cause and location channels generates a set of normalized probabilities, where the maximum probability corresponds to the second language model's confidence in the label. The risk level channel directly generates a real-number score between 0 and 1.

[0169] The maximum probability of the cause, the maximum probability of the location, and the normalized risk level are weighted averaged using empirical weights. This is then calibrated using a temperature curve on the validation set to ensure that the weighted result most closely matches the binary label of "whether the risk was ultimately confirmed on-site." This resulting single confidence level combines the consistency of information from multiple sources and takes into account the severity of the accident. The closer the value is to 1, the more confident the second language model is that the current comprehensive risk hypothesis is true.

[0170] The empirical weights are determined through a data-driven calibration process based on the differences in the explanatory power of "causal confidence," "location confidence," and "risk level" for the final accident confirmation results on a historical validation set. After adjusting the second language model, researchers first calculated the mutual information (MI) of each of the three indicators with respect to the binary label "whether the accident actually occurred" on a set of validation samples with known conclusions. In real-world data, causal confidence, which directly maps to the consistency between sensor trends and visual signs, had the highest MI, followed by location confidence. While risk level is important, it is not linearly correlated with accident confirmation and has the lowest MI. Based on these results, a grid search was used to set multiple candidate weights for the three indicators in the 0–1 range and force them to add to 1. The AUC was then calculated for each weighted average of the three indicators and the accident authenticity label. The weighted combination that achieved the highest AUC was selected as the empirical weight.

[0171] For example, when the cause confidence, location confidence, and normalized risk level are weighted at a ratio of 0.45:0.35:0.20, the system achieves the highest discrimination for real accidents. This weighting maintains a stable advantage during subsequent rolling validations, so it is configured as the global default. This setting logic balances quantitative data-driven approaches with common sense about accident mechanisms: cause confidence contributes most to risk authenticity, followed by location confidence, and risk level serves as a controllable metric for supplementary reference. Ultimately, this ensures that the overall confidence reflects the consistency of multi-source data while not overly amplifying the noise of a single path.

[0172] The preset methods for confidence thresholds include:

[0173] Calculate the confidence of historical comprehensive risk assumptions to obtain a confidence sample set.

[0174] The receiver operating characteristic curve is calculated based on the confidence sample set to obtain the receiver operating characteristic curve data.

[0175] The confidence value with the largest Youden index was selected from the receiver operating characteristic curve data and set as the confidence threshold.

[0176] Methods for obtaining receiver operating characteristic curve data include:

[0177] First, the constructed confidence sample set is read. The confidence sample set is the confidence value of each record in the historical comprehensive risk hypothesis and its corresponding binary manual label of "whether the risk is finally confirmed to exist"; then the sample set is completely sorted from high to low according to the confidence, and each confidence value is selected as the critical point in the sorting sequence in turn. Records greater than or equal to the critical point are marked as positive examples, and the rest are marked as negative examples. The true positive rate and false positive rate are counted at a single critical point; the critical point is continuously slid along the sorting sequence and the same statistical process is repeated. The resulting set of true positive rate-false positive rate coordinate pairs are sequentially written into the memory buffer. After completion, the receiver operating characteristic curve data is obtained.

[0178] Methods for selecting the maximum confidence value of the Youden index from the receiver operating characteristic curve data and setting it as the confidence threshold include:

[0179] Traverse each coordinate pair in the receiver operating characteristic curve data, read the true positive rate and false positive rate, and instantly calculate the difference between the two to obtain the corresponding Youden index; during the traversal process, continuously compare the current Youden index with the maximum value saved before the iteration, and when it is found that the new difference is larger, the maximum value and its corresponding critical point confidence are updated synchronously; at the end of the traversal, the critical point confidence saved in the register is the global optimal value of the Youden index, which is written into the configuration file and declared as the global confidence threshold, so that it can be used in the online inference stage to determine whether the comprehensive risk hypothesis triggers an early warning.

[0180] See also Figure 4 As shown in the figure, early warning information is a structured notification pushed to the operation and maintenance end when the confidence level of the comprehensive risk hypothesis exceeds the threshold. It contains at least five elements: the first is the cause of the risk, which uses a brief text to indicate the potential accident type inferred by the second language model; the second is spatial positioning, which gives the specific section of the corridor, cabin or camera field of view coordinates; the third is the risk level, which uses a "five-level system" or a 0-1 floating point to indicate the urgency of the situation; the fourth is the confidence level, which is used to illustrate the reliability of the second language model's own inference; and the fifth is the response suggestion, which provides the on-duty personnel with operational instructions for the next inspection or linkage action. For example, when the system detects a simultaneous slow rise in temperature and humidity and visually shows water stains spreading on the wall, it will send a message: "Warning: S3-Eastern section suspected of slow leakage - thermal and moisture coupling, risk level 0.78, confidence level 0.83, it is recommended to immediately send personnel to check the valves in this section and perform insulation tests on adjacent cables." In this way, operation and maintenance personnel no longer need to browse various curves and alarm lists, and can know the type of accident, exact location and response plan before the situation is still in its infancy, greatly reducing the probability of slowly evolving hidden dangers escalating into major failures.

[0181] Example 2

[0182] See also Figure 2As shown, this embodiment provides an integrated pipe gallery operation and maintenance method based on an AI large model, including:

[0183] Obtain the continuous time series data generated by the distributed environmental sensors in the tunnel and the monitoring image sequence of the corresponding time period;

[0184] Calculate the change feature vector of continuous time series data within a preset sliding time window to form a trend vector, and map the trend vector into a textual trend summary;

[0185] Input the textual trend summary into the pre-trained first language model and output the first anomaly inference result with confidence;

[0186] Perform differential calculation on the surveillance image sequence of the same time window to obtain a differential image sequence, and input the obtained differential image sequence into the pre-trained visual semantic model to generate a visual change description;

[0187] The first anomaly inference result and the visual change description are aligned and spliced ​​according to the timestamp, and then input into the pre-trained second language model to obtain a comprehensive risk hypothesis including cause, location, and risk level;

[0188] If the confidence level of the comprehensive risk hypothesis is higher than the preset confidence threshold, early warning information will be output.

[0189] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0190] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A comprehensive pipe gallery operation and maintenance method based on AI large model, characterized by: include: Obtain the continuous time series data generated by the distributed environmental sensors in the tunnel and the monitoring image sequence of the corresponding time period; Calculate the change feature vector of continuous time series data within a preset sliding time window to form a trend vector, and map the trend vector into a textual trend summary; Input the textual trend summary into the pre-trained first language model and output the first anomaly inference result with confidence; Perform differential calculation on the surveillance image sequence of the same time window to obtain a differential image sequence, and input the obtained differential image sequence into the pre-trained visual semantic model to generate a visual change description; The first anomaly inference result and the visual change description are aligned and spliced ​​according to the timestamp, and then input into the pre-trained second language model to obtain a comprehensive risk hypothesis including cause, location, and risk level; If the confidence level of the comprehensive risk hypothesis is higher than the preset confidence threshold, early warning information will be output.

2. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: Methods for acquiring continuous time series data and monitoring image sequences include: Align and combine the continuously sampled temperature data series, humidity data series, and gas concentration data series according to timestamps to obtain continuous time series data; The camera frame images are captured at a preset frame rate to obtain single-frame images; the single-frame images are spliced ​​in chronological order to obtain a monitoring image sequence.

3. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 2 is characterized in that: Methods for forming trend vectors include: Calculate the weighted average of continuous time series data within the preset sliding time window to obtain the mean sequence within the window; Perform the first-order difference on the mean sequence within the window to obtain the difference sequence; The pre-built slight threshold comparison table is queried based on the sensing type to obtain the slight threshold, the difference sequence is compared with the slight threshold element by element, and a Boolean over-threshold mark sequence is output; The longest length and total number of consecutive segments with a value of 1 in the super-threshold marker sequence are counted to obtain the change feature vector; The change feature vector is concatenated with the terminal value of the mean sequence within the window to generate a trend vector.

4. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: Methods for mapping trend vectors into textual trend summaries include: Call the preset field mapping dictionary to map each dimension of the trend vector to the corresponding indicator-value key-value pair; Call the pre-built natural language template library to fill each indicator-value key-value pair to generate a sentence fragment; The sentence fragments are spliced ​​together in the order of the timestamps generated by each dimension of the trend vector in the data buffer to obtain a textual trend summary.

5. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: The training methods for the first language model and the second language model include: Constructing a first data set, the first data set including a historical textual trend summary and corresponding pre-labeled anomaly categories and confidence levels, to obtain anomaly corpus; Perform incremental adjustments on the general pre-trained language model based on the abnormal corpus, and output a first task model as the first language model; Constructing a second data set, the second data set including the concatenated text of the historical first anomaly inference results and the visual change descriptions and their manually annotated causes, locations, and risk levels to obtain a fused corpus; A multi-task learning approach is used to continue adjusting the first task model on the fused corpus, and the fused task model is output as the second language model.

6. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: Methods for obtaining a differential image sequence include: The surveillance image sequence within the same time window is rearranged in ascending order of timestamps to obtain an ordered image sequence; Pair adjacent frames of the ordered image sequence to obtain a set of frame pairs; Calculate the absolute difference pixel by pixel for each pair of adjacent frames in the frame pair set to generate the corresponding single-frame difference image; All single-frame differential images are stitched together in the order of timestamps to obtain a differential image sequence.

7. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: The training methods for visual semantic models include: S101: Collect historical data including differential image sequences and their corresponding pre-labeled visual change descriptions to construct a training dataset; S102: Input the training dataset into the initialized semantic segmentation network to obtain a predicted visual change description; S103: Calculating a loss function between the predicted visual change description and the pre-labeled visual change description to obtain a loss value; S104: Perform back propagation on the semantic segmentation network according to the loss value and update the parameters to obtain an updated semantic segmentation network; S105: Repeat steps S102 to S104 until the loss value meets the convergence condition, and obtain a converged visual semantic model.

8. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: The preset methods for confidence thresholds include: Calculate the confidence of historical comprehensive risk assumptions to obtain a confidence sample set; Calculate the receiver operating characteristic curve according to the confidence sample set to obtain receiver operating characteristic curve data; The confidence value with the largest Youden index was selected from the receiver operating characteristic curve data and set as the confidence threshold.

9. The integrated pipe gallery operation and maintenance method based on AI large model according to claim 1 is characterized in that: The early warning information shall include at least the following elements: The first is the risk cause, which is to use short text to indicate the potential accident type inferred by the second language model; The second is spatial positioning, which is to give the coordinates of the specific sections, cabins or camera fields of view of the corridor; the third is the risk level, which is to use a five-level system or a 0-1 floating point to indicate the urgency of the situation; the fourth is the confidence level, which is to explain the reliability of the second language model's own inference; the fifth is the response suggestion, which is to provide the on-duty personnel with the operational instructions for the next inspection or linkage action.

10. An integrated pipe gallery operation and maintenance system based on an AI large model, used to implement an integrated pipe gallery operation and maintenance method based on an AI large model as described in any one of claims 1 to 9, characterized in that: include: The sensor acquisition module is used to obtain the continuous time series data generated by the distributed environmental sensors in the pipeline corridor and the monitoring image sequence of the corresponding time period; The trend summary module is used to calculate the change feature vector of continuous time series data within a preset sliding time window, form a trend vector, and map the trend vector into a textual trend summary; A model inference module is used to input the textual trend summary into the pre-trained first language model and output a first anomaly inference result with confidence; The visual difference module is used to perform difference calculation on the surveillance image sequence of the same time window to obtain a difference image sequence, and then input the obtained difference image sequence into the pre-trained visual semantic model to generate a visual change description; The fusion assessment module is used to align and concatenate the first anomaly inference result and the visual change description according to the timestamp, input the result into the pre-trained second language model, and obtain a comprehensive risk hypothesis including cause, location, and risk level; The early warning output module is used to output early warning information if the confidence level of the comprehensive risk hypothesis is higher than a preset confidence threshold.

Images