A system for health monitoring, intelligent prediction and early warning disposal of ancient buildings

Abstract

The application discloses an ancient building health monitoring, intelligent prediction and early warning treatment system, which comprises a sensor module, an AI-PLC cooperation module, an early warning module and a treatment module, the AI-PLC cooperation module comprises an AI module and a PLC module connected through a network cable; the PLC module is configured to generate an execution command according to a prediction result obtained from the AI module, and send the execution command to the treatment module, generate an alarm command according to the prediction result, and send the alarm command to the early warning module; the AI module is configured to obtain collected data from the sensor module, obtain a prediction result according to the collected data obtained from the sensor module, an LSTM algorithm and an InceptionV3 model; the AI module comprises a sliding window length calculation module, a learning rate calculation module, a fire image recognition module and a fire prediction and judgment module. The AI module cooperates with the PLC module to make decisions, generate a prediction result and an execution command, drive the treatment module to work, and the early warning module to give an early warning.

Description

Technical Field

[0001] This invention relates to the field of ancient building health assessment technology, and in particular to an ancient building health monitoring, intelligent prediction and early warning system. Background Technology

[0002] Taking ancient architecture as an example, ancient buildings carry and transmit precious historical, artistic, and scientific values. They are the continuation of China's five-thousand-year-old civilization and a solid cornerstone for demonstrating cultural confidence. my country has a large number of ancient architectural heritage sites, with a total of 5,058 designated key cultural relics protection units, including approximately 1,500 wooden structures. These wooden structures, having endured hundreds or even thousands of years, face increasingly severe threats from material aging, biological erosion, and natural disasters such as earthquakes. They also face threats from human factors such as fires, rising carbon dioxide concentrations, and increased stress on load-bearing components. Therefore, it is necessary to construct a safety monitoring, intelligent early warning, and response system for ancient buildings.

[0003] Ancient buildings are precious cultural heritages that carry historical, artistic, and scientific value. However, ancient buildings are threatened by various natural and man-made factors such as material aging, biological erosion, earthquakes, and fires after years of erosion; in particular, wooden ancient buildings are particularly vulnerable to fire risks and structural health problems.

[0004] A search revealed a Chinese patent with publication number CN121350470A, which discloses a real-time assessment system for the structural health of ancient buildings based on digital twins. The system includes: a first acquisition module, used to acquire point cloud data of the external space of the ancient building based on 3D laser scanning, UAV aerial photography, and close-range measurement technology, and to construct a digital model of the overall space of the ancient building using reverse geometric reconstruction and BIM model conversion technology; a second acquisition module, used to deploy a sensor network at key parts of the ancient building to collect physical parameter data such as strain, displacement, tilt, temperature, and humidity in real time, and to acquire material property data and environmental data; a construction module, used to construct a digital twin model by combining the digital model of the overall space of the ancient building, material property data, and environmental data; an analysis module, used to analyze the acquired data using deep learning algorithms to achieve anomaly detection, damage identification, and predict the remaining lifespan of the structure based on historical data; and an assessment and early warning module, used to perform real-time assessment and early warning of the structural health status of the ancient building based on the dynamic coupling of real-time monitoring data and the digital twin model.

[0005] However, the above-mentioned technical solutions still have at least the following drawbacks: low early warning accuracy, insufficient closed-loop control from monitoring to disposal, and difficulty in meeting the monitoring and early warning needs of the ancient architecture field. Summary of the Invention

[0006] In view of the above situation and to overcome the defects of the existing technology, the present invention provides a monitoring and early warning system for the health status of ancient buildings, including a sensor module, a treatment module, an early warning module, and also includes an AI-PLC collaborative module, wherein the AI-PLC collaborative module includes an AI module and a PLC module connected by a network cable;

[0007] The PLC module is configured to generate an execution command based on the prediction results obtained from the AI ​​module, and send the execution command to the processing module; and to generate an alarm command based on the prediction results, and send the alarm command to the early warning module.

[0008] The AI ​​module is configured to acquire collected data from the sensor module and obtain prediction results based on the collected data acquired from the sensor module, the LSTM algorithm, and the InceptionV3 model.

[0009] The AI ​​module includes: a sliding window length calculation module, a learning rate calculation module, a fire image recognition module, and a fire prediction and judgment module.

[0010] More preferably, the collected data includes: the natural frequency range of the ancient building's wooden structure, the sensor sampling frequency range, and image data;

[0011] The sliding window length calculation module is configured to determine the candidate range of window length based on the obtained natural frequency range of the ancient wooden structure and the sampling frequency range of the sensor.

[0012] Obtain the monitoring samples and sample lengths of the sensor, and obtain the initial window length based on the monitoring samples, the sample lengths, and the candidate window length range;

[0013] The sliding window length is obtained based on the window length candidate range and the initial window length.

[0014] Further preferred, based on the obtained natural frequency range of the ancient wooden structure and the sensor sampling frequency range, a candidate range for the window length is determined, including:

[0015] Based on the range of natural frequencies of the ancient wooden structure, the range of natural periods of the ancient wooden structure is obtained.

[0016] Obtain the minimum complete learning natural vibration period range of the AI ​​module, and obtain the minimum physical duration range based on the minimum complete learning natural vibration period range and the natural vibration period range of the ancient wooden structure.

[0017] The time step range is obtained based on the sensor sampling frequency range and the minimum physical duration range;

[0018] Expanding the range of time steps yields the candidate range of window length.

[0019] More preferably, the initial window length is obtained based on the monitored sample, sample length, and window length candidate range, including:

[0020] Based on the autocovariance formula, the monitored sample, and the sample length, the autocovariance of the lag-order sample and the autocovariance of the non-lag-order sample are obtained.

[0021] Based on the autocovariance of the lag-order samples and the autocovariance of the unlag-order samples, the sample autocorrelation coefficient is obtained;

[0022] The initial window length is obtained based on the sample autocorrelation coefficient and the first threshold.

[0023] More preferably, the initial window length is obtained based on the sample autocorrelation coefficient and the first threshold, including:

[0024] The sample autocorrelation coefficients with different lag orders are compared one by one with the first threshold. When the absolute value of the sample autocorrelation coefficient is less than the first threshold, the value of the smallest lag order is the initial window length.

[0025] More preferably, the sliding window length is obtained based on the candidate window length range and the initial window length, including:

[0026] The initial window length is compared with the window length candidate range. If the value of the initial window length is less than the minimum value in the window length candidate range, the sliding window length is the minimum value in the window length candidate range.

[0027] If the initial window length is greater than the maximum value in the candidate window length range, the sliding window length is the maximum value in the candidate window length range;

[0028] If the value of the initial window length is within the range of window length options, the sliding window length is the initial window length.

[0029] More preferably, the learning rate calculation module is configured to: determine the effective learning rate range based on the collected data, the initial maximum learning rate, and the minimum effective learning rate;

[0030] The learning rate for each training round is dynamically set according to the learning rate decay formula and the effective learning rate range.

[0031] After each fixed training cycle, the learning rate value will be reset to the initial maximum learning rate;

[0032] The learning rate decay formula is shown in equation (1).

[0033] (1);

[0034] in, The learning rate for the current training epoch. The initial maximum learning rate, To minimize the effective learning rate, This is the current training round number. This represents the total number of training rounds.

[0035] More preferably, the fire image recognition module is configured to: divide the image data into fire and no fire, and store the two types of images containing fire and no fire in two subfolders of the training set and the validation set, respectively;

[0036] Traverse the entire training set data, count the number of image samples in the two categories of "with fire" and "without fire", and obtain the number of image samples in the "with fire" category, the number of image samples in the "without fire" category, and the total number of samples.

[0037] Weights are assigned to image samples in the "on fire" and "off fire" categories based on the number of image samples in each category.

[0038] More preferably, the fire prediction and judgment module is configured to: divide the image data into several datasets according to the number of consecutive frames;

[0039] The fire prediction and judgment module is further configured to output a fire probability value for each frame of image using the InceptionV3 model, and to determine whether a fire has occurred based on the comparison result of the fire probability value with preset warning thresholds and alarm thresholds.

[0040] More preferably, the sensor module includes at least one of an accelerometer, a tilt sensor, a displacement sensor, a strain sensor, a video acquisition device, a smoke sensor, a carbon monoxide concentration detector, a carbon dioxide concentration detector, and a temperature and humidity sensor;

[0041] The treatment module includes a water pump, a spray system, a fan, pipelines, and an AMD control system;

[0042] The early warning module includes an alarm;

[0043] Both the processing module and the early warning module are electrically connected to the PLC module, and the sensor module is electrically connected to the AI ​​module.

[0044] Compared to existing technologies, this invention, by setting up an AI-PLC collaborative module, utilizes the LSTM algorithm to accurately capture changes in structural stress states or environmental changes through sensors to predict potential risks. It also uses the InceptionV3 model to analyze image data to identify fire hazards, thus fusing multimodal data and improving the accuracy and timeliness of early warnings. Simultaneously, by directly collaborating the AI ​​module with the PLC module for decision-making, it generates predictive results and executes commands to drive the remedial action module and the early warning module to issue warnings. Attached Figure Description

[0045] Figure 1 This is a topology diagram of a health monitoring, intelligent prediction and early warning system for ancient buildings according to the present invention;

[0046] Figure 2 This is a flowchart illustrating the sliding window length calculation module of the present invention;

[0047] Figure 3 This is a flowchart illustrating the learning rate calculation module of the present invention;

[0048] Figure 4 This is a flowchart illustrating the fire image recognition module of the present invention;

[0049] Figure 5 This is a flowchart illustrating the fire prediction and judgment module of the present invention;

[0050] Figure 6 This is a schematic diagram illustrating the results of fire identification and early warning in an embodiment of the present invention;

[0051] Figure 7 This is a schematic diagram of the results of structural health monitoring and prediction in an embodiment of the present invention;

[0052] Figure 8 These are screenshots of the interface of the software system in an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] To facilitate understanding of the methods and systems provided in the embodiments of this application, the background of the embodiments of this application will be introduced before introducing the embodiments of this application.

[0055] Taking ancient architecture as an example, ancient buildings carry and transmit precious historical, artistic, and scientific values. They are the continuation of China's five-thousand-year-old civilization and a solid cornerstone for demonstrating cultural confidence. my country has a large number of ancient architectural heritage sites, with a total of 5,058 designated key cultural relics protection units, including approximately 1,500 wooden structures. These wooden structures, having endured hundreds or even thousands of years, face increasingly severe threats from material aging, biological erosion, and natural disasters such as earthquakes. They also face threats from human factors such as fires, rising carbon dioxide concentrations, and increased stress on load-bearing components. Therefore, it is necessary to construct a safety monitoring, intelligent early warning, and response system for ancient buildings.

[0056] Ancient buildings are precious cultural heritages that carry historical, artistic, and scientific value. However, ancient buildings are threatened by various natural and man-made factors such as material aging, biological erosion, earthquakes, and fires after years of erosion; in particular, wooden ancient buildings are particularly vulnerable to fire risks and structural health problems.

[0057] A search revealed a Chinese patent with publication number CN121350470A, which discloses a real-time assessment system for the structural health of ancient buildings based on digital twins. The system includes: a first acquisition module, used to acquire point cloud data of the external space of the ancient building based on 3D laser scanning, UAV aerial photography, and close-range measurement technology, and to construct a digital model of the overall space of the ancient building using reverse geometric reconstruction and BIM model conversion technology; a second acquisition module, used to deploy a sensor network at key parts of the ancient building to collect physical parameter data such as strain, displacement, tilt, temperature, and humidity in real time, and to acquire material property data and environmental data; a construction module, used to construct a digital twin model by combining the digital model of the overall space of the ancient building, material property data, and environmental data; an analysis module, used to analyze the acquired data using deep learning algorithms to achieve anomaly detection, damage identification, and predict the remaining lifespan of the structure based on historical data; and an assessment and early warning module, used to perform real-time assessment and early warning of the structural health status of the ancient building based on the dynamic coupling of real-time monitoring data and the digital twin model.

[0058] However, the above-mentioned technical solutions still have at least the following drawbacks: low early warning accuracy, insufficient closed-loop control from monitoring to disposal, and difficulty in meeting the monitoring and early warning needs of the ancient architecture field.

[0059] like Figures 1 to 8 As shown, some embodiments of the ancient building health status monitoring and early warning system of the present invention include a sensor module, a processing module, and an early warning module, including:

[0060] AI-PLC collaborative module, the AI-PLC collaborative module includes an AI module and a PLC module connected via a network cable;

[0061] The PLC module is configured to generate an execution command based on the prediction results obtained from the AI ​​module, and send the execution command to the processing module; and to generate an alarm command based on the prediction results, and send the alarm command to the early warning module.

[0062] The AI ​​module is configured to acquire collected data from the sensor module and obtain prediction results based on the collected data acquired from the sensor module, the LSTM algorithm, and the InceptionV3 model.

[0063] The AI ​​module includes: a sliding window length calculation module, a learning rate calculation module, a fire image recognition module, a fire prediction and judgment module, and an algorithm module.

[0064] In these embodiments, the present invention, by setting up an AI-PLC collaborative module, utilizes the LSTM algorithm (located in the algorithm module within the AI ​​module) to accurately capture environmental changes through sensors to predict potential risks, and uses the InceptionV3 model (located in the algorithm module within the AI ​​module) to analyze image data to identify fire hazards. This multimodal data fusion improves the accuracy and timeliness of early warnings. Simultaneously, by directly collaborating the AI ​​module with the PLC module for decision-making, predictive results can be generated, and execution commands can be generated to drive the remedial action module for remedial action and to drive the early warning module to issue warnings.

[0065] The AI-PLC collaborative module is responsible for data processing, intelligent analysis, prediction, and control decision-making. Physically, the AI-PLC collaborative module consists of an AI module responsible for AI calculations and a PLC module responsible for logic control. The two communicate via network cable and the MC protocol. In this embodiment, the PLC module uses a Mitsubishi PLC main unit as the core control unit, and the AI ​​module establishes a communication connection with the PLC through the installed HslCommunication library functions.

[0066] Furthermore, the core function of the AI ​​module is to use deep learning algorithms to analyze and predict monitoring data, mainly including sub-functions such as fire image recognition, structural stress prediction, and gas concentration prediction.

[0067] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the collected data includes: the natural frequency range of the wooden structure of the ancient building, the sensor sampling frequency range, and image data;

[0068] The sliding window length calculation module is configured to determine the candidate range of window length based on the obtained natural frequency range of the ancient wooden structure and the sampling frequency range of the sensor.

[0069] Obtain the monitoring samples and sample lengths of the sensor, and obtain the initial window length based on the monitoring samples, the sample lengths, and the candidate window length range;

[0070] The sliding window length is obtained based on the window length candidate range and the initial window length.

[0071] In some implementations of monitoring and early warning systems for the health status of ancient buildings, a candidate range for window length is determined based on the acquired range of natural vibration frequencies of the ancient building's wooden structure and the sensor sampling frequency range, including:

[0072] Based on the range of natural frequencies of the ancient wooden structure, the range of natural periods of the ancient wooden structure is obtained.

[0073] Obtain the minimum number of complete learning natural vibration periods of the AI ​​module, and obtain the minimum physical duration range based on the range of the minimum complete learning natural vibration periods and the range of the natural vibration periods of the ancient wooden structure.

[0074] The time step range is obtained based on the sensor sampling frequency range and the minimum physical duration range;

[0075] Expanding the range of time steps yields the candidate range of window length.

[0076] In these implementations, the sliding window length is a key parameter when using LSTM models to process time series data. If the window length is too short, the model will not be able to learn the complete dynamic features, while if the window length is too long, it will introduce too much irrelevant information and increase the computational burden.

[0077] Furthermore, based on the structural characteristics of ancient buildings (such as materials, dimensions, and structural form), the range of natural frequencies of their wooden structures is determined through theoretical calculations or engineering experience. , From this, we can obtain the range of its natural vibration period. , ].

[0078] Furthermore, since the LSTM model needs to learn at least one or more complete vibration cycles to effectively capture its dynamic characteristics, a minimum number of complete learning natural vibration cycles is set (which can be 1.5, 2, etc.). Multiplying this number of cycles by the range of natural vibration cycles yields the minimum physical duration range required for model learning. , ].

[0079] Furthermore, to increase robustness, the time step range is expanded by a preset expansion ratio (e.g., 20% expansion both upwards and downwards) to obtain the final candidate range for window length. , All subsequently calculated sliding window lengths should fall within this range.

[0080] Specifically, taking a typical wooden ancient building as an example, its natural frequency range = 0.5 Hz = 3Hz; take the minimum number of complete learning natural cycles as 2; then the range of natural cycles is [ , ] = [0.33s, 2s], minimum physical duration range [ , [0.67s, 4s]. Assuming the minimum sampling frequency range is 10 Hz and the maximum is 50 Hz, the time step range is [10 × 0.67, 50 × 4] = [6.7, 200]. After expanding upwards and downwards by 20%, the window length selection range is obtained as [5, 240].

[0081] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the initial window length is obtained based on the monitoring sample, sample length, and a candidate range of window lengths, including:

[0082] Based on the autocovariance formula, the monitored sample, and the sample length, the autocovariance of the lag-order sample and the autocovariance of the non-lag-order sample are obtained.

[0083] Based on the autocovariance of the lag-order samples and the autocovariance of the unlag-order samples, the sample autocorrelation coefficient is obtained;

[0084] The initial window length is obtained based on the sample autocorrelation coefficient and the first threshold.

[0085] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the initial window length is obtained based on the sample autocorrelation coefficient and a first threshold, including:

[0086] The sample autocorrelation coefficients with different lag orders are compared one by one with the first threshold. When the absolute value of the sample autocorrelation coefficient is less than the first threshold, the value of the smallest lag order is the initial window length.

[0087] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the sliding window length is obtained based on the candidate window length range and the initial window length, including:

[0088] The initial window length is compared with the window length candidate range. If the value of the initial window length is less than the minimum value in the window length candidate range, the sliding window length is the minimum value in the window length candidate range.

[0089] If the initial window length is greater than the maximum value in the candidate window length range, the sliding window length is the maximum value in the candidate window length range;

[0090] If the value of the initial window length is within the range of window length options, the sliding window length is the initial window length.

[0091] In these implementations, a representative monitoring sample is acquired, such as a segment of stable vibration data collected by a sensor, and its length is recorded as... Then, the autocorrelation of the monitored sample is calculated using the autocovariance formula. For a time series { }, its lag Sample autocovariance of order And the sample autocovariance without lag (order 0) As shown in equations (2) and (3):

[0092] (2);

[0093] (3);

[0094] in, for Sensors collect data at all times. for Sensors collect data at all times. The average value of all monitored samples. The lag order is... To monitor the total length of the sample, Lagging The sample autocovariance of order, The sample autocovariance is without lag.

[0095] Furthermore, based on the aforementioned autocovariance, the sample autocorrelation coefficients for different lag orders are calculated according to equation (4). This coefficient reflects the sequence's autocorrelation coefficient at different lag orders. Time and The degree of linear correlation at any given time.

[0096] (4);

[0097] in, For lag The sample autocorrelation coefficient of order, For lag The sample autocovariance of order, The autocovariance of the samples without lag.

[0098] Furthermore, the absolute values ​​of the autocorrelation coefficients of samples with different lag orders (starting from 1) are compared one by one with a preset first threshold (e.g., 0.05 or 0.1, representing a relatively weak correlation); when the absolute value of the sample autocorrelation coefficient first appears to be less than the first threshold, the value of the corresponding minimum lag order is determined as the initial window length. Preferably, the first threshold is 0.2.

[0099] By fully calculating the autocorrelation function, the strength of the linear dependence of the time series across different time spans was quantified. This provides a direct basis for subsequently using a first threshold to automatically locate the critical point where linearly related information disappears.

[0100] By comparing the absolute values ​​of the autocorrelation coefficients of samples with different lag orders with a first threshold, the boundary points of natural decay of linear memory are automatically identified, thereby determining the initial window length. This avoids the subjectivity and arbitrariness of manually setting the window based on experience. Using absolute value comparisons prevents negative correlations from being missed, ensuring that the search is correctly terminated regardless of whether the linear dependency is positive or negative.

[0101] By comparing the initial window length with a range of alternative window lengths, the final sliding window length is obtained, thus integrating statistical results with engineering constraints. Setting a lower limit prevents the window from becoming too short due to noise or short-term stationary deviations, ensuring that at least one complete physical change cycle is captured. Setting an upper limit restricts the over-inclusion of weak, long-latency related information, avoiding redundant features and overfitting risks associated with excessively long windows. This approach respects statistical patterns while meeting the real-world monitoring requirements for alarm timeliness.

[0102] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the learning rate calculation module is configured to: determine the effective learning rate range based on the collected data, the initial maximum learning rate, and the minimum effective learning rate;

[0103] The learning rate for each training round is dynamically set according to the learning rate decay formula and the effective learning rate range.

[0104] After each fixed training cycle, the learning rate value will be reset to the initial maximum learning rate;

[0105] The learning rate decay formula is shown in equation (1).

[0106] (1);

[0107] in, The learning rate for the current training epoch. The initial maximum learning rate, To minimize the effective learning rate, This is the current training round number. This represents the total number of training rounds.

[0108] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the fire image recognition module is configured to: divide the image data into fire and no fire, and store the two types of images, including fire and no fire, in two subfolders of the training set and the validation set, respectively.

[0109] Traverse the entire training set data, count the number of image samples in the two categories of "with fire" and "without fire", and obtain the number of image samples in the "with fire" category, the number of image samples in the "without fire" category, and the total number of samples.

[0110] Weights are assigned to image samples in the "on fire" category based on the number of image samples in the "on fire" and "off fire" categories.

[0111] In these implementations, during the training of deep learning models (such as the InceptionV3 model for fire detection and LSTM models for structure and environment prediction), the learning rate directly affects the model's convergence speed and final accuracy. Traditional fixed learning rates often struggle to balance fast convergence and fine-grained optimization. This invention addresses this by configuring a learning rate calculation module within the AI ​​module and employing a cosine annealing with warm restarts strategy to dynamically adjust the learning rate.

[0112] Specifically, this module first sets the initial maximum learning rate based on the collected data and model characteristics. (For example, set to 0.001 in the initial stage of transfer learning for the fire identification model) and minimum effective learning rate. (For example, set to 0.00001 during the fine-tuning phase) to determine the effective learning rate range. , ].

[0113] During training, the learning rate is dynamically set for each training round according to the learning rate decay formula. Furthermore, after each fixed training cycle, the module resets the learning rate to the initial maximum learning rate and then decays it again according to the cosine curve in the learning rate decay formula. This allows the model to escape local optima in the later stages of training and find a smoother, more generalizable global optimum, thereby improving the model's ability to extract and predict the complex environmental features of ancient buildings.

[0114] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the fire image recognition module is configured to: divide the image data into fire and no fire, and store the two types of images, including fire and no fire, in two subfolders of the training set and the validation set, respectively.

[0115] Traverse the entire training set data, count the number of image samples in the two categories of "with fire" and "without fire", and obtain the number of image samples in the "with fire" category, the number of image samples in the "without fire" category, and the total number of samples.

[0116] Weights are assigned to image samples in the "on fire" and "off fire" categories based on the number of image samples in each category.

[0117] First, the fire image recognition module preprocesses the acquired image data, dividing it into two categories: "on fire" and "no fire," and stores the images containing both categories in the corresponding subfolders under the training set and validation set, respectively.

[0118] Next, before model training, the entire training set data is traversed to count the number of image samples in the fire category and the number of image samples in the non-fire category, and the total number of samples is calculated.

[0119] Finally, based on the statistically obtained sample size, class weights (class_weight) are automatically set for the "fire" and "no fire" categories using an algorithm library (e.g., Scikit-learn). In this embodiment, the weight of the "fire" category is calculated according to equation (5):

[0120] (5);

[0121] in, For fire-related categories, weights are assigned. This represents the total number of samples. This represents the number of image samples in the fire category.

[0122] The weight of the fireless category is calculated according to formula (6):

[0123] (6);

[0124] in, The weight of the fireless category, This represents the total number of samples. This represents the number of image samples in the non-fire category.

[0125] As shown in equations (5) and (6), in the application scenario of ancient building fire monitoring, the number of images with fire is much smaller than the number of images without fire, so the weight of the fire category is significantly greater than the weight of the no-fire category. By assigning higher weights to the minority class (images with fire), the model pays more attention to fire characteristics during training, effectively reducing the false negative rate of fires.

[0126] Since the number of image samples in the "fire" category is less than the number of image samples in the "no fire" category, assigning higher weights to images with fire makes the model pay more attention to fire features during training, effectively reducing the false negative rate of fires.

[0127] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the fire prediction and judgment module is configured to divide the image data into several datasets according to the number of consecutive frames.

[0128] The fire prediction and judgment module is further configured to output a fire probability value for each frame of image using the InceptionV3 model, and to determine whether a fire has occurred based on the comparison result of the fire probability value with preset warning thresholds and alarm thresholds.

[0129] In these implementations, in order to improve the accuracy of fire alarms and filter out instantaneous false alarms caused by changes in light, red objects, etc., the fire prediction and judgment module introduces judgment logic based on multi-level thresholds and the consistency of continuous frame timing.

[0130] Specifically, this module divides the continuous video stream images captured by the video acquisition device into several datasets according to a preset number of consecutive frames (e.g., 5). During processing, a frame skipping strategy (e.g., processing 1 frame every 3 frames) can be used to improve processing speed.

[0131] The InceptionV3 model outputs a fire probability value between 0 and 1 for each frame of image. The fire prediction and judgment module then makes a comprehensive judgment based on the comparison between the fire probability value of each frame in the dataset and preset warning thresholds (e.g., 0.4) and alarm thresholds (e.g., 0.7), according to the following rules:

[0132] (1) When the fire probability values ​​of consecutive frames in the dataset all exceed the alarm threshold, the fire prediction and judgment module confirms that a fire has occurred and triggers the highest level fire alarm.

[0133] (2) When there are frames in the dataset whose fire probability value exceeds the warning threshold but is lower than the alarm threshold, or exceeds the alarm threshold but the number of consecutive frames is insufficient, the fire prediction and judgment module will only trigger a warning signal to prompt the management personnel to pay attention;

[0134] (3) When the fire probability value of all frames in the dataset is lower than the warning threshold, the fire prediction and judgment module does not trigger any alarm.

[0135] The aforementioned dual verification mechanism, which combines probability amplitude and time series, greatly improves the reliability of the system and effectively reduces the false alarm rate.

[0136] In some implementations of monitoring and early warning systems for the health status of ancient buildings, the sensor module includes at least one of an accelerometer, a tilt sensor, a displacement sensor, a strain sensor, a video acquisition device, a smoke sensor, a carbon monoxide concentration detector, a carbon dioxide concentration detector, and a temperature and humidity sensor.

[0137] The treatment module includes a water pump, a spray system, a fan, pipelines, and an AMD control system;

[0138] The early warning module includes an alarm;

[0139] Both the processing module and the early warning module are electrically connected to the PLC module, and the sensor module is electrically connected to the AI ​​module.

[0140] In these implementations, the early warning module is responsible for presenting the analysis results of the AI ​​module to management personnel in an intuitive and hierarchical manner, providing a basis for emergency decision-making. In this implementation, the early warning module includes physical alarms (such as audible and visual alarms) and a software-level early warning network.

[0141] Furthermore, based on real-time monitoring data and AI algorithms, the system dynamically generates early warning signals of different risk levels for ancient buildings, including structural safety, fire risk, and environmental erosion (such as temperature, humidity, vibration, and settlement), according to relevant regulations and contingency plans. This embodiment employs a four-level early warning system:

[0142] (1) Blue (Attention Level): The monitoring data shows a slight anomaly, but it has not reached the warning threshold. The system prompts the management personnel to pay attention.

[0143] (2) Yellow (warning level): When the predicted indicator is about to or just exceeds the safety threshold, such as when the fire probability value exceeds the warning threshold (e.g., 0.4), the system issues a warning and pushes information such as the location and nature of the risk;

[0144] (3) Orange (Severe): The predicted indicators significantly exceed the safety threshold, the situation is relatively urgent, and the system issues a serious warning;

[0145] (4) Red (emergency level): When a fire is confirmed (e.g., the fire probability value of multiple consecutive frames exceeds the alarm threshold of 0.7) or the structure experiences dangerous vibration, the system issues the highest level emergency alarm and automatically activates the response module.

[0146] Furthermore, the warning command issues an audible and visual alert via an alarm, and immediately pushes detailed information including the risk location, nature, level, and preliminary response guidelines to relevant management personnel through a warning network (such as mobile phone text messages, APP push notifications, and monitoring center large screen pop-ups).

[0147] Furthermore, the treatment module is the terminal actuator that implements physical intervention. In this embodiment, the treatment module mainly includes water pumps, pipelines, a sprinkler system, a fan, and an AMD active quality drive control system.

[0148] Furthermore, in the event of a fire, the response module receives a signal from the PLC module and initiates the fire extinguishing procedure, simultaneously cutting off non-fire-fighting power. When the ambient humidity or carbon dioxide concentration exceeds the tolerance range of the ancient building (for example, excessively high carbon dioxide concentration can affect the painted decorations on the ancient building), the response module receives a signal from the PLC module and adjusts the ventilation. When the structure vibrates and exceeds a set threshold, the response module adaptively modifies the control parameters of the AMD active mass drive control system based on the structural and seismic characteristics to ensure the seismic safety of the ancient building.

[0149] The sensor module is responsible for collecting real-time data on the structural health and environmental status of the ancient building. Specifically, an ANSYS finite element model is constructed based on statistical information of the ancient building, and finite element analysis is performed to identify weak points in the ancient building. The placement locations of different types of sensors are then determined by combining the finite element analysis results with expert experience.

[0150] Furthermore, the sensor module includes a health monitoring module and an environmental monitoring module. The health monitoring module monitors parameters such as structural deformation, stress, and vibration, and employs sensors including, but not limited to, accelerometers, tilt sensors, displacement sensors, and strain sensors. The environmental monitoring module monitors parameters such as fire, carbon monoxide concentration, carbon dioxide concentration, temperature, and humidity, and employs sensors including, but not limited to, video acquisition devices, smoke sensors, carbon monoxide concentration detectors, carbon dioxide concentration detectors, and temperature and humidity sensors.

[0151] Furthermore, this embodiment employs the InceptionV3 model based on a convolutional neural network (CNN) to perform fire identification on the image data acquired by the video acquisition device. Its core functions include data preprocessing (loading, normalization, serialization), model building, model training, model evaluation, fire probability prediction, and alarm.

[0152] Specifically, the InceptionV3 model pre-trained on the ImageNet dataset is used as the base model, excluding its top fully connected layers. The input image is set to a preset pixel size RGB three-channel image (in this embodiment, the input image is uniformly scaled to 300×300 pixels), and data augmentation such as normalization and random horizontal flipping is performed. During the rapid training transfer learning phase, the bottom convolutional layers of the InceptionV3 model are frozen, and training parameters consisting of a training data generator, a validation data generator, and class weights are configured to train the added top-level classifier. The class weights are used to address the imbalance between the number of fire samples and non-fire samples (see the fire image recognition module below for details). During the fine-tuning phase, several convolutional layers near the top of the InceptionV3 model (such as the top 10 layers) are unfrozen for fine-tuning, while the remaining bottom convolutional layers remain frozen.

[0153] In the fire detection logic, this embodiment sets the following parameters: FRAME_SKIP (e.g., a value of 3 indicates that 1 frame is processed every 3 frames), ALARM_THRESHOLD (e.g., a value of 0.7), CONSECUTIVE_FRAMES (e.g., a value of 5, used to determine whether fire characteristics continue to appear), HISTORY_LENGTH (e.g., a value of 30, used for trend chart display), and WARNING_THRESHOLD (e.g., a value of 0.4, which triggers an alarm when the fire probability value exceeds this value but is lower than the alarm threshold).

[0154] For example, such as Figure 6 As shown, after training the model for 5 rounds, the training set loss decreased from 0.1582 to 0.0992, and the accuracy increased from 0.9390 to 0.9634, indicating that the model's predictive ability on the training set gradually improved. As shown in Figure 3, based on image analysis of real-time video, the system triggered a fire alarm when the fire probability value output by the InceptionV3 model was 77.93%.

[0155] Furthermore, for time-series data such as structural stress, displacement, and gas concentration, this embodiment employs a Long Short-Term Memory (LSTM) network for state prediction. Its core functions include data preprocessing (loading, normalization, and serialization), model building (including input layer, LSTM layer, Dropout layer, and Dense output layer), training using the Adam optimizer and mean squared error (MSE) loss function (with Early Stopping mechanism), model evaluation, concentration prediction, and early warning.

[0156] Furthermore, the key parameters of the LSTM model are set as follows: sliding window length is 30 (i.e., prediction is made using data from the past 30 time steps); training epochs are 100; batch size is 32; learning rate is 0.0005; number of LSTM units is [64, 32], i.e., 64 units in the first layer and 32 units in the second layer; dropout ratio is 0.2; validation set ratio is 0.1; early stopping patience value is 10, i.e., when the validation loss no longer improves, the model continues training for 10 more epochs before stopping training and restoring the optimal weights.

[0157] Furthermore, the LSTM model construction process is as follows: Initialize the sequential model; add an input layer and specify the input shape (time step, number of features). Add the first LSTM layer and set the return sequence to connect to the next LSTM layer; add a Dropout layer to prevent overfitting; add a second LSTM layer; add another Dropout layer; finally, add a fully connected layer as the output layer to output a predicted value. The LSTM model is compiled using the Adam optimizer and the mean squared error loss function. During training, an early stopping callback function is set to stop training and restore the optimal weights when the verification loss no longer improves; the model is trained using training data without shuffling the data (because the data is a time series).

[0158] For example, such as Figure 7 As shown, the structural health monitoring and prediction results indicate that the root mean square error (RMSE) between the actual and predicted values ​​is 9.53, and the mean absolute percentage error (MAPE) is 0.98%, demonstrating that the model can predict the structural health status effectively. Furthermore, the system sets threshold points and change threshold points; exceeding these thresholds triggers corresponding alarms. The threshold settings are primarily based on structural modeling, stress analysis, and expert judgment.

[0159] To fully realize the monitoring and early warning functions, the software system of this invention is logically divided into a data acquisition layer, a data processing and prediction layer, a control execution layer, and a system data display module.

[0160] First, the data acquisition layer

[0161] The data acquisition layer establishes a communication connection with the lower-level PLC via a Python program, specifically using the Mitsubishi MC protocol (e.g., through the MelsecA1ENet library), to read the register data corresponding to each sensor in the PLC in real time. For example, it reads register D80 to obtain smoke data, register D81 to obtain temperature data, register D82 to obtain carbon dioxide concentration data, and register D83 to obtain carbon monoxide concentration data.

[0162] The original analog data read (e.g., values ​​ranging from 0 to 16000) is a dimensionless digital quantity and cannot be directly used for subsequent analysis and early warning. To convert it into an engineering value with physical meaning, the system processes it using a linear conversion algorithm, as shown in equation (7):

[0163] (7);

[0164] in, These are the quantities after linear transformation. The raw analog values ​​read from the PLC. and These are the minimum and maximum values ​​of the analog quantity range (e.g., 0 and 16000 respectively). and These represent the minimum and maximum values ​​of the corresponding physical quantities within their actual engineering range (e.g., for carbon dioxide concentration, the range is 0-2000 ppm; for temperature, the range is -50℃ to 200℃). Using this algorithm, the system can accurately convert the PLC's internal values ​​into on-site physical quantities that management personnel can understand.

[0165] Meanwhile, the acquisition layer has a preliminary data verification function, which truncates abnormal data that exceeds the minimum or maximum value of the analog quantity and records it in the log.

[0166] Second, the data processing and prediction layer.

[0167] The data processing and prediction layer is the core of the system, responsible for cleaning, normalizing, and predicting historical and real-time data. For data preprocessing, the Pandas library is used to handle missing values, and Scikit-learn's MinMaxScaler is used to normalize the data, scaling it to a range suitable for the model.

[0168] In terms of LSTM time series prediction, the system constructs the LSTM model as described above, based on environmental data such as carbon dioxide concentration and temperature, as well as structural stress data. During real-time operation, the system maintains a fixed-length sliding window (e.g., window_size=30), organizing the latest 30 time-step data into the 3D tensor input of the LSTM model, inferring the predicted value for the next time step, and then inversely scaling it back to the actual physical units.

[0169] In terms of InceptionV3 image recognition, the system loads the pre-trained InceptionV3 model as described above, scales the input image to 300×300 pixels and performs normalization and data augmentation, and completes model training using a two-stage fine-tuning strategy, outputting a Sigmoid fire probability value in the range of 0 to 1.

[0170] Third, the control execution layer

[0171] The control execution layer generates control commands based on the prediction results and feeds them back to the field equipment. The system compares the predicted values ​​output by the LSTM model (such as predicted carbon dioxide and temperature values) with preset safety thresholds (such as a carbon dioxide threshold of 430 ppm and a temperature threshold of 23 ℃). When the predicted value exceeds the threshold, the Python host computer program writes an action signal (such as a prediction anomaly flag) into a specific data register (such as D100) of the PLC through the communication interface. After detecting the change in the state of the D100 register, the ladder logic program inside the PLC immediately triggers the corresponding hard-wired logic to drive the actuator (such as starting the fan for ventilation and cooling, starting the water pump for spraying, or adjusting the AMD control system parameters), thereby realizing automated closed-loop handling of abnormal states.

[0172] Fourth, system data display module

[0173] To facilitate intuitive monitoring of the ancient building's status by management personnel, a web-based visualization interface was developed. Utilizing the Dash framework and Plotly graphics library, the system caches real-time data in memory using a double-ended queue (e.g., maintaining the most recent 500 data entries), and dynamically renders waveforms comparing the "actual values" and "predicted values" of indicators such as carbon dioxide concentration and temperature on the webpage front end. When an anomaly occurs, the interface simultaneously flashes red alarm text (such as "CO2 Exceeds Standard" or "Temperature Too High"), providing intuitive information for emergency decision-making.

[0174] In one specific embodiment of this application, in terms of system control, at the lower-level machine, a Mitsubishi PLC host is used as the core control unit, responsible for receiving sensor data, executing control logic, and communicating and interacting with other devices. A PLC control program is written using GX Works2 software to implement logic for switching between manual / automatic modes, judging environmental parameter thresholds, and controlling equipment linkage.

[0175] At the host computer level, the graphical programming software KingSCADA (the configuration function of the giant control module) is used to realize the graphical editing, operation and status monitoring of the PLC control system.

[0176] (1) Functions related to macro control

[0177] The GRM550 series modules are PLC wireless communication modules from Jukong Technology. These modules support functions such as remote PLC program download and debugging, configuration monitoring, SMS alarms, mobile APP, and video monitoring. They also support multiple networking methods including 4G, Wi-Fi, and Ethernet, and provide an Ethernet interface. They are compatible with mainstream PLCs such as Siemens, Mitsubishi, and Omron, and are suitable for industrial automation, remote equipment maintenance and monitoring, and equipment networking scenarios.

[0178] (2) Communication between the control unit and the PLC

[0179] Communication Basics: The control module supports Mitsubishi FX3U's A-compatible 1E protocol communication. The Mitsubishi FX3U main unit is equipped with an FX3U-ENENT-ADP module (network port module). The control module and the PLC's network port module are connected via a network cable.

[0180] Communication settings on the PLC side: Communication between the two is achieved through the network port module. Set the communication parameters of the PLC-side network port module, set the IP address to 192.168.5.250, and set the port to 5005.

[0181] Communication settings on the control unit: Open the communication port editor in the GRM developer software, select MITSUBISHI_FX3G_TCP. This mode supports 1E frame protocol communication of Mitsubishi MC protocol. Select TCP mode as the protocol type below.

[0182] After creating the new device, fill in the IP address and network port, which should be consistent with the PLC parameter settings. Set the IP address to 192.168.5.250 and the port to 5005. The control module and the PLC can then communicate.

[0183] (3) This module integrates the function of creating a configuration interface. Open the GRM WebGUI developer software, create a new device manager, fill in the module device ID and enter the password to associate the configuration function with the module. Then you can edit the screen as needed.

[0184] Furthermore, the final monitoring interface is as follows: Figure 8 As shown.

[0185] The embodiments and functional operations of the subject matter described in this specification can be implemented in the following ways: digital electronic circuits, tangibly implemented computer software or firmware, computer hardware, including the structures disclosed in this specification and their equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more modules of computer program instructions encoded on one or more tangible non-transitory program carriers, for execution by a data processing device or to control the operation of the data processing device.

[0186] Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are then generated as coded information to be transmitted to an appropriate receiver device executed by data processing equipment. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations of the above.

[0187] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather as descriptions of features that can embody specific embodiments of a particular invention. Specific features described in this specification within the context of an independent embodiment may also be implemented in combination with a single embodiment. Conversely, various features described within the context of a single embodiment may also be implemented independently in multiple embodiments, or in any suitable sub-combination. Furthermore, while features may be described for combination and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and the claimed combination may be redirected to a sub-combination or a variation thereof.

[0188] Similarly, although operations are described in the accompanying drawings in a specific order, it should not be construed as requiring that such operations be performed in the specific order shown or in sequential order, or that all illustrated operations be performed, in order to achieve the desired result. In certain cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that program components and systems can generally be integrated into a single software product or packaged into multiple software products.

[0189] Specific implementations of the subject matter have been described. Other implementations are within the scope of the following claims. For example, the activities described in the claims can be performed in a different order and still achieve the desired result. As an example, the processes described in the drawings do not necessarily require a specific order or sequence to be shown in order to achieve the desired result. In certain implementations, multitasking and parallel processing may be advantageous.

Claims

1. A monitoring and early warning system for the health status of ancient buildings, comprising a sensor module, a processing module, and an early warning module, characterized in that, Also includes: AI-PLC collaborative module, the AI-PLC collaborative module includes an AI module and a PLC module connected via a network cable; The PLC module is configured to generate an execution command based on the prediction results obtained from the AI ​​module, and send the execution command to the processing module; and to generate an alarm command based on the prediction results, and send the alarm command to the early warning module. The AI ​​module is configured to acquire collected data from the sensor module and obtain prediction results based on the collected data acquired from the sensor module, the LSTM algorithm, and the InceptionV3 model. The AI ​​module includes: a sliding window length calculation module, a learning rate calculation module, a fire image recognition module, and a fire prediction and judgment module.

2. The monitoring and early warning system for the health status of ancient buildings according to claim 1, characterized in that, The collected data includes: the natural frequency range of the ancient wooden structure, the sensor sampling frequency range, and image data; The sliding window length calculation module is configured to determine the candidate range of window length based on the obtained natural frequency range of the ancient wooden structure and the sampling frequency range of the sensor. Obtain the monitoring samples and sample lengths of the sensor, and obtain the initial window length based on the monitoring samples, the sample lengths, and the candidate window length range; The sliding window length is obtained based on the window length candidate range and the initial window length.

3. The monitoring and early warning system for the health status of ancient buildings according to claim 2, characterized in that, Based on the obtained natural frequency range of the ancient wooden structure and the sensor sampling frequency range, the candidate range for the window length is determined, including: Based on the range of natural frequencies of the ancient wooden structure, the range of natural periods of the ancient wooden structure is obtained. Obtain the minimum complete learning natural vibration period range of the AI ​​module, and obtain the minimum physical duration range based on the minimum complete learning natural vibration period range and the natural vibration period range of the ancient wooden structure. The time step range is obtained based on the sensor sampling frequency range and the minimum physical duration range; Expanding the range of time steps yields the candidate range of window length.

4. The monitoring and early warning system for the health status of ancient buildings according to claim 3, characterized in that, Based on the monitored samples, sample length, and the candidate window length range, the initial window length is obtained, including: Based on the autocovariance formula, the monitored sample, and the sample length, the autocovariance of the lag-order sample and the autocovariance of the non-lag-order sample are obtained. Based on the autocovariance of the lag-order samples and the autocovariance of the unlag-order samples, the sample autocorrelation coefficient is obtained; The initial window length is obtained based on the sample autocorrelation coefficient and the first threshold.

5. The monitoring and early warning system for the health status of ancient buildings according to claim 4, characterized in that, The initial window length is obtained based on the sample autocorrelation coefficient and the first threshold, including: The sample autocorrelation coefficients with different lag orders are compared one by one with the first threshold. When the absolute value of the sample autocorrelation coefficient is less than the first threshold, the value of the smallest lag order is the initial window length.

6. The monitoring and early warning system for the health status of ancient buildings according to claim 5, characterized in that, The sliding window length is obtained based on the candidate window length range and the initial window length, including: The initial window length is compared with the window length candidate range. If the value of the initial window length is less than the minimum value in the window length candidate range, the sliding window length is the minimum value in the window length candidate range. If the initial window length is greater than the maximum value in the candidate window length range, the sliding window length is the maximum value in the candidate window length range; If the value of the initial window length is within the range of window length options, the sliding window length is the initial window length.

7. The monitoring and early warning system for the health status of ancient buildings according to claim 1, characterized in that, The learning rate calculation module is configured to determine the effective learning rate range based on the collected data, the initial maximum learning rate, and the minimum effective learning rate. The learning rate for each training round is dynamically set according to the learning rate decay formula and the effective learning rate range. After each fixed training cycle, the learning rate value will be reset to the initial maximum learning rate; The learning rate decay formula is shown in equation (1). （1）； in, The learning rate for the current training epoch. The initial maximum learning rate, To minimize the effective learning rate, This is the current training round number. This represents the total number of training rounds.

8. The monitoring and early warning system for the health status of ancient buildings according to claim 7, characterized in that, The fire image recognition module is configured to: divide the image data into fire and no fire, and store the two types of images containing fire and no fire in two subfolders of the training set and the validation set, respectively. Traverse the entire training set data, count the number of image samples in the two categories of "with fire" and "without fire", and obtain the number of image samples in the "with fire" category, the number of image samples in the "without fire" category, and the total number of samples. Weights are assigned to image samples in the "on fire" and "off fire" categories based on the number of image samples in each category.

9. The monitoring and early warning system for the health status of ancient buildings according to claim 8, characterized in that, The fire prediction and judgment module is configured to divide the image data into several datasets according to the number of consecutive frames. The fire prediction and judgment module is further configured to output a fire probability value for each frame of image using the InceptionV3 model, and to determine whether a fire has occurred based on the comparison result of the fire probability value with preset warning thresholds and alarm thresholds.

10. The monitoring and early warning system for the health status of ancient buildings according to claim 1, characterized in that, The sensor module includes at least one of the following: an accelerometer, a tilt sensor, a displacement sensor, a strain sensor, a video acquisition device, a smoke sensor, a carbon monoxide concentration detector, a carbon dioxide concentration detector, and a temperature and humidity sensor. The treatment module includes a water pump, a spray system, a fan, pipelines, and an AMD control system; The early warning module includes an alarm; Both the processing module and the early warning module are electrically connected to the PLC module, and the sensor module is electrically connected to the AI ​​module.

Images