Intelligent early warning system for cultural relic safety based on AI large model

Abstract

The application belongs to the technical field of security alarm systems and relates to an AI large model-based cultural relic safety intelligent early warning system. Environmental sensor temperature and humidity and light timing signals and camera cultural relic surface image signals are extracted; a pre-trained visual language large model encoder is used to map the image signals to a high-dimensional feature space, and a time domain convolution network is used to map the timing signals to the same high-dimensional feature space to complete cross-modal feature alignment and output fused features; a space-time heterogeneous graph is constructed with cultural relic surface areas as nodes and environmental influence factors as edges, the fused features are input into a graph attention network to calculate node space-time evolution deviation degrees, and when the deviation degrees exceed a dynamic self-adaptive baseline, a degradation type early warning signal frame is generated and sent through a security alarm bus. The application overcomes the defect that single sensor threshold alarm cannot identify multi-factor coupled degradation, realizes early perception of slight progressive degradation trends, and improves system anti-interference capability and complex degradation mode recognition capability.

Description

Technical Field

[0001] This invention belongs to the technical field of security alarm systems, and relates to an intelligent early warning system for cultural relic safety based on an AI large model. Background Technology

[0002] Current cultural relic safety monitoring systems primarily rely on various independent sensors deployed within the cultural relic preservation environment to achieve status awareness and alarm functions. Conventional solutions typically involve deploying temperature, humidity, and light sensors within the artifact display cases or storage rooms, along with video surveillance equipment at specific locations. During system operation, each sensor continuously collects environmental data at a fixed sampling frequency, while the video surveillance equipment continuously records images of the artifact's surface. After data collection, the system compares the real-time temperature, humidity, and light intensity values ​​with preset threshold ranges. When an environmental parameter exceeds the preset upper or lower threshold, the system generates a corresponding alarm signal and sends it to the monitoring center via a security communication bus. Video surveillance mainly serves as a visual recording source for post-event verification, with personnel manually observing for anomalies or triggering alarms through simple moving target detection algorithms when drastic changes occur in the image.

[0003] At the data processing and logical judgment level, existing technologies employ a paradigm of isolated processing of data from each channel and hard-coding of rules. Time-series data collected by environmental sensors and image data collected by video surveillance flow along independent processing links within the system, with no correlation or interaction at the feature level between the two. For time-series data, the system only performs comparisons between single-point instantaneous values ​​and thresholds, or uses simple moving average filtering followed by comparison, lacking in-depth analysis of time-series fluctuation patterns. For image data, the system typically uses image processing methods based on pixel-level difference or background subtraction, only generating alarm trigger signals when large-area, high-contrast physical damage or drastic displacement occurs on the surface of the artifact. The intrinsic connection between minute fluctuations in temperature and humidity and microscopic changes on the artifact's surface is severed; the system cannot establish a mapping relationship between environmental stress and the artifact's response, and the alarm decision logic is strictly limited to fixed thresholds and single-dimensional conditional judgment rules.

[0004] Because existing technologies treat environmental time-series data and visual image data as independent events for isolated processing and threshold judgment, they sever the coupling relationship between multiple physical quantities in the process of cultural relic deterioration. This leads to a core technical problem in existing technologies: they cannot identify the subtle, gradual deterioration of cultural relics caused by the coupling of multiple factors. Cultural relic deterioration, such as efflorescence and flaking, is usually the result of the combined effects of minor fluctuations in environmental temperature and humidity, long-term cumulative light exposure, and stress on specific materials. Fluctuations in a single environmental parameter often do not exceed preset thresholds, or even if they do, the degree of exceedance is extremely small and insufficient to trigger alarm logic based on rigid thresholds. Simultaneously, the changes in surface texture in the early stages of deterioration are extremely subtle, and conventional pixel-level image comparisons are unable to capture such microscopic feature changes. The hidden coupling relationships between multi-source sensor data are discarded by the isolated processing mechanism of existing technologies. The system cannot perceive the spatiotemporal co-evolution state between environmental factors and the surface appearance of cultural relics, resulting in missed detections in the early stages of gradual deterioration caused by the coupling of multiple factors. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent early warning system for cultural relic safety based on a large AI model, which can solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] The intelligent early warning system for cultural relic safety based on an AI large-scale model includes: a multi-source signal acquisition end, which extracts temperature, humidity, and illumination time-series signals collected by environmental sensors and image signals of the cultural relic surface collected by monitoring cameras; a cross-modal alignment processing end, which uses an encoder of a pre-trained visual-language large-scale model to map the image signals to a high-dimensional feature space, and maps the time-series signals to the same high-dimensional feature space through a temporal convolutional network to complete cross-modal feature alignment and output fused features; a spatiotemporal graph calculation end, which constructs a spatiotemporal heterogeneous graph with the surface area of ​​the cultural relic as nodes and environmental influencing factors as edges, inputs the fused features into a graph attention network, and calculates the spatiotemporal evolution deviation of the node states; and an early warning signal output end, which generates an early warning signal frame of the corresponding degradation type when the spatiotemporal evolution deviation exceeds the dynamic adaptive baseline, and sends the early warning signal frame through a security alarm bus.

[0008] Preferably, the temporal convolutional network includes a multi-scale dilated convolutional layer and a channel attention layer; the multi-scale dilated convolutional layer uses parallel convolutional kernels with different dilation coefficients to process the temperature, humidity, and illumination time-series signals respectively, and extracts multi-scale temporal fluctuation features; the channel attention layer performs global average pooling and fully connected mapping on the multi-scale temporal fluctuation features in the channel dimension to generate a channel weight vector, multiplies the channel weight vector with the multi-scale temporal fluctuation features channel by channel, outputs weighted temporal features, and maps the weighted temporal features to the high-dimensional feature space.

[0009] Preferably, the encoder of the pre-trained visual-language large model includes a local masking reconstruction sub-network; the surface image signal of the cultural relic is segmented into a sequence of non-overlapping image patches, a portion of the image patch sequence is randomly masked, the unmasked image patch sequence is input into the local masking reconstruction sub-network, the local masking reconstruction sub-network predicts the pixel values ​​of the masked image patch sequence based on the unmasked image patch sequence, calculates the reconstruction error between the predicted pixel values ​​and the real pixel values, backpropagates the reconstruction error to update the parameters of the encoder of the pre-trained visual-language large model, and outputs high-dimensional image features containing micro-texture features.

[0010] Preferably, the cross-modal alignment processing end performs cross-modal contrast alignment; the temperature, humidity, and illumination time series signals within the same time window and the surface image signal of the cultural relic are used to form positive sample pairs, and signals from different time windows are used to form negative sample pairs; the positive sample pairs and the negative sample pairs are projected onto the shared high-dimensional feature space, the cosine similarity between positive sample feature vectors and the cosine similarity between negative sample feature vectors are calculated, the distance between positive sample feature vectors is reduced and the distance between negative sample feature vectors is increased based on the contrastive learning loss function, and the aligned fused features are output.

[0011] Preferably, the graph attention network includes a heterogeneous message passing layer; in the spatiotemporal heterogeneous graph, the edges between the artifact surface area nodes and the environmental influencing factor nodes are divided into spatial edges and temporal edges; the heterogeneous message passing layer sets independent attention weight calculation matrices for the spatial edges and the temporal edges respectively, performs weighted aggregation of the fusion features of adjacent nodes based on the attention weight calculation matrices, introduces an environmental stress transmission attenuation factor to correct the attention weights, inputs the corrected weighted aggregation features into an activation function, and outputs the spatiotemporal evolution deviation of the node state.

[0012] Preferably, the multi-scale dilated convolutional layer outputs multiple temporal fluctuation features using parallel convolutional kernels with different dilation coefficients; the feature variance of each of the multiple temporal fluctuation features in the time dimension is calculated, and the feature variance is input into a softmax normalization function to generate dynamic scale weight coefficients; each of the multiple temporal fluctuation features is multiplied by the corresponding dynamic scale weight coefficient and then concatenated; the concatenated features are then dimensionality-reduced using a one-dimensional convolutional layer to output a temporal feature representation fused with multi-scale dynamic weights, which is then input into the channel attention layer.

[0013] Preferably, the cross-modal alignment processing end constructs a cultural relic deterioration knowledge graph, which defines the mapping relationship between temperature and humidity change patterns and image appearance deterioration patterns; retrieves easily confused image appearance deterioration patterns that match the current temperature and humidity change pattern from the cultural relic deterioration knowledge graph, constructs hard negative sample pairs between the current cultural relic surface image signal and the image signal corresponding to the easily confused image appearance deterioration pattern; inputs the hard negative sample pairs into the contrastive learning loss function to perform feature space extrapolation calculation, and updates the distribution of the fused features.

[0014] Preferably, the environmental stress transmission attenuation factor is calculated based on spatial physical properties; physical distance data and physical barrier material property data are collected between the nodes of the cultural relic surface area and the nodes of the environmental influencing factors; a distance attenuation function is constructed based on the physical distance data; a material attenuation coefficient is obtained by querying the material barrier coefficient table based on the physical barrier material property data; the output of the distance attenuation function is multiplied by the material attenuation coefficient to generate the environmental stress transmission attenuation factor; and the factor is multiplied by the attention weight in the heterogeneous message passing layer.

[0015] Preferably, the warning signal output terminal includes protocol encapsulation logic and hierarchical routing logic; the protocol encapsulation logic queries a preset risk level mapping table to obtain a risk level identifier according to the degradation type, and encapsulates the risk level identifier and the degradation type into the frame header field of the warning signal frame; the hierarchical routing logic parses the risk level identifier in the frame header field, and when the risk level identifier is the highest level, switches the primary and backup communication links of the security alarm bus to a dedicated high-priority link, and sends the warning signal frame through the dedicated high-priority link.

[0016] Preferably, the warning signal output terminal includes baseline update logic; the baseline update logic slides along the time axis to extract the spatiotemporal evolution deviation sequence within the historical time window, performs STL time series decomposition on the spatiotemporal evolution deviation sequence to separate the trend term, periodic term and residual term; removes the periodic term and the residual term, retains the trend term, calculates the quantile value of the trend term at the current time step as the dynamic adaptive baseline, and compares the spatiotemporal evolution deviation calculated in real time with the dynamic adaptive baseline.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] 1. This invention extracts time-series signals of temperature, humidity, and illumination from environmental sensors and image signals of artifact surfaces from monitoring cameras. It utilizes a pre-trained visual-language large-scale model encoder to map the image signals to a high-dimensional feature space. The time-series signals are then mapped to the same high-dimensional feature space via a temporal convolutional network to achieve cross-modal feature alignment, outputting fused features. This constructs a spatiotemporal heterogeneous graph with artifact surface areas as nodes and environmental influencing factors as edges. The fused features are input into a graph attention network to calculate the spatiotemporal evolution deviation of node states. When the deviation exceeds a dynamic adaptive baseline, a warning signal frame is generated. This processing method maps the originally isolated environmental time-series data and visual image data to a unified high-dimensional feature space for cross-modal alignment, establishing a feature-level correlation between environmental stress and the artifact's physical appearance. Based on this, a spatiotemporal heterogeneous graph is constructed, and the spatiotemporal evolution deviation is calculated through a graph attention network. This transforms multi-source heterogeneous sensor data into multi-factor coupled correlation features, overcoming the defect that single sensor threshold alarms are easily affected by environmental noise. This enables the system to capture the hidden coupling correlation between small fluctuations in temperature and humidity and changes in visual appearance, achieving early perception of small, gradual degradation trends caused by multi-factor coupling, and improving the early warning system's ability to identify complex degradation patterns and its anti-interference capability.

[0019] 2. The scheme extracts multi-scale temporal fluctuation features through multi-scale dilated convolutional layers and generates channel weight vectors by combining them with a channel attention layer. This enables dynamic weighted extraction of multi-scale fluctuation features from time-series signals, enhancing the ability to characterize subtle environmental fluctuations. A local mask reconstruction subnetwork is used to perform mask prediction on image patch sequences and calculate reconstruction errors. This forces the model to learn the intrinsic relationship between unmasked and masked regions in the image, outputting high-dimensional image features containing micro-texture characteristics, thus improving the sensitivity to early subtle deterioration textures on the surface of cultural relics. Based on spatial physical properties, the environmental stress transmission attenuation factor is calculated. The attention weights are corrected in the heterogeneous message passing layer, incorporating the physical barriers and distance attenuation laws of the cultural relic preservation environment into the graph calculation process. This ensures that the calculation of the spatiotemporal evolution deviation of node states conforms to the stress transmission laws of the real physical field, improving the physical accuracy of the deviation calculation. The historical spatiotemporal evolution deviation sequence is decomposed into a time series, the periodic term and residual term are removed, and the trend term is retained to calculate the quantile value as a dynamic adaptive baseline. This eliminates the interference of environmental periodic fluctuations and random noise, so that the baseline can be adaptively adjusted with the trend evolution of the long-term preservation state of cultural relics, and avoids false alarms caused by regular environmental fluctuations. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the overall operation of the intelligent early warning system for cultural relic safety according to the present invention.

[0021] Figure 2This is a flowchart of the multi-scale temporal feature dynamic extraction process of the temporal convolutional network of the present invention;

[0022] Figure 3 This is a flowchart of the image extraction and mask reconstruction fine-tuning process of the pre-trained visual language encoder of the present invention;

[0023] Figure 4 This is a flowchart of the cross-modal feature comparison alignment + degraded knowledge graph hard negative sample optimization process of the present invention;

[0024] Figure 5 This is a flowchart of the spatiotemporal heterogeneous graph construction and graph attention network feature evolution calculation of the present invention;

[0025] Figure 6 This is a flowchart illustrating the early warning signal encapsulation, routing, distribution, and dynamic adaptive baseline update process of the present invention. Detailed Implementation

[0026] refer to Figure 1 In one embodiment, the AI-based intelligent early warning system for cultural relic safety operates on a local monitoring server in the cultural relic display case or warehouse. The system establishes communication connections with multi-source sensing devices deployed in the cultural relic preservation environment via wired or wireless means. The multi-source signal acquisition end continuously acquires raw data output by the sensing devices according to a preset sampling frequency, extracting temperature and humidity, illumination time-series signals collected by environmental sensors, and surface image signals of the cultural relic collected by monitoring cameras. The sampling frequency of the temperature and humidity sensor is set to 1 time / minute, the sampling frequency of the illumination sensor is set to 1 time / 5 minutes, and the sampling frequency of the monitoring camera is set to 1 frame / hour. The length of each time window is set to 24 hours, meaning that each time window contains 1440 temperature and humidity sampling points, 288 illumination sampling points, and 24 frames of cultural relic surface images. The multi-source signal acquisition end preprocesses the acquired raw data, performing moving average filtering on the temperature and humidity time-series signal to remove high-frequency noise, outlier removal processing on the illumination time-series signal to eliminate instantaneous strong light interference, and grayscale conversion and histogram equalization processing on the cultural relic surface image signal to enhance image contrast. After preprocessing, the multi-source signal acquisition end packages the temperature and humidity, light timing signals and the surface image signal of the cultural relic within the same time window and sends them to the cross-modal alignment processing end.

[0027] The cross-modal alignment processing unit receives data packets from the multi-source signal acquisition unit and performs feature extraction and mapping on both image and temporal signals. It loads a pre-trained visual-language large-scale model encoder, which employs a Transformer-based structure with 12 Transformer encoder layers, each with 12 attention heads and a hidden layer dimension of 768. The cross-modal alignment processing unit resizes the pre-processed artifact surface image to 224×224 pixels and divides it into a 16×16 sequence of non-overlapping image patches. Each patch is flattened into a 768-dimensional vector, and a learnable positional encoding is added to each vector to preserve the spatial location information of the image. The image patch sequence with added positional encoding is then input into the pre-trained visual-language large-scale model encoder. The encoder calculates the correlation features between image patches using a multi-head self-attention mechanism. After layer-by-layer processing by the 12 Transformer encoder layers, the output is a high-dimensional image feature with a dimension of 768. Simultaneously, the cross-modal alignment processing unit inputs the time-series signals of temperature, humidity, and illumination within the same time window into a temporal convolutional network. This network performs multi-scale feature extraction and weighting on the temporal signals, then maps the temporal features to a 768-dimensional high-dimensional feature space through a fully connected layer, placing them in the same feature space as the image's high-dimensional features. The cross-modal alignment processing unit then concatenates the mapped image high-dimensional features with the temporal high-dimensional features, performing feature fusion through a 1536×768-dimensional fully connected layer, outputting a 768-dimensional fused feature.

[0028] The spatiotemporal graph computation unit receives the fused features output from the cross-modal alignment processing unit and constructs a spatiotemporal heterogeneous graph with the artifact surface area as nodes and environmental influencing factors as edges. The spatiotemporal graph computation unit divides the artifact surface into nine rectangular regions of equal area, each region serving as an artifact surface area node; simultaneously, it sets three environmental influencing factor nodes: temperature, humidity, and illumination, resulting in a total of 12 nodes in the spatiotemporal heterogeneous graph. The edges between nodes are divided into spatial edges and temporal edges. Spatial edges connect each artifact surface area node to all environmental influencing factor nodes, representing the direct impact of environmental factors on the artifact area; temporal edges connect each artifact surface area node to its corresponding node in adjacent time windows, representing the temporal evolution of the area's state. The spatiotemporal graph computation unit uses the fused features corresponding to each node as the node's initial feature vector and loads a graph attention network, which contains two heterogeneous message passing layers. The node features and edge information of the spatiotemporally heterogeneous graph are input into a graph attention network. The first heterogeneous message passing layer aggregates the features of the first-order neighbors of each node, and the second heterogeneous message passing layer further aggregates the features of the second-order neighbors of each node. After two layers of message passing, the final state feature of each node is output. The spatiotemporal graph computation end calculates the spatiotemporal evolution deviation of the node state based on the final state feature of the node. It compares the state features of all nodes at the current time step with the state features of the corresponding nodes at the previous time step, calculates the Euclidean distance between the two, and performs a weighted average of the distances of all nodes to obtain the spatiotemporal evolution deviation.

[0029] The early warning signal output terminal receives the spatiotemporal evolution deviation from the spatiotemporal map calculation terminal and compares the real-time calculated spatiotemporal evolution deviation with the dynamic adaptive baseline. When the spatiotemporal evolution deviation exceeds the dynamic adaptive baseline, the early warning signal output terminal identifies the corresponding degradation type based on the change pattern of node state characteristics and generates an early warning signal frame containing information such as degradation type, timestamp, and deviation value. The early warning signal output terminal sends the early warning signal frame to the display terminal and storage device of the monitoring center through the security alarm bus. The personnel on duty at the monitoring center take corresponding processing measures based on the content of the early warning signal frame. When the spatiotemporal evolution deviation does not exceed the dynamic adaptive baseline, the early warning signal output terminal stores the spatiotemporal evolution deviation of the current time step into the historical database for subsequent baseline update calculations.

[0030] This embodiment establishes a feature-level correlation between environmental stress and the physical appearance of cultural relics by mapping environmental time-series data and visual image data to a unified high-dimensional feature space for cross-modal alignment. A spatiotemporal heterogeneous graph is constructed, and spatiotemporal evolution deviation is calculated using a graph attention network. This transforms multi-source heterogeneous sensor data into multi-factor coupled correlation features, enabling the system to capture the coupled correlation between minute fluctuations in temperature and humidity and changes in visual appearance.

[0031] refer to Figure 2In one embodiment, the temporal convolutional network includes a multi-scale dilated convolutional layer and a channel attention layer. The multi-scale dilated convolutional layer receives preprocessed temperature, humidity, and illumination time-series signals. The input signal has a dimension of T×3, where T is the number of sampling points within the time window, and 3 represents the three signal channels: temperature, humidity, and illumination. The multi-scale dilated convolutional layer uses parallel convolutional kernels with different dilation coefficients to process the input time-series signals and extract multi-scale temporal fluctuation features. The parameter configuration of the multi-scale dilated convolutional layer is shown in Table 1.

[0032] Table 1. Parameter Configuration Table for Multiscale Dilated Convolutional Layers

[0033]

[0034] The four parallel convolutional kernel branches of the multi-scale dilated convolutional layer use the same kernel size but different dilation coefficients to capture fluctuation features at different time scales. The branch with a dilation coefficient of 1 captures short-term instantaneous fluctuations at a time scale of 3 sampling points, the branch with a dilation coefficient of 2 captures medium-term fluctuations at a time scale of 7 sampling points, the branch with a dilation coefficient of 4 captures long-term trend fluctuations at a time scale of 15 sampling points, and the branch with a dilation coefficient of 8 captures ultra-long-term cumulative effects at a time scale of 31 sampling points. The stride of each convolutional kernel branch is set to 1, and the same padding method is used to ensure that the time dimension of the output feature is consistent with the time dimension of the input signal. The output features of the four branches are concatenated along the channel dimension to obtain multi-scale temporal fluctuation features with a dimension of T×256.

[0035] Multi-scale temporal fluctuation features are input into the channel attention layer. The channel attention layer performs global average pooling and fully connected mapping on the multi-scale temporal fluctuation features along the channel dimension to generate a channel weight vector. The weight calculation process of the channel attention layer is shown in the following equation:

[0036]

[0037] in, The weighting coefficient for the c-th channel is... It is the sigmoid activation function. This is the weight matrix for the first fully connected layer, with dimensions 256×32. This is the weight matrix for the second fully connected layer, with dimensions 32×256. It is a linear rectified activation function. This is a global average pooling operation. The multi-scale time-domain fluctuation characteristics of the c-th channel have a dimension of T×1.

[0038] Global average pooling compresses the T×1 dimensional features of each channel into a scalar, representing the global feature information of that channel. After dimensionality reduction mapping in the first fully connected layer and nonlinear transformation by the ReLU activation function, the feature dimension is reduced from 256 to 32. Then, after dimensionality increase mapping in the second fully connected layer, the feature dimension is restored to 256. Finally, the sigmoid function maps the values ​​of each dimension to between 0 and 1, generating a 256-dimensional channel weight vector. The channel attention layer multiplies the channel weight vector with the multi-scale temporal fluctuation features channel by channel, multiplying the feature of each channel with its corresponding weight coefficient to obtain a T×256-dimensional weighted temporal feature. The weighted temporal feature is input into a 256×768-dimensional fully connected layer, mapped to a 768-dimensional high-dimensional feature space, and output to the feature fusion module at the cross-modal alignment processing end.

[0039] This embodiment extracts temporal fluctuation features at different time scales through multi-scale dilated convolutional layers, and dynamically weights the features of different channels by combining a channel attention layer. This can enhance the expression of features related to the deterioration of cultural relics in time-series signals and suppress interference from irrelevant noise.

[0040] refer to Figure 3 In another embodiment, the encoder of the pre-trained vision-language large model includes a local masking reconstruction subnetwork. The cross-modal alignment process resizes the pre-processed artifact surface image to 224×224 pixels and divides it into a 16×16 sequence of non-overlapping image patches, resulting in 196 image patches, each 14×14 pixels. The cross-modal alignment process randomly masks 30% of the image patch sequence; the masked patches are replaced with zero vectors, while the unmasked patches retain their original pixel values. The unmasked image patch sequence is flattened into a 768-dimensional vector, and after adding learnable positional encodings, it is input into the local masking reconstruction subnetwork.

[0041] The local masking reconstruction subnetwork adopts the same Transformer structure as the pre-trained vision-language large model encoder, containing 8 Transformer encoder layers, each with 8 attention heads, and a hidden layer dimension of 768. The local masking reconstruction subnetwork receives a sequence of undisturbed image patch vectors, calculates the correlation features between undisturbed image patches through a multi-head self-attention mechanism, and outputs a predicted feature vector for each image patch location after layer-by-layer processing by the 8 Transformer encoder layers. The local masking reconstruction subnetwork maps the predicted feature vectors to pixel value vectors through a fully connected layer of dimension 768×196, predicting the pixel values ​​of the masked image patches.

[0042] The local masking reconstruction subnetwork calculates the mean squared error between the predicted and actual pixel values ​​as the reconstruction error. The calculation process involves subtracting the predicted pixel value of the masked image patch from its corresponding actual pixel value, squared the result, and then averaging the squared errors across all masked image patches. This reconstruction error is backpropagated to the encoder of the pre-trained visual-language large-scale model, updating all encoder parameters and enabling the encoder to learn the intrinsic relationship between unmasked and masked regions in the image. After fine-tuning through the local masking reconstruction task, the pre-trained visual-language large-scale model encoder can extract microscopic texture features from the surface of artifacts and output high-dimensional image features containing microscopic texture information.

[0043] This embodiment fine-tunes the encoder of the pre-trained visual-language large model through a local mask reconstruction task, forcing the model to learn the intrinsic connections between different regions in the image, enabling it to extract the micro-texture features of the artifact surface and improve its ability to perceive early minor degradation.

[0044] refer to Figure 4 In a preferred embodiment, the cross-modal alignment processing end performs cross-modal contrast alignment. The cross-modal alignment processing end constructs positive sample pairs by combining temperature, humidity, and illumination time-series signals within the same time window with image signals from the artifact surface, and negative sample pairs by combining time-series signals from different time windows with image signals. Each positive sample pair contains a time-series feature vector and a corresponding image feature vector, and each negative sample pair contains a time-series feature vector and an image feature vector from a different time window. The cross-modal alignment processing end randomly selects 1024 samples from the historical database to construct a contrast learning dataset containing 1024 positive sample pairs and 1024 × 1023 negative sample pairs.

[0045] The cross-modal alignment process projects positive and negative sample pairs onto a shared high-dimensional feature space, calculating the cosine similarity between positive and negative feature vectors. The cosine similarity is calculated by dividing the dot product of the two feature vectors by the product of their L2 norms. The cross-modal alignment process then uses a contrastive learning loss function to reduce the distance between positive and negative feature vectors and increase the distance between them. The contrastive learning loss function is calculated as follows:

[0046]

[0047] in, To compare the learning loss function values, The number of samples in a batch. Let be the temporal feature vector of the i-th sample. Let i be the image feature vector corresponding to the i-th sample. This is the cosine similarity calculation function. This is the temperature coefficient, with a value of 0.07.

[0048] The contrastive learning loss function employs InfoNCE loss. For each temporal feature vector, its corresponding image feature vector is considered a positive sample, while all other image feature vectors are considered negative samples. The loss function maximizes the ratio of the similarity between positive sample pairs to the sum of the similarities of all sample pairs, thus bringing positive sample feature vectors closer together in the high-dimensional feature space and distancing negative sample feature vectors from each other. In the cross-modal alignment process, the contrastive learning loss is backpropagated to the temporal convolutional network and the encoder of the pre-trained vision-language large model, updating the parameters of both networks and optimizing the distribution of the feature space. After cross-modal contrastive alignment, temporal features and image features within the same time window have high similarity in the high-dimensional space, while features from different time windows have low similarity, resulting in the output of aligned fused features.

[0049] This embodiment constructs positive and negative sample pairs for comparative learning, which can bring environmental temporal features and image features within the same time window closer in high-dimensional space, and push features from different time windows further apart, thereby achieving effective alignment of cross-modal features and establishing feature-level correlation between environmental stress and cultural relic appearance.

[0050] refer to Figure 5 Furthermore, the graph attention network includes a heterogeneous message passing layer. In the spatiotemporal heterogeneous graph, the edges between nodes on the surface of the artifact and nodes representing environmental influencing factors are classified as spatial edges, and the edges between nodes on the surface of the artifact within adjacent time windows are classified as temporal edges. The heterogeneous message passing layer sets independent attention weight calculation matrices for spatial edges and temporal edges respectively. The attention weight calculation matrix for spatial edges has a dimension of 768×768, and the attention weight calculation matrix for temporal edges also has a dimension of 768×768.

[0051] The heterogeneous message passing layer receives node features and edge information from the spatiotemporal heterogeneous graph. For each node, it collects the feature vectors of all its neighboring nodes. For each neighboring node, it selects the corresponding attention weight calculation matrix based on the edge type, and maps the feature vectors of the central node and its neighboring nodes to the attention space through linear transformations. It calculates the attention score between the central node and its neighboring nodes by concatenating the mapped features of the central node and its neighboring nodes, multiplying this by the attention vector, and then applying the LeakyReLU activation function. Finally, it performs softmax normalization on the attention scores of all neighboring nodes for each node to obtain the initial attention weights.

[0052] The heterogeneous message passing layer introduces an environmental stress propagation attenuation factor to correct the initial attention weights. The initial attention weights are multiplied by the corresponding environmental stress propagation attenuation factor to obtain the corrected attention weights. The feature vectors of each adjacent node are multiplied by the corrected attention weights and then summed to obtain the aggregated features. The aggregated features are input into the ReLU activation function, and the updated features of the center node are output. The message passing process of the heterogeneous message passing layer is shown in the following equation:

[0053]

[0054] in, Let i be the updated feature vector of the center node i. Let i be the set of all neighboring nodes of the central node i. Let i be the initial attention weight between the center node i and its neighboring node j. The environmental stress transfer attenuation factor between the central node i and the adjacent node j. Calculate the attention weight matrix for the corresponding edge type. Let be the feature vector of the neighboring node j. It is a linear rectification activation function.

[0055] After processing through two heterogeneous message passing layers, the graph attention network outputs the final state feature of each node. The spatiotemporal graph computation end calculates the spatiotemporal evolution deviation based on the final state feature of the nodes, as shown in the following formula:

[0056]

[0057] in, Let be the spatiotemporal evolution deviation at time step t. This represents the total number of nodes in the spatiotemporal heterogeneity graph. Let be the weight coefficient for the i-th node. The weight coefficient for the cultural relic surface area node is set to 0.15, and the weight coefficient for the environmental impact factor node is set to 0.05. Let be the feature vector of the i-th node at time step t. Let i be the feature vector of the i-th node at time step t-1. It is an L2 norm.

[0058] This embodiment sets independent attention weight calculation matrices for spatial edges and temporal edges respectively, which can capture the feature correlations in the spatial and temporal dimensions respectively; it introduces an environmental stress transmission attenuation factor to correct the attention weights, so that the graph calculation process conforms to the stress transmission law of the real physical field and improves the accuracy of node state evolution calculation.

[0059] In one embodiment, the multi-scale dilated convolutional layer outputs multiple temporal fluctuation features using parallel convolutional kernels with different dilation coefficients. The four convolutional kernel branches of the multi-scale dilated convolutional layer each output four temporal fluctuation features, each with a dimension of T×64. The feature variance of each temporal fluctuation feature in the time dimension is calculated as follows: the average of all values ​​for each feature in the time dimension is calculated, each value is subtracted from the average, the square is taken, and the average of all squared values ​​is calculated. The variances of the four features are input into the softmax normalization function to generate dynamic scale weight coefficients. The calculation process for the dynamic scale weight coefficients is shown in the following formula:

[0060]

[0061] in, The dynamic scale weight coefficients of the k-th convolutional kernel branch are... Let be the variance of the temporal fluctuation characteristics of the output of the k-th convolutional kernel branch. This is the index of the convolution kernel branch, with values ​​ranging from 1 to 4.

[0062] The dynamic scale weight coefficients range from 0 to 1, and the sum of the weight coefficients for the four branches is 1. Branches with larger variance correspond to time scales with more volatile fluctuations and have larger weight coefficients; branches with smaller variance correspond to time scales with more gradual fluctuations and have smaller weight coefficients. Each temporal fluctuation feature is multiplied by its corresponding dynamic scale weight coefficient to obtain a weighted four-way feature. These four weighted features are then concatenated along the channel dimension to obtain a temporal feature representation with a dimension of T×256, fused with multi-scale dynamic weights. This temporal feature representation is then input into a 256×256 one-dimensional convolutional layer for dimensionality reduction mapping, outputting a feature of dimension T×256, which is then input into the channel attention layer.

[0063] This embodiment generates dynamic scale weight coefficients by calculating the variance of the time-domain fluctuation characteristics of each path. It can automatically adjust the weights of each branch according to the intensity of fluctuations at different time scales, thereby enhancing the ability to extract features from abnormal fluctuations.

[0064] In another embodiment, a cross-modal alignment processing stage constructs a knowledge graph of cultural relic deterioration. The knowledge graph is stored using a triplet structure, where each triple takes the form of <temperature and humidity change pattern, resulting in, image appearance deterioration pattern>. Temperature and humidity change patterns include rapid temperature rise, rapid temperature fall, rapid humidity rise, rapid humidity fall, periodic temperature fluctuations, and periodic humidity fluctuations; image appearance deterioration patterns include surface flaking, efflorescence, mold growth, color fading, and surface cracking. The knowledge graph defines the mapping relationship between each temperature and humidity change pattern and its corresponding image appearance deterioration pattern, and also stores similarity scores between different deterioration patterns.

[0065] The cross-modal alignment processing unit extracts the temperature and humidity change patterns within the current time window, calculates the first and second differences of the temperature and humidity time-series signals, and identifies the current temperature and humidity change pattern based on the absolute value and trend of the differences. It then retrieves all image appearance degradation patterns matching the current temperature and humidity change pattern from the cultural relic degradation knowledge graph, calculates the feature similarity between these degradation patterns and the current cultural relic surface image signal, and selects the three degradation patterns with the highest similarity as easily confused image appearance degradation patterns. The cross-modal alignment processing unit then obtains the image signals corresponding to these easily confused image appearance degradation patterns from the historical database, constructing hard negative sample pairs between the current cultural relic surface image signal and the image signals corresponding to these easily confused image appearance degradation patterns.

[0066] The cross-modal alignment processing stage performs feature space propagation calculations on the hard negative sample pairs input contrastive learning loss function, adding weight coefficients for hard negative samples to the original contrastive learning loss function, thus increasing the proportion of the loss for hard negative sample pairs in the total loss. The updated loss function is then backpropagated to the temporal convolutional network and the encoder of the pre-trained vision-language large model to update network parameters and further optimize the distribution of the feature space, ensuring that easily confused degraded modes are separated from each other in the feature space.

[0067] This embodiment constructs a knowledge graph of cultural relic deterioration and retrieves easily confused image deterioration patterns that match the current temperature and humidity change patterns as hard negative samples. This can further optimize the distribution of the feature space, improve the accuracy of cross-modal feature alignment, and reduce false alarms.

[0068] In a preferred embodiment, the environmental stress transmission attenuation factor is calculated based on spatial physical properties. The spatiotemporal map calculation terminal pre-collects spatial layout data of the artifact's preservation environment, including the coordinates of various areas on the artifact's surface, the installation coordinates of environmental sensors, the structural dimensions of the display case, and the material distribution data inside the display case. Based on the coordinate data of the nodes on the artifact's surface and the nodes of environmental influencing factors, the Euclidean distance between them is calculated as the physical distance data. Based on the material distribution data inside the display case, the physical barrier material properties between the nodes on the artifact's surface and the nodes of environmental influencing factors are determined.

[0069] The spatiotemporal map calculation unit constructs a distance decay function based on physical distance data. The distance decay function adopts an exponential decay form, and the calculation process is shown in the following formula:

[0070]

[0071] in, This is the distance attenuation coefficient. This is the distance attenuation constant, with a value of 0.01. The distance between nodes on the surface of the cultural relic and nodes representing environmental factors is expressed in centimeters.

[0072] The spatiotemporal diagram calculation terminal queries the material barrier coefficient table based on the physical barrier material property data to obtain the corresponding material attenuation coefficient. The barrier coefficients of common cultural relic preservation environment materials are shown in Table 2.

[0073] Table 2. Barrier Coefficients of Common Cultural Relics Preservation Environments

[0074]

[0075] The material barrier coefficient ranges from 0 to 1, with a higher value indicating a stronger barrier to the corresponding environmental factors. The appropriate barrier coefficient is selected based on the type of environmental factor: temperature barrier coefficient for temperature, humidity barrier coefficient for humidity, and light barrier coefficient for illumination. The distance attenuation coefficient is multiplied by the material attenuation coefficient to generate the environmental stress transmission attenuation factor. In the heterogeneous message passing layer, the calculated environmental stress transmission attenuation factor is multiplied by the corresponding initial attention weight to correct the attention weights.

[0076] This embodiment calculates the environmental stress transmission attenuation factor based on spatial physical properties, taking into account the influence of physical distance and material barriers on environmental stress transmission. It can more accurately reflect the magnitude of environmental stress on different surface areas of cultural relics and improve the physical rationality of the calculation of spatiotemporal evolution deviation.

[0077] refer to Figure 6 Furthermore, the early warning signal output terminal includes protocol encapsulation logic and hierarchical routing logic. The protocol encapsulation logic predefines the format of the early warning signal frame, which consists of a frame header and a frame body. The frame header includes a synchronization word, frame length, risk level identifier, degradation type code, timestamp, and checksum field. The synchronization word field is 2 bytes long and has a fixed value of 0xAA55; the frame length field is 2 bytes long and indicates the total number of bytes in the entire early warning signal frame; the risk level identifier field is 1 byte long and has a value of 1, 2, or 3, corresponding to the three risk levels of general, important, and urgent, respectively; the degradation type code field is 1 byte long and corresponds to different types of cultural relic degradation; the timestamp field is 8 bytes long and indicates the time when the early warning signal was generated; the checksum field is 2 bytes long and is used to verify the integrity of the frame data. The frame body includes information such as the spatiotemporal evolution deviation value, the state characteristics of each node, and the coordinates of abnormal areas. The length of the frame body is dynamically adjusted according to the actual content.

[0078] The protocol encapsulation logic queries a preset risk level mapping table based on the identified degradation type to obtain the corresponding risk level identifier. The mapping relationship between cultural relic degradation types and risk levels is shown in Table 3.

[0079] Table 3 Mapping Table of Cultural Relics Deterioration Types and Risk Levels

[0080]

[0081] The protocol encapsulation logic writes the acquired risk level identifier and degradation type code into the frame header field of the warning signal frame, and writes information such as the spatiotemporal evolution deviation value, node status characteristics and abnormal area coordinates into the frame body field. It also calculates the checksum of the frame data and writes it into the checksum field to complete the encapsulation of the warning signal frame.

[0082] The hierarchical routing logic parses the risk level identifier in the header of the warning signal frame and selects the corresponding communication link based on the risk level. The security alarm bus has three communication links: a main communication link, a backup communication link, and a dedicated high-priority link. The main and backup communication links are used to transmit general and important level warning signals and routine monitoring data. The dedicated high-priority link is an independent physical channel used only for transmitting emergency level warning signals. When the risk level identifier is level 3 (the highest level), the hierarchical routing logic switches the security alarm bus's main and backup communication links to the dedicated high-priority link, sending the warning signal frame through the dedicated high-priority link. When the risk level identifier is level 1 or 2, the hierarchical routing logic sends the warning signal frame through the main communication link. If the main communication link fails, it automatically switches to the backup communication link.

[0083] This embodiment achieves standardized transmission of early warning information by mapping degradation type to risk level and encapsulating it into the frame header field of the early warning signal frame; the hierarchical routing logic switches communication links according to risk level, which can ensure the timely delivery of high-priority early warning signals.

[0084] In one embodiment, the early warning signal output includes baseline update logic. The baseline update logic slides along the time axis to extract the spatiotemporal evolution deviation sequence within a historical time window. The sliding time window size is set to 30 days, meaning that it extracts the spatiotemporal evolution deviation values ​​for all time steps within the past 30 days, forming a deviation sequence of length 720 (24 time steps per day). The baseline update logic performs STL time series decomposition on this deviation sequence, decomposing the sequence into three parts: a trend term, a periodic term, and a residual term.

[0085] STL time series decomposition employs a locally weighted regression scatter smoothing method. First, Loess smoothing is used to extract the periodic term of the series, with the length of the periodic term set to 7 days, corresponding to the cyclical fluctuations of the environment. Then, the periodic term is subtracted from the original series to obtain a series free of periodic fluctuations. Next, Loess smoothing is performed on the series free of periodic fluctuations to extract the trend term, which reflects the long-term trend of deviation. Finally, the trend term is subtracted from the series free of periodic fluctuations to obtain the residual term, which reflects the impact of random noise.

[0086] The baseline update logic removes periodic and residual terms, retaining only the trend term sequence. The 95th percentile value of the trend term sequence at the current time step is calculated as the dynamic adaptive baseline, as shown in the following formula:

[0087]

[0088] in, For the dynamic adaptive baseline at time step t, The function for calculating the 95th quantile. This is the trend term sequence corresponding to the t-th time step.

[0089] The 95th percentile indicates that 95% of the values ​​in the trend term sequence are less than or equal to this quantile value. The baseline update logic performs a baseline update operation daily, recalculating the dynamic adaptive baseline using the latest 30-day deviation sequence and replacing the old baseline value. The warning signal output compares the real-time calculated spatiotemporal evolution deviation with the updated dynamic adaptive baseline to determine whether to generate a warning signal.

[0090] This embodiment performs STL time series decomposition on the historical spatiotemporal evolution deviation sequence, removes periodic and residual terms, retains the trend term and calculates the quantile as a dynamic adaptive baseline. This can eliminate the interference of environmental periodic fluctuations and random noise, and make the baseline adaptively adjust with the changes in the preservation status of cultural relics.

Claims

1. A smart early warning system for cultural relic safety based on an AI large-scale model, characterized in that: include: The multi-source signal acquisition end extracts temperature, humidity, and light timing signals collected by environmental sensors and image signals of the artifact surface collected by monitoring cameras; In the cross-modal alignment processing end, the encoder of the pre-trained visual-language large model is used to map the image signal to a high-dimensional feature space, and the temporal signal is mapped to the same high-dimensional feature space through a temporal convolutional network to complete cross-modal feature alignment and output fused features; The spatiotemporal graph calculation terminal constructs a spatiotemporal heterogeneous graph with the surface area of ​​cultural relics as nodes and environmental influencing factors as edges. The fused features are input into the graph attention network to calculate the spatiotemporal evolution deviation of the node states. At the warning signal output end, when the spatiotemporal evolution deviation exceeds the dynamic adaptive baseline, a warning signal frame of the corresponding degradation type is generated and the warning signal frame is sent through the security alarm bus.

2. The intelligent early warning system for cultural relic safety based on an AI large-scale model as described in claim 1, characterized in that, The temporal convolutional network includes multi-scale dilated convolutional layers and channel attention layers; The multi-scale dilated convolutional layer uses parallel convolutional kernels with different dilation coefficients to process the temperature, humidity, and illumination time-series signals respectively, and extracts multi-scale temporal fluctuation features. The channel attention layer performs global average pooling and fully connected mapping on the multi-scale temporal fluctuation features in the channel dimension to generate a channel weight vector. The channel weight vector is then multiplied with the multi-scale temporal fluctuation features channel by channel to output a weighted temporal feature. The weighted temporal feature is then mapped to the high-dimensional feature space.

3. The intelligent early warning system for cultural relic safety based on an AI large-scale model as described in claim 1, characterized in that, The encoder of the pre-trained vision-language large model includes a local mask reconstruction subnetwork; The image signal of the artifact surface is segmented into a sequence of non-overlapping image patches. A portion of the image patch sequence is randomly masked. The unmasked image patch sequence is input into the local masking reconstruction sub-network. The local masking reconstruction sub-network predicts the pixel values ​​of the masked image patch sequence based on the unmasked image patch sequence, calculates the reconstruction error between the predicted pixel values ​​and the actual pixel values, backpropagates the reconstruction error to update the parameters of the encoder of the pre-trained visual-language large model, and outputs high-dimensional image features containing micro-texture features.

4. The intelligent early warning system for cultural relic safety based on an AI large-scale model as described in claim 1, characterized in that, The cross-modal alignment processing terminal performs cross-modal comparison alignment; The temperature, humidity, and light timing signals within the same time window are paired with the image signals of the artifact surface to form positive sample pairs, while signals from different time windows are paired with negative sample pairs. The positive sample pairs and the negative sample pairs are projected onto the shared high-dimensional feature space, and the cosine similarity between the positive sample feature vectors and the cosine similarity between the negative sample feature vectors are calculated. Based on the contrastive learning loss function, the distance between the positive sample feature vectors is reduced and the distance between the negative sample feature vectors is increased, and the aligned fused features are output.

5. The intelligent early warning system for cultural relic safety based on an AI large-scale model according to claim 1, characterized in that, The graph attention network includes a heterogeneous message passing layer; In the spatiotemporal heterogeneous diagram, the edges connecting the cultural relic surface area nodes and the environmental influencing factor nodes are divided into spatial edges and temporal edges; The heterogeneous message passing layer sets independent attention weight calculation matrices for the spatial edge and the temporal edge, respectively. Based on the attention weight calculation matrices, the fusion features of adjacent nodes are weighted and aggregated, and an environmental stress transmission attenuation factor is introduced to correct the attention weights. The corrected weighted aggregated features are input into the activation function, and the spatiotemporal evolution deviation of the node state is output.

6. The intelligent early warning system for cultural relic safety based on an AI large model according to claim 2, characterized in that, The multi-scale dilated convolutional layer outputs multiple temporal fluctuation characteristics from parallel convolutional kernels with different dilation coefficients. Calculate the feature variance of the time-domain fluctuation features of each of the multiple paths in the time dimension, and input the feature variance into the softmax normalization function to generate dynamic scale weight coefficients. The multi-path temporal fluctuation features of each path are multiplied by the corresponding dynamic scale weight coefficients and then concatenated. The concatenated features are then dimensionality-reduced and mapped through a one-dimensional convolutional layer to output a temporal feature representation that fuses multi-scale dynamic weights, which is then input into the channel attention layer.

7. The intelligent early warning system for cultural relic safety based on an AI large model as described in claim 4, characterized in that, The cross-modal alignment processing end constructs a knowledge graph of cultural relic deterioration, which defines the mapping relationship between temperature and humidity change patterns and image appearance deterioration patterns. Retrieve easily confused image appearance degradation patterns that match the current temperature and humidity change pattern from the knowledge graph of cultural relic degradation, and construct hard negative sample pairs by combining the current cultural relic surface image signal with the image signal corresponding to the easily confused image appearance degradation pattern. The hard negative sample pairs are input into the contrastive learning loss function to perform feature space extrapolation calculations, thereby updating the distribution of the fused features.

8. The intelligent early warning system for cultural relic safety based on an AI large model as described in claim 5, characterized in that, The environmental stress transmission attenuation factor is calculated based on spatial physical properties. Collect physical distance data between the nodes on the surface of the cultural relic and the nodes of the environmental influencing factors, as well as physical barrier material property data; A distance attenuation function is constructed based on the physical distance data. A material attenuation coefficient is obtained by querying the material barrier coefficient table based on the physical barrier material attribute data. The output of the distance attenuation function is multiplied by the material attenuation coefficient to generate the environmental stress transmission attenuation factor, which is then multiplied by the attention weight in the heterogeneous message passing layer.

9. The intelligent early warning system for cultural relic safety based on an AI large-scale model according to claim 1, characterized in that, The warning signal output terminal includes protocol encapsulation logic and hierarchical routing logic; The protocol encapsulation logic queries a preset risk level mapping table to obtain a risk level identifier based on the degradation type, and encapsulates the risk level identifier and the degradation type into the frame header field of the warning signal frame. The hierarchical routing logic parses the risk level identifier in the frame header field. When the risk level identifier is the highest level, it switches the primary and backup communication links of the security alarm bus to a dedicated high-priority link and sends the warning signal frame through the dedicated high-priority link.

10. The intelligent early warning system for cultural relic safety based on an AI large model according to claim 1, characterized in that, The warning signal output terminal includes baseline update logic; The baseline update logic slides along the time axis to extract the spatiotemporal evolution deviation sequence within the historical time window, performs STL time series decomposition on the spatiotemporal evolution deviation sequence, and separates the trend term, periodic term and residual term; The periodic term and the residual term are removed, the trend term is retained, the quantile value of the trend term at the current time step is calculated as the dynamic adaptive baseline, and the spatiotemporal evolution deviation calculated in real time is compared with the dynamic adaptive baseline.

Images