Scenic area facility safety monitoring method and system based on Internet of Things

Abstract

The invention discloses a scenic spot facility safety monitoring method and system based on the Internet of Things, and the method comprises the steps: collecting facility state data through a fixed sensor node, and extracting deformation trend features through a time sequence difference algorithm; crowdsourcing perception data is collected through a tourist terminal, and facility micro-vibration characteristics are extracted through quality control processing; constructing a three-level heterogeneous communication architecture of a cellular network, a LoRa Mesh network and a satellite short message to transmit data; a cross-modal attention mechanism is adopted to fuse vibration features and video features, and vibration sources are distinguished through semantic consistency verification; and constructing a space-time heterogeneous graph comprising facility nodes, tourist area nodes and environment nodes, and performing space-time feature learning by adopting a graph neural network to output a risk score. According to the invention, wide-area perception, reliable transmission, accurate identification and coupling evaluation of scenic spot facility safety monitoring are realized.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) technology, and in particular to a method and system for safety monitoring of scenic area facilities based on IoT. Background Technology

[0002] With the rapid development of tourism, the number of facilities such as glass walkways, viewing platforms, and cable car stations in scenic areas is constantly increasing, making facility safety monitoring a crucial aspect of scenic area management. The Internet of Things (IoT) technology provides a new technical means for monitoring the safety of scenic area facilities. By deploying sensor networks to collect facility status data in real time, potential safety hazards can be detected promptly.

[0003] The existing safety monitoring technology for scenic area facilities has the following shortcomings: First, the sensing coverage is insufficient. Existing technologies mainly rely on fixed sensor nodes to collect facility status data. Due to limitations in deployment costs and construction conditions, the number of sensor nodes is limited, making it difficult to achieve full-area coverage monitoring of the facility.

[0004] Second, data transmission is unreliable. Mountainous scenic areas have complex terrain, and cellular networks have signal blind spots in valleys and canyons. Traditional wireless sensor network routing algorithms do not consider the impact of mountainous terrain on signal propagation, resulting in a decrease in data transmission success rate.

[0005] Third, anomaly identification is inaccurate. Existing multi-sensor data fusion technologies typically employ a simple superposition method at the signal or feature level, which cannot distinguish between vibrations caused by visitor behavior and vibrations caused by structural anomalies in facilities, resulting in a high false alarm rate.

[0006] Fourth, the risk assessment has a single dimension. Existing technologies treat facility status monitoring, visitor management, and environmental monitoring as independent systems, failing to consider the coupled impact of visitor load and environmental conditions on facility safety risks.

[0007] The aforementioned technical deficiencies are interconnected. Insufficient sensing coverage leads to incomplete data collection, unreliable data transmission results in missing information, inaccurate anomaly identification leads to the misjudgment and filtering of valid information, and a single risk assessment dimension makes it impossible to accurately predict facility safety risks. Therefore, there is an urgent need to provide an IoT-based method and system for monitoring the safety of scenic area facilities, which should collaboratively optimize multiple aspects such as sensing, transmission, fusion, and assessment to achieve accurate monitoring and timely early warning of safety risks to scenic area facilities. Summary of the Invention

[0008] To achieve the above objectives, in a first aspect, the present invention provides a method for safety monitoring of scenic area facilities based on the Internet of Things, comprising the following steps: S1. Fixed Node Data Acquisition and Deformation Trend Extraction: Fixed sensor nodes deployed at key locations of scenic area facilities collect facility status data and environmental parameter data. Displacement data is processed using a time-series difference algorithm to extract deformation trend features. The time-series difference algorithm eliminates periodic systematic errors by performing periodic difference on the displacement measurement sequence, extracts smooth trend signals through adaptive Kalman filtering, and determines the existence of significant deformation trends through statistical significance testing. The deformation trend features are feature vectors characterizing the deformation state of the facilities, including deformation rate, deformation direction, and confidence level. S2. Crowdsourced Data Collection and Quality Control: Crowdsourced sensing data is collected through visitor terminals. The crowdsourced sensing data includes acceleration signals and their collection location information. The crowdsourced sensing data is processed sequentially by frequency band separation, equipment normalization, spatial clustering, and outlier removal to extract facility micro-vibration features. The facility micro-vibration features are feature vectors characterizing the vibration state of the facility, including dominant frequency, secondary frequency, spectral centroid, root mean square value of vibration, peak factor, and waveform factor. S3. Heterogeneous network transmission: Construct a three-level heterogeneous communication architecture of cellular network, LoRa Mesh network and satellite short message. Select the transmission channel according to the network status and data priority, and transmit the deformation trend characteristics obtained in step S1, the facility micro-vibration characteristics obtained in step S2, the video monitoring data collected by the video acquisition equipment deployed in the scenic area, the tourist data and environmental data provided by the scenic area management system to the cloud. S4. Cross-modal semantic fusion: Multimodal feature encoding is performed on the facility micro-vibration features and video monitoring data transmitted to the cloud in step S3. A cross-modal attention mechanism is used to establish the semantic correspondence between the vibration feature vector and the video feature vector. Semantic consistency verification is used to distinguish between vibrations caused by tourist behavior and vibrations caused by facility structural abnormalities, and the fused features are output. The fused features include fused feature vector, vibration feature type, behavior recognition result and semantic consistency judgment result. S5. Three-element coupling risk assessment: Construct a spatiotemporal heterogeneous graph, which is a graph structure containing three different types of nodes: facility nodes, visitor area nodes, and environment nodes. The node features of the facility nodes include the deformation trend features obtained in step S1 and the fusion features obtained in step S4. The node features of the visitor area nodes include visitor data, and the node features of the environment nodes include environmental data. A graph neural network is used to learn the spatiotemporal features of the spatiotemporal heterogeneous graph, outputting a risk score for each facility node, and generating safety monitoring and early warning information based on the risk scores.

[0009] Further, in step S1, the time-series difference algorithm includes: The displacement measurement sequence is subjected to periodic difference calculation, with the difference period set to 24 sampling points, corresponding to 24 hours; An adaptive Kalman filter is applied to the difference sequence, and the state vector includes the estimated periodic displacement and the rate of change of periodic displacement; the process noise covariance matrix is ​​dynamically adjusted by monitoring the variance of the innovation sequence within the sliding window. A statistical significance test is performed on the periodic displacement change rate of the filtered output. When the absolute value of the test statistic exceeds the preset significance threshold and the duration exceeds the preset number of days threshold, a significant deformation trend is determined to exist.

[0010] Furthermore, in step S2, the quality control of the crowdsourced perception data includes: A bandpass filter is used to separate the original acceleration signal into frequency bands, preserving the characteristic frequency bands of the facility's structural vibration. The root mean square value of vibration is calculated from the data collected by multiple tourist terminals within the same facility area and the same time window. The root mean square value of vibration is then converted into a standard score for equipment normalization. The DBSCAN clustering algorithm is used to spatially cluster the data collection locations and remove isolated data points that do not belong to any cluster. A robust statistical method based on the absolute deviation of the median is used to identify and remove characteristic outliers.

[0011] Furthermore, in step S3, the LoRa Mesh network uses an altitude-weighted routing algorithm for route optimization. The altitude-weighted routing algorithm introduces an altitude penalty factor in the link cost calculation, and the altitude penalty factor is positively correlated with the altitude difference between two adjacent nodes.

[0012] Furthermore, in step S3, the selection strategy for the transmission channel is as follows: When the cellular network signal strength is higher than a preset signal strength threshold, transmission is made through the cellular network; When the cellular network signal strength is lower than the preset signal strength threshold or the connection fails, transmission is carried out through the LoRa Mesh network; When facility status data is detected to exceed a preset abnormal threshold and both the cellular network and LoRa Mesh network are unavailable, it is transmitted via satellite short message.

[0013] Furthermore, in step S4, the cross-modal attention mechanism includes: A one-dimensional convolutional neural network is used to encode the acceleration signal corresponding to the micro-vibration characteristics of the facility, and output the vibration feature vector. Video segments corresponding to the time periods of facility micro-vibration characteristics were extracted, and each frame of the image was encoded using a convolutional neural network to obtain the video feature matrix; Using the vibration feature vector as the query and each row of the video feature matrix as the key and value, we calculate the attention weights and perform weighted aggregation to output the fused feature vector.

[0014] Furthermore, in step S4, the semantic consistency verification includes: Abnormal vibration events are classified into vibration types based on peak factor, peak duration, and spectral characteristics. Vibration characteristic types include impact type, rhythmic type, and continuous type. Behavior recognition is performed on video surveillance data corresponding to the time period of the abnormal vibration event, and the probability of each behavior category is output. The source of vibration is determined by matching the vibration characteristic type with the behavior recognition result. If they match, the vibration is determined to be caused by the behavior of tourists and is filtered out. If they do not match, the vibration is determined to be a suspected structural abnormality of the facility and is included in the risk assessment.

[0015] Further, in step S5, the spatiotemporal heterogeneous graph includes facility-facility edges, visitor-facility edges, and environment-facility edges; facility-facility edges are used to connect two facility nodes that have a physical connection relationship; visitor-facility edges are used to connect visitor area nodes and facility nodes located in that area, and the edge weight is positively correlated with visitor density; environment-facility edges are used to connect environment nodes and facility nodes affected by that environmental factor.

[0016] Furthermore, in step S5, the graph neural network includes a heterogeneous graph attention network and a gated recurrent unit; the heterogeneous graph attention network configures different feature transformation matrices for different types of nodes and aggregates neighbor node information through the attention mechanism to learn spatial features; the gated recurrent unit performs temporal modeling of the feature vector sequences of each facility node at multiple time steps.

[0017] Secondly, the present invention also provides an Internet of Things-based scenic area facility safety monitoring system for performing the above-mentioned method, comprising: The fixed sensing module is used to execute step S1. It is deployed at key locations of scenic area facilities, collects facility status data and environmental parameter data through fixed sensor nodes, processes displacement data by executing a time-series difference algorithm, and outputs deformation trend characteristics. The crowdsourced perception module, which is used to execute step S2, is deployed in the cloud, receives crowdsourced perception data uploaded by tourist terminals, performs frequency band separation, equipment normalization, spatial clustering and outlier removal processing, and outputs facility micro-vibration characteristics. The heterogeneous transmission module is used to perform step S3, which includes three communication methods: cellular network communication, LoRa Mesh communication and satellite short message communication. It uses an altitude-weighted routing algorithm to optimize the routing of the LoRa Mesh network and automatically selects the transmission channel according to the network status and data priority. The cross-modal fusion module, which is used to execute step S4, is deployed in the cloud. It performs multimodal feature encoding on facility micro-vibration characteristics and video surveillance data, establishes the semantic correspondence between vibration feature vectors and video feature vectors using a cross-modal attention mechanism, distinguishes vibration sources through semantic consistency verification, and outputs fused features. The risk assessment module, used to execute step S5, is deployed in the cloud and constructs a spatiotemporal heterogeneous graph containing facility nodes, visitor area nodes, and environmental nodes. It uses a graph neural network to learn the spatiotemporal features of the spatiotemporal heterogeneous graph, outputs a risk score for each facility node, and generates safety monitoring and early warning information based on the risk score.

[0018] Beneficial effects: 1. Expanding the monitoring coverage of facilities: This invention expands the sensing coverage by combining fixed sensor nodes with visitor terminals. Fixed sensor nodes enable accurate monitoring of key locations, while visitor terminals utilize the natural distribution of visitors on the facilities to achieve wide-area coverage. The temporal difference algorithm extracts high-precision deformation trend features from low-precision measurement data, and crowdsourced data quality control processing solves the problems of equipment differences and location deviations, thus addressing the problem of insufficient sensing coverage in existing technologies.

[0019] 2. Improve the success rate of data transmission in mountainous environments: This invention improves transmission reliability through a three-level heterogeneous communication architecture of cellular network, LoRa Mesh network and satellite short message. The altitude-weighted routing algorithm prioritizes routing through nodes with similar altitudes to adapt to mountainous terrain. The three-level channel selection strategy automatically switches transmission channels according to network status and data priority, solving the problem of unreliable data transmission in existing technologies.

[0020] 3. Reduce false alarm rate caused by tourist behavior: This invention establishes a correspondence between vibration data and video data at the semantic level through a cross-modal attention mechanism. Semantic consistency verification determines the source of vibration based on the matching relationship between vibration type and behavior recognition results. Vibrations caused by tourist behavior are filtered out before risk assessment, thus solving the problem of inaccurate anomaly recognition in the prior art.

[0021] 4. Achieving a three-dimensional joint modeling of tourists, facilities, and environment: This invention constructs a spatiotemporal heterogeneous graph containing facility nodes, tourist area nodes, and environmental nodes. Tourist behavior and environmental factors are used as related variables affecting facility risk for joint modeling. A graph neural network learns the spatiotemporal features of the spatiotemporal heterogeneous graph to achieve a three-dimensional risk assessment of tourists, facilities, and environment, thus solving the problem of a single dimension of risk assessment in existing technologies. Attached Figure Description

[0022] Figure 1 A flowchart illustrating the steps of the method described in this invention is shown. Figure 2A schematic diagram of the system architecture described in this invention is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0024] This invention provides a method and system for safety monitoring of scenic area facilities based on the Internet of Things. By constructing a ubiquitous sensing network combining fixed sensor nodes and tourist terminals, a three-level heterogeneous communication architecture combining cellular networks, LoRa Mesh networks, and satellite short messages, a cross-modal semantic fusion mechanism combining vibration features and video features, and a spatiotemporal heterogeneous graph combining facility nodes, tourist area nodes, and environmental nodes, it achieves wide-area perception, reliable transmission, accurate identification, and coupled evaluation for safety monitoring of scenic area facilities.

[0025] This embodiment uses a mountainous scenic area as an example for application. The scenic area includes 47 facilities that need to be monitored, such as glass walkways, viewing platforms, wooden corridors, and cable car stations. The total area of ​​the scenic area is 12 square kilometers, with an elevation difference of 800 meters and an average daily visitor volume of 8,000.

[0026] Combination Figure 1 In a first aspect, the present invention provides a method for safety monitoring of scenic area facilities based on the Internet of Things, the specific steps of which are as follows: S1. Fixed Node Data Acquisition and Deformation Trend Extraction This step involves collecting facility status data and environmental parameter data by deploying fixed sensor nodes at key locations within the scenic area's facilities. The displacement data is then processed using a time-series difference algorithm to extract deformation trend features.

[0027] Each fixed sensor node includes a GNSS positioning module, a MEMS triaxial accelerometer, a MEMS dual-axis tilt sensor, an environmental sensor module, and an ultrasonic anemometer. The GNSS positioning module collects the three-dimensional position coordinates of key points in the facility; the MEMS triaxial accelerometer collects vibration acceleration signals; the MEMS dual-axis tilt sensor collects the tilt angle; the environmental sensor module collects temperature, humidity, and air pressure data; and the ultrasonic anemometer collects wind speed data. In this embodiment, 2 to 4 fixed sensor nodes are deployed for each monitoring facility, for a total of 156 nodes. The sampling period for the GNSS positioning module is set to 1 hour, the sampling frequency for the MEMS accelerometer is set to 100Hz, the sampling period for the tilt sensor is set to 10 minutes, and the sampling periods for the environmental sensor module and the anemometer are set to 5 minutes.

[0028] The civilian-grade GNSS positioning module used in this embodiment has a single measurement accuracy of ±2 meters, which cannot directly meet the millimeter-level accuracy requirements for facility deformation monitoring. Existing technologies typically use high-precision RTK positioning modules to solve this problem, but the cost of a single device can reach tens of thousands of yuan, making large-scale deployment in scenic areas difficult. This invention proposes a time-series differential algorithm, which utilizes the progressive accumulation characteristics of facility deformation and the periodicity of measurement errors. Through differential processing of long-term series data, it extracts high-precision deformation trend features from low-precision measurement data, significantly reducing equipment costs while ensuring monitoring accuracy.

[0029] The temporal difference algorithm includes three processing steps: periodic difference, adaptive Kalman filtering, and statistical significance testing.

[0030] Perform periodic difference calculation on the GNSS position measurement sequence: in, For the first The difference values ​​at each sampling time point, in meters; For the first The location measurement values ​​at each sampling time are in meters. For the first The location measurement values ​​at each sampling time are in meters. For the differential period, this embodiment uses 24 sampling points. Since the sampling period is 1 hour, the 24 sampling points correspond to 24 hours. Because the ionospheric delay and tropospheric delay in GNSS measurement errors are related to atmospheric temperature and have a significant diurnal periodicity, the periodic systematic errors related to temperature changes cancel each other out during the differential process through diurnal period differential calculation, resulting in a differential value. It reflects the cumulative displacement over 24 hours.

[0031] The GNSS positioning module outputs position coordinates in the X, Y, and Z directions, and performs periodic difference and adaptive Kalman filtering independently on each of the three directions. The filtering process is illustrated below using a single direction as an example.

[0032] An adaptive Kalman filter is applied to the difference sequence to extract the smoothed trend signal. The state vector is defined as follows: in, For the first The state vector at each moment; For the first The estimated periodic displacement at each moment represents a smoothed estimate of the cumulative displacement over 24 hours, in meters. For the first The rate of change of periodic displacement at a given moment represents the trend of the estimated periodic displacement over time, expressed in meters per hour.

[0033] The state transition equation is: in, This is the state transition matrix; For the first The state vector at each moment; For the first The process noise vector at each time step follows a function with zero mean and a covariance matrix of . The state transition matrix is ​​defined as follows: (The distribution is Gaussian.) in, The time interval between two consecutive filters is expressed in hours; in this embodiment, it is set to 1. This state transition model indicates that the estimated periodic displacement value accumulates according to the rate of change of periodic displacement, and the rate of change of periodic displacement remains stable over a short period of time.

[0034] The observation equation is: in, For the first The current value of the difference sequence is taken from the observations at each time point. The unit is meters; For the observation matrix, take The superscript T indicates matrix transpose, and represents the difference value. For the estimated periodic displacement Direct observation; For the first The observation noise at each time step follows a mean of zero and a variance of . Gaussian distribution, Based on the nominal accuracy setting of the GNSS module, this embodiment takes... square meters.

[0035] Initial process noise covariance matrix Set as a diagonal matrix: in, The process noise variance of the estimated periodic displacement is taken in this embodiment. square meters; The process noise variance is the periodic displacement change rate. In this embodiment, we take... Square meters per square hour.

[0036] Standard Kalman filtering uses a fixed process noise covariance matrix, and its filtering performance degrades when the system's dynamic characteristics change. This invention employs an adaptive mechanism to dynamically adjust the process noise covariance matrix, judging system dynamic changes and adjusting filtering parameters accordingly by monitoring the statistical characteristics of the innovation sequence. Innovation is defined as the difference between the actual observed value and the predicted observed value. in, For the first The information at any given moment is expressed in meters. According to the first The state estimate at time step i is for the i-th time step n. The predicted value of the state at each time point.

[0037] Calculate the variance of the innovation sequence within the sliding window: in, For the first The sliding window variance of the information sequence at each time step, in square meters; The length of the sliding window is 24 in this embodiment; For the first The information at any given moment; the summation operation pairs from arrive conduct.

[0038] The adaptive adjustment method for the process noise covariance matrix is ​​as follows: in, For the first The process noise covariance matrix at each time step; For the first The adaptive adjustment factor at each time point is calculated as follows: in, The reference value for the new information variance is expressed in square meters. It is the average of the new information variance during the first month of stable system operation. In this embodiment, it is taken as... Square meters. When the innovation variance increases, it indicates that the deviation between the observed value and the model prediction increases. The adaptive adjustment factor increases, which makes the filter more confident in the observed value. When the innovation variance is within the normal range, the adaptive adjustment factor remains at 1, and the filter maintains its confidence in the model prediction.

[0039] A statistical significance test is performed on the rate of change of periodic displacement in the filtered output. The null hypothesis is that the rate of change of periodic displacement is equal to zero, and the alternative hypothesis is that the rate of change of periodic displacement is not equal to zero. The test statistic is calculated as follows: in, The test statistic is dimensionless. To test the mean of the rate of change of periodic displacement within the window, the unit is meters per hour; To test the standard deviation of the rate of change of periodic displacement within the window, the unit is meters per hour; To test the sample size within the window. When the test statistic... When the absolute value of the value exceeds a preset significance threshold and the duration exceeds a preset number of days threshold, it is determined that the facility corresponding to the fixed sensor node has a significant deformation trend. In this embodiment, the test window length is set to 168 samples, corresponding to 7 days of data, the significance threshold is set to 2.576, corresponding to a 99% confidence level, and the number of days threshold is set to 7 days.

[0040] The deformation rate is calculated from the estimated periodic displacement. (Estimated periodic displacement) This represents the cumulative deformation over 24 hours. The deformation rate is defined as the deformation per unit time and is calculated as follows: The unit is meters per hour.

[0041] After processing by the time-series difference algorithm, the deformation trend characteristics of each fixed sensor node are output. These characteristics include: 3-dimensional estimates of periodic displacement in the X, Y, and Z directions; 3-dimensional deformation rates in the X, Y, and Z directions; 3-dimensional confidence scores of deformation rates in the X, Y, and Z directions; 1-dimensional composite amplitude of three-dimensional displacement; 1-dimensional composite amplitude of three-dimensional rates; 2-dimensional azimuth and elevation angles of the deformation direction; 1-dimensional rate of change of periodic displacement; 1-dimensional deformation trend significance marker; and 1-dimensional cumulative monitoring days. The feature vector dimension is 16. These deformation trend characteristics are transmitted to the cloud in step S3 and used as input data for the ternary coupling risk assessment in step S5.

[0042] S2, Crowdsourced Data Collection and Quality Control Step S1 achieves precise monitoring of key locations through fixed sensor nodes, but deployment costs limit its ability to cover the entire facility area. This step collects crowdsourced sensing data through visitor terminals, leveraging the natural distribution of visitors on the facility to expand monitoring coverage, and extracts reliable facility micro-vibration characteristics from the crowdsourced data through quality control processing.

[0043] This embodiment utilizes the scenic area's official application to collect crowdsourced data. When the application detects that a visitor's terminal's GPS location has entered a predefined facility monitoring area, it automatically triggers data collection from the visitor's terminal's built-in inertial measurement unit. The inertial measurement unit's sampling frequency is set to 100Hz, with continuous data collection for 5 seconds each trigger, simultaneously recording the GPS location coordinates and the collection timestamp. The crowdsourced sensing data only includes acceleration signals and their collection location information; sensitive information such as audio, video, and contact lists is not collected.

[0044] Crowdsourced sensing data originates from tourist terminals of different models and in different states, resulting in issues such as significant differences in device sensitivity, inaccurate location tracking, and interference from human walking signals. Existing technologies that directly use crowdsourced data lead to unreliable monitoring results. This invention addresses the quality issues of crowdsourced data through four processing steps: frequency band separation, device normalization, spatial clustering, and outlier removal.

[0045] The acceleration signal generated by human walking is mainly distributed in the 1Hz to 3Hz frequency band, while the characteristic frequency of structural vibration is usually distributed in the 5Hz to 50Hz range. In this embodiment, a fourth-order Butterworth bandpass filter is used to separate the original acceleration signal into frequency bands. The lower cutoff frequency of the passband is set to 5Hz, and the upper cutoff frequency of the passband is set to 50Hz. A forward-backward bidirectional filtering method is used to eliminate phase distortion.

[0046] Different models of visitor terminals exhibit variations in sensitivity and zero bias in their accelerometers. This embodiment performs statistical normalization on data collected from multiple visitor terminals within the same facility area and time window. The root mean square value of vibration is calculated for each collected sample. in, This is the root mean square value of the vibration, expressed in meters per square second. This represents the total number of sampling points; in this embodiment, it is set to 500. The first of the filtered acceleration signal Each sampling point value is represented in meters per square second; the summation operation is performed on... From 1 to conduct.

[0047] The root mean square (RMS) values ​​of vibration for all samples collected within the same facility area and the same time window were statistically analyzed, and the mean value was calculated. and standard deviation Convert the root mean square value of vibration for each sample into a standard score: in, The normalized standard score is dimensionless. This is the root mean square value of vibration for the current sample, in meters per square second. This is the mean of the root mean square values ​​of vibration for all samples within this time window in this region, expressed in meters per square second. This represents the standard deviation of the root mean square values ​​of vibration for all samples within this time window in this region, expressed in meters per second squared. The time window length is set to 10 minutes.

[0048] The GPS positioning accuracy of the tourist terminal is limited, and the collected locations may be inaccurate. This embodiment uses the DBSCAN clustering algorithm to spatially cluster the collected locations, with a neighborhood radius parameter set to 3 meters and a minimum neighborhood sample number parameter set to 5. Each cluster corresponds to a valid monitoring area on the facility, and the center coordinates of the cluster serve as the representative location of that area. Isolated data points not belonging to any cluster are marked as location anomalies and removed.

[0049] This embodiment employs a robust statistical method based on the absolute deviation of the median to identify and remove characteristic outliers. For the data samples within each spatial cluster, the median of the standard score sequence is calculated. Absolute deviation from the median: in, The absolute deviation of the median is dimensionless. For the first Standard scores for each sample, dimensionless; The median of the standard score sequence is dimensionless. The absolute deviation of the median is converted to the equivalent standard deviation: in, The standard deviation is the equivalent standard deviation, dimensionless; 1.4826 is the conversion factor between the absolute deviation of the median and the standard deviation under a normal distribution. When the sample standard scores... With median The absolute value of the difference is greater than When this happens, the sample is marked as a feature outlier and removed.

[0050] After removing outliers, the facility's micro-vibration features are extracted, including the dominant frequency, secondary frequency, spectral centroid, root mean square value of vibration, peak factor, and waveform factor. These facility micro-vibration features, along with the deformation trend features output in step S1, are transmitted to the cloud in step S3. The facility micro-vibration features will also serve as input data for cross-modal semantic fusion in step S4.

[0051] S3, heterogeneous network transmission Steps S1 and S2 complete data collection and feature extraction at the facility end. This step is responsible for reliably transmitting deformation trend features, facility micro-vibration features, video surveillance data, tourist data and environmental data to the cloud, and at the same time, sending the early warning information generated by the cloud to the management terminal.

[0052] This embodiment constructs a three-tiered heterogeneous communication architecture consisting of a cellular network, a LoRa Mesh network, and satellite short messages. The cellular network, as the primary channel, adopts the 4G LTE Cat.1 network standard, with an uplink bandwidth of 5Mbps and an end-to-end latency of 50 to 100 milliseconds, carrying real-time transmission of all types of data within the cellular signal coverage area. The LoRa Mesh network, as the secondary channel, uses LoRa modulation technology, operates in the 470MHz to 510MHz frequency band, has a transmit power of 20dBm, a receive sensitivity of -137dBm, and a single-hop transmission distance of 2 to 5 kilometers in mountainous environments. The Mesh topology networking allows each fixed sensor node to function as both a terminal and a router. Satellite short messages, as the tertiary channel, utilize the BeiDou-3 short message service, covering the Asia-Pacific region, with a maximum single-transmission capacity of 78 bytes.

[0053] Cellular networks in mountainous scenic areas often experience signal blind spots in valleys and canyons, where LoRa Mesh networks must undertake crucial data transmission tasks. Traditional LoRa Mesh routing algorithms use the expected number of transmissions as a link cost metric, considering only packet loss rate and neglecting the impact of mountainous terrain on wireless signal propagation. When there is a significant altitude difference between two nodes, the signal needs to cross valleys, and attenuation and multipath effects lead to a decline in actual communication quality. This invention proposes an altitude-weighted routing algorithm that introduces an altitude penalty factor into link cost calculation, prioritizing routing through nodes at similar altitudes to avoid signal transmission across valleys. in, For nodes With nodes The weighted cost of the links between them is dimensionless. For nodes With nodes The expected number of transmissions between them is dimensionless and is obtained by taking the reciprocal of the historical transmission success rate. The altitude weighting coefficient is dimensionless; in this embodiment, it is set to 0.3. For nodes The altitude, in meters; For nodes The altitude, in meters; The reference elevation difference is in meters; in this embodiment, it is set to 100. When the elevation difference between two adjacent nodes is 100 meters, the link weighted cost increases by 30%, and the routing algorithm will tend to select other nodes with similar elevations as the next hop.

[0054] The channel selection strategy automatically switches based on network status and data priority. For routine monitoring data, when the cellular network signal strength is above -110dBm, the data is uploaded to the cloud in real time via the cellular network. When the cellular network signal strength is below -110dBm or the connection fails, the system automatically switches to the LoRa Mesh network. The data is then compressed and encoded before being transmitted via multi-hop routing to the gateway node deployed in the cellular network coverage area. For anomaly warning data, when a fixed sensor node detects that the facility status data exceeds a preset anomaly threshold and both the cellular network and the LoRa Mesh network are unavailable, the satellite short message channel is triggered to transmit the warning data.

[0055] The data transmitted to the cloud in this step includes: deformation trend characteristics output in step S1, facility micro-vibration characteristics output in step S2, video surveillance data collected by video acquisition equipment, and tourist and environmental data provided by the scenic area management system. The facility micro-vibration characteristics and video surveillance data will serve as inputs to step S4, while the deformation trend characteristics, tourist data, and environmental data will serve as inputs to step S5.

[0056] S4, cross-modal semantic fusion The data transmitted to the cloud in step S3, including facility micro-vibration features and video surveillance data, needs further fusion processing to distinguish the sources of vibration. This step performs multimodal feature encoding on facility micro-vibration features and video surveillance data, uses a cross-modal attention mechanism to establish a semantic correspondence between vibration feature vectors and video feature vectors, and uses semantic consistency verification to distinguish between vibrations caused by visitor behavior and vibrations caused by facility structural anomalies, outputting fused features.

[0057] Existing multi-sensor data fusion technologies typically employ simple signal-level or feature-level superposition, failing to determine whether vibration signals are caused by normal tourist behaviors such as jumping or running, or by structural abnormalities in the facility, leading to a high false alarm rate. This invention establishes a semantic correspondence between vibration data and video data through a cross-modal attention mechanism. It utilizes behavioral information from the video to interpret the source of vibration signals, filtering out vibrations caused by tourist behavior before conducting a risk assessment.

[0058] In this embodiment, video capture devices are deployed in each facility monitoring area, and the shooting range of the video capture devices covers the main areas of visitor activity within that area. When the spatial clustering results in step S2 determine that there is valid crowdsourced data in a certain area, the corresponding video surveillance data for that area is retrieved for cross-modal fusion analysis.

[0059] For vibration feature encoding, a one-dimensional convolutional neural network is used to extract time-frequency features. The network input is the filtered acceleration signal corresponding to the facility micro-vibration features output in step S2, with a signal length of 500 sampling points. The one-dimensional convolutional neural network includes three convolutional layers: the first convolutional layer has 32 kernels, a kernel size of 7, and a stride of 2; the second convolutional layer has 64 kernels, a kernel size of 5, and a stride of 2; and the third convolutional layer has 128 kernels, a kernel size of 3, and a stride of 2. Each convolutional layer is followed by a batch normalization layer and a ReLU activation function, and then a global average pooling layer and a fully connected layer. The network output is a 128-dimensional vibration feature vector. .

[0060] For video feature encoding, based on the timestamp of the facility's micro-vibration features, a 5-second video segment is extracted, consisting of 2.5 seconds before and after the corresponding time period. With a sampling interval of 0.5 seconds, 10 frames are obtained. A ResNet-18 network is used to extract a 512-dimensional frame-level feature vector for each frame. The feature vectors from the 10 frames are then arranged row-wise to obtain the video feature matrix. , dimension .

[0061] The cross-modal attention mechanism uses the vibration feature vector as the query and the video features of each frame as the key and value. It learns the correlation between the vibration signal and each video frame through attention computation. The query vector is obtained by linearly projecting the vibration feature vector. in, The query vector has dimensions of 1. ; To query the projection matrix, the dimension is ; This is a vibration feature vector with a dimension of 128; For the attention dimension, this embodiment uses 32.

[0062] Linear projection of each row of the video feature matrix yields the key matrix and value matrix: in, The key matrix has dimensions of . ; The video feature matrix has dimensions of . ; Let be the key projection matrix, with dimension . ; superscript This indicates the matrix transpose.

[0063] in, It is a value matrix with dimension . ; The value projection matrix has dimensions of . .

[0064] Calculate the similarity between the query vector and the key vectors of each frame, and normalize the similarity to obtain the attention weights: in, This is an attention weight vector with a dimension of 10, and the sum of its elements is 1; attention weights The element Indicates the vibration signal and the first The correlation between video frames. Weighted aggregation of the value matrix based on attention weights yields single-head cross-modal fusion features: in, This is a single-head cross-modal fusion feature vector with dimension [missing information]. ; It is the transpose of the value matrix, with dimension . .

[0065] This embodiment employs a multi-head attention mechanism, using four attention heads to compute in parallel. Each attention head independently learns different attention patterns, and the output dimension of each attention head is 32. The outputs of the four attention heads are concatenated to obtain a 128-dimensional vector, which is then processed through a multi-head attention mechanism. linear projection matrix A 128-dimensional fusion feature vector was obtained. : in, The feature vectors are fused, with a dimension of 128; , , , The outputs are from four attention heads, each with a dimension of 32. This is a vector concatenation operation.

[0066] Semantic consistency verification determines the source of vibration based on the matching relationship between vibration feature types and video behavior recognition results. When an abnormal vibration event is detected in the facility's micro-vibration features, video surveillance data corresponding to the time period of the abnormal vibration event is extracted for behavior recognition. Behavior recognition employs a spatiotemporal convolutional network, which includes spatial and temporal convolutional branches. The spatial convolutional branch uses ResNet-18 to extract spatial features from each frame, while the temporal convolutional branch uses three layers of one-dimensional convolution to model the temporal relationship between frames. The kernel sizes are 3, 3, and 3, and the number of channels are 256, 256, and 512, respectively. The features from the two branches are concatenated and output through a fully connected layer to determine the probability of each behavior category. The behavior categories include normal walking, jumping, running, crowding, and no obvious activity.

[0067] Vibration events are classified into vibration types. The peak factor is defined as the ratio of the absolute value of the signal peak to the root mean square value. in, The peak factor is dimensionless. This represents the maximum absolute value of the acceleration signal at each sampling point, expressed in meters per second squared. This is the root mean square value of the vibration, expressed in meters per second squared. Peak duration. It is defined as the continuous duration during which the amplitude of an acceleration signal exceeds 80% of its peak absolute value, and is measured in seconds.

[0068] The rule for determining the vibration characteristic type is: if the peak factor Greater than 5 and peak duration If the time is less than 0.2 seconds, it is determined to be an impulse type; if the ratio of the peak value to the mean value of the power spectral density is greater than 10 and the frequency corresponding to the peak value is in the range of 1Hz to 4Hz, it is determined to be a rhythmic type; otherwise, it is determined to be a continuous type.

[0069] The nature of the vibration is determined based on the matching relationship between vibration feature types and behavior recognition results. Let the recognition probability of jumping behavior be... The probability of recognizing running behavior is Confidence threshold Set to 0.7. If the vibration characteristic type is impact type and If the abnormal vibration event is determined to be caused by the tourist's jumping behavior, it will be filtered out. If the vibration characteristic type is rhythmic and If the vibration pattern does not match the behavior identification result, the abnormal vibration event is determined to be caused by the tourist running and is filtered out. If the vibration pattern does not match the behavior identification result, the abnormal vibration event is determined to be a suspected structural abnormality of the facility and is included in the subsequent risk assessment.

[0070] After semantic consistency verification, the fused features are output, including the fused feature vector, vibration feature type, behavior recognition result, and semantic consistency judgment result. These fused features, together with the deformation trend features from step S1, serve as the input data for the ternary coupling risk assessment in step S5.

[0071] S5, Tripartite Coupling Risk Assessment The fusion features output in step S4 have distinguished the sources of vibration. In this step, the fusion features are combined with the deformation trend features in step S1, the tourist data and environmental data transmitted in step S3 to conduct a comprehensive analysis, construct a spatiotemporal heterogeneous map and use a graph neural network to learn the spatiotemporal features, output the risk score of each facility node, and generate safety monitoring and early warning information based on the risk score.

[0072] Existing scenic area facility safety monitoring technologies treat facility status monitoring, visitor management, and environmental monitoring as independent systems, failing to consider the coupled impact of visitor load and environmental conditions on facility safety risks. For example, during peak hours, visitor density increases significantly, leading to a substantial rise in the dynamic load on facilities; in windy weather, wind-induced vibrations of suspended facilities intensify, potentially posing safety risks when combined with visitor load. This invention constructs a spatiotemporal heterogeneous graph comprising three types of nodes: facility nodes, visitor area nodes, and environmental nodes. It jointly models visitor behavior and environmental factors as correlated variables affecting facility risks, achieving a risk assessment based on the three-dimensional coupling of visitors, facilities, and environment.

[0073] Facility nodes are used to represent monitored scenic area facilities. This embodiment includes 47 facility nodes. The node features of the facility nodes are formed by concatenating the deformation trend features obtained in step S1 and the fusion features obtained in step S4. The deformation trend features are 16-dimensional, and the fusion features are 128-dimensional. The dimension of the concatenated feature vector is [dimension missing]. The visitor area node represents the area divided within the scenic area. In this embodiment, the scenic area is divided into 12 visitor areas. The node features of the visitor area node include real-time visitor density, average stay duration, pedestrian movement speed, and density change trend determined based on the visitor data transmitted in step S3. The feature vector dimension is [missing information]. Environmental nodes are used to represent environmental factors affecting facility loads. In this embodiment, three environmental nodes are set to correspond to wind load, temperature stress, and humidity effects, respectively. The node features of the environmental nodes include wind speed, temperature, humidity, air pressure, and their rates of change, determined based on the environmental data transmitted in step S3. The feature vector dimension is... .

[0074] Facility-facility edges connect two facility nodes that are physically connected; there are 63 such edges in this embodiment. Visitor-facility edges connect visitor area nodes to facility nodes located within that area; the edge weight is positively correlated with the visitor density of the corresponding visitor area node. in, The edge weights for the visitor-facility pair are dimensionless. Real-time visitor density for visitor area nodes, in person per square meter; This represents the historical maximum visitor density for the area, expressed in person-per-square-meter units. Environment-facility edges connect environment nodes to facility nodes affected by that environmental factor. The node characteristics of the spatiotemporal heterogeneous graph change dynamically over time; the characteristics of visitor area nodes and environment nodes are updated every 5 minutes.

[0075] Since the spatiotemporal heterogeneous graph contains three different types of nodes with different feature dimensions, this embodiment configures different feature transformation matrices for different types of nodes to map them to a unified feature space. Let the output feature dimension be... Transformation matrix of facility nodes Dimensions Transformation matrix of tourist area nodes Dimensions Transformation matrix of environment nodes Dimensions .

[0076] For facility nodes Its feature vector is updated by aggregating information from neighboring nodes: in, For facility nodes The updated feature vector has a dimension of ; For ELU activation function; For facility nodes The set of neighboring nodes includes adjacent facility nodes, visitor area nodes of the region to which it belongs, and related environmental nodes; For the edge The attention weights are dimensionless. For neighboring nodes The feature transformation matrix corresponding to the type, based on the node type take , or ; For neighboring nodes eigenvectors; summation operation on the set of neighboring nodes All nodes conduct.

[0077] The attention score is calculated as follows: in, For the edge Attention score, dimensionless; The activation function is LeakyReLU, with a negative slope set to 0.2; The attention vector has a dimension of . ; For nodes The feature transformation matrix corresponding to the type; This is a vector concatenation operation; superscript This represents the transpose of a vector.

[0078] The normalization method for attention weights is as follows: in, These are the normalized attention weights; It is a natural exponential function; the summation operation is performed on the set of neighboring nodes. All nodes This embodiment employs four attention heads for parallel computation, each configured with an independent attention vector. The network consists of three spatial convolutional layers, with each layer outputting a feature dimension. The numbers are 128, 128, and 64, respectively.

[0079] A gated cyclic unit is used to perform time-series modeling of the feature vector sequences of each facility node at multiple time steps. Let the time window length be... In this embodiment, a time step is taken. This corresponds to 1 hour of historical data, with a time step interval of 5 minutes. The hidden state dimension of the gated loop unit is set to... .

[0080] The calculation method for updating the gate is as follows: in, For the first The update gate vector at each time step has a dimension of . ; Use the Sigmoid activation function; To update the gate weight matrix, the dimension is... This embodiment is ; For the first The spatial aggregated feature vector at each time step has a dimension of . In this embodiment, the value is 64; For the first The hidden state vector at each time step has a dimension of . ; To update the bias vector of the gate, the dimension is... ; This is a vector concatenation operation.

[0081] The calculation method for resetting the door is as follows: in, For the first The reset gate vector at each time step has a dimension of . ; To reset the weight matrix of the gate, the dimension is... ; The bias vector for resetting the gate has a dimension of . .

[0082] The candidate hidden state is calculated as follows: in, For the first The candidate hidden state vector at each time step has a dimension of . ; It is the hyperbolic tangent activation function; Here is the weight matrix of the candidate hidden states, with dimension 1. ; Let be the bias vector of the candidate hidden state, with dimension . ; This is an element-wise multiplication operation.

[0083] The method for updating hidden states is as follows: in, For the first The hidden state vector at each time step has a dimension of . ; It is a vector of all 1s with dimension . .

[0084] The hidden states of each facility node are mapped to risk scores through a two-layer fully connected network: in, Risk scores are assigned to facility nodes, with values ​​ranging from 0 to 100. Here is the weight matrix of the first fully connected layer, with dimension 1. ; This is the bias vector of the first fully connected layer, with a dimension of 32; The weight vector of the second fully connected layer has a dimension of . ; This is the bias scalar for the second fully connected layer; For the gated loop unit in the first The hidden state vector at each time step; This is the ReLU activation function.

[0085] The graph neural network model is trained using supervised learning. The training data comes from the scenic area's historical operational data, with abnormal facility events marked in the maintenance records used as positive samples and data from normal operating periods used as negative samples. The loss function used is the binary cross-entropy loss function. in, The value is the loss function value, which is dimensionless. This represents the number of training samples; For the first The true label for each sample is 1 for abnormal events and 0 for normal states; For the first The predicted probability of each sample is calculated using the risk score. Divide by 100; Summation operation From 1 to conduct.

[0086] Facilities are classified into different risk levels based on risk scores: a risk score of 80 or higher is a red alert, a risk score between 60 and 79 is an orange alert, a risk score between 40 and 59 is a yellow alert, and a risk score below 40 is a normal state. The alert information includes the name, number, and location coordinates of the risky facility, its risk level, main risk factors, and recommended measures. The alert information is transmitted via the heterogeneous transmission module in step S3 to the scenic area management center's large screen, management personnel's mobile terminals, and on-site audio-visual alarm devices.

[0087] Combination Figure 2 The present invention also provides an Internet of Things-based scenic area facility safety monitoring system for performing the above methods, including a fixed sensing module, a crowdsourced sensing module, a heterogeneous transmission module, a cross-modal fusion module, and a risk assessment module.

[0088] The fixed sensing module is used to execute step S1. It is deployed at key locations of scenic area facilities, collects facility status data and environmental parameter data through fixed sensor nodes, processes displacement data by executing a time-series difference algorithm, and outputs deformation trend characteristics.

[0089] The crowdsourced perception module, deployed in the cloud, is used to execute step S2. It receives crowdsourced perception data uploaded by tourist terminals, performs frequency band separation, equipment normalization, spatial clustering and outlier removal, and outputs facility micro-vibration characteristics.

[0090] The heterogeneous transmission module is used to perform step S3, which includes three communication methods: cellular network communication, LoRa Mesh communication, and satellite short message communication. It uses an altitude-weighted routing algorithm to optimize the routing of the LoRa Mesh network and automatically selects the transmission channel according to the network status and data priority to realize the uploading of monitoring data and the distribution of early warning information.

[0091] The cross-modal fusion module, deployed in the cloud, is used to perform step S4. It encodes multimodal features of facility micro-vibration characteristics and video surveillance data, establishes semantic correspondence between vibration feature vectors and video feature vectors using a cross-modal attention mechanism, distinguishes vibration sources through semantic consistency verification, and outputs fused features.

[0092] The risk assessment module, used to execute step S5, is deployed in the cloud and constructs a spatiotemporal heterogeneous graph containing facility nodes, visitor area nodes, and environmental nodes. It uses a heterogeneous graph attention network and gated recurrent units to learn the spatiotemporal features of the spatiotemporal heterogeneous graph, outputs the risk score of each facility node, and generates safety monitoring and early warning information based on the risk score.

[0093] The above description is merely a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A scenic spot facility safety monitoring method based on Internet of Things, characterized in that, The method comprises the following steps: S1, fixed node data acquisition and deformation trend extraction: collecting facility state data and environmental parameter data through fixed sensor nodes deployed at key positions of the scenic spot facilities, processing the displacement data using a time series difference algorithm to extract deformation trend characteristics; the time series difference algorithm eliminates periodic system errors by performing periodic difference on the displacement measurement sequence, extracts a smooth trend signal through adaptive Kalman filtering, and determines whether there is a significant deformation trend through statistical significance test; the deformation trend characteristics are feature vectors representing the deformation state of the facility, including deformation rate, deformation direction and confidence level; S2, crowd-sourced data acquisition and quality control: collecting crowd-sourced perception data through a tourist terminal, the crowd-sourced perception data including acceleration signals and their collection location information; performing frequency band separation, device normalization, spatial clustering and outlier rejection processing on the crowd-sourced perception data in sequence to extract facility micro-vibration characteristics; the facility micro-vibration characteristics are feature vectors representing the vibration state of the facility, including main frequency, secondary frequency, frequency spectrum center of gravity, vibration root mean square value, peak factor and waveform factor; S3, heterogeneous network transmission: constructing a three-level heterogeneous communication architecture of cellular network, LoRa Mesh network and satellite short message, selecting a transmission channel according to network status and data priority, and transmitting the deformation trend characteristics obtained in step S1, the facility micro-vibration characteristics obtained in step S2, video monitoring data collected by video acquisition devices deployed in the scenic spot, tourist data and environmental data provided by the scenic spot management system to the cloud; S4, cross-modal semantic fusion: performing multi-modal feature encoding on the facility micro-vibration characteristics and the video monitoring data transmitted to the cloud in step S3, establishing a semantic correspondence between the vibration feature vector and the video feature vector using a cross-modal attention mechanism, distinguishing between vibrations caused by tourist behavior and vibrations caused by facility structure abnormalities through semantic consistency verification, and outputting fusion features; the fusion features include fusion feature vectors, vibration feature types, behavior recognition results and semantic consistency determination results; S5, ternary coupling risk assessment: constructing a spatio-temporal heterogeneous graph, the spatio-temporal heterogeneous graph being a graph structure containing three different types of nodes, namely facility nodes, tourist area nodes and environmental nodes; the node features of the facility nodes include the deformation trend characteristics obtained in step S1 and the fusion features obtained in step S4, the node features of the tourist area nodes include the tourist data, and the node features of the environmental nodes include the environmental data; performing spatio-temporal feature learning on the spatio-temporal heterogeneous graph using a graph neural network to output risk scores of each facility node, and generating safety monitoring and early warning information according to the risk scores. 2.The method of claim 1, wherein, In step S1, the time series difference algorithm comprises: performing periodic difference operation on the displacement measurement sequence, and the difference period is set to 24 sampling points, corresponding to 24 hours; applying adaptive Kalman filtering to the difference sequence, and the state vector includes the periodic displacement estimate and the periodic displacement change rate; by monitoring the variance of the innovation sequence in the sliding window, the process noise covariance matrix is dynamically adjusted; The periodic displacement rate of the filtered output is subjected to a statistical significance test, and when the absolute value of the test statistic exceeds a preset significance threshold and the duration exceeds a preset day threshold, it is determined that there is a significant deformation trend. 3.The method of claim 1, wherein, In step S2, the quality control of the crowd-sourced perception data includes: A band-pass filter is used to separate the frequency bands of the original acceleration signal, and the characteristic frequency band of the facility structure vibration is retained; The root mean square values of the data collected by multiple tourist terminals in the same facility area and in the same time window are calculated, and the root mean square values are converted into standard scores for device normalization processing; A DBSCAN clustering algorithm is used to perform spatial clustering on the collection positions, and isolated data points that do not belong to any cluster are removed; A robust statistical method based on median absolute deviation is used to identify and remove characteristic outliers. 4.The method of claim 1, wherein, In step S3, the LoRa Mesh network uses an altitude-weighted routing algorithm for routing optimization, which introduces an altitude penalty factor in the link cost calculation, which is positively correlated with the altitude difference between adjacent two nodes. 5.The method of claim 1, wherein, In step S3, the selection strategy of the transmission channel is: When the cellular network signal strength is higher than the preset signal strength threshold, it is transmitted through the cellular network; When the cellular network signal strength is lower than the preset signal strength threshold or the connection fails, it is transmitted through the LoRa Mesh network; When the facility state data exceeds the preset abnormal threshold and both the cellular network and the LoRa Mesh network are unavailable, it is transmitted through the satellite short message. 6.The method of claim 1, wherein, In step S4, the cross-modal attention mechanism includes: A one-dimensional convolutional neural network is used to encode the acceleration signal corresponding to the facility micro-vibration feature, outputting a vibration feature vector; The video clip of the corresponding period of the facility micro-vibration feature is intercepted, and a convolutional neural network is used to encode each frame of image to obtain a video feature matrix; The vibration feature vector is used as a query, and each row of the video feature matrix is used as a key and a value, the attention weight is calculated and weighted aggregated, and the fusion feature vector is output. 7.The method of claim 1, wherein, In step S4, the semantic consistency verification includes: According to the peak factor, peak duration and spectral characteristics, the abnormal vibration event is classified by vibration type, and the vibration feature type includes impact type, rhythm type and continuous type; The video monitoring data corresponding to the abnormal vibration event in the corresponding period is subjected to behavior recognition, and the probability of each behavior category is output; According to the matching relationship between the vibration feature type and the behavior recognition result, the vibration source is determined, if matched, it is determined as vibration caused by tourist behavior and filtered, if not matched, it is determined as suspected facility structure anomaly and included in risk assessment. 8.The method of claim 1, wherein, In step S5, the spatio-temporal heterogeneous graph includes facility-facility edges, tourist-facility edges and environment-facility edges; the facility-facility edge is used to connect two facility nodes that have a physical connection relationship; the tourist-facility edge is used to connect the tourist area node and the facility node located in the area, and the edge weight is positively correlated with the tourist density; the environment-facility edge is used to connect the environment node and the facility node affected by the environment factor. 9.The method of claim 1, wherein, In step S5, the graph neural network includes a heterogeneous graph attention network and a gated recurrent unit; the heterogeneous graph attention network configures different feature transformation matrices for different types of nodes, and aggregates neighbor node information through the attention mechanism to learn spatial features; The gated loop unit performs time-series modeling of the feature vector sequences of each facility node at multiple time steps.

10. An IoT-based scenic spot facility safety monitoring system for performing the method of any one of claims 1 to 9, characterized in that, include: The fixed sensing module is used to execute step S1. It is deployed at key locations of scenic area facilities, collects facility status data and environmental parameter data through fixed sensor nodes, processes displacement data by executing a time-series difference algorithm, and outputs deformation trend characteristics. The crowdsourced perception module, which is used to execute step S2, is deployed in the cloud, receives crowdsourced perception data uploaded by tourist terminals, performs frequency band separation, equipment normalization, spatial clustering and outlier removal processing, and outputs facility micro-vibration characteristics. The heterogeneous transmission module is used to perform step S3, which includes three communication methods: cellular network communication, LoRa Mesh communication and satellite short message communication. It uses an altitude-weighted routing algorithm to optimize the routing of the LoRa Mesh network and automatically selects the transmission channel according to the network status and data priority. The cross-modal fusion module, which is used to execute step S4, is deployed in the cloud. It performs multimodal feature encoding on facility micro-vibration characteristics and video surveillance data, establishes the semantic correspondence between vibration feature vectors and video feature vectors using a cross-modal attention mechanism, distinguishes vibration sources through semantic consistency verification, and outputs fused features. The risk assessment module, used to execute step S5, is deployed in the cloud and constructs a spatiotemporal heterogeneous graph containing facility nodes, visitor area nodes, and environmental nodes. It uses a graph neural network to learn the spatiotemporal features of the spatiotemporal heterogeneous graph, outputs a risk score for each facility node, and generates safety monitoring and early warning information based on the risk score.

Images