A distributed environmental multimodal intelligent monitoring system

Abstract

This invention discloses a distributed environmental multimodal intelligent monitoring system, belonging to the field of environmental monitoring technology. It includes a sensor node module, a perception and control module, a communication synchronization module, a data acquisition and fusion module, a data modeling module, a modal analysis module, a simulation and maintenance module, a visual decision-making module, and a security protection module. This invention ensures the time accuracy of cross-node data transmission and event correlation, can maintain network self-healing and continuous operation when some nodes fail or communication is interfered with, achieves long-term autonomous operation, reduces maintenance costs, and enables the monitoring network to adapt to different working conditions and structural changes.

Description

Technical Field

[0001] This invention relates to the field of environmental monitoring technology, and in particular to a distributed environmental multimodal intelligent monitoring system. Background Technology

[0002] With the continuous development of aerospace technology, modern launch platforms are gradually evolving towards larger scale, greater intelligence, and greater automation. Launch site environments encompass complex conditions such as high-temperature gases, strong acoustic and vibrational impacts, high-speed airflow, severe thermal gradients, electromagnetic interference, and extreme weather conditions. These conditions exhibit extremely strong dynamic changes, are highly non-uniform in spatial distribution, and their temporal evolution scale spans from milliseconds to hours. Simultaneously, various facilities on the platform (such as fuel pipelines, support structures, transmission mechanisms, cooling systems, and electrical subsystems) are tightly coupled physically and logically. Any local anomaly (such as valve stagnation, temperature surges, or structural cracks) can potentially evolve into a global risk within a short period. Against this backdrop, constructing a distributed, multimodal intelligent environmental monitoring system capable of achieving full-time and spatial coverage, ultra-high timeliness, and high reliability has become a crucial element in ensuring launch mission safety.

[0003] Existing distributed environmental multimodal intelligent monitoring systems cannot guarantee the time accuracy of cross-node data transmission and event correlation. They cannot maintain network self-healing and continuous operation when some nodes fail or communication is interfered with, increasing maintenance costs. Furthermore, the monitoring network cannot adapt to different working conditions and structural changes. Therefore, we propose a distributed environmental multimodal intelligent monitoring system. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by proposing a distributed environmental multimodal intelligent monitoring system.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A distributed environmental multimodal intelligent monitoring system includes a sensor node module, a perception and control module, a communication synchronization module, a data acquisition and fusion module, a data modeling module, a modal analysis module, a simulation and maintenance module, a visual decision-making module, and a security protection module.

[0007] The sensing node module collects multimodal environmental data through a sensor network and adaptively adjusts the sampling frequency and communication strategy according to the energy level.

[0008] The perception control module is used to comprehensively evaluate the energy margin and link quality of each sensing node in the sensor network and select the optimal path to transmit data.

[0009] The communication synchronization module is used to establish a secure channel between each sensor node, conduct secure communication, and synchronize the time of each sensor node.

[0010] The acquisition and fusion module is used to preprocess data from various modalities and map different types of data to a unified feature space.

[0011] The data modeling module establishes a corresponding sensor map structure based on the sensor network structure and updates the sensor map structure in real time to obtain the spatial topology changes of the launch platform under different operating conditions.

[0012] The modal analysis module is used to predict the environmental change trends and potential anomalies of the launch platform, and automatically performs root cause analysis after an anomaly is detected.

[0013] The simulation maintenance module is used to build a digital twin of the launch platform in the cloud, and at the same time predict changes in airflow, heat distribution and structural stress in the launch environment, and dynamically optimize the maintenance strategy.

[0014] The visual decision-making module is used to provide a multi-level visual interface to provide decision support suggestions for operation and maintenance personnel.

[0015] The security protection module is used to encrypt and control access to data throughout the entire process of data acquisition, transmission, and storage.

[0016] As a further aspect of the present invention, the specific steps of the sensing node module adaptively adjusting the sampling frequency and communication strategy according to the energy level are as follows:

[0017] P1.1: When each sensor node in the sensor network is powered on or periodically woken up, it reads the real-time energy storage of the power subsystem, the instantaneous voltage of the supercapacitor, the current output of the light / heat / vibration collector, and the available power status reported by the power management chip. Then, it establishes a corresponding short history window based on a preset sampling period and uses this short history window to count the energy inflow and outflow.

[0018] P1.2: Calculate the current net energy growth rate of the corresponding sensor node based on the statistical results. Then, write the timestamp, sample array, and estimated current net energy growth rate into the local energy state structure of the sensor node. At the same time, calculate the available energy ratio of each sensor node based on the information. The specific formula for calculating the available energy ratio is as follows:

[0019] ;

[0020] In the formula, Represents the available energy ratio of the sensing node; This represents the current energy stored in the sensor node; The minimum safe energy storage that a node needs to retain; This represents the upper limit of the rated capacity of the energy storage of the sensing node; Represents the smallest positive number;

[0021] P1.3: When the sensor node starts to collect data, each sensor node first enters the self-test mode. If the output of the multi-source energy acquisition unit of each sensor node is zero for a long time or the voltage of the supercapacitor energy storage unit is abnormal, it is determined that the power supply of the sensor node is faulty, and it enters the low power safety mode and reports to the gateway.

[0022] P1.4: Based on the physical importance, recent observation volatility, and task-level requirements of different modes (temperature, vibration, gas, acoustic, and image) for each sensor, a priority score for each mode is periodically calculated. These scores are then sorted from highest to lowest to generate an initial sampling queue. High-priority modes are allocated to time slices with higher sampling frequencies. Simultaneously, the available energy ratio of each sensor node is checked to determine if it meets the minimum workload for high-energy-consuming modes. If not, the sampling frequency of the corresponding mode is reduced, or it is converted to event-triggered sampling. The specific formula for calculating the priority score is as follows:

[0023] ;

[0024] In the formula, Representative mode Priority rating; Representative mode The static importance weights; The amplification factor representing the variance term; Representative mode Estimation of the observation variance within the current sliding window; Representative mode Additional preferences;

[0025] P1.5: Based on the ratio of the current sampling queue to the available energy of each sensor node, calculate the target sampling period for each mode, and allocate a smaller sampling period for high-priority modes with rapid fluctuations, while increasing the sampling period or switching to event-triggered sampling for low-priority or energy-intensive modes. Then, based on the current energy status, the amount of data to be sent, and the link quality assessment, select one of the following communication strategies: immediate upload, aggregated upload, compressed upload, or only sending event digests.

[0026] P1.6: Calculate the total consumption of direct transmission and transmission after local processing. If the transmission savings of transmission after local processing exceed the computational energy consumption and the processing delay is within the preset tolerance range, then local processing is performed. Otherwise, direct transmission or downgrade to digest upload is performed. Then, using the most recent energy collection sample, short-term prediction is performed through exponential smoothing, and the predicted net energy increment for the future time window is generated in each decision period.

[0027] P1.7: If the predicted net energy increment is negative, the corresponding sensor nodes will be reduced in advance or the event trigger threshold will be increased to conserve energy. When a multi-node congestion scenario occurs, the number of retransmissions, ACK waiting timeout and backoff time will be dynamically adjusted according to the current energy ratio of the sensor node and the link quality.

[0028] In addition, it should be noted that the sensor node described in P1.1 has a built-in multi-source energy harvesting unit and a supercapacitor energy storage unit. The multi-source energy harvesting unit specifically includes light energy, heat energy and vibration energy harvesters, etc.

[0029] As a further aspect of the present invention, the specific steps for the perception control module to select the optimal path for data transmission are as follows:

[0030] S1.1: When the sensor network starts or the routing cycle begins, the gateway or area controller scans all currently sensed sensor nodes and links, and constructs a set of candidate paths from the source node to the destination node, assigning an initial attraction value to each candidate path.

[0031] S1.2: Based on the real-time measured energy consumption estimate, end-to-end delay estimate and link reliability, calculate the fitness of each candidate path. Then, for each candidate path, randomly select an intermediate node on the path and replace it with its backup neighbor node to generate a new path variant, and calculate the fitness of the new path variant.

[0032] S1.3: If the fitness of the new path variant is better than that of the original candidate path, then the original candidate path is updated to the path variant; otherwise, the original solution is retained and the abandonment count of the path variant is increased. Then, the selection probability is assigned based on the normalized fitness of each candidate path. Then, according to the selection probability of each candidate path, any candidate path is randomly selected and subjected to multiple local perturbations.

[0033] S1.4: Evaluate the multiple path variants generated by the perturbation and select the path variant with the highest fitness. If the fitness value of the selected path variant is greater than that of the original candidate path, update the path variant as the optimal variant. Otherwise, retain the original candidate path and continue to accumulate the abandonment count of the corresponding path variant.

[0034] S1.5: If the abandonment count of a path variant exceeds a preset threshold, the path is abandoned, and a new path variant is generated to fill its position by selecting a node with high remaining energy and good link quality. The abandonment count is then cleared, and the initial score of the new path variant is calculated. Otherwise, the historical stability of the corresponding path variant is evaluated by retaining the score function. If the evaluation result is higher than the preset threshold, it is retained; otherwise, the replacement process is executed.

[0035] S1.6: After each round of path search, replacement and update is completed, based on the latest fitness and acceptance records, the candidate path table is sorted and updated from high to low fitness, and the top-ranked paths that meet the energy-reliability threshold are selected as the actual data forwarding paths, and the corresponding paths are sent to the nodes for forwarding settings in the next cycle.

[0036] As a further aspect of the present invention, the specific calculation formula for the initial attraction value in S1.1 is as follows:

[0037] ;

[0038] In the formula, The initial attractiveness score represents the candidate path; Normalized value representing the minimum remaining energy of nodes on the path; Normalized value representing the average link quality along the path; Represents a positive constant bias;

[0039] The specific formula for calculating fitness as described in S1.2 is as follows:

[0040] ;

[0041] In the formula, The fitness of the path; This represents the sum of the total remaining energy along the path; The estimated end-to-end transmission delay represents the path. This represents the packet loss rate estimate for the path.

[0042] As a further aspect of the present invention, the communication synchronization module establishes a secure channel between each sensing node, and the specific steps for secure communication are as follows:

[0043] S2.1: Install and connect entangled photon pair transmitters and single-photon detectors between each pair of key nodes that need to establish a secure channel and synchronization. At the same time, equip each node with a quantum random number generator and a local clock oscillator. Then measure the emission rate, photon spectral width and polarization reference of the photon pair transmitters, and then measure the dark count rate and detection efficiency of the photon detectors and record them as the initial hardware parameters.

[0044] S2.2: Write the hardware parameters and initial observation data into the local initialization table and notify the peer node to enter the "pending distribution" state. Each endpoint outputs its own generation basis selection sequence according to the quantum random number generator, and uses the local clock oscillator to record the timestamp of each transmission or measurement, and sets the maximum allowable time difference threshold.

[0045] S2.3: Record the sequence of detection events that occur within each time window. The entangled photon pair transmitter emits entangled photon pairs at a set rate. The photons at both ends are transmitted to two different sets of nodes. The single-photon detector records the timestamp of the detection event and the measurement basis used based on the set time window. Afterwards, each endpoint exchanges the necessary basis selection information and timestamp index through the classical channel and calculates the coincidence count within the decision time window.

[0046] S2.4: Calculate the histogram of the time difference distribution of the paired timestamp data to obtain the corresponding time correlation function, fit the main peak of the time correlation function to estimate the peak position, calculate the statistical uncertainty of the peak position by repeating sampling and fitting multiple times, and evaluate the confidence interval required for synchronization correction based on the statistical uncertainty. The peak position generated by multiple fittings is used as the preliminary clock offset estimate and passed to the correction stage.

[0047] S2.5: Reciprocal measurement is adopted, and time stamp signals are sent from endpoint A to endpoint B in turn and measured in reverse. The fixed delay difference of the optical path and the node clock deviation are obtained. The clock deviation estimate is obtained by fitting with the least squares method. Then, the clock deviation estimate is converted into a local clock adjustment command and the local clock is corrected. After each correction, a short entanglement distribution is performed again and a new time peak is calculated. If the deviation exceeds the preset safety threshold, it is rolled back to the previous stable configuration and retried with the adjustment step size.

[0048] S2.6: In or in parallel with the time synchronization process, calculate the bit error rate of the qubits based on the paired measurement results. If the bit error rate of the qubits exceeds the threshold allowed by the protocol, an alarm should be triggered and the current key extraction should be stopped. Then, the bit error rate of the original bit string that is still retained after passing the basis screening is estimated, and error correction and privacy amplification are performed at the same time to obtain a usable symmetric key. The generated key is then recorded together with the synchronization state.

[0049] As a further aspect of this invention, during the peak position fitting stage of time synchronization, the communication synchronization module performs correlation analysis between the quantum bit error rate (QBER) and the clock offset, and establishes a QBER-clock offset coupling model. When QBER > 2%, the communication synchronization module automatically adjusts the clock correction step size and extends the entangled photon sampling time window to ensure that the clock synchronization accuracy remains stable within ±1 ns under complex electromagnetic interference environments. Subsequently, based on the length of the original bit string after basis filtering and the bit error rate, the hash function parameters for privacy amplification are dynamically adjusted. When the bit error rate is ≤1%, the SHA-256 hash function is used, with a compression ratio of 2:1; when 1% < bit error rate ≤3%, the SHA-512 hash function is used, with a compression ratio of 3:1; when the bit error rate >3%, dual privacy amplification is activated, first using SHA-512 and then the SM3 algorithm, with a compression ratio of 5:1. This effectively ensures the security strength of the final generated symmetric key, and the key generation efficiency is significantly improved compared to the fixed compression ratio scheme. Simultaneously, during the entangled photon transmission stage, the dark count rate of the photon detector is monitored in real time. When the dark count rate >5 × 10⁻⁶, the dark count rate is increased. -6 When the polarization state of entangled photons is switched automatically, a backup photon detector is activated to ensure the connectivity of the quantum channel.

[0050] As a further aspect of the present invention, the specific steps for the data modeling module to establish the corresponding sensor map structure are as follows:

[0051] S3.1: Each sensor node periodically broadcasts a data packet. The gateway or regional agent listens for each data packet and creates a new node entry in the graph database when a new ID is received. Then, each sensor node is written to the local edge table and synchronized to the temporary table of the graph engine. After registration, an initial "activity tag" and version number are immediately assigned to each sensor node.

[0052] S3.2: For each sensor node with positioning capability, directly read its output three-dimensional coordinates in the Cartesian coordinate system. For sensor nodes without positioning capability, collect the received signal strength indication (RSSI) values ​​between them and their surrounding positioned neighboring nodes. Then, use the neighboring RSSI and other ranging quantities to calculate the relative distance between the sensor node with missing positioning information and its neighboring nodes. Then, use the trilateration method for coarse positioning. Recursive positioning based on Kalman filtering fuses historical and current measurements and outputs the position estimate and covariance. Nodes whose position estimate covariance exceeds a preset threshold are marked as "position uncertain" and the position confidence index is saved in the node attributes of the graph.

[0053] S3.3: Calculate the geometric distance between each pair of sensor nodes, obtain the expected received power between each pair of sensor nodes based on the calculated geometric distance, and make a link reachability prediction. If the expected received power between two groups of nodes meets the reachability threshold, add an edge to the candidate edge set, and add a corresponding timestamp, initial state and observation source to each candidate edge.

[0054] S3.4: Collect multi-source indicators for each candidate edge, construct edge weights using a weighted linear combination, store the weight matrix in a sparse format, maintain the weight value of each edge using a sliding time window, perform an exponentially weighted moving average on the weight value of each edge, return the updated weight value to the graph structure, and trigger the version number to increment to generate the latest sensor graph structure.

[0055] As a further aspect of the present invention, the data packet in S3.1 specifically includes a unique node ID, device type code, local coordinate estimation, remaining energy, most recent sampling timestamp, and antenna received power estimation, etc.

[0056] As a further aspect of the present invention, the specific steps of the modal analysis module in predicting the environmental change trends and potential anomalies of the launch platform are as follows:

[0057] S4.1: Read the sensor graph structure at the current time and multiple past sampling periods, obtain the multimodal observation vector of each node and the weight of the corresponding edge, align the node observations at each time according to the node ID, and then perform missing value repair and normalization on the multimodal observation vectors of each sensor node to generate the corresponding temporal node feature tensor.

[0058] S4.2: Extract the weights of each edge in the current sensor graph structure. Based on the extracted edge weights, construct an adjacency matrix with self-loops. Then calculate the corresponding degree matrix. Based on the adjacency matrix and the degree matrix, construct a normalized adjacency matrix using the normalized symmetric Laplace approximation. Calculate the normalized adjacency matrix for each time step. Simultaneously, perform time smoothing on the normalized adjacency matrix for each time step and output the spatial propagation matrix for each time step.

[0059] S4.3: Extract the standardized node feature matrix at each time step of the sensor graph structure. Then, perform graph convolution on the node feature matrix and the spatial propagation matrix to aggregate information. After that, perform activation processing with a nonlinear activation function. After each round of graph convolution processing, perform batch normalization and residual connection processing to finally output a set of temporal spatial embedding sequences.

[0060] S4.4: Stack the temporal spatial embedding sequences into a three-dimensional sequence in time, extract the time series of each sensing node, and input the time series of all sensing nodes into the LSTM encoder with shared parameters in parallel. The LSTM encoder updates the hidden state and outputs the temporal context vector at each time step, and finally generates the temporal encoded representation of each node.

[0061] S4.5: The time code of each sensing node is concatenated with the spatial embedding of the current time to form a comprehensive representation vector. Then, the comprehensive representation vector is input into the feedforward neural network. The feedforward neural network then processes the comprehensive representation vector layer by layer through the forward propagation algorithm and outputs the predicted value of the continuous environmental quantity and the anomaly probability score.

[0062] S4.6: Compare each anomaly probability score with the historical baseline. If it exceeds the historical baseline, mark the corresponding sensor node as "anomaly". At the same time, check the synchronization anomaly indication, device energy anomaly and physical consistency from neighboring nodes. If multiple pieces of evidence meet the preset support conditions, a formal anomaly event is triggered. At the same time, calculate the confidence score for each judgment and record the trigger time and the observation dimensions involved.

[0063] As a further aspect of the present invention, the specific steps for the simulation maintenance module to construct a digital twin of the launch platform in the cloud are as follows:

[0064] P2.1: Import the launch platform simulation model from CAD, BIM or laser point cloud, and perform noise reduction, overlapping surface merging, hole repair, thin wall identification and key details on the established launch platform simulation model. After preprocessing, divide the processed launch platform simulation model into fluid domain and solid domain, and determine the geometric surface set of the interface between the two domains and label the boundary condition type.

[0065] P2.2: Set up local fine meshes near nozzles, heat flow concentration areas, thin-walled structures and measuring points; use coarse meshes in the far field or non-critical areas, and output the element index table, node coordinates, interface mapping table and other mesh metadata. Generate a unique version ID and metadata for each mesh. Then, perform a quality check on the generated mesh data. If it fails, automatically trigger local re-subdivision or order reduction processing based on the mesh quality index.

[0066] P2.3: Identify the nearest CFD cells or FEA node set for each sensor node, and use distance-weighted pre-calculated interpolation or projection weights. Construct a sparse mapping matrix based on the calculation results, align the inflow values ​​of each sensor node by timestamp, and remove high-frequency noise through a low-pass filter. At the same time, output the corresponding simulation-readable time series file or memory shared buffer.

[0067] P2.4: If a sensor node loses connection at any time, the replacement value is calculated by exponential smoothing using the most recent k value, and the confidence level of the boundary time series is marked. Then, based on the launch platform simulation model, multi-step simulation is performed according to the set physical time steps. Then, the latest mapped observation data is collected at the synchronization time point. After that, a localized incremental method is used to calculate the assimilation gain using the observation covariance matrix and the model prior covariance, and the simulation state is corrected.

[0068] P2.5: If there is a systematic deviation between the observed data and the simulation data, the self-calibration process of the launch platform simulation model parameters is triggered, and the residual statistics are written into the model error database. After assimilation, fluid-structure interaction iteration is used to iterate until the interface converges in each coupling time step of the launch platform simulation model simulation process, and key prediction quantities are output. At the same time, each key prediction quantity is post-processed to generate engineering-readable prediction indicators, and risk indicators are calculated according to engineering rules and safety thresholds.

[0069] As a further aspect of the present invention, the visual decision-making module provides a multi-level visualization interface, and the specific steps for providing decision support suggestions to operation and maintenance personnel are as follows:

[0070] P3.1: Streams from spatiotemporal prediction, digital twins, sensor observations, alarm streams, and maintenance records are written to different topics according to event type and marked with a unified timestamp and source identifier when entering the database. All incoming data first enters a short-term memory queue for real-time consumption, and is simultaneously asynchronously written to the time series database and event warehouse. Timeliness indicators are calculated for each data record and a data freshness table is maintained. When data has not been updated for a preset tolerance delay, it is marked as stale and the corresponding visualization component is grayed out. Then, based on each group of data, the backend maintains an aggregated view.

[0071] P3.2: The front-end renders the real-time monitoring panel based on the aggregated view cached by the back-end. The panel is then organized hierarchically, with each layer displaying the current value of key indicators, short-term trend bars, and data freshness indicators. When the user clicks on any area card, the secondary view expands, and the original time series, digital twin prediction curve, and historical abnormal event list for that point are displayed in a side pane on the same page. Drill-down requests trigger the back-end to load more detailed time series on demand, return differential data for the front-end to present, temporarily overwrite the thresholds, and record the changes as operation comments.

[0072] P3.3: The real-time generated risk probabilities and uncertainties are mapped to the heatmap rendering grid according to their physical coordinates. Weighted spatial interpolation is used, combined with the interpolation confidence field, to generate a continuous heatmap of the blank area. The corresponding level switching function is added to the generated heatmap. For detected abnormal events, event cards are generated on the interface, and the possible propagation paths of each abnormal event are displayed in a graph structure. Nodes and edges are colored according to their contribution and support layer-by-layer expansion. The playback mode can play back the numerical field and heatmap along the time axis and simultaneously highlight the key points on the causal link.

[0073] P3.4: Short-term forecasts, historical failure rates, current energy / performance indicators, and maintenance history are integrated into a corresponding health score according to predetermined weights and classified according to preset thresholds. At the same time, based on the historical health score, a corresponding line-line historical view is generated, and an actionable set of suggestions is generated based on the health score and the short-term forecast of the digital twin.

[0074] As a further aspect of the present invention, the security protection module performs encryption processing and access control on data throughout the entire chain of acquisition, transmission, and storage, with the following specific steps:

[0075] P4.1: Deploy QKD links and certified classical channels at two communication ends. When the system starts, the two sides conduct a QKD session to generate multiple sets of original qubit pairs. After basis filtering, a shared original key string is obtained. Then, the qubit error rate is calculated using the sampled bit pairs.

[0076] P4.2: If the bit error rate of the quantum bits is within an acceptable threshold, error correction and privacy amplification are performed to generate common symmetric session key material. Then, the quantum key is used to sign the classical authentication data with a message authentication code. After mutual verification, the session is marked as "authenticated". When any session authentication fails, it is immediately interrupted and an anomaly is reported. Then, the session key is updated according to the lifetime strategy period, and the metadata of each session is written to the security audit log.

[0077] P4.3: Using the symmetric material produced by QKD as a seed, a multi-purpose subkey slot is generated according to the predetermined key derivation function. An end-to-end encrypted channel is established using AES-GCM. Then, each piece of data to be transmitted is encrypted with the subkey, a MAC is generated, and immutable metadata is added to the header. Large data is also segmented and encrypted.

[0078] P4.4: Deploy access control points and policy decision points in the cloud and at the edge respectively. When each resource request is received, the access control point verifies the requester's quantum authentication session at the session layer, checks the message MAC, and then forwards the attributes and resource identifier to the policy decision point for policy evaluation. After that, the policy decision point returns the decision, validity period, and whether additional multi-factor confirmation is required. When the decryptor's attribute set meets the policy, the resource data is decrypted, and the access decision and decryption log are synchronously written to the tamper-proof audit chain.

[0079] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0080] In the startup phase, this invention involves a gateway or area controller scanning sensor nodes and links to construct a candidate path set. Fitness is calculated based on energy consumption, end-to-end latency, and link reliability. Path variants are generated through random node replacement and local perturbations. Paths with high fitness are updated and retained, while inefficient paths are replaced or reconstructed. Finally, paths with sufficient energy and high reliability are selected as the actual forwarding pathways. Quantum entangled photon pair transmitters and single-photon detectors are deployed between nodes. Quantum random number generation and clock correction are combined to achieve quantum key distribution and nanosecond-level time synchronization. Time correlation function fitting and clock deviation estimation are performed on entangled photon detection events, and precise synchronization is achieved through reciprocal measurement and least squares correction. After synchronization, the qubit error rate is calculated from the measurement results, and a secure key is generated through error correction and privacy amplification. Subsequently, a dynamically updated graph structure is constructed, fusing localization, RSSI, and multi-source indicators to generate a weighted adjacency matrix. Multimodal observation data, after missing data repair and normalization, is input into a graph convolutional network and LSTM combined model to extract spatial and temporal features. The model outputs environmental quantity predictions and anomaly probability scores. If the anomaly probability exceeds the baseline and meets the multidimensional support conditions, it automatically identifies the abnormal event and records the confidence level and time information, ensuring the time accuracy of cross-node data transmission and event correlation. It can maintain network self-healing and continuous operation when some nodes fail or communication is interfered with, achieve long-term autonomous operation, reduce maintenance costs, and enable the monitoring network to adapt to different working conditions and structural changes. Attached Figure Description

[0081] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0082] Figure 1 This is a system block diagram of a distributed environmental multimodal intelligent monitoring system proposed in this invention. Detailed Implementation

[0083] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Example

[0084] Reference Figure 1 A distributed environmental multimodal intelligent monitoring system includes a sensor node module, a perception and control module, a communication synchronization module, a data acquisition and fusion module, a data modeling module, a modal analysis module, a simulation and maintenance module, a visual decision-making module, and a security protection module.

[0085] The sensor node module collects multimodal environmental data through the sensor network and adaptively adjusts the sampling frequency and communication strategy according to the energy level; the perception control module is used to comprehensively evaluate the energy margin and link quality of each sensor node in the sensor network and select the optimal path to transmit data.

[0086] Specifically, at the start of the sensor network startup or routing cycle, the gateway or area controller scans all currently sensed sensor nodes and links, constructs a set of candidate paths from the source node to the destination node, assigns an initial attraction value to each candidate path, and calculates the fitness of each candidate path based on real-time measured energy consumption estimates, end-to-end delay estimates, and link reliability. Then, for each candidate path, an intermediate node is randomly selected and replaced with its backup neighbor node to generate a new path variant. The fitness of the new path variant is calculated. If the fitness of the new path variant is better than the original candidate path, the original candidate path is updated to the path variant; otherwise, the original solution is retained and the abandonment count of the path variant is increased. Then, selection probabilities are assigned based on the normalized fitness of each existing candidate path. Next, based on the selection probabilities of each candidate path, any candidate path is randomly selected and subjected to multiple local perturbations. The multiple path variants generated by the perturbations are evaluated. The path variant with the highest fitness is selected. If the fitness value of the selected path variant is greater than that of the original candidate path, the path variant is updated to the optimal variant. Otherwise, the original candidate path is retained and the abandonment count of the corresponding path variant continues to accumulate. If the abandonment count of a path variant exceeds a preset threshold, the path is abandoned, and a new path variant is generated to fill its position by selecting a node with high remaining energy and good link quality. The abandonment count is then cleared, and the initial score of the new path variant is calculated. Otherwise, the historical stability of the corresponding path variant is evaluated by retaining the score function. If the evaluation result is higher than the preset threshold, it is retained; otherwise, the replacement process is executed. After each round of path search, replacement, and update, the candidate path table is sorted and updated from high to low fitness based on the latest fitness and acceptance records. The top-ranked paths that meet the energy-reliability threshold are selected as the actual data forwarding paths, and the corresponding paths are distributed to the nodes for the next cycle of forwarding settings.

[0087] In addition, it should be noted that the specific formula for calculating the initial attraction value is as follows:

[0088] ;

[0089] In the formula, The initial attractiveness score represents the candidate path; Normalized value representing the minimum remaining energy of nodes on the path; Normalized value representing the average link quality along the path; Represents a positive constant bias;

[0090] The specific formula for calculating fitness as described in S1.2 is as follows:

[0091] ;

[0092] In the formula, The fitness of the path; This represents the sum of the total remaining energy along the path; The estimated end-to-end transmission delay represents the path. This represents the packet loss rate estimate for the path.

[0093] The communication synchronization module is used to establish a secure channel between each sensor node, conduct secure communication, and synchronize the time of each sensor node.

[0094] Specifically, entangled photon pair transmitters and single-photon detectors are installed and connected between each pair of key nodes requiring secure channels and synchronization. Each node is equipped with a quantum random number generator and a local clock oscillator. The emission rate, photon spectral width, and polarization reference of the photon pair transmitters are then measured, along with the dark count rate and detection efficiency of the photon detectors. These are recorded as initial hardware parameters. The hardware parameters and initial observation data are written into a local initialization table, and the peer node is notified to enter a "pending distribution" state. Each endpoint selects its own generation basis sequence based on the output of the quantum random number generator and utilizes... The local clock oscillator records the timestamp of each transmission or measurement, sets a maximum allowable time difference threshold, and records the sequence of detection events occurring within each time window. The entangled photon pair transmitter emits entangled photon pairs at a set rate, with the photons at both ends transmitted to two different sets of nodes. The single-photon detector records the timestamp of the detection event and the measurement basis used based on the set time window. Then, each endpoint exchanges necessary basis selection information and timestamp indices through a classical channel, calculates the coincidence count within the decision time window, calculates the histogram of the time difference distribution of the paired timestamp data to obtain the corresponding time correlation function, fits the main peak of the time correlation function to estimate the peak position, and calculates the statistical uncertainty of the peak position by repeating sampling and fitting multiple times. Based on this statistical uncertainty, the confidence interval required for synchronization correction is evaluated. The peak position generated by multiple fittings is used as a preliminary clock offset estimate and transmitted to the correction stage. Reciprocal measurement is used, with endpoint A sending time stamp signals to endpoint B alternately and measuring in reverse to obtain the fixed optical path delay difference and node clock deviation. The clock deviation estimate is obtained by least squares fitting, and then the clock deviation estimate is converted into a local clock adjustment command and the local time is corrected. After each calibration, a short period of entanglement distribution is performed and a new time peak is calculated. If the deviation exceeds the preset safety threshold, it rolls back to the previous stable configuration and tries again with an adjusted step size. In or in parallel with the time synchronization process, the bit error rate of the qubits is calculated based on the paired measurement results. If the bit error rate of the qubits exceeds the threshold allowed by the protocol, an alarm should be triggered and the current round of key extraction should be stopped. Then, the bit error rate of the original bit string that is still retained after the basis screening is estimated, and error correction and privacy amplification are performed at the same time to obtain a usable symmetric key. The generated key is then recorded together with the synchronization state.

[0095] The acquisition and fusion module is used to preprocess data from various modalities and map different types of data to a unified feature space. Example

[0096] Reference Figure 1 A distributed environmental multimodal intelligent monitoring system includes a sensor node module, a perception and control module, a communication synchronization module, a data acquisition and fusion module, a data modeling module, a modal analysis module, a simulation and maintenance module, a visual decision-making module, and a security protection module.

[0097] Based on the sensor network structure, the data modeling module establishes a corresponding sensor map structure and updates the sensor map structure in real time to obtain the spatial topology changes of the launch platform under different operating conditions.

[0098] Specifically, each sensor node periodically broadcasts a data packet. The gateway or regional agent listens for each data packet and creates a new node entry in the graph database upon receiving a new ID. Then, each sensor node is written to its local edge table and synchronized to the graph engine's temporary table. After registration, each sensor node is immediately assigned an initial "activity tag" and version number. For sensor nodes with positioning capabilities, their output Cartesian coordinates are directly read. For sensor nodes without positioning capabilities, the Received Signal Strength Indication (RSSI) values ​​between them and their positioned neighbors are collected. Then, the relative distance between the missing sensor node and its neighbors is calculated using distance measurements such as neighbor RSSI. Coarse positioning is then performed using trilateration. Recursive positioning based on Kalman filtering fuses historical and current measurements, outputting the position estimate and covariance. Nodes whose location estimation covariance exceeds a preset threshold are marked as "location uncertain," and the location confidence index is saved in the node attributes of the graph. The geometric distance between each pair of sensor nodes is calculated, and the expected received power between each pair of sensor nodes is obtained based on the calculated geometric distance. Link reachability is predicted. If the expected received power between two sets of nodes meets the reachability threshold, an edge is added to the candidate edge set. At the same time, a corresponding timestamp, initial state, and observation source are added to each candidate edge. Multi-source indicators for each candidate edge are collected, and edge weights are constructed using a weighted linear combination. Then, the weight matrix is ​​stored in a sparse format, and the weight value of each edge is maintained using a sliding time window. Then, an exponentially weighted moving average is performed on the weight value of each edge, and the updated weight value is returned to the graph structure. The version number is then incremented to generate the latest sensor graph structure.

[0099] The data packet specifically includes the node's unique ID, device type code, local coordinate estimate, remaining energy, most recent sampling timestamp, and antenna received power estimate.

[0100] The modal analysis module is used to predict environmental change trends and potential anomalies of the launch platform, and automatically performs root cause analysis after an anomaly is detected.

[0101] Specifically, the sensor graph structure of the current time and multiple past sampling periods is read, and the multimodal observation vector of each node and the weight of the corresponding edge are obtained. The node observations at each time are then sorted by node ID. After alignment, missing value repair and normalization are performed on the multimodal observation vectors of each sensor node to generate corresponding temporal node feature tensors. The weights of each edge in the current time-stamped sensor graph structure are extracted. Based on the extracted edge weights, an adjacency matrix with self-loops is constructed, and the corresponding degree matrix is ​​calculated. Then, based on the adjacency matrix and degree matrix, a normalized symmetric Laplacian approximation is used to construct a normalized adjacency matrix. The normalized adjacency matrix for each time step is calculated, and temporal smoothing is performed on the normalized adjacency matrix for each time step. The spatial propagation matrix for each time step is output. The standardized node feature matrix for each time step of the sensor graph structure is extracted. Then, the node feature matrix and spatial propagation matrix are processed through graph convolution for information aggregation, followed by activation processing using a nonlinear activation function. After each round of graph convolution, batch normalization and residual connection processing are performed, finally outputting a set of temporal spatial embedding sequences. These temporal spatial embedding sequences are stacked temporally to form a three-dimensional sequence. The sequence is processed, and the time series of each sensor node is extracted. The time series of all sensor nodes are input into a shared parameter LSTM encoder in parallel. The LSTM encoder updates the hidden state and outputs the time context vector at each time step. Finally, the time encoding representation of each node is generated. The time encoding of each sensor node is concatenated with the spatial embedding of the current time to form a comprehensive representation vector. The comprehensive representation vector is then input into a feedforward neural network. The feedforward neural network processes the comprehensive representation vector layer by layer through the forward propagation algorithm and outputs the predicted values ​​of continuous environmental quantities and anomaly probability scores. Each anomaly probability score is compared with the historical baseline. If it exceeds the historical baseline, the corresponding sensor node is marked as "abnormal". At the same time, the synchronization anomaly indication, device energy anomaly and physical consistency from neighboring nodes are checked. If multiple pieces of evidence meet the preset support conditions, a formal anomaly event is triggered. The confidence score of each judgment is calculated and the trigger time and the observation dimensions involved are recorded.

[0102] The simulation maintenance module is used to build a digital twin of the launch platform in the cloud, while predicting changes in airflow, heat distribution and structural stress in the launch environment, and dynamically optimizing maintenance strategies; the visual decision-making module is used to provide a multi-level visualization interface to provide decision support suggestions for operation and maintenance personnel; the security protection module is used to encrypt and control access to data throughout the entire chain of acquisition, transmission and storage.

Claims

1. A distributed environment multi-modal intelligent monitoring system, characterized in that, The sensing node module, the perception control module, the communication synchronization module, the acquisition fusion module, the data modeling module, the modal analysis module, the simulation maintenance module, the visual decision module, and the security protection module are included. The sensing node module collects multi-modal environmental data through a sensor network and adaptively adjusts the sampling frequency and communication strategy according to the energy level. The perception control module is used for comprehensively evaluating the energy margin and link quality of each sensing node in the sensor network and selecting an optimal path for transmitting data. The communication synchronization module is used for establishing a secure channel between each sensing node, performing secure communication, and synchronizing the time of each sensing node. The acquisition fusion module is used for preprocessing each modal data and mapping different types of data to a unified feature space. The data modeling module establishes a corresponding sensing graph structure according to the sensor network structure and updates the sensing graph structure in real time to obtain the spatial topology changes of the launch platform under different working conditions. The modal analysis module is used for predicting the environmental change trend and potential anomalies of the launch platform and automatically performing root cause analysis after detecting anomalies. The simulation maintenance module is used for constructing a digital twin of the launch platform in the cloud, predicting airflow changes, heat distribution, and structural stress of the launch environment, and dynamically optimizing maintenance strategies. The visual decision module is used for providing a multi-level visual interface to provide decision support suggestions for operation and maintenance personnel. The security protection module is used for encrypting data in the acquisition, transmission, and storage links and performing access control.

2. The distributed environment multi-modal intelligent monitoring system according to claim 1, wherein, The specific steps of the perception control module selecting an optimal path for transmitting data are as follows: S1.1: When the sensor network starts or the routing period begins, the gateway or regional controller scans all the currently perceived sensor nodes and links and constructs a set of candidate path collections from the source node to the destination node, and assigns an initial attraction value to each candidate path. S1.2: According to the real-time measured energy consumption estimate, end-to-end delay estimate, and link reliability, the fitness of each candidate path is calculated, and then for each candidate path, a middle node on the path is randomly selected and replaced with its backup neighbor node to generate a new path variant, and the fitness of the new path variant is calculated. S1.3: If the fitness of the new path variant is better than that of the original candidate path, the original candidate path is updated to the path variant, otherwise the original solution is retained and the abandonment count of the path variant is increased, then the selection probability is assigned based on the normalized fitness of each candidate path, and then a random candidate path is selected according to the selection probability of each candidate path and subjected to multiple local perturbations. S1.4: The multiple path variants generated by perturbation are evaluated, and the path variant with the highest fitness is selected. If the fitness value of the selected path variant is greater than that of the original candidate path, the path variant is updated to the optimal variant, otherwise the original candidate path is retained and the abandonment count of the corresponding path variant is continued to accumulate. S1.5: If the discard count of the path variant exceeds the preset threshold, the path is discarded, and a new path variant with high remaining energy and good link quality is generated to fill the position, and the discard count is cleared, and the initial score of the new path variant is calculated, otherwise, the historical stability of the corresponding path variant is evaluated by the reserved score function, if the evaluation result is higher than the preset threshold, it is reserved, otherwise, the replacement process is executed; S1.6: After each round of path search, replacement and update is completed, based on the latest fitness and acceptance record, the candidate path table is sorted and updated according to the fitness from high to low, and multiple paths with high ranking and meeting the energy-reliability threshold are selected as the actual data forwarding path, and the corresponding path is issued to the node for the next cycle of forwarding setting.

3. The distributed environment multi-modal intelligent monitoring system of claim 2, wherein, The communication synchronization module establishes a secure channel between each sensing node, and the specific steps of secure communication are as follows: S2.1: Install and connect the entangled photon pair transmitter and single photon detector between each pair of key nodes that need to establish a secure channel and synchronization, and equip each node with a quantum random number generator and a local clock oscillator, then measure the emission rate, photon spectral width and polarization reference of the entangled photon pair transmitter photon pair, and then measure the dark count rate and detection efficiency of the photon detector, and record the initial hardware parameters; S2.2: Write the hardware parameters and initial observation data into the local initialization table and notify the opposite node to enter the "to be distributed" state, each end point generates its own base selection sequence according to the quantum random number generator, and records the time stamp of each transmission or measurement using the local clock oscillator, and sets the maximum allowed time difference threshold; S2.3: Record the detection event sequence occurring in each time window, the entangled photon pair transmitter emits entangled photon pairs at a set rate, the photons at both ends are transmitted to two different groups of nodes, and the single photon detector records the detection event time stamp and the measurement base used based on the set time window, then each end point exchanges the necessary base selection information and time stamp index through the classical channel, and calculates the coincidence count in the determination time window; S2.4: Calculate the histogram of the time difference distribution of the paired timestamp data to obtain the corresponding time correlation function, fit the main peak of the time correlation function to estimate the peak position, calculate the statistical uncertainty of the peak position by repeating multiple sampling and fitting, and evaluate the confidence interval required for synchronization correction according to the statistical uncertainty, and the peak position generated by multiple fitting is transmitted to the correction stage as the preliminary clock offset estimate; S2.5: Use reciprocal measurement, alternately send time marker signals from endpoint A to endpoint B and measure in reverse, and obtain the optical path fixed delay difference and node clock deviation, obtain the clock deviation estimate value by least squares fitting, then convert the clock deviation estimate value into a local clock adjustment command, and correct the local clock, after each correction, perform entanglement distribution again for a short time and calculate a new time peak, if the deviation exceeds the preset safety threshold, roll back to the last stable configuration and try again with an adjustment step. S2.6: In or parallel to the time synchronization process, the quantum bit error rate is calculated based on the paired measurement results, if the quantum bit error rate exceeds the protocol allowed threshold, an alarm should be triggered and the current round of key extraction should be aborted, then the raw bit string that still remains after the base screening is subjected to error rate estimation, at the same time error correction is performed and privacy amplification is carried out to obtain the available symmetric key, and the generated key is recorded together with the synchronization state.

4. The distributed environment multi-modal intelligent monitoring system of claim 3, wherein, The specific steps of establishing the corresponding sensing graph structure by the data modeling module are as follows: S3.1: Each sensing node periodically broadcasts a data packet, and the gateway or regional agent listens to each data packet, and when a new ID is received, a node entry is created in the graph database, and then each sensing node is written into the local edge table and synchronized to the temporary table of the graph engine, and after registration is completed, an initial "active label" and version number are assigned to each sensing node; S3.2: For each sensing node with positioning capability, directly read its output three-dimensional coordinates in Cartesian coordinate system, for the sensing nodes without positioning capability, collect the received signal strength indication values RSSI of the neighboring nodes with positioning capability, then calculate the relative distance between the sensing node without positioning information and the neighboring nodes using the ranging quantities such as neighbor RSSI, and then perform coarse positioning using the trilateration method, and fuse the historical and current measurements based on the recursive positioning of Kalman filter, output the position estimate and covariance, and mark the nodes with covariance of position estimate exceeding the preset threshold as "position uncertain", and save the position reliability index in the node attribute of the graph; S3.3: Calculate the geometric distance between each pair of sensing nodes, and based on the calculated geometric distance, obtain the expected received power between each pair of sensing nodes, and perform link reachability prediction, if the expected received power between the two groups of nodes meets the reachability threshold, add an edge in the candidate edge set, and add the corresponding timestamp, initial state and observation source to each candidate edge; S3.4: Collect the multi-source indicators of each candidate edge, and use weighted linear combination to construct the edge weight, then store the weight matrix in sparse format, use sliding time window to maintain the weight value of each edge, perform exponential weighted moving average on the weight value of each edge, return the updated weight value to the graph structure, and trigger the version number to increment, to generate the latest sensing graph structure.

5. The distributed environment multi-modal intelligent monitoring system of claim 4, wherein, The specific steps of predicting the environmental change trend and potential anomalies of the launch platform by the modal analysis module are as follows: S4.1: Read the sensing graph structure at the current time and in the past multiple sampling periods, obtain the multi-modal observation vector of each node and the weight of the corresponding edge, align the node observations at each time according to the node ID, then perform missing value repair and normalization processing on the multi-modal observation vector of each sensing node to generate the corresponding time series node feature tensor; S4.2: Extract the weight of each edge in the current time step of the sensor graph structure, construct the adjacency matrix with self-loop based on the extracted weight of each edge, then calculate the degree matrix, and then based on the adjacency matrix and the degree matrix, construct the normalized adjacency matrix using the normalized symmetric Laplace approximation, calculate the normalized adjacency matrix of each time step, and at the same time, perform time smoothing processing on the normalized adjacency matrix of each time step, and output the spatial propagation matrix of each time step; S4.3: Extract the normalized node feature matrix of each time step of the sensor graph structure, then perform information aggregation by graph convolution processing on the node feature matrix and the spatial propagation matrix, then perform activation processing by a nonlinear activation function, and after each round of graph convolution processing, perform batch normalization and residual connection processing, and finally output a set of time sequence spatial embedding sequences; S4.4: Stack the time sequence spatial embedding sequences into a three-dimensional sequence according to time, and extract the time sequence of each sensor node, and input the time sequence of all sensor nodes into the shared parameter LSTM encoder in parallel, the LSTM encoder updates the hidden state at each time step and outputs the time context vector, and finally generates the time encoding representation of each node; S4.5: Concatenate the time encoding of each sensor node with the spatial embedding of the current time to form a comprehensive representation vector, then input the comprehensive representation vector into the feedforward neural network, then the feedforward neural network processes the comprehensive representation vector layer by layer through the forward propagation algorithm, and outputs the predicted value of the continuous environmental quantity and the abnormal probability score; S4.6: Compare each abnormal probability score with the historical baseline, if it exceeds the historical baseline, mark the corresponding sensor node as "abnormal", and at the same time check the synchronous abnormal indication from the neighbor nodes, the device energy abnormality and the physical consistency, if multiple evidences meet the preset support condition, trigger the formal abnormal event, at the same time calculate the confidence score of each determination, and record the triggering time and the involved observation dimension.

Images