Multimodal fire safety monitoring method and system integrated with edge computing

Abstract

The application provides a multi-modal fire safety monitoring method and system integrated with edge computing, which comprises the following steps: obtaining original environmental state data of each monitoring node and constructing multi-modal standardized features; determining potential paths of smoke or heat diffusion between edge nodes according to the multi-modal standardized features and a spatial adjacency graph structure constructed based on physical constraint driving, and generating an adjacency matrix; calculating an instantaneous risk value which is time-unmemorable and based on current features only; calculating a current integrated risk value based on the instantaneous risk value through a trend-oriented risk integration structure; determining an alarm level of a current node according to a comparison difference between the current integrated risk value and a risk integration threshold value for a corresponding local edge node, and performing a preset response action according to the corresponding alarm level. Through the above improvement, the application can systematically solve a plurality of key problems in the current fire safety monitoring field.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of edge computing, and particularly relates to a multi-modal fire safety monitoring method and system integrated with edge computing. BACKGROUND

[0002] At present, the fire safety monitoring system has gradually developed from the local detection mode driven by a single sensor to the multi-source information fusion and intelligent linkage direction. In various types of buildings, industrial facilities, transportation hubs and other scenes, multi-modal sensors such as smoke, temperature, gas and wind speed have become a basic configuration, and it is expected to improve the early identification ability of fire risk. However, there are several key technical shortcomings in the actual application of the existing fire safety monitoring system. On the one hand, the system mostly relies on the central server to process the data reported by the sensor, and needs to return the original information to the cloud for judgment, which causes large communication pressure and high response time delay, especially in the communication-unfriendly areas such as underground space and old buildings, the system is easy to fail due to network anomalies, and lacks local autonomy. On the other hand, although multi-modal sensors (such as smoke, temperature, gas, and wind speed) have gradually become popular, most systems still use fixed weight or rule-triggered methods for data fusion, which is difficult to model the dynamic correlation between modalities, and is prone to false positives and false negatives in complex ventilation environments. In addition, the current mainstream system generally uses a "instantaneous threshold trigger" alarm mechanism, which lacks the ability to model the evolution trend of fire, and cannot judge non-urgent risk situations such as sustained warming and slow diffusion, and it is also difficult to perform hierarchical response or dynamic linkage control. At the same time, limited by the computing power and power consumption of the edge node, most complex algorithms cannot be directly deployed on the front-end device, and there is still a lack of lightweight, multi-modal intelligent analysis scheme adapted to the fire monitoring scene. The above problems seriously restrict the performance improvement of the intelligent fire safety system in terms of timeliness, accuracy and response linkage, and a new method with edge deployment capability, supporting multi-modal information modeling and dynamic risk response is urgently needed. SUMMARY

[0003] The purpose of the present application is to provide a multi-modal fire safety monitoring method and system integrated with edge computing, which can systematically solve a number of key problems in the current fire safety monitoring field.

[0004] In order to achieve the above purpose, in the first aspect of the present application, a multi-modal fire safety monitoring method integrated with edge computing is provided, the method comprising:

[0005] Obtaining the original environmental state data of each monitoring node, and constructing multi-modal standardized features;

[0006] According to the multi-modal standardized features and the constructed spatial adjacency graph structure driven by physical constraints, a potential path of smoke or heat diffusion between edge nodes is determined by generating an adjacency matrix, wherein the graph structure includes an edge node set and an adjacency matrix set, and the adjacency matrix set includes several wind direction enhanced adjacency weights;

[0007] According to the multi-modal standardized features, the spatial adjacency graph structure, and the trained smoke concentration prediction model, a spatial smoke concentration prediction value of an area without deployed sensors is generated, and a time-unaware, current-feature-based instantaneous risk value is calculated again in combination with the multi-modal standardized features;

[0008] A current integrated risk value is calculated based on the instantaneous risk value through a trend-oriented risk integration structure, serving as an alarm judgment variable;

[0009] For a corresponding local edge node, an alarm level of the current node is determined according to a comparison difference between the current integrated risk value and a risk integration threshold, and a preset response action is performed according to the corresponding alarm level.

[0010] Preferably, the original environmental state data includes smoke concentration, temperature, gas concentration, and air flow rate.

[0011] The multi-modal standardized features are constructed as follows:

[0012] Original environmental state data is obtained.

[0013] All modal samples are aligned to a fixed time window, and for missing data within the window, a local reconstruction is performed using a reconstruction value obtained by averaging the values of the available sampling points before and after the corresponding node within the window.

[0014] Each type of aligned data is normalized to convert it into a dimensionless feature representation, and the four types of original environmental state data are combined in a weighted manner to form the multi-modal standardized features of the corresponding node at the current time.

[0015] Preferably, the spatial adjacency graph structure driven by physical constraints is constructed as follows:

[0016] The spatial coordinates of all monitoring nodes are obtained, and the Euclidean distance between nodes is calculated based on the spatial coordinates.

[0017] According to the Euclidean distance between nodes and the wind direction angle of air flow rate, a wind direction enhanced adjacency weight is determined.

[0018]

[0019] wherein p (i) , p(j) respectively, are the coordinates of nodes i, j; σ is a spatial Gaussian attenuation factor; d0 is the maximum connection distance threshold beyond which the nodes are considered not conductive; θ ij is the angle between the current wind direction of node i and the unit directional vector from i to j; cos + (θ ij ) = max(0, cos(θ ij )) represents the positive incentive effect of wind direction on the diffusion path; is an indicator function, which takes 1 if the connection ij crosses the obstacle region, otherwise 0; λ is the obstacle term penalty coefficient, representing the weakening of the spatial blockage to the diffusion path;

[0020] An adjacency matrix is obtained according to the wind direction enhanced adjacency weight.

[0021] Preferably, the spatial adjacency graph structure based on physical constraint driving is also provided with a wind direction offset penalty function: when the angle between the current wind direction of the current node i and the unit directional vector from the current node i to node j is the smallest compared with the angle between the average wind direction of the current node i at the previous n time steps, the wind direction offset penalty reaches the maximum, and the matching correlation probability is the highest, which is used to cope with the sudden change of environmental wind direction or the influence of sensor abnormality.

[0022] Preferably, the training process of the smoke concentration prediction model comprises:

[0023] obtaining multi-modal standardized features, an adjacency matrix and historical spatial smoke concentration values; each element A ij of the adjacency matrix represents the physical diffusion potential from node j to node i; the historical spatial smoke concentration values include corresponding historical air flow rate and corresponding time;

[0024] mapping the multi-modal standardized features of node i or j to the local concentration initial estimate value using the edge side lightweight model; aggregating the feature multi-modal standardized features of the adjacent nodes j; a physical consistency residual term is also designed to punish abnormal trends that do not conform to the diffusion physical law in the prediction; the prediction formula is:

[0025]

[0026] wherein C(v i ,t) is the spatial smoke concentration prediction value, f θ (·) is the edge side lightweight model, which is used to map the features of node i or j to the local concentration initial estimate value; ω0 is the local perception weight; A ij is the adjacency weight; represents the set of graph structure adjacent nodes of node i; β is the physical consistency adjustment coefficient; For physical consistency residual term, used to punish abnormal trends that do not conform to the diffusion physical law in prediction.

[0027] Preferably, the edge side lightweight model is a lightweight two-layer neural network with normalized activation, each layer uses ReLU activation function, and the last two minutes of historical concentration samples are locally fine-tuned with mean square loss; the edge side lightweight model only uses node local multi-modal standardized features for prediction, meeting the edge low power consumption deployment conditions.

[0028] Preferably, according to the multi-modal standardized features, spatial adjacency graph structure and trained smoke concentration prediction model, the spatial smoke concentration prediction value of the region where the sensor is not deployed is generated, and the multi-modal standardized features are combined again to calculate the instantaneous risk value based on the current features without memory to time, including:

[0029] The spatial smoke concentration prediction value and the smoke concentration, temperature and gas concentration in the corresponding multi-modal standardized features are fused according to the actual risk contribution degree to form a risk density function, and the current integral risk value is calculated; wherein the current integral risk value is processed by regular suppression to improve the stability of risk expression in space; wherein the regular suppression processing is:

[0030]

[0031] Wherein, φ(v i ,t) is the instantaneous risk value of node i, λ is the local regular strength coefficient; is the graph adjacency set of node i; v j is the jth node, φ(v j ,t) is the instantaneous risk value of node j

[0032] The regular suppression processing adjusts the instantaneous risk value φ(v i ,t) of node i to approach the mean value of its neighborhood, suppresses the false alarm risk caused by isolated high value, and retains the overall risk space difference.

[0033] Preferably, the current integral risk value is calculated based on the instantaneous risk value through a trend-oriented risk integral structure, which is used as an alarm judgment variable, including:

[0034] According to the instantaneous risk value and the historical instantaneous risk value of the corresponding node, the trend enhancement gating function is used to amplify the growth;

[0035] The previous time integral value and the instantaneous risk value of the corresponding node are acquired, the trend-enhanced instantaneous risk value and the spatial decoupling regularization term are combined, and the current integral risk value is calculated through state update analysis; the spatial decoupling regularization term is used for punishing the continuous anomaly on the isolated node; and the current integral risk value contains composite variables of historical information, trend state and spatial structure alignment.

[0036] Preferably, the preset response action includes local prompting, starting sound and light alarm, executing high-level linkage and not triggering any action, and only maintaining a listening state.

[0037] In another aspect of the application, an integrated edge computing multi-modal fire safety monitoring system is provided, which comprises a data acquisition module, a spatial adjacency graph construction module, an instantaneous risk value determination module, a trend risk value analysis module and an alarm module.

[0038] The data acquisition module is used for acquiring the original environmental state data of each monitoring node and constructing multi-modal standardized features.

[0039] The spatial adjacency graph construction module is used for determining the potential path of smoke or heat diffusion between edge nodes according to the multi-modal standardized features and the constructed spatial adjacency graph structure based on physical constraint driving, by generating an adjacency matrix; wherein the graph structure comprises an edge node set and an adjacency matrix set, and the adjacency matrix set comprises several wind direction enhanced adjacency weights.

[0040] The instantaneous risk value determination module is used for generating a spatial smoke concentration prediction value of a region without deploying a sensor in space according to the multi-modal standardized features, the spatial adjacency graph structure and a trained smoke concentration prediction model, and calculating an instantaneous risk value without memory to time and only based on the current features again in combination with the multi-modal standardized features.

[0041] The trend risk value analysis module is used for calculating a current integral risk value based on the instantaneous risk value through a trend-oriented risk integral structure, as an alarm judgment variable.

[0042] The alarm module is used for determining the alarm level of the current node according to the comparison difference between the current integral risk value and the risk integral threshold value for the corresponding local edge node, and executing a preset response action according to the corresponding alarm level.

[0043] The application has at least the following beneficial technical effects:

[0044] In view of the above problems, the application provides a multi-modal fire safety monitoring method and system integrated with edge computing, and an intelligent monitoring mechanism capable of independent operation on the edge side and having space understanding and time evolution perception capability is constructed. The application simultaneously fuses multiple sensing data (including smoke, temperature, gas, air flow, etc.) on the edge node, and introduces an intelligent modeling mechanism for risk trend identification, and no longer relies on instantaneous judgment of a single mode, but considers the dynamic consistency and change trajectory among multi-modal signals, thereby effectively suppressing false positives and enhancing early fire identification capability. On this basis, the application further introduces an interpretable risk accumulation mechanism, constructs a fire evolution risk field by spatial modeling and integral judgment of sensing signals in different regions and time periods, and realizes hierarchical response and regional guidance linkage based on the field information, thereby improving the overall intelligence, flexibility and controllability of the system. In addition, the designed algorithms all have lightweight characteristics and can be embedded and run on resource-limited edge nodes, realize local perception, reasoning and response without the center server, and truly meet the actual deployment requirements of the fire safety system in terms of low power consumption, high real-time performance and high reliability. Through the above improvements, the application can systematically solve multiple key problems in the current fire safety monitoring field. BRIEF DESCRIPTION OF DRAWINGS

[0045] The application is further described by means of the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the following drawings.

[0046] Figure 1 A multi-modal fire safety monitoring method integrated with edge computing is provided for the embodiment of the application.

[0047] Figure 2 A multi-modal fire safety monitoring system framework integrated with edge computing is provided for the embodiment of the application. DETAILED DESCRIPTION

[0048] The embodiments of the application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are only used to explain the application, and cannot be understood as a limitation on the application.

[0049] As shown in the drawings, Figure 1 A multi-modal fire safety monitoring method integrated with edge computing is provided for the embodiment of the application, and the method comprises the following steps S1-S5:

[0050] S1, acquiring original environment state data of each monitoring node, and constructing multi-modal standardized features.

[0051] Specifically, in the edge device, the original environmental state data of each monitoring node i is collected by deploying a sensing module, and multi-modal standardized features are constructed, including smoke concentration (unit ppm), temperature (unit ℃), gas concentration (unit ppm), and air flow rate (unit m / s vector), each of which has different sampling frequencies and time delays. Due to the phenomenon of asynchronous sampling between sensors, data loss, etc. in the fire environment, the system performs uniform time slot alignment and missing value filling on the collected sequence to avoid the interference of single-modal data loss on multi-modal joint modeling.

[0052] To this end, the present application first aligns all modal samples to a fixed time window (for example, every 60 seconds), and for the missing data in the window, the "neighbor point average reconstruction strategy" is used for local reconstruction. Specifically, for the missing smoke sensor value , the reconstructed value is obtained by averaging the values of the available sampling points before and after it in the window:

[0053]

[0054] where δ represents the distance between the nearest previous and subsequent valid sampling intervals (in seconds) from time point t; and are the smoke concentration values (unit ppm) of the node at the nearest previous and subsequent sampling time points, respectively. This reconstruction strategy does not introduce additional modeling burden, but only relies on the limited historical cache data stored locally by the edge node, ensuring that high-integrity input construction can be achieved under the condition of limited node resources.

[0055] Next, each type of aligned data is normalized to convert it into dimensionless feature representation to adapt to subsequent model input. The normalization method uses the minimum and maximum values of the sliding time window in the node to avoid introducing global dependencies. At the same time, considering that different modalities have different risk contributions in early fire identification, the system combines the four types of normalized features in a weighted manner to form the multi-modal standardized feature of node i at time t

[0056] where represents the normalized smoke concentration value, which is normalized by the minimum and maximum interval in the most recent time period; is the normalized result of the temperature value, which is normalized by is the normalized result of the gas concentration value, which is normalized by ​is the normalized scalar value of the wind speed vector, which is derived from w s ,w T ,w G ,w V are the modal risk-sensitive weighting coefficients, which are set as {0.4, 0.3, 0.2, 0.1} by default, and can be adjusted according to the actual scene (such as kitchen, warehouse, office area, etc.) in the deployment phase.

[0057] The final generated multi-modal standardized features As the standardized multi-modal input features of the node at the current time, it will be directly input into the modeling section in the subsequent steps for concentration field prediction and risk modeling calculations.

[0058] S2, according to the multi-modal standardized features and the constructed space adjacency graph structure driven by physical constraints, the potential paths of smoke or heat diffusion between the edge nodes are determined by generating an adjacency matrix, wherein the graph structure includes an edge node set and an adjacency matrix set, and the adjacency matrix set includes several wind direction enhanced adjacency weights.

[0059] Specifically, this step aims to construct a spatial topological structure graph G=(V,A) with directionality and environmental constraint capability for the spatial physical complexity in the fire monitoring system, to describe the potential paths of smoke or heat diffusion between the edge nodes, and to provide a structural basis for subsequent concentration propagation modeling. The establishment of this graph structure is not simply a "spatial proximity judgment", but combines node deployment location, air flow direction, building ventilation layout and other fire scene specific information to construct a physical constraint driven adjacency relationship model. This step directly depends on the multi-modal standardized features output by step 1 wherein the wind speed vector is the key parameter for spatial connectivity determination in this step.

[0060] First, read the spatial coordinates p (i) =(x (i) ,y (i) ) of all monitoring nodes, which are obtained once through building CAD drawings or laser mapping systems during system deployment, and are used to calculate the Euclidean distance between nodes ||p (i) -p (j) ||. In addition, the system introduces a building structure layer M(x,y) to encode the unreachable areas (such as walls, enclosed spaces) in the building in matrix form, to constrain the legality of the space connection path.

[0061] In traditional adjacency graph construction methods, the connection weights between nodes often only consider spatial distance. However, in actual fire scenarios, the smoke diffusion path is highly dependent on the airflow direction. Therefore, we designed a wind-enhanced adjacency weight function A. ij , used to describe the smoke reachability from node i to node j, is defined as follows:

[0062]

[0063] in,

[0064] p (i) p (j) : These are the coordinates of nodes i and j (in meters);

[0065] σ: Spatial Gaussian attenuation factor (default value 3 meters);

[0066] d0: Maximum connection distance threshold (default 6 meters). If this is exceeded, the nodes are considered to be non-connectable.

[0067] θ ij : Current wind direction of node i (source) In The angle between (i) and the unit direction vector pointing from i to j;

[0068] cos + (θ ij )=max(0,cos(θ ij ): Indicates the positive excitation effect of wind direction on the diffusion path;

[0069] Indicator functions, if connected If the obstacle is crossed, the value is 1; otherwise, it is 0.

[0070] λ: The obstacle penalty coefficient, representing the weakening effect of spatial obstruction on the diffusion path (typical value λ = 1.0).

[0071] This weighted model integrates three layers of physical elements in a fire scenario: spatial distance, airflow direction, and building structural obstacles. It captures the actual probability of smoke propagation through explicit structural modeling, rather than relying solely on data-driven learning.

[0072] In this graphical model, the adjacent edges of each node not only represent spatial connections but also reflect the directional path of smoke transmission. For example, if two nodes are physically adjacent but have opposing wind directions, are blocked by walls, or have their airflow disturbed by building structures, then their A... ij It will be significantly suppressed or even reduced to 0, to avoid the generation of physically unreasonable propagation paths in the subsequent diffusion modeling of the system.

[0073] In particular, to enhance the robustness and deployment flexibility of the system, we introduce a wind direction anomaly suppression term in the graph construction process to deal with sudden changes in environmental wind direction or sensor anomalies. This mechanism introduces a wind direction offset penalty function p ij , defined as:

[0074]

[0075] is the average wind direction of node i in the previous n time steps;

[0076] is the wind direction offset penalty strength coefficient (typical value is 0.5);

[0077] This term will be multiplied into A ij , dynamically adjusting the connection stability and reducing false connections caused by temporary wind interference.

[0078] The final adjacency matrix A will be used in the subsequent diffusion modeling step to guide the spatial propagation path of the predicted value C(v i ,t). Each edge of the graph structure G = (V, A) has a real meaning in physical space, and the construction method does not require complex fluid simulation throughout, with low computational complexity, edge device friendliness, and adaptability to fire scenarios. This step plays a key role in connecting physical space information and model prediction structure in the patent scheme, and is the basis for subsequent implementation of spatial accurate diffusion prediction and dynamic risk transmission.

[0079] S3, according to the multi-modal standardized features, spatial adjacency graph structure and trained smoke concentration prediction model, generating spatial smoke concentration prediction values in areas where sensors have not been deployed, and again combining the multi-modal standardized features, calculating the instantaneous risk value based on the current features without memory.

[0080] Specifically, this step aims to construct a smoke concentration prediction model C(v ,t) based on the spatial adjacency graph structure G = (V, A) and multi-modal feature vector i generated in the previous step, to achieve smoke concentration estimation in areas where sensors have not been deployed, and to provide key input for subsequent fire risk density modeling. Unlike the traditional sensor "just-in-time" method, this patent addresses the characteristics of strong spatial correlation, weak point sampling density and dynamic interference flow field in building interior smoke diffusion, and proposes an "edge smoke prediction modeling mechanism guided by graph", which comprehensively realizes spatial completion, time series filtering and physical consistency.

[0081] The inputs include: multi-modal standardized features where the smoke, temperature, gas and wind speed data have been standardized; adjacency matrix where each element Aij denotes the physical diffusion potential from node j to node i, which combines the wind direction, spatial distance and structural barrier information; historical concentration estimation value before the current time t for judging local diffusion trend and filtering out isolated anomalies.

[0082] To construct a reasonable estimate of smoke concentration, this patent proposes a physically consistent residual regulation model (PhysCons-ResReg), whose core prediction mechanism is as follows:

[0083]

[0084] f θ (·) is an edge-side lightweight model used to map the features of node i or j to the local concentration initial estimate;

[0085] ω0is the local perception weight, which is set to 0.5 by default, and the rest is contributed by graph diffusion;

[0086] A ij is the adjacency weight defined in step 2;

[0087] denotes the set of graph structure adjacent nodes of node i;

[0088] βis the physical consistency adjustment coefficient, set to 0.1-0.3;

[0089] is the physically consistent residual term, which is used to punish abnormal trends that do not conform to the diffusion physical law in prediction, defined as follows:

[0090]

[0091] This residual term reflects whether the concentration change at node i at the previous time is consistent with the surrounding area, and is used to avoid "isolated jump" false positives (such as external interference, single-point sensor fluctuations, etc.). By incorporating into the prediction formula, the system can take into account the neighborhood average trend in spatial diffusion modeling, avoiding prediction errors driven by non-diffusion.

[0092] In actual operation, all f θ (·) is implemented as a lightweight two-layer neural network with normalized activation (such as [4-8-1] network structure), each layer uses ReLU activation function, and the last two minutes of historical concentration samples are fine-tuned locally with mean square loss. This model only uses the node's Prediction is made to meet the edge low-power deployment conditions. The structure of the prediction model regards the local concentration estimation of each node in the graph as a combined process: on the one hand, based on the local perception prediction of the current features, and on the other hand, from the physical diffusion contribution of the adjacent nodes based on the wind direction and the structure conduction, and finally through the physical consistency term to suppress the outliers. Such design makes the system have stronger spatial continuity expression ability, especially suitable for practical scenarios such as the existence of partial blind area in the building, wind field disturbance, and serious sensor fluctuation. The final output C(v i ,t) is used to build the risk density function φ(v i ,t) for the next step, and can also be used as an input for dynamic visualization of the concentration map, for front-end interface rendering or auxiliary personnel to identify potential risk areas.

[0093] Further, the spatial smoke concentration prediction value C(v i ,t) is fused with the multi-modal feature vector x t (i) of the node at the current time to build a spatially continuous risk density function φ(v i ,t) for representing the fire risk intensity of the node v i at time t. The function will be used as an input for the next step of integral field modeling to determine the starting value of the time accumulation process.

[0094] The input variables include: C(v i ,t): representing the predicted concentration value of the node v i at the current time, with the unit of ppm; from step 1, representing the standardized features collected by the node; current smoke sensor raw value (ppm), not predicted; current temperature (℃), normalized; current flammable gas concentration (ppm), normalized; current air flow speed module length (m / s), normalized.

[0095] To maintain physical interpretability, this step designs a modal risk-oriented fusion structure to fuse C(v i ,t) and each variable in x

[0096]

[0097] w C is the main weight of the concentration prediction, and the recommended value is 0.5;

[0098] w T ,w G ,w S{0.2, 0.2, 0.1} respectively, recommended values are {0.2, 0.2, 0.1} in turn;

[0099] Note: here we use the original value as an auxiliary input to prevent the prediction model from being slow to respond to rapid concentration rises due to historical errors; It is not directly involved in modeling because its role has already affected the graph structure diffusion path in step 2, and should not repeat its role here to avoid variable conflicts.

[0100] To avoid extreme values dominating the overall risk expression, the system introduces a local mean constraint term in this step to regularize the instantaneous risk value φ(v i ,t) and improve the stability of the risk expression in space:

[0101]

[0102] λ is the local regularization strength coefficient (e.g. λ = 0.3);

[0103] is the graph adjacency set of node v i from step 2;

[0104] This formula adjusts φ(v i ,t) to approach the mean value of its neighborhood, suppressing false positive risks caused by isolated high values, while preserving the overall risk space differences.

[0105] The final output of this step φ(v i ,t) is an instantaneous risk value that does not remember time and is only based on current features, satisfying multiple requirements such as continuous, smooth, interpretable, and suitable for subsequent integral modeling.

[0106] S4, based on the instantaneous risk value, calculate the current integral risk value through a trend-oriented risk integral structure, as an alarm judgment variable.

[0107] Specifically, the task of this step is to construct the risk integral value R(v i ,t) of each node v i at time t, which represents the fire risk accumulation in the time dimension. This integral field is the core dynamic quantity in this patent, not only responsive to sudden risks, but also able to distinguish between "short-term disturbances" and "persistent fire evolution". Unlike traditional instantaneous trigger mechanisms based on thresholds, this integral mechanism realizes the comprehensive modeling of multi-modal risk trend accumulation, response inertia, and spatial risk diffusion correlation.

[0108] The input is the risk density function φ(v i ,t) output by step 3i , t) is the instantaneous fire intensity representation at time t, which is based on the multi-modal feature fusion of concentration, temperature, and gas signals. It does not have the ability to express the trend of "risk evolution" by itself, so a time series memory structure R(v i , t) is constructed for subsequent alarm level judgment and response decision.

[0109] This step proposes a trend-oriented risk accumulation structure (Trend-Gated Risk Accumulator, TGRA for short), and the construction formula is as follows:

[0110]

[0111] Among them:

[0112] R(v i , t) represents the current integrated risk value, which is finally used as an alarm judgment variable;

[0113] R(v i , t-1) is the integrated value at the previous moment, which is recursively calculated;

[0114] ρ is the historical retention coefficient (default value 0.85), which represents the residual of past risk contribution at the current moment;

[0115] φ(v i , t) is the current risk density;

[0116] is a trend enhancement gating function, defined as follows:

[0117]

[0118] It means that the trend bonus is only introduced when the risk is rising, and x 2 amplifies significant growth;

[0119] η is the trend enhancement coefficient (e.g. η = 0.5);

[0120] is a spatial decoupling regularization term, used to punish persistent anomalies on isolated nodes:

[0121]

[0122] λ is the spatial regularization coefficient (typical value is 0.3);

[0123] is the set of nodes adjacent to v i in the graph structure;

[0124] This term encourages R(v i,t) does not deviate from the evolution trend of surrounding nodes, avoiding false alarms of isolated points.

[0125] Compared with φ(v) in the previous step i The values ​​of ,t) are different, R(v) i (t) is a composite variable that incorporates historical information, trend status, and spatial structure alignment. Its main functions include:

[0126] To characterize whether the fire risk at the location of the node has a "continuous accumulation" characteristic;

[0127] Determine whether the risk is continuously amplified by external disturbances;

[0128] Through space regularization Suppressing single-point anomalous risk values ​​and maintaining the coherence of the overall spatial risk field;

[0129] This forms a dynamic risk evolution map that is smooth in space and coherent in time.

[0130] This integral structure is particularly well-suited to the complex structural building environments addressed in this patent, including:

[0131] Smoke sensors are sparsely deployed in areas where signals may be intermittent;

[0132] Locations where gases are easily diffused, local disturbances are frequent, but the overall fire develops slowly;

[0133] Frequent wind speed disturbances, but these are not abnormal phenomena caused by a fire.

[0134] In the above situation, relying solely on instantaneous thresholds is prone to false alarms, but the R(v) constructed using TGRA... i The response can be triggered only after a certain amount of risk energy has accumulated, which reduces the false alarm rate and improves the trend recognition capability. The final output is R(v). i ,t) will be used for the next step of alarm level judgment and response strategy control.

[0135] S5. For the corresponding local edge node, determine the alarm level of the current node based on the difference between the current integral risk value and the risk integral threshold, and execute the preset response action according to the corresponding alarm level.

[0136] Specifically, the goal of this step is to convert the risk integral value R(v) output from step 4 into a specific value. i The alarm level (t) is transformed into a clear and actionable alarm level, triggering corresponding physical linkage response actions. This step is the end point of the entire patent process, and its core task is to form a clear behavioral closed loop from "risk assessment" to "emergency control," ensuring that edge nodes have local decision-making and response capabilities, and completing the practical implementation of the multimodal monitoring system.

[0137] Input variable R(v)i t) represents the node v i The risk integration value at time t, derived from the risk integration modeling of the previous step, is a key judgment index that has fused concentration, temperature, gas, and other multi-modal information and has a trend attribute. The system sets fixed classification response thresholds θ1, θ2, θ3, corresponding to the prompt, warning, and critical levels, respectively.

[0138] The response judgment logic is as follows:

[0139]

[0140] Level(v i ,t) : the alarm level of the current node;

[0141] θ1, θ2, θ3: risk integration thresholds configured by system experience or training data, for example, θ1 = 10, θ2 = 30, θ3 = 60;

[0142] All judgments are completed locally at the node and do not depend on the cloud system.

[0143] Each alarm level corresponds to a set of preset response actions, defined by the response mapping table Definition:

[0144] Level = 1: local prompt, such as LED flashing or device screen warning;

[0145] Level = 2: start sound and light alarm and send message push to the management terminal;

[0146] Level = 3: execute high-level linkage, including access control, smoke exhaust system start, broadcast voice evacuation instruction, etc.;

[0147] Level = 0: do not trigger any action, only maintain listening state.

[0148] All linkage operations are completed through the control interfaces integrated by the edge node, such as GPIO output end, relay module, Modbus / RS485 protocol communication, etc., to ensure the response consistency and control scalability between different types of devices. To enhance system stability, the system supports the "redundancy cooperative response" mechanism under high-level alarm conditions: if R(v i ,t) is lower than θ3, but more than half of its neighbor nodes have reached Level = 3, then the node will also be considered as high risk and some key actions will be activated (such as starting video recording or broadcasting guidance). This logic does not introduce new variables, but is based only on the response level of the neighbors, maintaining the independence of edge decision-making. The output of this step includes: Level(v i ,t) : alarm level result; Corresponding linkage action execution signal; optional response log (including timestamp, trigger level, control object ID, etc.) for system filing and analysis.

[0149] As shown in FIG. 1, the system according to the present application comprises a data acquisition module 101, a spatial adjacency graph construction module 102, a transient risk value determination module 103, a trend risk value analysis module 104, and an alarm module 105. Figure 2

[0150] The data acquisition module 101 is configured to acquire original environmental state data of each monitoring node, and construct multi-modal standardized features.

[0151] The spatial adjacency graph construction module 102 is configured to determine potential paths of smoke or heat diffusion between edge nodes according to the multi-modal standardized features and a constructed spatial adjacency graph structure based on physical constraint driving, by generating an adjacency matrix.

[0152] The transient risk value determination module 103 is configured to generate a spatial smoke concentration prediction value of a region without deployed sensors in the space according to the multi-modal standardized features, the spatial adjacency graph structure, and a trained smoke concentration prediction model, and calculate a transient risk value without memory to time and only based on current features again in combination with the multi-modal standardized features.

[0153] The trend risk value analysis module 104 is configured to calculate a current integral risk value based on the transient risk value through a trend-oriented risk integral structure, as an alarm judgment variable.

[0154] The alarm module 105 is configured to determine an alarm level of a current node according to a comparison difference between the current integral risk value and a risk integral threshold value for a corresponding local edge node, and execute a preset response action according to the corresponding alarm level.

[0155] In addition, technical details not described in detail in the present embodiment can be referred to the parameter operation method provided by any embodiment of the present application, which will not be described here.

[0156] ​It should be noted that, in this document, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0157] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0158] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, and of course can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium (such as a read-only memory / random access memory, a magnetic disk, or an optical disk), and includes a plurality of instructions for making a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device) execute the methods described in the various embodiments of the present application.

[0159] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent flow transformation made by using the content of the specification and drawings, or directly or indirectly applied to other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multimodal fire safety monitoring method integrating edge computing, characterized in that, The method includes: Obtain the raw environmental status data of each monitoring node and construct multimodal standardized features; Based on the multimodal normalization features and the constructed physical constraint-driven spatial adjacency graph structure, the potential paths for smoke or heat diffusion between edge nodes are determined by generating an adjacency matrix; wherein, the graph structure includes a set of edge nodes and a set of adjacency matrices, and the set of adjacency matrices includes several wind-enhanced adjacency weights. Based on the multimodal normalized features, spatial adjacency graph structure, and trained smoke concentration prediction model, the predicted spatial smoke concentration in areas where no sensors are deployed is generated. Then, combined with the multimodal normalized features, the instantaneous risk value that has no memory of time and is based solely on the current features is calculated. Based on the instantaneous risk value, the current integral risk value is calculated using a trend-oriented risk integral structure and used as an alarm judgment variable; For the corresponding local edge node, the alarm level of the current node is determined by comparing the current integral risk value with the risk integral threshold, and a preset response action is executed according to the corresponding alarm level.

2. The multimodal fire safety monitoring method integrating edge computing according to claim 1, characterized in that, The raw environmental data includes smoke concentration, temperature, gas concentration, and air velocity. The multimodal normalized features are constructed as follows: Obtain raw environmental state data; All modal samples are aligned to a fixed time window. For missing data within the window, local reconstruction is performed using the reconstructed values. The reconstructed values ​​are obtained by averaging the available sample point values ​​before and after the corresponding node within the window. Each type of aligned data is normalized and transformed into a dimensionless feature representation. At the same time, the four types of original environmental state data are combined in a weighted manner to form the multimodal standardized features of the corresponding node at the current time.

3. The multimodal fire safety monitoring method integrating edge computing according to claim 1, characterized in that, The construction steps of the physical constraint-driven spatial adjacency graph structure include: Obtain the spatial coordinates of all monitoring nodes, and calculate the Euclidean distance between the nodes based on the spatial coordinates; Determine the wind-enhanced adjacency weight based on the Euclidean distance between nodes and the wind direction angle between the airflow velocity and the nodes: Where, p (i) p (j) θ represents the coordinates of nodes i and j, respectively; σ is the spatial Gaussian decay factor; d0 is the maximum connection distance threshold, beyond which nodes are considered non-connectable; θ ij Let cos(i) be the angle between the current wind direction at node i and the unit direction vector pointing from i to j. + (θ ij )=max(0,cos(θ ij )) indicates the positive excitation effect of wind direction on the diffusion path; For indicator functions, if the connection... If the obstacle is crossed, the value is 1; otherwise, it is 0. λ is the obstacle penalty coefficient, which represents the weakening effect of spatial obstruction on the diffusion path. The adjacency matrix is ​​obtained based on the wind-direction-enhanced adjacency weight.

4. The multimodal fire safety monitoring method integrating edge computing according to claim 3, characterized in that, The spatial adjacency graph structure driven by physical constraints also includes a wind direction offset penalty function: when the difference between the angle between the current wind direction of the current node i and the unit direction vector pointing from the current node i to the node j and the angle between the angle between the current node i and the average wind direction of the current node i in the previous n time steps is the smallest, the wind direction offset penalty reaches its maximum, that is, the probability of matching association is the highest, which is used to cope with sudden changes in environmental wind direction or the impact of sensor anomalies.

5. The multimodal fire safety monitoring method integrating edge computing according to claim 1, characterized in that, The training process of the smoke concentration prediction model includes: Obtain multimodal normalized features, adjacency matrix, and historical spatial smoke concentration values; each element A of the adjacency matrix ij This represents the physical diffusion potential from node j to node i; the historical spatial smoke concentration value includes the corresponding historical air velocity and the corresponding time. A lightweight edge-side model is used to map the multimodal normalized features of node i or j to an initial local concentration estimate; the multimodal normalized features of adjacent node j are aggregated; a physical consistency residual term is also designed to penalize abnormal trends in the prediction that do not conform to the physical laws of diffusion; the prediction formula is: Wherein, C(v) i f(t) represents the predicted spatial smog concentration. θ (·) represents a lightweight edge-side model used to normalize the multimodal features of node i or j. Mapped to the initial local concentration estimate; ω0 is the local perception weight; A ij Adjacency weight; β represents the set of adjacent nodes in the graph structure of node i; β is the physical consistency adjustment coefficient. This is a physical consistency residual term used to penalize abnormal trends in the prediction that do not conform to the physical laws of diffusion.

6. The multimodal fire safety monitoring method integrating edge computing according to claim 5, characterized in that, The edge-side lightweight model is a lightweight two-layer neural network with normalized activation. Each layer uses the ReLU activation function and performs local fine-tuning on the historical concentration samples from the past two minutes using mean square loss. The edge-side lightweight model only uses the node's local multimodal normalized features for prediction, which meets the requirements for low-power edge deployment.

7. The multimodal fire safety monitoring method integrating edge computing according to claim 1, characterized in that, The process involves generating predicted spatial smoke concentration values ​​for sensor-free areas based on the multimodal normalized features, spatial adjacency graph structure, and a trained smoke concentration prediction model. Then, combining these with the multimodal normalized features, an instantaneous risk value is calculated that is memoryless and based solely on current features. This includes: The predicted spatial smoke concentration is fused with the actual risk contribution of smoke concentration, temperature, and gas concentration in the corresponding multimodal normalized features to form a risk density function, and the current integrated risk value is calculated. The current integrated risk value is then subjected to regularization suppression processing to improve the stability of the risk representation in space. The regularization suppression processing is as follows: Wherein, φ(v i ,t) represents the instantaneous risk value of node i, and λ is the local regularity intensity coefficient; v is the graph adjacency set of node i; j For the j-th node, φ(v) j ,t) represents the instantaneous risk value of node j; The regularization suppression process adjusts the instantaneous risk value φ(v) of node i. i The value of ,t) moves closer to the mean of its neighborhood, suppressing the risk of false alarms caused by isolated high values, while preserving the overall risk space variability.

8. The multimodal fire safety monitoring method integrating edge computing according to claim 1, characterized in that, The step of calculating the current integral risk value based on the instantaneous risk value using a trend-oriented risk integral structure, and using it as an alarm judgment variable, includes: Based on the instantaneous risk value of the corresponding node and the historical instantaneous risk value, growth is amplified through a trend-enhancing gating function; The previous integral value and instantaneous risk value of the corresponding node are obtained. Combined with the trend-enhancing instantaneous risk value and the spatial decoupling regularization term, the current integral risk value is calculated through state update analysis. The spatial decoupling regularization term is used to penalize persistent anomalies on isolated nodes. The current integral risk value contains a composite variable of historical information, trend state, and spatial structure alignment.

9. The multimodal fire safety monitoring method integrating edge computing according to claim 1, characterized in that, The preset response actions include providing a local prompt, activating an audible and visual alarm, executing a high-level linkage, and not triggering any action while maintaining a listening state.

10. A multimodal fire safety monitoring system integrating edge computing, characterized in that, The system includes a data acquisition module, a spatial adjacency graph construction module, an instantaneous risk value determination module, a trend risk value analysis module, and an alarm module; The data acquisition module is used to acquire the original environmental state data of each monitoring node and construct multimodal standardized features; The spatial adjacency graph construction module is used to determine the potential paths for smoke or heat diffusion between edge nodes by generating an adjacency matrix based on the multimodal normalization features and the constructed physical constraint-driven spatial adjacency graph structure. The graph structure includes a set of edge nodes and a set of adjacency matrices, and the set of adjacency matrices includes several wind-enhanced adjacency weights. The instantaneous risk value determination module is used to generate a spatial smoke concentration prediction value for an area in space where no sensors are deployed, based on the multimodal normalization features, the spatial adjacency graph structure, and the trained smoke concentration prediction model. Then, it combines the multimodal normalization features to calculate an instantaneous risk value that has no memory of time and is based solely on the current features. The trend risk value analysis module is used to calculate the current integral risk value based on the instantaneous risk value through a trend-oriented risk integral structure, and use it as an alarm judgment variable. The alarm module is used to determine the alarm level of the corresponding local edge node based on the difference between the current integral risk value and the risk integral threshold, and to execute a preset response action according to the corresponding alarm level.

Images