VOCs leakage gas multi-modal data monitoring and data management method and system based on artificial intelligence

Abstract

The invention discloses a VOCs leaked gas multi-modal data monitoring and data management method and system based on artificial intelligence. The method comprises the steps that infrared monitoring images and gas sensor data of a target scene are preprocessed and then input into a first branch and a second branch for multi-scale feature extraction; fusing the feature data of each scale in the two branch outputs based on a first strategy to obtain a fusion enhanced feature set; according to the fusion enhanced feature set, obtaining a VOCs gas leakage monitoring classification identification result of the target scene; the first strategy comprises the steps of performing interactive enhancement on feature data from the first branch and the second branch through an interactive fusion Mangbar model to obtain first data and second data respectively, and performing linear projection after splicing the first data and the second data to obtain fusion enhancement features.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of computer vision, environmental monitoring and artificial intelligence, and in particular to an artificial intelligence-based method and system for multimodal data monitoring and management of VOCs leaked gas. Background Technology

[0002] VOCs (volatile organic compounds) are widespread pollutants in industrial production, possessing toxicity, flammability, and strong diffusion properties. Their leakage not only wastes resources and damages the ecological environment but can also trigger major safety accidents such as explosions and poisoning, seriously threatening industrial production safety and public health. Therefore, achieving rapid, accurate, and real-time monitoring of VOCs leaks is a key technological requirement for ensuring safe industrial production and promoting environmental governance, and it has irreplaceable application value in scenarios such as chemical industrial parks, oil and gas pipelines, and industrial workshops.

[0003] Current VOCs leak detection technologies are mainly divided into two categories: single-modal detection and multi-modal fusion detection, but both have significant technical bottlenecks. In single-modal detection, infrared image-based methods rely on the absorption characteristics of VOCs to specific wavelengths of infrared light for imaging. While this enables visual monitoring, infrared images generally suffer from low resolution, blurred gas boundaries, and susceptibility to interference from environmental heat sources, making it difficult to distinguish between real leaks and background thermal radiation artifacts, and resulting in weak identification capabilities for low-concentration VOCs leaks. On the other hand, VOCs gas sensor-based detection methods offer fast response times and high sensitivity, but are susceptible to cross-interference from mixed gases. Furthermore, sensor data is often time-series, and traditional models based on convolutional neural networks (CNNs) struggle to effectively model long-range dependencies in the data, leading to high false alarm and false negative rates. Both single-modal methods inherently suffer from a limited information dimension, making them unsuitable for the complex environments of industrial settings characterized by dynamic occlusion, changing lighting, and multiple coexisting interference sources. In comparison, existing multimodal fusion detection technologies attempt to combine spatial visual information from infrared images with temporal concentration information from sensors for VOCs leakage prediction. Compared to single-modal methods, the accuracy has been improved, but key technical shortcomings still exist: First, the fusion strategies are inefficient, often employing simple splicing or weighted fusion, failing to fully explore the complementary relationship between the two types of data and unable to achieve deep interaction at the feature level; second, most of these methods use computationally complex Transformer architectures. These fusion methods heavily rely on self-attention mechanisms, and the computational load increases quadratically with the data dimension, making it difficult to efficiently process high-dimensional multimodal data and unable to meet the needs of real-time detection.

[0004] In recent years, State Space Models (SSMs), as an efficient sequence modeling method, have gradually demonstrated strong potential in natural language processing and computer vision. SSMs model long-range dependencies with linear complexity, effectively handling global information in high-dimensional data. The Mamba model, as a novel architecture based on SSM, introduces a selective scanning mechanism, dynamically adjusting model parameters according to input, significantly improving model performance in long sequence tasks and exhibiting high efficiency and robustness in processing high-dimensional data. However, existing research mainly focuses on single-modal models and has not yet applied the Mamba model to multimodal fusion VOCs leak gas detection tasks.

[0005] Therefore, there is an urgent need for a VOCs leakage monitoring and data management technology that can overcome the limitations of single-modality, achieve efficient multimodal feature fusion, and adapt to complex industrial environments, in order to solve the core technical problems of high computational complexity, insufficient long-range dependency modeling, and inefficient multimodal information fusion in existing methods. Summary of the Invention

[0006] This invention provides an artificial intelligence-based method and system for monitoring and managing multimodal data of VOCs leaked gas, which solves the core technical problems of high computational complexity, insufficient long-range dependency modeling, and inefficient multimodal information fusion in existing VOCs leak monitoring and management methods.

[0007] To achieve the above objectives, this invention provides an artificial intelligence-based method for multimodal data monitoring and management of VOCs leaking gas, comprising:

[0008] Acquire infrared monitoring images and gas sensor data of the target scene; after preprocessing the infrared monitoring images and gas sensor data, input them into the first branch and the second branch respectively for multi-scale feature extraction.

[0009] The first branch extracts features based on visual selective scanning of state space blocks, and the second branch extracts features based on selective scanning of state space blocks; both the first and second branches output feature data at preset multi-scales.

[0010] The feature data at each scale in the outputs of the first and second branches are fused based on the first strategy to obtain a fused enhanced feature set; the VOCs gas leakage monitoring classification and identification results of the target scene are obtained by decoding based on the fused enhanced feature set.

[0011] The first strategy includes: using the Mamba model to interactively enhance the feature data from the first branch and the second branch, obtaining the first data and the second data respectively, and then performing linear projection after concatenating the first data and the second data to obtain the fused enhanced features.

[0012] Preferably, acquiring infrared monitoring images and gas sensor data of the target scene includes:

[0013] Acquiring infrared monitoring images ;

[0014] Acquiring gas sensor data .

[0015] in, This represents a height of H, a width of W, and a number of channels. Matrix; This represents a matrix with height L and width d.

[0016] Preferably, the preprocessing includes:

[0017] The infrared monitoring images are standardized, including:

[0018] .

[0019] in, This represents the standardized infrared monitoring image; Represents the original infrared monitoring image; This represents the mean of the original infrared monitoring image; This represents the standard deviation of the original infrared monitoring image.

[0020] Standardization processing of gas sensor data includes:

[0021] .

[0022] in, This represents the standardized gas sensor data; This represents the average value of the sensor data across all channels; This represents the standard deviation of each channel of the sensor data.

[0023] The standardized infrared monitoring images were augmented using random rotation and color jitter.

[0024] The standardized gas sensor data is augmented using time-series jitter.

[0025] The normalized and data-enhanced infrared monitoring image is segmented into local blocks using a Vision Transformer-style image segmentation module and mapped to a high-dimensional feature space, including:

[0026] .

[0027] in, This represents the image data output by the image segmentation module. , Indicates height as , width is The number of channels is C k The matrix; k represents the scale level; This indicates the trimming module.

[0028] The enhanced gas sensor data is segmented into local temporal blocks. Spatial long-range contextual features are extracted from the gas sensor data by selectively scanning the state space blocks at multiple scales, and then mapped to a high-dimensional feature space, including:

[0029] .

[0030] in, This represents the high-dimensional features after mapping. , Indicates height as , width is The matrix; k represents the scale level; This represents the temporal block and linear projection module.

[0031] Preferably, the feature data from the first branch and the second branch are interactively enhanced through the interactive fusion Mamba model to obtain the first data and the second data, respectively, including:

[0032] By using a linear projection layer and layer normalization, the infrared monitoring image features from the first branch and the gas sensor data features from the second branch are mapped to a unified feature dimension, and scale differences are eliminated, resulting in the projection-normalized infrared intermediate feature. and intermediate quantity characteristics of gas sensors :

[0033] ;

[0034] .

[0035] in, The projection weight matrix is ​​the feature of the infrared monitoring image. For the corresponding bias term, (•) indicates layer normalization operation; This is the projection weight matrix of the gas sensor data features. For the corresponding bias term, As a functional identifier, it represents projection.

[0036] Will and The spatial context information and cross-modal enhancement information are extracted by inputting them into the selective scanning state space model block and the gating mechanism module, respectively, to obtain the first result and the second result, including:

[0037] ;

[0038] ;

[0039] .

[0040] in, This represents the projection matrix of the discretized system. Represents an exponential function; The adjustment parameter indicates the time step size; Represents the system state transition matrix; Represents the system projection matrix; This represents the projection matrix of the discretized system. Represents the hidden state at time step t; This represents the hidden state at time step t-1; This represents the input at time step t; This represents the system matrix at time step t; This represents the selective scan output of the state at time step t.

[0041] ;

[0042] ;

[0043] ;

[0044] .

[0045] in, , These are the gating weight matrices for infrared features and sensor features, respectively; (•) is the Sigmoid activation function; This indicates that the temporal features of the sensor are reshaped into a 2D feature map format to fit the convolution operation; Conv(•) indicates a 3×3 convolution operation; (•) indicates global average pooling, which compresses the infrared 2D feature map into a 1D sequence; and For the gated weight matrix, and This is a gated bias term; As a functional identifier, it indicates gate control; This represents element-wise multiplication; A proportion coefficient for preserving the original features is used to balance the contributions of the filtered features and the original features.

[0046] Will and Separately and infrared intermediate quantity characteristics and intermediate quantity characteristics of gas sensors Perform fusion enhancement to obtain the first and second data:

[0047] ;

[0048] .

[0049] Preferably, the first and second data are concatenated and then linearly projected to obtain the fused enhanced features. :

[0050] .

[0051] in, This indicates a splicing operation.

[0052] Preferably, the VOCs gas leakage monitoring classification results obtained by decoding based on the fused enhanced feature set for the target scene include:

[0053] Decoding is performed based on the fusion enhancement features at each scale in the fusion enhancement feature set, combined with a channel-aware selective scanning state-space decoder, including:

[0054] Suppose that the set of fusion enhancement features includes fusion enhancement features at M scales, that is, it includes M fusion enhancement features, where the scale of the 1st to the Mth fusion enhancement features decreases.

[0055] The Mth fusion enhancement feature is enhanced and upsampled, and its scale is restored to the scale of the (M-1)th fusion enhancement feature. After being concatenated with the (M-1)th fusion enhancement feature, it is input into the channel sensing selective scanning state space for decoding. The channel sensing selective scanning state space decoder outputs the first decoded data.

[0056] The first decoded data is upsampled to restore the scale to the scale of the (M-2)th fused enhancement feature, and then concatenated with the (M-2)th fused enhancement feature before being input into the channel-aware selective scanning state space decoder for decoding. The channel-aware selective scanning state space decoder outputs the second decoded data.

[0057] Based on the second decoded data, upsampling, feature splicing, and decoding continue until the scale of the decoded data is restored to the scale of the first fused enhanced feature. The channel-aware selective scanning state space decoder then outputs the decoded data.

[0058] After decoding is complete, the decoded fused and enhanced feature data is input into the Softmax classifier, which outputs the VOCs gas leakage identification and classification results for the target scene.

[0059] The present invention also provides an artificial intelligence-based VOCs leak gas multimodal data monitoring and data management system for the method of the present invention, the system comprising a first module, a second module, a third module and a fourth module.

[0060] The first module is used to acquire infrared monitoring images and VOCs gas sensor data of the target scene.

[0061] The second module is used to preprocess the infrared monitoring image and input it into the first branch for multi-scale feature extraction; the feature extraction of the first branch is based on visual selective scanning state space blocks, and the feature extraction of the second branch is based on selective scanning state space blocks; both the first branch and the second branch output feature data under preset multi-scale conditions.

[0062] The third module is used to fuse the feature data at each scale in the outputs of the first and second branches based on the first strategy to obtain a fused and enhanced feature set.

[0063] The first strategy includes: interactively enhancing the feature data from the first and second branches through the interactive fusion Mamba model to obtain the first data and the second data respectively; and then performing linear projection after concatenating the first data and the second data to obtain the fused enhanced features.

[0064] The fourth module is used to decode the fused enhanced feature set to obtain the VOCs gas leakage monitoring classification results for the target scene.

[0065] The present invention has the following beneficial effects:

[0066] This invention presents an AI-based method for multimodal data monitoring and management of VOCs leaks. By selectively scanning state space blocks from infrared monitoring images to extract multi-scale spatial features and selectively scanning state space blocks from VOCs gas sensor data to extract multi-scale temporal features, it comprehensively captures the spatial distribution details and dynamic concentration changes of VOCs leaks. The extracted dual-modal features are interactively and adaptively fused using an interactive fusion module, promoting complementary enhancement and redundancy suppression of cross-modal features, achieving deep integration of VOCs multimodal information. This method provides an efficient and accurate monitoring solution for problems such as low-concentration leaks, environmental interference, and cross-influence of mixed gases in complex industrial scenarios. This method overcomes the shortcomings of existing methods, such as inefficient multimodal leak information fusion, insufficient long-range dependency modeling, and high computational complexity, enabling accurate monitoring and systematic data management of VOCs leaks in industrial scenarios.

[0067] The AI-based VOCs leak gas multimodal data monitoring and data management system of the present invention, used in the method of the present invention, has the same beneficial effects as the method of the present invention.

[0068] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0069] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0070] Figure 1 This is a schematic diagram of the method flow of a preferred embodiment of the present invention.

[0071] Figure 2 This is a schematic diagram of the VOCs multimodal detection and identification process according to a preferred embodiment of the present invention.

[0072] Figure 3 This is a schematic diagram of the first strategy of a preferred embodiment of the present invention. Detailed Implementation

[0073] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0074] See Figures 1 to 2 In a preferred embodiment of the present invention, an artificial intelligence-based method for multimodal data monitoring and management of VOCs leaking gas is provided, comprising:

[0075] P1. Acquire infrared monitoring images and gas sensor data of the target scene. P1 specifically includes:

[0076] Acquiring infrared monitoring images .

[0077] Acquiring gas sensor data .

[0078] in, This represents a height of H, a width of W, and a number of channels. Matrix; This represents a matrix with height L and width d.

[0079] P2. After preprocessing the infrared monitoring images and gas sensor data, input them into the first branch and the second branch respectively for multi-scale feature extraction.

[0080] In a preferred embodiment of the present invention, the preprocessing includes:

[0081] Standardize the infrared monitoring images and gas sensor data:

[0082] ;

[0083] in, This represents the standardized infrared monitoring image; Represents the original infrared monitoring image; This represents the mean of the original infrared monitoring image; This represents the standard deviation of the original infrared monitoring image.

[0084] Standardization processing of gas sensor data includes:

[0085] .

[0086] in, This represents the standardized gas sensor data; This represents the average value of the sensor data across all channels; This represents the standard deviation of each channel of the sensor data.

[0087] The standardized infrared monitoring images were augmented by random rotation (±20°) and color jitter (brightness ±15%, contrast ±10%).

[0088] The standardized gas sensor data is augmented using timing jitter.

[0089] The normalized and data-enhanced infrared monitoring image is segmented into local blocks using a Vision Transformer-style image segmentation module and mapped to a high-dimensional feature space, including:

[0090] .

[0091] in, This represents the image data output by the image segmentation module. , Indicates height as , width is The number of channels is C k The matrix; k represents the scale level; This indicates the trimming module.

[0092] In a preferred embodiment of the present invention, when performing multi-scale feature extraction, it is preferable to perform feature extraction at four levels of scale, that is, the preset multi-scale includes four levels of scale. , .

[0093] In a preferred embodiment of the present invention, the feature extraction of the first branch is based on the Visual Selective Scan State Space (VS4), and the feature extraction of the second branch is based on the Selective Scan State Space (S4); both the first branch and the second branch output feature data at preset multi-scale.

[0094] In a preferred embodiment of the present invention, in the visual selective scanning state space block of the first branch, the input features are first processed by layer normalization, linear projection, and depthwise convolution to extract the infrared global information of the monitoring scene and enhance the spatial structure. Subsequently, a Selective Scan 2D (SS2D) module is introduced to fully mine and model the long-range context space and channel information in the infrared monitoring image features. Specifically, the SS2D module will shape... The input features are flattened into four parts along four directions (top left to bottom right, bottom right to top left, top right to bottom left, bottom left to top right). The sequences were processed, and their long-range dependencies in multiple directions were extracted using four different selective scanning modules. Finally, these sequences were reshaped back to their original shape and summed to integrate the multi-directional information. The feature representation was then enhanced through residual connections, thus completing the effective modeling of the input infrared monitoring image data.

[0095] In a preferred embodiment of the present invention, in the selective scan state space block of the second branch, the input features are first processed by layer normalization, linear projection, and depthwise convolution to extract global information of the gas monitoring data in the monitoring scene and enhance the spatial structure. Subsequently, a selective scan (SS) module is introduced to fully mine and model the long-range context space and channel information in the gas monitoring data features. Specifically, the SS module will... The input features are flattened into two along two directions (from beginning to end, from end to beginning). The sequences were analyzed, and their multi-directional long-range dependencies were extracted using two different selective scanning modules. Finally, the two sequences were reconstructed back to their original shapes and summed. The feature representation was then enhanced using residual connections, thus completing the effective modeling of the input gas monitoring data.

[0096] P3. The feature data at each scale in the output of the first branch and the second branch are fused based on the first strategy to obtain a fused enhanced feature set; the VOCs gas leakage monitoring classification and identification results of the target scene are obtained by decoding based on the fused enhanced feature set.

[0097] In a preferred embodiment of the present invention, the first strategy includes: interactively enhancing the feature data from the first branch and the second branch through an interactive fusion Mamba model to obtain first data and second data respectively; and then performing linear projection after splicing the first data and the second data to obtain fused enhanced features.

[0098] In a preferred embodiment of the present invention, the feature data from the first branch and the second branch are interactively enhanced by the interactive fusion Mamba model to obtain the first data and the second data respectively, including:

[0099] See Figure 3 Taking feature data from the first and second branches as input, the infrared monitoring image features from the first branch and the gas sensor data features from the second branch are mapped to a unified feature dimension based on a linear projection layer and layer normalization, and scale differences are eliminated to obtain the projection-normalized infrared intermediate features. and intermediate quantity characteristics of gas sensors :

[0100] ;

[0101] .

[0102] in, The projection weight matrix is ​​the feature of the infrared monitoring image. For the corresponding bias term, ( This indicates the layer normalization operation; This is the projection weight matrix of the gas sensor data features. For the corresponding bias term, As a functional identifier, it represents projection.

[0103] Will and The first results obtained by extracting spatial context information from the selectively scanned state space model block include:

[0104] ;

[0105] ;

[0106] .

[0107] in, , This represents the projection matrix of the discretized system. Represents an exponential function; The adjustment parameter indicates the time step size; Represents the system state transition matrix; Represents the system projection matrix; This represents the projection matrix of the discretized system. Represents the hidden state at time step t; This represents the hidden state at time step t-1; This represents the input at time step t; This represents the system matrix at time step t; This represents the selective scan output of the state at time step t.

[0108] Will and The input to the gating mechanism module extracts cross-modal enhancement information, resulting in the following second results:

[0109] ;

[0110] ;

[0111] ;

[0112] .

[0113] in, , These are the gating weight matrices for infrared features and sensor features, respectively; (•) is the Sigmoid activation function; This indicates that the temporal features of the sensor are reshaped into a 2D feature map format to fit the convolution operation; Conv(•) indicates a 3×3 convolution operation; (•) indicates global average pooling, which compresses the infrared 2D feature map into a 1D sequence; and For the gated weight matrix, and This is a gated bias term; As a functional identifier, it indicates gate control; This represents element-wise multiplication; A proportion coefficient for preserving the original features is used to balance the contributions of the filtered features and the original features.

[0114] Will and Separately and infrared intermediate quantity characteristics and intermediate quantity characteristics of gas sensors Perform fusion enhancement to obtain the first and second data:

[0115] ;

[0116] .

[0117] In a preferred embodiment of the present invention, the first data and the second data are concatenated and then linearly projected to obtain the fusion enhancement feature. :

[0118] .

[0119] in, This indicates a splicing operation.

[0120] In a preferred embodiment of the present invention, the VOCs gas leakage monitoring and classification identification result of the target scene obtained by decoding based on the fused enhanced feature set includes:

[0121] Decoding is performed based on the fusion enhancement features at each scale in the fusion enhancement feature set, combined with the Channel-Aware Selective Scan State Space decoder (CAS4), including:

[0122] Suppose that the set of fusion enhancement features includes fusion enhancement features at M scales, that is, it includes M fusion enhancement features, where the scale of the 1st to the Mth fusion enhancement features decreases.

[0123] The Mth fusion enhancement feature is enhanced and upsampled, and its scale is restored to the scale of the (M-1)th fusion enhancement feature. The result is then concatenated with the (M-1)th fusion enhancement feature and input into the channel-aware selective scanning state space decoder for decoding. The channel-aware selective scanning state space decoder outputs the first decoded data.

[0124] The first decoded data is upsampled to restore the scale to the scale of the (M-2)th fused enhancement feature, and then concatenated with the (M-2)th fused enhancement feature before being input into the channel-aware selective scanning state space decoder for decoding. The channel-aware selective scanning state space decoder outputs the second decoded data.

[0125] Based on the second decoded data, upsampling, feature splicing, and decoding continue until the scale of the decoded data is restored to the scale of the first fused enhanced feature. The channel-aware selective scanning state space decoder then outputs the decoded data.

[0126] After decoding is complete, the decoded data is upsampled and input into the Softmax classifier. The Softmax classifier outputs the VOCs gas leakage identification and classification results for the target scene.

[0127] In a preferred embodiment of the present invention, in the channel-aware selective scanning state space decoder, the input features are first extracted through selective scanning state space blocks; then, the features are normalized through layer normalization; and finally, local feature information is extracted through convolution. The processed features are then input into max pooling and average pooling, respectively, to generate multi-scale features through different pooling methods; then, a residual skip connection mechanism is used to fuse features of different scales, improving the multi-scale feature fusion effect.

[0128] In the channel-aware selective scanning state-space decoder, a channel attention mechanism is used to upsample and stitch multi-scale fusion enhancement features, thereby enhancing the expressive power of cross-modal spatial-temporal information. The decoder's built-in channel-aware selective scanning state-space processing mechanism strengthens the decoding expression effect of leakage features of fused features through selective scanning, and optimizes monitoring accuracy and concentration estimation accuracy by combining a weighted combination of cross-entropy loss and MSE loss. Data decoding based on the channel-aware selective scanning state-space decoder can significantly improve the recovery capability of low-concentration leakage features and dynamic concentration change features in complex industrial scenarios, solving the problem of monitoring accuracy fluctuations in scenarios with environmental heat source interference, sensor noise, and cross-influence of mixed gases, which are common in traditional methods.

[0129] This invention presents an AI-based method for multimodal data monitoring and management of VOCs leaking gases. By extracting multi-scale features from infrared monitoring images and VOCs gas sensor data using visual selective scanning state space blocks and selective scanning state space blocks, it comprehensively captures multi-scale long-range spatial distribution details and temporal concentration changes. The extracted dual-modal features are fused using a first strategy with multi-granularity interaction and gating, promoting cross-modal complementary enhancement and redundancy suppression, achieving adaptive deep fusion and enhanced feature representation of VOCs multimodal information in the target scene. By decoupling the spatial features of infrared images and the temporal features of gas sensor data and modeling them independently, it overcomes the limitations of traditional methods in cross-modal long-range dependency modeling. This invention provides an efficient and accurate monitoring solution for problems such as low-concentration leaks, environmental interference, and cross-influence of mixed gases in complex industrial scenarios. This invention overcomes the existing shortcomings of inefficient multimodal fusion, insufficient long-range dependency modeling, and high computational complexity, enabling accurate monitoring and systematic data management of VOCs leaks in industrial scenarios.

[0130] In a preferred embodiment of the present invention, an artificial intelligence-based VOCs leak gas multimodal data monitoring and data management system is also provided for use in the method of the present invention. The system includes a first module, a second module, a third module, and a fourth module.

[0131] The first module is used to acquire infrared monitoring images and VOCs gas sensor data of the target scene.

[0132] The second module is used to preprocess the infrared monitoring image and input it into the first branch for multi-scale feature extraction; the feature extraction of the first branch is based on visual selective scanning state space blocks, and the feature extraction of the second branch is based on selective scanning state space blocks; both the first branch and the second branch output feature data under preset multi-scale conditions.

[0133] The third module is used to fuse the feature data at each scale in the outputs of the first and second branches based on the first strategy to obtain a fused and enhanced feature set.

[0134] The first strategy includes: interactively enhancing the feature data from the first and second branches through the interactive fusion Mamba model to obtain the first data and the second data respectively; and then performing linear projection after concatenating the first data and the second data to obtain the fused enhanced features.

[0135] The fourth module is used to decode the fused enhanced feature set to obtain the VOCs gas leakage monitoring classification results for the target scene.

[0136] The AI-based VOCs leak gas multimodal data monitoring and data management system of the present invention, used in the method of the present invention, has the same beneficial effects as the method of the present invention.

[0137] Verification section:

[0138] The effectiveness of the method of this invention was verified on the MultimodalGasData and a self-made hybrid dataset. This dataset was collected using seven MOX sensors and a thermal imaging camera, covering five scenarios: VOCs, perfume mist, incense smoke, mixed gases, and fresh air. The sensors output numerical signals, and the thermal images have a resolution of 32×32 pixels. The dataset contains a total of 8000 samples, 1600 per category, collected at fixed 2-second intervals, with different gas release durations to ensure data diversity. Detailed category distribution, sensor readings, and thermal image sample examples are shown in Table 1.

[0139] Table 1. Statistical data of the dataset

[0140] Gas type Training samples Test samples Label Total quantity NoGas 1120 480 0 1600 Perfume 1120 480 1 1600 Smoke 1120 480 2 1600 Mixture 1120 480 3 1600 VOC 1120 480 4 1600 total 5600 2400 - 8000

[0141] The method of this invention was implemented using PyTorch in an Ubuntu environment. Training and inference were performed on two RTX4090 GPUs. During training, the AdamW optimizer with an initial learning rate of 6e-5 and a weight decay of 0.01 was used. The model was trained for 300 iterations with a batch size of 8. Furthermore, it was compared with several state-of-the-art methods. Table 2 shows the mainstream models used in the preferred embodiments of this invention and models based on the method of this invention, where VOCMamba is a model based on the method of this invention. Table 3 shows the quantitative experimental data on accuracy of all selected models on MultimodalGasData and a self-made mixed dataset, as well as the evaluation results on accuracy (ACC), precision (PR), recall (RC), and F1 score (F1). Table 4 shows the efficiency and accuracy experimental results and the evaluation results on parameter count, computational complexity, and inference speed.

[0142] Table 2. Mainstream Models and Models Based on the Method of this Invention

[0143] Method (Model) backbone Published in journals (conferences) Publication date CNN-BiGRU ResNet-101 JLPPI 2025 ShuffleNet ShuffleNet-101 Sensors 2024 SCG ShuffleNet-101 Journal of Instrumentation 2024 CNN-LSTM MobileNetV3 Applied System Innovation 2021 LDA-CNN ResNet50 Chemometrics and Intelligent Laboratory Systems 2023 Pvt-TabTransformer MiT-B4 ICCV 2021 Swin-MLP-Transformer Swin-B5 Journal of Electronic Measurement and Instrumentation 2025 VOCMamba VMamba-B - -

[0144] Table 3. Quantitative experimental data on the model's accuracy on MultimodalGasData and a self-made hybrid dataset.

[0145] Method (Model) Accuracy ACC Precision Rate (PR) Recall Rate (RC) F1 value CNN-BiGRU 95.10 95.12 95.10 93.40 ShuffleNet 97.86 97.81 97.86 97.53 SCG 94.64 94.98 94.64 94.83 CNN-LSTM 95.21 95.25 95.21 95.19 LDA-CNN 93.00 93.25 93.00 93.25 Pvt-TabTransformer 97.76 97.79 97.76 97.76 Swin-MLP-Transformer 97.92 97.93 97.92 97.90 VOCMamba 99.12 98.90 99.12 98.87

[0146] Table 4. Quantitative experimental data on the model's efficiency on MultimodalGasData and a self-made hybrid dataset.

[0147] Method (Model) Number of parameters (M) Computational complexity (GFlops) Reasoning speed (s) CNN-BiGRU 78.51 133.79 8.0 ShuffleNet 67.24 138.05 8.2 SCG 58.67 164.49 9.8 CNN-LSTM 56.06 127.73 7.8 LDA-CNN 62.91 144.56 9.6 Pvt-TabTransformer 239.86 323.10 13.9 Swin-MLP-Transformer 288.29 291.88 11.6 VOCMamba 51.77 85.26 2.9

[0148] Analysis of Tables 3 and 4 shows that:

[0149] (1) Accuracy comparison: VOCMamba’s ACC reaches 99.12%, which is 1.2% better than the current best method.

[0150] (2) Efficiency comparison: The number of parameters is reduced by 8%, the index FLOPs are reduced by 40%, and the inference speed is increased by about 2.5 times.

[0151] (3) Robustness verification: In a complex mixed gas interference environment, facing infrared mixed gas interference and gas sensor data jitter, the detection and recognition accuracy of VOC fluctuates by less than 5%, which is significantly better than traditional methods.

[0152] This invention utilizes a selective scanning state-space model and a multimodal fusion strategy to achieve accurate monitoring and systematic data management of VOCs leakage in complex industrial scenarios, providing an innovative solution for smart industrial environmental monitoring and protection. Experiments show that this method achieves industry-leading levels in accuracy, efficiency, and robustness.

[0153] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for monitoring and managing multimodal data of VOCs leaking gas based on artificial intelligence, characterized in that, include: Acquire infrared monitoring images and gas sensor data of the target scene; After preprocessing, the infrared monitoring images and gas sensor data are input into the first branch and the second branch respectively for multi-scale feature extraction. The feature extraction of the first branch is based on visual selective scanning of state space blocks, and the feature extraction of the second branch is based on selective scanning of state space blocks; both the first and second branches output feature data under preset multi-scale conditions. The feature data at each scale in the outputs of the first and second branches are fused based on the first strategy to obtain a fused and enhanced feature set; The VOCs gas leak monitoring and classification results for the target scene are obtained by decoding the fused and enhanced feature set. The first strategy includes: interactively enhancing feature data from the first branch and the second branch through an interactive fusion Mamba model to obtain first data and second data respectively; then concatenating the first data and the second data and performing linear projection to obtain fused enhanced features.

2. The method for multimodal data monitoring and management of VOCs leaking gas based on artificial intelligence according to claim 1, characterized in that, The acquisition of infrared monitoring images and gas sensor data of the target scene includes: Acquiring infrared monitoring images ; Acquiring gas sensor data ; in, This represents a height of H, a width of W, and a number of channels. Matrix; This represents a matrix with height L and width d.

3. The method for multimodal data monitoring and management of VOCs leaking gas based on artificial intelligence according to claim 2, characterized in that, The preprocessing includes: The infrared monitoring images are standardized, including: ； in, This represents the standardized infrared monitoring image; Represents the original infrared monitoring image; This represents the mean value of the original infrared monitoring image; This represents the standard deviation of the original infrared monitoring image; Standardization processing of gas sensor data includes: ； in, This represents the standardized gas sensor data; This represents the average value of the sensor data across all channels; This represents the standard deviation of each channel of the sensor data; The standardized infrared monitoring images were augmented using random rotation and color jitter. The standardized gas sensor data is augmented using time-series jitter. The normalized and data-enhanced infrared monitoring image is segmented into local blocks using a Vision Transformer-style image segmentation module and mapped to a high-dimensional feature space, including: ； in, This represents the image data output by the image segmentation module. , Indicates height as , width is The number of channels is C k The matrix; k represents the scale level; This indicates the trimming module; The enhanced gas sensor data is segmented into local temporal blocks. Spatial long-range contextual features are extracted from the gas sensor data by selectively scanning the state space blocks at multiple scales, and then mapped to a high-dimensional feature space, including: ； in, This represents the high-dimensional features after mapping. , Indicates height as , width is The matrix; k represents the scale level; This represents the temporal block and linear projection module.

4. The method for multimodal data monitoring and management of VOCs leaking gas based on artificial intelligence according to claim 3, characterized in that, The step of interactively enhancing the feature data from the first and second branches through the interactive fusion Mamba model to obtain the first and second data respectively includes: By using a linear projection layer and layer normalization, the infrared monitoring image features from the first branch and the gas sensor data features from the second branch are mapped to a unified feature dimension, and scale differences are eliminated, resulting in the projection-normalized infrared intermediate feature. and intermediate quantity characteristics of gas sensors : ； ； in, The projection weight matrix is ​​the feature of the infrared monitoring image. For the corresponding bias term, (•) indicates layer normalization operation; This is the projection weight matrix of the gas sensor data features. For the corresponding bias term, As a functional identifier, it represents projection; Will and The spatial context information and cross-modal enhancement information are extracted by inputting them into the selective scanning state space model block and the gating mechanism module, respectively, to obtain the first result and the second result, including: ； ； ； in, , This represents the projection matrix of the discretized system. Represents an exponential function; The adjustment parameter indicates the time step size; Represents the system state transition matrix; Represents the system projection matrix; This represents the projection matrix of the discretized system. Represents the hidden state at time step t; This represents the hidden state at time step t-1; This represents the input at time step t; This represents the system matrix at time step t; Selective scan output representing the projected weight moments of the infrared monitoring image features at time step t; ； ； ； ； in, , These are the gating weight matrices for infrared features and sensor features, respectively; (•) is the Sigmoid activation function; This indicates that the temporal features of the sensor are reshaped into a 2D feature map format to fit the convolution operation; Conv(•) indicates a 3×3 convolution operation; (•) indicates global average pooling, which compresses the infrared 2D feature map into a 1D sequence; and For the gated weight matrix, and This is a gated bias term; As a functional identifier, it indicates gate control; This represents element-wise multiplication; A proportion coefficient for preserving the original features is used to balance the contributions of the filtered features and the original features. Will and Separately and infrared intermediate quantity characteristics and intermediate quantity characteristics of gas sensors Perform fusion enhancement to obtain the first and second data: ； 。 5. The method for multimodal data monitoring and data management of VOCs leaking gas based on artificial intelligence according to claim 4, characterized in that, The first and second data are concatenated and then linearly projected to obtain the fused enhanced features. : ； in, This indicates a splicing operation.

6. The method for multimodal data monitoring and management of VOCs leaking gas based on artificial intelligence according to claim 5, characterized in that, The VOCs gas leakage monitoring classification results obtained by decoding based on the fused and enhanced feature set for the target scene include: Decoding is performed based on the fusion enhancement features at each scale in the fusion enhancement feature set, combined with a channel-aware selective scanning state-space decoder, including: Assume that the set of fusion enhancement features includes fusion enhancement features at M scales, that is, it includes M fusion enhancement features, where the scale of the 1st to the Mth fusion enhancement features decreases; The Mth fusion enhancement feature is enhanced and upsampled, and its scale is restored to the scale of the (M-1)th fusion enhancement feature. After being concatenated with the (M-1)th fusion enhancement feature, it is input into the channel-aware selective scanning state space decoder for decoding. The channel-aware selective scanning state space decoder outputs the first decoded data. The first decoded data is upsampled to restore the scale to the scale of the (M-2)th fusion enhancement feature, and then concatenated with the (M-2)th fusion enhancement feature before being input into the channel-aware selective scanning state space decoder for decoding. The channel-aware selective scanning state space decoder outputs the second decoded data. Based on the second decoded data, upsampling, feature splicing and decoding continue until the scale of the decoded data is restored to the scale of the first fused enhanced feature. The channel-aware selective scanning state space decoder outputs the decoded data. After decoding is complete, the decoded fused and enhanced feature data is input into the Softmax classifier, which outputs the VOCs gas leakage identification and classification results for the target scene.

7. An artificial intelligence-based VOCs leak gas multimodal data monitoring and management system, used in the method described in any one of claims 1 to 6, characterized in that, The system includes a first module, a second module, a third module, and a fourth module; The first module is used to acquire infrared monitoring images and VOCs gas sensor data of the target scene; The second module is used to preprocess the infrared monitoring image and input it into the first branch for multi-scale feature extraction; the feature extraction of the first branch is based on visual selective scanning state space blocks, and the feature extraction of the second branch is based on selective scanning state space blocks; both the first branch and the second branch output feature data under preset multi-scale conditions. The third module is used to fuse the feature data at each scale in the output of the first branch and the second branch based on the first strategy to obtain a fused enhanced feature set; The first strategy includes: interactively enhancing feature data from the first branch and the second branch through an interactive fusion Mamba model to obtain first data and second data respectively; concatenating the first data and the second data and then performing linear projection to obtain fused enhanced features; The fourth module is used to decode the fused enhanced feature set to obtain the VOCs gas leakage monitoring and classification results of the target scene.

Images