An emission compliance self-checking and self-correcting system based on multi-modal fusion

Abstract

The application discloses a kind of based on multi-modal fusion's emission compliance self-checking and self-correcting system, comprising: multi-modal data acquisition module, for collecting the multi-source data of enterprise emission facilities and pre-processing;Multi-modal feature coding module, for respectively feature coding to multi-modal data set;Double-channel feature construction module, for constructing current channel feature and contrast channel feature;Contrast space-time cross attention feature extraction module, for by double-channel contrast space-time cross attention network, output comprehensive space-time difference feature representation;Regulation knowledge graph matching module, for generating compliance determination result and regulation clause information;Self-correcting task generation module, for generating self-correcting task;Task execution and closed-loop management module, for recording self-correcting task execution process.The application adopts double-channel contrast space-time cross attention network, realizes emission behavior intelligent identification and automatic compliance self-checking and self-correcting.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and environmental monitoring technology, and in particular to an emission compliance self-inspection and self-correction system based on multimodal fusion. Background Technology

[0002] With the rapid development of the Industrial Internet and environmental monitoring technologies, digital monitoring of enterprise emission processes has become an important means of achieving emission compliance management. Existing emission compliance detection systems are mostly based on single-modal data input, such as using sensor numerical signals for over-limit analysis or using video image recognition technology to detect emission status.

[0003] However, such systems have limitations in the fusion and processing of multi-source heterogeneous data, and cannot effectively associate images, text and time series, resulting in the abnormal emission identification results being greatly affected by data noise and sampling bias. At the same time, although some deep learning-based monitoring models can capture complex features, they lack the ability to model the spatial dependence and temporal evolution of emission facilities, making it difficult to identify hidden anomalies caused by equipment linkage or emission process lag.

[0004] Furthermore, existing emission compliance assessment methods generally rely on rule templates or fixed thresholds, which lack flexibility when dealing with emission data from multiple scenarios and across different time periods, making it difficult to achieve intelligent matching and differentiated judgment based on regulatory clauses. For the self-inspection and rectification phase, traditional systems mostly rely on manual aggregation and assignment, lacking a mechanism for automatically generating rectification tasks and implementing closed-loop management.

[0005] Therefore, how to provide an emission compliance self-inspection and self-correction system based on multimodal fusion is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a multimodal fusion-based emission compliance self-inspection and rectification system. This invention utilizes multimodal feature encoding, a dual-channel comparative spatiotemporal cross-attention network, and regulatory knowledge graph matching technology to construct a complete process from emission data collection, feature fusion, spatiotemporal difference analysis to automatic matching of regulatory clauses and generation of self-rectification tasks. By fusing image, numerical, and text data, it achieves multidimensional dynamic modeling of emission behavior and accurately captures the spatial correlation and temporal evolution characteristics of abnormal emissions using a comparative spatiotemporal cross-attention mechanism. Simultaneously, by introducing a regulatory knowledge graph, it achieves semantic-level matching between emission parameters and regulatory clauses, automatically outputting compliance judgment results and generating targeted rectification tasks. This invention possesses advantages such as high anomaly identification accuracy, high level of intelligent compliance matching, and high efficiency in self-rectification closed-loop, effectively improving the intelligent level of enterprise emission supervision and compliance management.

[0007] According to an embodiment of the present invention, an emission compliance self-inspection and self-correction system based on multimodal fusion includes:

[0008] The multimodal data acquisition module is used to collect multi-source data from enterprise emission facilities, perform preprocessing, and obtain a multimodal dataset.

[0009] The multimodal feature encoding module is used to encode features of the multimodal dataset separately to obtain a multimodal unified feature tensor;

[0010] The dual-channel feature construction module is used to construct the current channel features and the comparison channel features based on the multimodal unified feature tensor, so as to obtain a dual-channel input sample set.

[0011] The spatiotemporal cross-attention feature extraction module is used to receive the current channel features and the comparison channel features, and output a comprehensive spatiotemporal difference feature representation through a dual-channel spatiotemporal cross-attention network;

[0012] The regulatory knowledge graph matching module is used to input the comprehensive spatiotemporal difference feature representation into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information;

[0013] The self-correction task generation module is used to generate self-correction tasks based on compliance judgment results and regulatory clause information, and push them to the mobile terminal for execution.

[0014] The task execution and closed-loop management module is used to record the self-correction task execution process, monitor the rectification completion status and retest emission data, and update compliance status information.

[0015] Optionally, modules can be integrated using the following methods:

[0016] Collect multi-source data on enterprise emission facilities, preprocess the multi-source data to obtain a multimodal dataset;

[0017] The features of the multimodal dataset are encoded separately, and then weighted and fused through a feature fusion layer to obtain a unified multimodal feature tensor.

[0018] Based on the multimodal unified feature tensor, the current channel features and the comparison channel features are constructed to obtain a dual-channel input sample set;

[0019] The dual-channel input sample set is input into a dual-channel comparative spatiotemporal cross-attention network to extract emission behavior features, forming a comprehensive spatiotemporal difference feature representation;

[0020] The comprehensive spatiotemporal difference feature representation is input into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information;

[0021] Based on the compliance assessment results and regulatory information, identify the facilities with abnormal emissions and the causes of the abnormalities, generate self-correction tasks, and push them to mobile terminals for execution.

[0022] Optionally, the process of collecting multi-source data from enterprise emission facilities and preprocessing the multi-source data to obtain a multimodal dataset specifically includes:

[0023] Deploy multiple types of data acquisition units, including exhaust gas sensors, wastewater sensors, and noise sensors, at various key nodes of the enterprise's emission facilities;

[0024] Each sensor performs periodic data acquisition operations at a fixed sampling frequency, and adds a timestamp to the sensor output data at each time point, and summarizes them in chronological order to form a continuous emission data sequence;

[0025] By deploying camera equipment at the emission facility site, the emission outlet, pipeline connection and flue gas diffusion area are continuously photographed at a set frame rate, and image frames corresponding to each time step are collected. The image frames are then arranged in chronological order to obtain an image sequence covering the entire detection cycle.

[0026] Text data containing inspection records of emission facilities is exported from the enterprise inspection system, text data of permit information related to emission facilities is exported from the pollutant discharge permit management system, and text data of environmental regulations applicable to the current enterprise emission type is extracted from the environmental management database. The above three types of text data are classified and organized according to the data source, and uniformly formatted to form a text data sequence.

[0027] The emission numerical data sequence, image sequence and text data sequence are aligned according to the time index and synchronized using a unified timestamp identifier to ensure that the data under different modes have consistency at the same time step, and generate a time-synchronized data sequence.

[0028] Noise processing and data cleaning are performed on the time-synchronized data sequence. For the emission numerical data, a sliding window method is used to perform local averaging at each time step to smooth out abrupt changes and reduce abnormal fluctuations.

[0029] For the image data portion, blurry image frames caused by lighting interference, occlusion, or sensor malfunction are removed, and images whose clarity meets the threshold requirements are retained;

[0030] For the text data portion, invalid placeholders, duplicate records, and abnormally formatted content are removed, the encoding format is standardized, and semantically complete text records are retained to obtain a multimodal time synchronization data sequence;

[0031] The pixel values ​​of each frame in the image sequence are normalized to map the original pixel values ​​to a uniform numerical range, reduce the interference caused by differences in brightness and contrast, and uniformly adjust the size of the images, converting each frame in the image sequence into a tensor format image data set with consistent dimensions.

[0032] Punctuation cleanup is performed on each text record in the text data sequence, and a dictionary-based word segmentation method is used to divide the sentence into word units. Each word is standardized and encoded to unify word form and spelling, generating a set of structured text vectors consistent with the number of text records.

[0033] The emission numerical data sequence after noise processing, the standardized image sequence, and the text vector sequence after structured processing are indexed and matched according to a unified timestamp. The data of different modalities are correlated at the time step level, and the data format of each modality is uniformly encoded to ensure that it has a consistent input dimension and time index.

[0034] After matching and format unification are completed, the image frame, emission value and text vector of each time step are combined into a multimodal sample. The multimodal samples of all time steps are arranged in chronological order to finally obtain the multimodal dataset.

[0035] Optionally, the step of performing feature encoding on the multimodal datasets separately and then weightedly fusing them through a feature fusion layer to obtain a unified multimodal feature tensor specifically includes:

[0036] Image data at each time step in the multimodal dataset is input into a convolutional neural network. The image is then passed through convolutional layers, pooling layers, and fully connected layers to extract spatial features and obtain corresponding image feature tensors. Each image feature tensor has the same dimensionality.

[0037] The emission numerical data at each time step in the multimodal dataset is input into a multilayer perceptron network. The data is then processed sequentially through the input layer, two or more hidden layers, and the output layer using a nonlinear activation function to extract the numerical expression features and obtain a set of numerical feature vectors.

[0038] The text vectors at each time step in the multimodal dataset are input into a language model pre-trained on a corpus. The embedding layer, encoder layer, and attention mechanism module are used to extract semantic-level contextual features to obtain the corresponding text semantic feature tensor.

[0039] Image feature tensors, numerical feature vector sets, and text semantic feature tensors are matched according to time steps and input into the feature fusion layer. The features of the three modalities are fused using a weighted fusion mechanism to form a unified fused feature tensor.

[0040] Optionally, the step of constructing the current channel features and the comparison channel features based on the multimodal unified feature tensor to obtain the dual-channel input sample set specifically includes:

[0041] Based on the multimodal unified feature tensor, all feature data are divided according to the monitoring period according to the time index. Feature data within the current monitoring period is extracted and marked as the feature tensor of the current monitoring period, and feature data within the historical compliance period is extracted and marked as the feature tensor of the historical compliance period.

[0042] The feature tensors of the current monitoring period and the historical compliance period are respectively input into the feature encoder with shared parameters. The local feature distribution is extracted by the convolutional transformation layer, the feature dimension is transformed linearly by the linear mapping layer, and the feature space is nonlinearly mapped by the nonlinear activation layer, thereby obtaining the encoded feature set of the current monitoring period and the encoded feature set of the historical compliance period respectively.

[0043] A feature alignment operation is performed on the coded feature set of the current monitoring period and the coded feature set of the historical compliance period. The dimension mapping function is used to unify the dimensions of the two types of coded features, mapping each feature vector to the same feature space, so that the two sets of features are consistent in numerical dimension and representation, and the aligned current channel features and aligned comparison channel features are obtained.

[0044] The aligned current channel features and the aligned contrast channel features are paired one-to-one according to the time step index. Each pair of features at the same time step is combined into a set of dual-channel sample pairs. Dual-channel sample pairs at all time steps are constructed in sequence and arranged in chronological order to form a set of dual-channel input samples.

[0045] Optionally, the step of inputting the dual-channel input sample set into a dual-channel comparative spatiotemporal cross-attention network for emission behavior feature extraction to form a comprehensive spatiotemporal difference feature representation specifically includes:

[0046] Receive a set of dual-channel input samples, input the spatial attention initialization layer of the dual-channel contrast spatiotemporal cross-attention network, perform channel mapping and batch normalization processing on the current channel features and the contrast channel features, and output the standardized current channel input features and the standardized contrast channel input features;

[0047] A topological adjacency matrix is ​​generated based on the physical topology of the emission facilities, and a propagation weight matrix is ​​generated based on historical anomaly propagation paths.

[0048] The topological adjacency matrix and the propagation weight matrix are weighted and fused according to node index to generate the causal propagation matrix;

[0049] The standardized current channel input features and the standardized comparison channel input features are weighted by the causal propagation matrix to obtain the spatial attention weight matrix.

[0050] Multiply the spatial attention weight matrix of the current channel with the corresponding set of spatial value vectors to obtain the current spatial features; multiply the spatial attention weight matrix of the comparison channel with the corresponding set of spatial value vectors to obtain the comparison spatial features.

[0051] The standardized current channel input features and the standardized contrast channel input features are input into the time attention layer, and a time-related weight matrix is ​​generated based on the time step sequence, specifically including:

[0052] Within each channel, the input feature vector is first transformed linearly to obtain the time query vector and time key vector respectively. By performing the inner product operation on the query vector and key vector of all time steps, the similarity score matrix between each pair of time steps is obtained.

[0053] Subsequently, each row of the similarity score matrix is ​​subjected to exponential normalization so that the relevance weights of the same time step to all time steps sum to one, thus forming a time relevance weight matrix.

[0054] Based on the standardized current channel input features and the standardized comparison channel input features, the significance score for each time step is calculated.

[0055] The significance score is used as a multiplicative weighting coefficient on the time correlation weight matrix to dynamically modulate the attention weights at each time step, generating an adjusted time attention matrix. The adjusted time attention matrix is ​​then weighted and calculated with the time value vector sets of the current channel and the comparison channel, respectively, to obtain the current time features and the comparison time features.

[0056] Input the current spatial features and current temporal features into the first spatiotemporal cross-attention channel, and input the contrasting spatial features and contrasting temporal features into the second spatiotemporal cross-attention channel. Calculate the cross-attention respectively, and output the current spatiotemporal fusion features and the contrasting spatiotemporal fusion features.

[0057] Receive the current spatiotemporal fusion features and the comparison spatiotemporal fusion features, extract the corresponding query matrix, key matrix and value matrix, and calculate the cross-attention matrix of the current channel and the comparison channel respectively. The cross-attention matrix of the current channel is calculated by multiplying the query matrix and key matrix of the current channel and then normalizing the result. The cross-attention matrix of the comparison channel is calculated by multiplying the query matrix and key matrix of the comparison channel and then normalizing the result.

[0058] Element-level difference operations are performed on the cross-attention matrices of the two channels to obtain the difference attention matrix. The time value vectors of the current channel and the comparison channel are then weighted using the difference attention matrix to generate differential activation features.

[0059] By concatenating the current spatiotemporal fusion features, the contrastive spatiotemporal fusion features, and the differential activation features along the channel dimension, a comprehensive spatiotemporal difference feature representation is obtained.

[0060] Optionally, the step of inputting the current spatial features and current temporal features into the first spatiotemporal cross-attention channel, and inputting the contrasting spatial features and contrasting temporal features into the second spatiotemporal cross-attention channel, calculating cross-attention respectively, and outputting the current spatiotemporal fusion features and contrasting spatiotemporal fusion features specifically includes:

[0061] Receive current spatial features, comparison spatial features, current time features, and comparison time features; input the current spatial features and current time features into the first spatiotemporal cross-attention channel; input the comparison spatial features and comparison time features into the second spatiotemporal cross-attention channel; and establish the spatiotemporal feature correspondence between the two channels.

[0062] In the first spatiotemporal cross-attention channel, the current spatial features are used as the input query matrix, and the current temporal features are used as the input key matrix and value matrix, respectively. The degree of attention association between the two is calculated by matrix multiplication, and normalization is performed after dividing by the square root of the feature dimension to obtain the cross-attention weight matrix of the first channel.

[0063] The current time features are weighted and summed using the cross-attention weight matrix of the first channel to generate the spatiotemporal fusion features of the first channel, and the output is the current spatiotemporal fusion features;

[0064] In the second spatiotemporal cross-attention channel, the spatial features of the comparison are used as the input query matrix, and the temporal features of the comparison are used as the input key matrix and value matrix, respectively. The correlation distribution between the query matrix and the key matrix is ​​calculated by matrix multiplication, and normalization is performed after dividing by the square root of the feature dimension to obtain the cross-attention weight matrix of the second channel.

[0065] The cross-attention weight matrix of the second channel is used to perform weighted summation of the contrast time features to generate the spatiotemporal fusion features of the second channel, and the output is the contrast spatiotemporal fusion features.

[0066] Optionally, the step of inputting the comprehensive spatiotemporal difference feature representation into the regulatory knowledge graph matching module to generate compliance determination results and corresponding regulatory clause information specifically includes:

[0067] The system receives a comprehensive spatiotemporal difference feature representation, inputs it into the regulatory knowledge graph matching module, decomposes the input features, and extracts a multi-dimensional vector set containing emission parameter features, facility node features, and time features.

[0068] The regulatory knowledge graph matching module consists of a knowledge representation layer, a graph attention reasoning layer, and a regulatory association decision layer. The knowledge representation layer is used to construct and store a knowledge graph structure containing emission parameter nodes, facility nodes, and regulatory clause nodes, and to establish a set of triplet relationships between emission parameter nodes, facility nodes, and regulatory clause nodes according to the relationships between nodes.

[0069] In the knowledge representation layer, node embedding operations are performed on emission parameter nodes, facility nodes, and regulatory clause nodes respectively, converting the structural and semantic information of each node into a low-dimensional vector representation, forming a node embedding set composed of emission parameter node embedding, facility node embedding, and regulatory clause node embedding.

[0070] In the graph attention inference layer, the attention weights between nodes are calculated based on the node embedding set. First, feature mapping and concatenation operations are performed on the embedding vectors of any pair of nodes. Then, the association strength is calculated through the weight vector with the leakage linear rectified function. All association strengths within the adjacent range of the same node are normalized to obtain the attention weight distribution between nodes.

[0071] The node embeddings are weighted and updated based on the weight distribution to generate a set of node features that includes emission parameter node update features, facility node update features, and regulatory clause node update features.

[0072] In the regulatory association decision layer, the cosine similarity between the emission parameter node update feature and the emission parameter node embedding, and the cosine similarity between the facility node update feature and the facility node embedding are calculated respectively to obtain the emission parameter matching similarity and the facility node matching similarity. The two types of similarity are weighted and fused according to the set weight coefficient to calculate the regulatory clause association score.

[0073] Based on the correlation score of the regulatory clauses, the updated set of embedded regulatory clause nodes is retrieved, the node with the highest score is selected, and the corresponding regulatory clause number, clause content and matching confidence score are output to generate a compliance judgment result and the corresponding regulatory clause information.

[0074] Optionally, the step of determining the abnormal emission facilities and causes based on the compliance assessment results and regulatory information, generating a self-correction task, and pushing it to the mobile terminal for execution specifically includes:

[0075] Receive the compliance determination result and the regulatory clause information, parse the non-compliant items in the determination result and the corresponding regulatory clause numbers, extract the facility node identifier, emission parameter characteristics and time segment information associated with the non-compliant items, and generate an abnormal association table containing facility identifier, emission index and corresponding regulatory clause;

[0076] In the abnormal association table, based on the degree of exceeding the emission limit conditions and emission parameter characteristics stipulated in the regulations, the corresponding set of emission abnormal facilities is determined. An abnormality score is calculated for each facility node in the set. The abnormality score is determined by the deviation ratio between the actual monitored value and the regulatory limit value and the correlation score of the regulations.

[0077] The difference between the actual monitored value and the regulatory limit reflects the extent of emission exceedance, while the score related to the regulatory clauses reflects the degree of compliance with regulatory constraints. The two are combined according to a weighted coefficient to obtain the anomaly score for each facility node.

[0078] The causes of anomalies are classified and determined based on the numerical range of the anomaly severity score. Anomalies below the threshold are marked as slight fluctuations, while anomalies above the threshold are marked as excessive emissions or data distortion. A set of rectification targets is generated in conjunction with the enforcement conditions of the corresponding regulations.

[0079] A self-correction task structure is constructed based on the set of rectification targets. Each task includes fields for rectification target, execution requirements, responsible personnel, and time constraints, and a task deadline is generated based on the rectification time limits stipulated in the regulations.

[0080] The generated self-correction task structure is pushed to the mobile terminal and assigned to the corresponding responsible person's account through the task scheduling interface. The task status is recorded as "pending execution" and the task completion progress and emission retest results are continuously updated during the execution process, ultimately forming a closed-loop management process for emission compliance self-inspection and self-correction.

[0081] The beneficial effects of this invention are:

[0082] This invention achieves deep-level correlation modeling of emission data by introducing multimodal fusion and a comparative spatiotemporal cross-attention mechanism. The system can establish a unified feature representation among multi-source data such as images, numerical data, and text, and simultaneously capture emission differences between the current cycle and historical cycles through a dual-channel input structure, enabling anomaly detection results to have higher spatiotemporal consistency and robustness.

[0083] In the spatial dimension, the system constructs a causal propagation matrix using facility topology and historical anomaly propagation paths, thereby modeling the propagation characteristics of abnormal emissions between equipment. In the temporal dimension, the system dynamically weights key time segments through a significance scoring mechanism, enabling the model to have higher sensitivity and stability when dealing with sudden changes in emissions.

[0084] In the compliance assessment phase, this invention establishes semantic relationships between emission parameters, facility nodes, and regulatory clause nodes through a regulatory knowledge graph matching module, enabling automatic matching and clause-level judgment of emission behaviors and regulatory constraints. Based on the assessment results, the system automatically generates self-correction tasks containing rectification objectives, implementation requirements, and time constraints, and pushes them to the terminal for execution, forming a closed-loop compliance management mechanism from anomaly detection to task implementation.

[0085] This method, while ensuring the accuracy of data analysis, automates regulatory response and rectification, thereby improving the efficiency and intelligence of enterprises' emissions compliance work. Attached Figure Description

[0086] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0087] Fig. 1 This is a flowchart of a method for an emission compliance self-inspection and self-correction system based on multimodal fusion proposed in this invention;

[0088] Fig. 2 This is a schematic diagram of the structure of an emission compliance self-inspection and self-correction system based on multimodal fusion proposed in this invention;

[0089] Fig. 3 This is a schematic diagram of the structure of a dual-channel comparison spatiotemporal cross-attention network in an emission compliance self-inspection and self-correction system based on multimodal fusion proposed in this invention. Detailed Implementation

[0090] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0091] refer to Figs. 1-3 A self-inspection and rectification system for emissions compliance based on multimodal fusion, comprising:

[0092] The multimodal data acquisition module is used to collect multi-source data from enterprise emission facilities, perform preprocessing, and obtain a multimodal dataset.

[0093] The multimodal feature encoding module is used to encode features of the multimodal dataset separately to obtain a multimodal unified feature tensor;

[0094] The dual-channel feature construction module is used to construct the current channel features and the comparison channel features based on the multimodal unified feature tensor, so as to obtain a dual-channel input sample set.

[0095] The spatiotemporal cross-attention feature extraction module is used to receive the current channel features and the comparison channel features, and output a comprehensive spatiotemporal difference feature representation through a dual-channel spatiotemporal cross-attention network;

[0096] The regulatory knowledge graph matching module is used to input the comprehensive spatiotemporal difference feature representation into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information;

[0097] The self-correction task generation module is used to generate self-correction tasks based on compliance judgment results and regulatory clause information, and push them to the mobile terminal for execution.

[0098] The task execution and closed-loop management module is used to record the self-correction task execution process, monitor the rectification completion status and retest emission data, and update compliance status information.

[0099] In this embodiment, the modules are interconnected using the following method:

[0100] Collect multi-source data on enterprise emission facilities, preprocess the multi-source data to obtain a multimodal dataset;

[0101] The features of the multimodal dataset are encoded separately, and then weighted and fused through a feature fusion layer to obtain a unified multimodal feature tensor.

[0102] Based on the multimodal unified feature tensor, the current channel features and the comparison channel features are constructed to obtain a dual-channel input sample set;

[0103] The dual-channel input sample set is input into a dual-channel comparative spatiotemporal cross-attention network to extract emission behavior features, forming a comprehensive spatiotemporal difference feature representation;

[0104] The comprehensive spatiotemporal difference feature representation is input into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information;

[0105] Based on the compliance assessment results and regulatory information, identify the facilities with abnormal emissions and the causes of the abnormalities, generate self-correction tasks, and push them to mobile terminals for execution.

[0106] In this embodiment, the process of collecting multi-source data from enterprise emission facilities and preprocessing the multi-source data to obtain a multimodal dataset specifically includes:

[0107] Multiple data acquisition units, including exhaust gas sensors, wastewater sensors, and noise sensors, are deployed at various key nodes of the enterprise's emission facilities. These units are used to collect in real time the concentrations of sulfur dioxide, nitrogen oxides, and particulate matter in the flue gas, the chemical oxygen demand, ammonia nitrogen, and total phosphorus in the wastewater, and the noise level in the working environment.

[0108] Each sensor performs periodic data acquisition operations at a fixed sampling frequency, and adds a timestamp to the sensor output data at each time point, and summarizes them in chronological order to form a continuous emission data sequence;

[0109] By deploying camera equipment at the emission facility site, the emission outlet, pipeline connection and flue gas diffusion area are continuously photographed at a set frame rate, and image frames corresponding to each time step are collected. The image frames are then arranged in chronological order to obtain an image sequence covering the entire detection cycle.

[0110] Text data containing inspection records of emission facilities is exported from the enterprise inspection system, text data of permit information related to emission facilities is exported from the pollutant discharge permit management system, and text data of environmental regulations applicable to the current enterprise emission type is extracted from the environmental management database. The above three types of text data are classified and organized according to the data source, and uniformly formatted to obtain a text data set. Each piece of text data corresponds to a structured text record, which is used to represent the compliance explanation content under a specific time step or rule scenario, and finally forms a text data sequence.

[0111] The emission numerical data sequence, image sequence, and text data sequence are aligned according to the time index and synchronized using a unified timestamp identifier to ensure that the data under different modes are consistent at the same time step, generating a time-synchronized data sequence. The data item at each time step consists of image frames, numerical data, and text data.

[0112] Noise processing and data cleaning are performed on the time-synchronized data sequence. For the emission numerical data, a sliding window method is used to perform local averaging at each time step to smooth out abrupt changes and reduce abnormal fluctuations.

[0113] For the image data portion, blurry image frames caused by lighting interference, occlusion, or sensor malfunction are removed, and images whose clarity meets the threshold requirements are retained;

[0114] For the text data portion, invalid placeholders, duplicate records, and abnormally formatted content are removed, the encoding format is standardized, and semantically complete text records are retained, ultimately resulting in a cleaned multimodal time synchronization data sequence with a standardized structure and low noise.

[0115] The pixel values ​​of each frame in the image sequence are normalized to map the original pixel values ​​to a uniform numerical range, reducing the interference caused by differences in brightness and contrast. The image size is also uniformly adjusted so that all image frames have the same height, width and number of channels. Finally, each frame in the image sequence is converted into a tensor format image data set with consistent dimensions.

[0116] For each text record in the text data sequence, punctuation cleaning is performed to remove invalid symbols and special characters. Then, the sentences are segmented and a dictionary-based word segmentation method is used to divide the sentences into word units. Subsequently, each word is standardized and encoded to unify word form and spelling, construct a word-level semantic structure representation, and finally generate a set of structured text vectors consistent with the number of text records.

[0117] The emission numerical data sequence after noise processing, the standardized image sequence, and the text vector sequence after structured processing are indexed and matched according to a unified timestamp. The data of different modalities are correlated at the time step level, and the data format of each modality is uniformly encoded to ensure that it has a consistent input dimension and time index.

[0118] After matching and format unification are completed, the image frame, emission value and text vector of each time step are combined into a multimodal sample. The multimodal samples of all time steps are arranged in chronological order to finally obtain the multimodal dataset.

[0119] In this embodiment, the step of performing feature encoding on the multimodal datasets respectively, and then weightedly fusing them through a feature fusion layer to obtain a unified multimodal feature tensor specifically includes:

[0120] Image data at each time step in the multimodal dataset is input into a convolutional neural network. The image is then passed through convolutional layers, pooling layers, and fully connected layers to extract spatial features and obtain corresponding image feature tensors. Each image feature tensor has the same dimensionality.

[0121] The emission numerical data at each time step in the multimodal dataset is input into a multilayer perceptron network. The data is then processed sequentially through the input layer, two or more hidden layers, and the output layer using a non-linear activation function to extract the numerical expression features, resulting in a set of numerical feature vectors, where each numerical feature vector has a consistent dimensional representation.

[0122] The text vectors at each time step in the multimodal dataset are input into a language model pre-trained on a corpus. The embedding layer, encoder layer, and attention mechanism module are used to extract semantic-level contextual features to obtain the corresponding text semantic feature tensor.

[0123] Image feature tensors, numerical feature vector sets, and text semantic feature tensors are matched according to time steps and input into the feature fusion layer. The features of the three modalities are fused using a weighted fusion mechanism to form a unified fused feature tensor.

[0124] In this embodiment, the step of constructing the current channel features and the comparison channel features based on the multimodal unified feature tensor to obtain the dual-channel input sample set specifically includes:

[0125] Based on the multimodal unified feature tensor, all feature data are divided according to the monitoring period according to the time index. Feature data within the current monitoring period is extracted and marked as the feature tensor of the current monitoring period, and feature data within the historical compliance period is extracted and marked as the feature tensor of the historical compliance period.

[0126] Both the current monitoring cycle feature tensor and the historical compliance cycle feature tensor are composed of image features, numerical features and text features, and they are continuous in the time dimension and correspond to each other in the modal dimension.

[0127] The feature tensors of the current monitoring period and the historical compliance period are respectively input into the feature encoder with shared parameters. The local feature distribution is extracted by the convolutional transformation layer, the feature dimension is transformed linearly by the linear mapping layer, and the feature space is nonlinearly mapped by the nonlinear activation layer. Thus, the encoded feature set of the current monitoring period and the encoded feature set of the historical compliance period are obtained respectively. The two types of encoded feature sets are consistent in structure.

[0128] A feature alignment operation is performed on the coded feature set of the current monitoring period and the coded feature set of the historical compliance period. The dimension mapping function is used to unify the dimensions of the two types of coded features, mapping each feature vector to the same feature space, so that the two sets of features are consistent in numerical dimension and representation, and the aligned current channel features and aligned comparison channel features are obtained.

[0129] The aligned current channel features and the aligned contrast channel features are paired one-to-one according to the time step index. Each pair of features at the same time step is combined into a set of dual-channel sample pairs. Dual-channel sample pairs at all time steps are constructed in sequence and arranged in chronological order to form a set of dual-channel input samples.

[0130] In this embodiment, the step of inputting the dual-channel input sample set into a dual-channel comparative spatiotemporal cross-attention network for emission behavior feature extraction to form a comprehensive spatiotemporal difference feature representation specifically includes:

[0131] Receive a set of dual-channel input samples, input the spatial attention initialization layer of the dual-channel contrast spatiotemporal cross-attention network, perform channel mapping and batch normalization processing on the current channel features and the contrast channel features, and output the standardized current channel input features and the standardized contrast channel input features;

[0132] A topological adjacency matrix is ​​generated based on the physical topology of the emission facilities, and a propagation weight matrix is ​​generated based on the historical anomaly propagation paths. The historical anomaly propagation paths consist of the sequence of associated anomalies recorded by the enterprise's emission monitoring system in previous monitoring periods. Each path consists of a trigger node, a propagation node, and a termination node. The influence intensity of any adjacent node in the path is calculated by weighting based on the frequency of anomaly occurrence and propagation delay.

[0133] The topological adjacency matrix and the propagation weight matrix are weighted and fused according to node index to generate the causal propagation matrix;

[0134] The standardized current channel input features and the standardized comparison channel input features are weighted by the causal propagation matrix to obtain the spatial attention weight matrix:

[0135] ;

[0136] in, Here is the spatial attention weight matrix. For a set of spatial query vectors, It is a set of spatial key vectors. This is the matrix transpose symbol. This is the element-wise multiplication operator. For causal propagation matrix, is a feature dimension constant, representing the feature dimension of the spatial key vector set;

[0137] Multiply the spatial attention weight matrix of the current channel with the corresponding set of spatial value vectors to obtain the current spatial features; multiply the spatial attention weight matrix of the comparison channel with the corresponding set of spatial value vectors to obtain the comparison spatial features.

[0138] The spatial query vector set, spatial key vector set, and spatial value vector set are generated by linear mapping between the standardized current channel input features and the standardized comparison channel input features.

[0139] The standardized current channel input features and the standardized comparison channel input features are input into the time attention layer, and a time correlation weight matrix is ​​generated based on the time step sequence; specifically including:

[0140] Within each channel, the input feature vector is first transformed linearly to obtain the time query vector and time key vector respectively. By performing the inner product operation on the query vector and key vector of all time steps, the similarity score matrix between each pair of time steps is obtained.

[0141] Subsequently, each row of the similarity score matrix is ​​subjected to exponential normalization so that the correlation weights of the same time step to all time steps sum to one, thereby forming a time correlation weight matrix, which is used to characterize the strength of the dependency relationship within the time series.

[0142] Based on the standardized current channel input features and the standardized contrast channel input features, calculate the significance score for each time step:

[0143] ;

[0144] in, For time step The significance score, For the Sigmoid function, This is the first learnable weight parameter matrix. This is the second learnable weight parameter matrix. For time step The input feature vector corresponds to the standardized input features of the current channel or the standardized input features of the contrasting channel in the time dimension. The average feature vector of the time series is calculated from the input features of all time steps, representing the average feature state over the entire period.

[0145] The significance score is used as a multiplicative weighting coefficient on the time correlation weight matrix to dynamically modulate the attention weights at each time step, generating an adjusted time attention matrix. The adjusted time attention matrix is ​​then weighted and calculated with the time value vector sets of the current channel and the comparison channel, respectively, to obtain the current time features and the comparison time features.

[0146] Input the current spatial features and current temporal features into the first spatiotemporal cross-attention channel, and input the contrasting spatial features and contrasting temporal features into the second spatiotemporal cross-attention channel. Calculate the cross-attention respectively, and output the current spatiotemporal fusion features and the contrasting spatiotemporal fusion features.

[0147] Receive the current spatiotemporal fusion features and the comparison spatiotemporal fusion features, extract the corresponding query matrix, key matrix and value matrix, and calculate the cross-attention matrix of the current channel and the comparison channel respectively. The cross-attention matrix of the current channel is calculated by multiplying the query matrix and key matrix of the current channel and then normalizing the result. The cross-attention matrix of the comparison channel is calculated by multiplying the query matrix and key matrix of the comparison channel and then normalizing the result.

[0148] Element-level difference operations are performed on the cross-attention matrices of the two channels to obtain the difference attention matrix. The time value vectors of the current channel and the comparison channel are then weighted using the difference attention matrix to generate differential activation features.

[0149] By concatenating the current spatiotemporal fusion features, the contrastive spatiotemporal fusion features, and the differential activation features along the channel dimension, a comprehensive spatiotemporal difference feature representation is obtained.

[0150] In this embodiment, the step of inputting the current spatial features and current temporal features into the first spatiotemporal cross-attention channel, and inputting the contrasting spatial features and contrasting temporal features into the second spatiotemporal cross-attention channel, calculating the cross-attention respectively, and outputting the current spatiotemporal fusion features and the contrasting spatiotemporal fusion features specifically includes:

[0151] Receive current spatial features, comparison spatial features, current time features, and comparison time features; input the current spatial features and current time features into the first spatiotemporal cross-attention channel; input the comparison spatial features and comparison time features into the second spatiotemporal cross-attention channel; and establish the spatiotemporal feature correspondence between the two channels.

[0152] In the first spatiotemporal cross-attention channel, the current spatial features are used as the input query matrix, and the current temporal features are used as the input key matrix and value matrix, respectively. The degree of attention association between the two is calculated by matrix multiplication, and normalization is performed after dividing by the square root of the feature dimension to obtain the cross-attention weight matrix of the first channel.

[0153] The current time features are weighted and summed using the cross-attention weight matrix of the first channel to generate the spatiotemporal fusion features of the first channel, and the output is the current spatiotemporal fusion features;

[0154] In the second spatiotemporal cross-attention channel, the spatial features of the comparison are used as the input query matrix, and the temporal features of the comparison are used as the input key matrix and value matrix, respectively. The correlation distribution between the query matrix and the key matrix is ​​calculated by matrix multiplication, and normalization is performed after dividing by the square root of the feature dimension to obtain the cross-attention weight matrix of the second channel.

[0155] The cross-attention weight matrix of the second channel is used to perform weighted summation of the contrast time features to generate the spatiotemporal fusion features of the second channel, and the output is the contrast spatiotemporal fusion features.

[0156] In this embodiment, the step of inputting the comprehensive spatiotemporal difference feature representation into the regulatory knowledge graph matching module to generate compliance determination results and corresponding regulatory clause information specifically includes:

[0157] The system receives a comprehensive spatiotemporal difference feature representation, inputs it into the regulatory knowledge graph matching module, decomposes the input features, and extracts a multi-dimensional vector set containing emission parameter features, facility node features, and time features.

[0158] The regulatory knowledge graph matching module consists of a knowledge representation layer, a graph attention reasoning layer, and a regulatory association decision layer. The knowledge representation layer is used to construct and store a knowledge graph structure containing emission parameter nodes, facility nodes, and regulatory clause nodes. It establishes a set of triple relationships between emission parameter nodes, facility nodes, and regulatory clause nodes according to the relationship between nodes. Each triple relationship represents the mapping relationship between emission parameters, facilities, and corresponding regulatory clauses.

[0159] In the knowledge representation layer, node embedding operations are performed on emission parameter nodes, facility nodes, and regulatory clause nodes respectively, converting the structural and semantic information of each node into a low-dimensional vector representation, forming a node embedding set composed of emission parameter node embedding, facility node embedding, and regulatory clause node embedding.

[0160] In the graph attention inference layer, the attention weights between nodes are calculated based on the node embedding set. First, feature mapping and concatenation operations are performed on the embedding vectors of any pair of nodes. Then, the association strength is calculated through the weight vector with the leakage linear rectified function. All association strengths within the adjacent range of the same node are normalized to obtain the attention weight distribution between nodes.

[0161] The node embeddings are weighted and updated based on the weight distribution to generate a set of node features that includes emission parameter node update features, facility node update features, and regulatory clause node update features.

[0162] In the regulatory association decision layer, the cosine similarity between the emission parameter node update feature and the emission parameter node embedding, and the cosine similarity between the facility node update feature and the facility node embedding are calculated respectively to obtain the emission parameter matching similarity and the facility node matching similarity. The two types of similarity are weighted and fused according to the set weight coefficient to calculate the regulatory clause association score, which is used to measure the association strength between the input feature and the regulatory clause node.

[0163] Based on the correlation score of the regulatory clauses, the updated set of embedded regulatory clause nodes is retrieved, the node with the highest score is selected, and the corresponding regulatory clause number, clause content and matching confidence score are output to generate a compliance judgment result and the corresponding regulatory clause information.

[0164] In this embodiment, the step of determining the abnormal emission facilities and causes based on the compliance assessment results and regulatory clause information, generating a self-correction task, and pushing it to the mobile terminal for execution specifically includes:

[0165] Receive the compliance determination result and the regulatory clause information, parse the non-compliant items in the determination result and the corresponding regulatory clause numbers, extract the facility node identifier, emission parameter characteristics and time segment information associated with the non-compliant items, and generate an abnormal association table containing facility identifier, emission index and corresponding regulatory clause;

[0166] In the abnormal association table, based on the degree of exceeding the emission limit conditions and emission parameter characteristics stipulated in the regulations, the corresponding set of emission abnormal facilities is determined. An abnormality score is calculated for each facility node in the set. The abnormality score is determined by the deviation ratio between the actual monitored value and the regulatory limit value and the correlation score of the regulations.

[0167] The difference between the actual monitored value and the regulatory limit reflects the extent of emission exceedance, while the score related to the regulatory clauses reflects the degree of compliance with regulatory constraints. The two are combined according to a weighted coefficient to obtain the anomaly score for each facility node.

[0168] The causes of anomalies are classified and determined based on the numerical range of the anomaly severity score. Anomalies below the threshold are marked as slight fluctuations, while anomalies above the threshold are marked as excessive emissions or data distortion. A set of rectification targets is generated in conjunction with the enforcement conditions of the corresponding regulations.

[0169] A self-correction task structure is constructed based on the set of rectification targets. Each task includes fields for rectification target, execution requirements, responsible personnel, and time constraints, and a task deadline is generated based on the rectification time limits stipulated in the regulations.

[0170] The generated self-correction task structure is pushed to the mobile terminal and assigned to the corresponding responsible person's account through the task scheduling interface. The task status is recorded as "pending execution" and the task completion progress and emission retest results are continuously updated during the execution process, ultimately forming a closed-loop management process for emission compliance self-inspection and self-correction.

[0171] Example 1:

[0172] To verify the feasibility of this invention in practice, it was applied to the emission monitoring and management system of a large manufacturing enterprise. This enterprise has multiple production lines and numerous emission nodes for gas, spraying, and waste gas treatment, with a daily monitoring data volume exceeding 30GB, involving three types of data: image monitoring streams, sensor numerical signals, and inspection record text. Traditional emission monitoring systems rely solely on numerical threshold alarms, failing to accurately identify hidden emission anomalies caused by equipment aging, pipeline coupling, or abnormal control strategies, and also lacking the ability to automatically assess compliance and distribute rectification tasks. After deployment in this scenario, this invention uses a multimodal data acquisition module to simultaneously collect and unify the formats of on-site camera video frames, real-time concentration values ​​from flue gas analyzers, and inspection text reports, forming a multimodal raw dataset. The system extracts image features, numerical features, and text semantic features through a multimodal feature encoding module, and performs weighted fusion at the feature fusion layer to generate a unified feature tensor for subsequent analysis.

[0173] In actual operation, the system automatically constructs feature samples of the current and historical channels every 5 minutes as a monitoring cycle, and extracts emission behavior features through a dual-channel comparative spatiotemporal cross-attention network. The system can simultaneously monitor the coupling relationship between emission facilities in space and the changing trend over time, and identify emission behaviors that deviate significantly from the historical baseline through difference modeling. When flue gas monitoring data fluctuates but does not exceed the traditional alarm threshold, the system can still detect potential anomalies based on comprehensive spatiotemporal difference features, and input the results into the regulatory knowledge graph matching module to achieve automatic association between emission behavior and regulatory provisions. The system finally outputs a compliance judgment result, generates a self-correction task containing rectification goals, implementation requirements, and rectification deadlines, and automatically pushes it to the on-site mobile terminal.

[0174] Statistical results after one week of continuous operation show that the system detected 152 abnormal emission events, of which 37.5% were minor emission exceedances that were missed by traditional systems. In the regulatory knowledge graph matching stage, the system accurately located relevant regulatory clauses and automatically generated 136 self-correction tasks, with an average task dispatch delay of only 2 minutes. Compared with the company's original manual review process, this invention improves compliance judgment efficiency by approximately 65%, and reduces the average response time for handling emission anomalies from 1.8 hours to 22 minutes. After rectification, the emission compliance rate during the monitoring period increased from 92.4% to 98.7%. These results demonstrate that this invention can achieve multimodal emission data fusion processing, accurate anomaly detection, and automated compliance closed-loop management in complex industrial environments, effectively improving the intelligence and compliance response capabilities of emission management.

[0175] Table 1 Performance Verification Data of Multimodal Emission Compliance Self-Inspection and Rectification System

[0176] As shown in Table 1 above, the emission compliance self-inspection and self-correction system based on multimodal fusion proposed in this invention significantly improves upon traditional emission monitoring methods in several key performance indicators. Firstly, regarding anomaly detection capability, the system identified 152 emission anomaly events within the same monitoring period, an increase of approximately 60% compared to the 95 events identified by the traditional system. This indicates that multimodal data fusion and the spatiotemporal cross-attention mechanism can more comprehensively capture the spatiotemporal correlation features in the emission process, thereby improving the sensitivity of anomaly identification. Secondly, in terms of the identification rate of minor anomalies, the system of this invention achieves 80.3%, higher than the 42.8% of the traditional system. This demonstrates that the system can effectively identify hidden over-limit emission problems caused by equipment aging, instantaneous fluctuations, or pipeline coupling, solving the problem of traditional methods being insensitive to low-intensity anomaly identification.

[0177] In terms of processing efficiency, the system of this invention, supported by a regulatory knowledge graph, achieves automated judgment and task generation. The average time for compliance judgment is reduced from 210 seconds to 73 seconds, a decrease of approximately 65%, and the average response time for anomaly handling is reduced from 108 minutes to 22 minutes, an improvement of nearly 80%, thus enhancing the efficiency of compliance response and task closure. In practical applications, the system can automatically generate 136 self-correction tasks, achieving intelligent assignment and full-process tracking of emission anomalies, reducing manual intervention.

[0178] Regarding emission compliance, the emission compliance rate of this invention's system increased to 98.7%, a 6.3 percentage point improvement over traditional systems, demonstrating that the system can effectively suppress long-term over-emission risks under continuous monitoring and dynamic rectification mechanisms. Simultaneously, the regulatory matching accuracy improved from 85.2% to 97.9%, proving that the regulatory knowledge graph can accurately correlate emission parameters, facility nodes, and regulatory clauses, achieving automated and refined compliance judgment. Overall, this invention, through multimodal fusion, comparative spatiotemporal attention modeling, and regulatory knowledge graph reasoning, constructs a high-precision, high-efficiency, closed-loop adaptive emission compliance self-inspection and self-correction system, demonstrating significant technical advantages and application value in improving the intelligence level of emission monitoring and enterprise compliance management capabilities.

[0179] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A self-inspection and rectification system for emission compliance based on multimodal fusion, characterized in that, Includes the following steps: The multimodal data acquisition module is used to collect multi-source data from enterprise emission facilities, perform preprocessing, and obtain a multimodal dataset. The multimodal feature encoding module is used to encode features of the multimodal dataset separately to obtain a multimodal unified feature tensor; The dual-channel feature construction module is used to construct the current channel features and the comparison channel features based on the multimodal unified feature tensor, so as to obtain a dual-channel input sample set. The spatiotemporal cross-attention feature extraction module is used to receive the current channel features and the comparison channel features, and output a comprehensive spatiotemporal difference feature representation through a dual-channel spatiotemporal cross-attention network; The regulatory knowledge graph matching module is used to input the comprehensive spatiotemporal difference feature representation into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information; The self-correction task generation module is used to generate self-correction tasks based on compliance judgment results and regulatory clause information, and push them to the mobile terminal for execution. The task execution and closed-loop management module is used to record the self-correction task execution process, monitor the rectification completion status and retest emission data, and update compliance status information.

2. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 1, characterized in that, The modules are connected in the following way: Collect multi-source data on enterprise emission facilities, preprocess the multi-source data to obtain a multimodal dataset; The features of the multimodal dataset are encoded separately, and then weighted and fused through a feature fusion layer to obtain a unified multimodal feature tensor. Based on the multimodal unified feature tensor, the current channel features and the comparison channel features are constructed to obtain a dual-channel input sample set; The dual-channel input sample set is input into a dual-channel comparative spatiotemporal cross-attention network to extract emission behavior features, forming a comprehensive spatiotemporal difference feature representation; The comprehensive spatiotemporal difference feature representation is input into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information; Based on the compliance assessment results and regulatory information, identify the facilities with abnormal emissions and the causes of the abnormalities, generate self-correction tasks, and push them to mobile terminals for execution.

3. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 2, characterized in that, The preprocessing includes time synchronization, noise reduction, data cleaning, and formatting. The formatting includes pixel normalization and resolution standardization of image data, and symbol cleaning and sentence / word segmentation of text data.

4. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 2, characterized in that, The process of encoding features from different multimodal datasets and then weighting and fusing them through a feature fusion layer to obtain a unified multimodal feature tensor specifically includes: Image data from a multimodal dataset is input into a convolutional neural network to extract spatial features of the images and obtain the corresponding image feature tensors. The emission numerical data from the multimodal dataset is input into a multilayer perceptron network to extract the numerical representation features and obtain a set of numerical feature vectors. Text vectors from a multimodal dataset are input into a language model pre-trained on a corpus to extract semantic-level contextual features, resulting in a text semantic feature tensor. The image feature tensor, the set of numerical feature vectors, and the text semantic feature tensor are matched according to time steps, and a weighted fusion mechanism is used to form a unified fused feature tensor.

5. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 2, characterized in that, The process of constructing the current channel features and the comparison channel features based on the multimodal unified feature tensor to obtain the dual-channel input sample set specifically includes: Based on the multimodal unified feature tensor, all feature data are divided according to the monitoring period according to the time index to obtain the feature tensor of the current monitoring period and the feature tensor of the historical compliance period. The current monitoring cycle feature tensor and the historical compliance cycle feature tensor are respectively input into the feature encoder with shared parameters to obtain the coded feature set of the current monitoring cycle and the coded feature set of the historical compliance cycle. Perform feature alignment on the coded feature set of the current monitoring period and the coded feature set of the historical compliance period to obtain the aligned current channel features and the aligned comparison channel features; The aligned current channel features and the aligned contrast channel features are paired one-to-one according to the time step index and arranged in chronological order to form a dual-channel input sample set.

6. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 2, characterized in that, The step of inputting the dual-channel input sample set into a dual-channel comparative spatiotemporal cross-attention network to extract emission behavior features and form a comprehensive spatiotemporal difference feature representation specifically includes: Receive a set of dual-channel input samples, input the spatial attention initialization layer of the dual-channel contrast spatiotemporal cross-attention network, perform channel mapping and batch normalization processing on the current channel features and the contrast channel features, and output the standardized current channel input features and the standardized contrast channel input features; A topological adjacency matrix is ​​generated based on the physical topology of the emission facilities, a propagation weight matrix is ​​generated based on the historical anomaly propagation paths, and weighted fusion is performed according to the node index to generate a causal propagation matrix. The standardized current channel input features and the standardized comparison channel input features are weighted by the causal propagation matrix to obtain the spatial attention weight matrix. Multiply the spatial attention weight matrix of the current channel with the corresponding set of spatial value vectors to obtain the current spatial features; multiply the spatial attention weight matrix of the comparison channel with the corresponding set of spatial value vectors to obtain the comparison spatial features. The standardized current channel input features and the standardized comparison channel input features are input into the time attention layer, and a time correlation weight matrix is ​​generated based on the time step sequence. Based on the standardized current channel input features and the standardized comparison channel input features, the significance score for each time step is calculated. Based on the saliency score, an adjusted time attention matrix is ​​generated, and then weighted and calculated with the time value vector set of the current channel and the time value vector set of the comparison channel respectively to obtain the current time feature and the comparison time feature; Input the current spatial features and current temporal features into the first spatiotemporal cross-attention channel, and input the contrasting spatial features and contrasting temporal features into the second spatiotemporal cross-attention channel. Calculate the cross-attention respectively, and output the current spatiotemporal fusion features and the contrasting spatiotemporal fusion features. Receive the current spatiotemporal fusion features and the comparison spatiotemporal fusion features, extract the corresponding query matrix, key matrix and value matrix, and calculate the cross-attention matrix of the current channel and the comparison channel respectively; Element-wise difference operations are performed on the cross-attention matrices of the two channels to obtain the difference attention matrix. The difference attention matrix is ​​then used to perform a weighted operation on the time value vectors of the current channel and the comparison channel to generate differential activation features. By concatenating the current spatiotemporal fusion features, the contrastive spatiotemporal fusion features, and the differential activation features along the channel dimension, a comprehensive spatiotemporal difference feature representation is obtained.

7. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 6, characterized in that, The specific steps of inputting the current spatial features and current temporal features into the first spatiotemporal cross-attention channel, inputting the contrasting spatial features and contrasting temporal features into the second spatiotemporal cross-attention channel, calculating the cross-attention respectively, and outputting the current spatiotemporal fusion features and contrasting spatiotemporal fusion features include: Receive current spatial features, comparison spatial features, current time features, and comparison time features; input the current spatial features and current time features into the first spatiotemporal cross-attention channel; input the comparison spatial features and comparison time features into the second spatiotemporal cross-attention channel. In the first spatiotemporal cross-attention channel, the current spatial features are used as the input query matrix, and the current temporal features are used as the input key matrix and value matrix, respectively, to calculate the cross-attention weight matrix of the first channel; The current time features are weighted and summed using the cross-attention weight matrix of the first channel to generate the spatiotemporal fusion features of the first channel, and the output is the current spatiotemporal fusion features; In the second spatiotemporal cross-attention channel, the spatial features of the comparison are used as the input query matrix, and the temporal features of the comparison are used as the input key matrix and value matrix, respectively, to calculate the cross-attention weight matrix of the second channel; The cross-attention weight matrix of the second channel is used to perform weighted summation of the contrast time features to generate the spatiotemporal fusion features of the second channel, and the output is the contrast spatiotemporal fusion features.

8. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 2, characterized in that, The step of inputting the comprehensive spatiotemporal difference feature representation into the regulatory knowledge graph matching module to generate compliance judgment results and corresponding regulatory clause information specifically includes: The system receives a comprehensive spatiotemporal difference feature representation, inputs it into the regulatory knowledge graph matching module, decomposes the input features, and extracts a multi-dimensional vector set. In the knowledge representation layer of the regulatory knowledge graph matching module, a knowledge graph structure containing emission parameter nodes, facility nodes, and regulatory clause nodes is constructed, and a set of triplet relationships is established according to the association between nodes. In the knowledge representation layer, node embedding operations are performed on emission parameter nodes, facility nodes, and regulatory clause nodes respectively to form a node embedding set; In the graph attention inference layer, attention weights between nodes are calculated based on the node embedding set, and the node embeddings are weighted and updated according to the attention weights to generate a node feature set. In the regulatory association decision-making layer, the similarity of emission parameter matching and facility node matching are calculated separately. The two types of similarity are weighted and fused according to the set weight coefficients to calculate the regulatory clause association score. Based on the correlation score of the regulatory clauses, the updated set of embedded regulatory clause nodes is retrieved to generate compliance judgment results and corresponding regulatory clause information.

9. The emission compliance self-inspection and self-correction system based on multimodal fusion according to claim 2, characterized in that, The process of determining the facilities and causes of abnormal emissions based on compliance assessment results and regulatory information, generating a self-correction task, and pushing it to the mobile terminal for execution specifically includes: Receive compliance assessment results and regulatory clause information, parse the non-compliant items in the assessment results and the corresponding regulatory clause numbers, and generate an anomaly association table; In the abnormal association table, based on the constraints of the regulations and the exceedance range of emission parameter characteristics, the corresponding set of abnormal emission facilities is determined, and an abnormality score is calculated for each abnormal facility node. The causes of anomalies are classified and determined based on the numerical range of the anomaly severity score, and a set of rectification targets is generated in combination with the execution conditions of the corresponding legal provisions. A self-correction task structure is constructed based on the set of rectification targets, and the self-correction task structure is pushed to the mobile terminal for execution.

Images