Harmful gas leakage detection method based on artificial intelligence

Abstract

The invention discloses a harmful gas leakage detection method based on artificial intelligence, relates to the technical field of harmful gas leakage detection, and effectively relieves the problems of slow gas diffusion and weak signals in the initial leakage stage through multi-modal data fusion and deep space-time modeling. According to the method, the time convolution network and the space convolution network are combined, the space-time dynamic features in continuous frames are extracted and fused into the composite feature vector, and weak leakage signals are fully captured; instance segmentation is carried out on a leakage gas area through an embedded branch network and a two-dimensional Gaussian model, pixels in the same leakage instance are effectively gathered into an oblique oval cluster, and the segmentation precision is improved; and an online correction mechanism is introduced, and the layout position and orientation of the sensor are adjusted in real time by using multi-modal data and a segmentation result, so that the influence caused by an initial layout error is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of harmful gas leakage detection, and in particular to a harmful gas leakage detection method based on artificial intelligence. Background Art

[0002] In industrial applications, in the initial stage of many harmful gas leaks, the signals are extremely weak. Relying solely on the chemical reactions or temperature changes of single-point sensors to identify leaks is affected by background noise, resulting in unstable signals, false alarms or missed alarms.

[0003] Currently, some solutions increase the sensor density and adopt signal amplification and multi-frame superposition processing methods to address the detection problems in the initial stage of leakage. However, this also increases the cost and maintenance difficulty, and may introduce more errors. On the other hand, in industrial environments, there are often various interference factors such as equipment vibration, temperature fluctuations and occlusions. Even with noise filtering or dynamic compensation techniques, it is difficult to accurately distinguish weak leakage signals from environmental noise in the initial stage of leakage. Therefore, there is an urgent need for a harmful gas leakage detection method that can quickly and accurately capture the weak changes in gas diffusion at the very beginning of the leakage to solve such problems. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] The present invention provides a harmful gas leakage detection method based on artificial intelligence to solve the problems that the current harmful gas leakage detection methods have poor detection effects in the initial stage of harmful gas leakage, and are prone to signal instability, false alarms or missed alarms.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] An embodiment of the present invention provides a harmful gas leakage detection method based on artificial intelligence, which includes:

[0008] Step S1, obtaining image data in a target area by using a sensor;

[0009] Step S2, performing spatial filtering, multi-frame superposition and temporal smoothing processing on the image data to obtain preprocessed data, and improving the distinguishability of low-concentration and low-contrast leakage signals under initial leakage conditions;

[0010] Step S3, based on the preprocessed data, using a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames in the data to form a composite feature vector;

[0011] Step S4, adopting an embedded branch network to map the composite feature vector obtained in step S3 to an embedded space, using a two-dimensional Gaussian model to describe the pixel distribution, and performing instance segmentation of the leakage gas area:

[0012] Output the offset and covariance parameters of each pixel to obtain its coordinates in the embedding space.

[0013] Cluster the pixels within the same leakage instance in an oblique ellipse manner within a two-dimensional Gaussian distribution region, and cluster the pixels in the embedding space based on the clustering loss function to obtain the segmentation results of each leakage instance.

[0014] Step S5: According to the original sensor data obtained in Step S1 and the processing and segmentation results of Steps S2 - S4, adjust the installation position and orientation of the sensor through an online calibration algorithm to achieve leakage localization based on multi-point data fusion.

[0015] As a preferred solution of the method for detecting harmful gas leakage based on artificial intelligence according to the present invention, wherein: the sensor includes an infrared spectral imaging sensor, a thermal imaging sensor, and a visible light sensor.

[0016] The infrared spectral imaging sensor detects the light absorption in a specific band based on the principle of non-dispersive infrared absorption and performs quantitative analysis according to Lambert-Beer's law; the thermal imaging sensor obtains a temperature distribution image; the visible light sensor is used to provide auxiliary visual information.

[0017] As a preferred solution of the method for detecting harmful gas leakage based on artificial intelligence according to the present invention, wherein: in Step S3, model the continuous frame data through a temporal convolutional network to extract temporal features; fuse the temporal features with spatial features to form a composite feature vector.

[0018] As a preferred solution of the method for detecting harmful gas leakage based on artificial intelligence according to the present invention, wherein: in Step S3, the step of using a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data is as follows:

[0019] Let the feature tensor extracted by the spatial convolutional neural network after preprocessing be denoted as F S ∈R T×H′×W′×D , where F S represents the feature tensor extracted after preprocessing, T represents the number of consecutive frames, H′ represents the height of the feature map, W′ represents the width of the feature map, and D represents the number of feature channels.

[0020] For the features of consecutive frames at each spatial position (i, j), use a temporal convolutional network for temporal modeling, and represent the features as:

[0021]

[0022] Among them, T(i, j, t) represents the temporal feature at position (i, j) at time t, i represents the index in the spatial horizontal direction, j represents the index in the spatial vertical direction, t represents the time index, k represents the convolution kernel index in the summation process, K represents the size of the temporal convolution kernel, w k represents the k-th weight parameter in the temporal convolution kernel, d represents the dilation coefficient, and b represents the bias parameter; the calculation formula for the convolution kernel weights is:

[0023]

[0024] Among them, w k represents the k-th weight parameter in the temporal convolution kernel, m represents the index in the summation process, M represents the total number of basis vectors, γ k,m represents the coefficient of the k-th convolution kernel on the m-th basis vector, u m represents the m-th basis vector.

[0025] As a preferred solution of the method for detecting harmful gas leakage based on artificial intelligence according to the present invention, wherein: in step S3, the step of using a deep neural network to extract the spatio-temporal dynamic features of the diffusion of leaked gas in consecutive frames of data further includes:

[0026] After obtaining the temporal features, a gating fusion strategy is adopted to fuse the temporal features with the original spatial features, and its gating function is defined as:

[0027]

[0028] Among them, G(i, j, t) represents the fusion gating coefficient at position (i, j) at time t, σ(·) represents the Sigmoid activation function, W g represents the fusion weight matrix,

[0029] represents the vector obtained by concatenating T(i, j, t) and F S (i, j, t) in the channel dimension, b g represents the fusion bias parameter;

[0030] The expression of the fusion weight matrix is: W g =[W g1 W g2 , where W g1 represents the fusion weight parameter corresponding to the temporal features, and W g2 represents the fusion weight parameter corresponding to the spatial features;

[0031] The two features are fused into a composite feature vector through the gating mechanism, and the expression is:

[0032] F fusion(i,j,t) = G(i,j,t) ⊙ T(i,j,t) + (1 - G(i,j,t)) ⊙ F S (i,j,t),

[0033] where F fusion (i,j,t) represents the composite feature vector at position (i,j) at time t, ⊙ represents element-wise product operation, G(i,j,t), T(i,j,t) and F S (i,j,t) represent the fusion gating coefficient, temporal feature and spatial feature respectively.

[0034] As a preferred scheme of the harmful gas leakage detection method based on artificial intelligence according to the present invention, wherein: in step S4, an embedded branch network is used to perform instance segmentation on the leakage gas area. By learning the two-dimensional Gaussian model parameters and designing a clustering loss function, the pixels within the same instance are gathered into an oblique elliptical area in the embedding space, thereby achieving high-precision instance segmentation.

[0035] As a preferred scheme of the harmful gas leakage detection method based on artificial intelligence according to the present invention, wherein: in step S4, the embedded branch network maps the composite feature vector F fusion (i,j,t) to obtain the offset of each pixel in the embedding space. The formula for the mapping process is:

[0036] δ(i,j,t) = f δ (F fusion (i,j,t)),

[0037] where δ(i,j,t) represents the offset of pixel (i,j) at time t, f δ (·) represents the mapping function used to predict the offset in the embedded branch network, F fusion (i,j,t) represents the composite feature vector obtained in step S3, i and j are the horizontal and vertical spatial indices respectively, and t is the time index;

[0038] According to the predicted offset, the original pixel coordinates p(i,j) are superimposed with the offset to obtain the pixel coordinates in the embedding space:

[0039] c(i,j,t) = p(i,j) + δ(i,j,t),

[0040] where c(i,j,t) represents the embedded coordinates of pixel (i,j) after offset at time t, and p(i,j) represents the spatial coordinates of pixel (i,j) in the original image;

[0041] To describe the distribution of pixels in the embedding space using a two-dimensional Gaussian model, the embedding branch network outputs covariance parameters simultaneously. The oblique ellipse model is adopted, and the covariance matrix of each pixel is parameterized by a rotation matrix, with the expression:

[0042]

[0043] where Σ(i,j,t) represents the covariance matrix of pixel (i,j) at time t, R(θ(i,i,t)) is the rotation matrix, θ(i,j,t) represents the predicted rotation angle used to determine the direction of the oblique ellipse, and σ1(i,j,t) and σ2(i,j,t) represent the standard deviations of the major and minor axes of the ellipse respectively. represents the transpose of the rotation matrix;

[0044] The rotation matrix is defined as:

[0045]

[0046] where cosθ(i,j,t) and sinθ(i,j,t) represent the cosine and sine values of the angle θ(i,j,t) respectively;

[0047] For the learning of the pixel distribution in the embedding space, a clustering loss function is adopted to promote the pixels within the same leakage instance to be closely clustered while maintaining a separation between different instances. The clustering loss function consists of three parts. One is the internal compactness loss, which uses a metric with Mahalanobis distance, and the formula is:

[0048]

[0049] where represents the internal compactness loss, is the set of pixels within the k-th leakage instance, N k is the number of pixels within the k-th instance, is the mean of the embedding coordinates of the k-th instance, and Σ(i,j,t) -1 is the inverse of the covariance matrix;

[0050] Another part is the inter-instance separation loss, which is defined as:

[0051]

[0052] where represents the separation loss between different instances, and are the embedding centers of the k-th and l-th instances respectively, Δ is the preset separation interval threshold, and ||·|| is the Euclidean norm;

[0053] Add a regularization term:

[0054] Among them, represents the covariance regularization loss, Ω is the set of all pixels, I is the identity matrix, and ||·|| F represents the Frobenius norm;

[0055] The overall clustering loss function is:

[0056]

[0057] Among them, λ intra , λ inter and λ cov are the weight coefficients of internal compactness, separation between instances, and covariance regularization loss, respectively.

[0058] As a preferred solution of the method for detecting harmful gas leakage based on artificial intelligence according to the present invention, wherein: in step S5, the multi-modal data of each sensor is fused with the segmentation result; according to the fusion result, the layout position and installation angle of the sensor are corrected in real time to maintain high sensitivity and fast response ability to gas leakage in different leakage stages.

[0059] As a preferred solution of the method for detecting harmful gas leakage based on artificial intelligence according to the present invention, wherein: in step S5, the step of fusing the multi-modal data of each sensor with the segmentation result is:

[0060] The multi-modal data fusion part integrates infrared spectral imaging data, thermal imaging data, visible light data, and the segmentation result into global features. Let the original data of each sensor be D IR , D T , D V , among which, D IR represents infrared spectral imaging data, D T represents thermal imaging data, D V represents visible light data;

[0061] Let the segmentation result be denoted as S seg , and the fusion process adopts weighted summation and feature splicing. The fusion function is defined as:

[0062]

[0063] Among them, F fuse represents the global feature obtained after fusion, represents the multi-modal data fusion function;

[0064] An online correction algorithm is constructed. Let the initial installation position and orientation of the sensor be p and θ, where p ∈ R 2 , θ ∈ R.

[0065] As a preferred solution of the harmful gas leakage detection method based on artificial intelligence described in the present invention, wherein: in step S5, the step of correcting the layout position and installation angle of the sensor in real time according to the fusion result is as follows:

[0066] Introduce adjustment amounts for online calibration, denoted as Δp and Δθ. The sensor parameter calibration uses a transformation function, and the formula is:

[0067] Where the rotation matrix R(θ) is defined as:

[0068]

[0069] The goal of online calibration is to make the fusion feature F fuse (i,j,t) match the predicted area of the sensor after calibration;

[0070] Construct the calibration loss function as:

[0071]

[0072] Where p(i,j) represents the coordinates of the pixel (i,j) in the original image, Ω is the set of pixels to be calibrated, Δp and Δθ respectively represent the adjustment amounts of the sensor position and orientation, ||·|| represents the Euclidean distance, and the calibration loss function is used to measure the deviation between the fusion feature and the predicted area after calibration;

[0073] Minimize through the gradient descent method to obtain the optimal adjustment amounts Δp * and Δθ * .

[0074] The beneficial effects of the present invention are as follows: Through multi-modal data fusion and deep spatio-temporal modeling, the present invention effectively alleviates the problems of slow gas diffusion and weak signals in the initial stage of leakage; the present invention combines a temporal convolutional network and a spatial convolutional network to extract spatio-temporal dynamic features in consecutive frames and fuse them into a composite feature vector to fully capture weak leakage signals; through an embedded branch network and a two-dimensional Gaussian model, instance segmentation of the leakage gas area is performed, effectively clustering pixels within the same leakage instance into an oblique elliptical cluster to improve the segmentation accuracy; an online calibration mechanism is introduced to use multi-modal data and the segmentation result to adjust the sensor layout position and orientation in real time, reducing the influence brought by the initial layout error.

[0075] The present invention not only improves the noise interference and low contrast problems in the initial detection of traditional methods, but also improves the real-time performance and accuracy of leakage positioning. Description of the Drawings

[0076] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0077] Figure 1 It is a schematic flowchart of the method for detecting harmful gas leakage based on artificial intelligence in Embodiment 1. Specific Embodiments

[0078] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the accompanying drawings of the specification.

[0079] In the following description, many specific details are set forth to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0080] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that can be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that mutually excludes other embodiments.

[0081] Embodiment 1, referring to Figure 1 , this embodiment provides a method for detecting harmful gas leakage based on artificial intelligence, including the following steps:

[0082] Step S1, use a sensor to obtain image data in the target area;

[0083] The sensor includes an infrared spectral imaging sensor, a thermal imaging sensor, and a visible light sensor;

[0084] The infrared spectral imaging sensor detects the light absorption in a specific band based on the principle of non-dispersive infrared absorption and performs quantitative analysis based on Lambert-Beer's law; the thermal imaging sensor obtains a temperature distribution image; the visible light sensor is used to provide auxiliary visual information;

[0085] Step S2, perform spatial filtering, multi-frame superposition, and temporal smoothing processing on the image data to obtain preprocessed data, and improve the recognizability of low-concentration and low-contrast leakage signals under initial leakage conditions;

[0086] Step S3: Based on the preprocessed data, use a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data, and form a composite feature vector;

[0087] In step S3, a temporal convolutional network is used to model the consecutive frame data to extract temporal features; the temporal features are fused with the spatial features to form a composite feature vector;

[0088] In step S3, the steps of using a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data are as follows:

[0089] Let the feature tensor extracted by the spatial convolutional neural network after preprocessing be denoted as F S ∈R T×H′×W′×D where F S represents the feature tensor extracted after preprocessing, T represents the number of consecutive frames, H′ represents the height of the feature map, W′ represents the width of the feature map, and D represents the number of feature channels;

[0090] For the features of consecutive frames at each spatial position (i,j), a temporal convolutional network is used for temporal modeling, and the features are represented as:

[0091]

[0092] where T(i,j,t) represents the temporal feature at position (i,j) at time t, i represents the index in the spatial horizontal direction, j represents the index in the spatial vertical direction, t represents the time index, k represents the convolutional kernel index in the summation process, K represents the size of the temporal convolutional kernel, w k represents the k-th weight parameter in the temporal convolutional kernel, d represents the dilation coefficient, and b represents the bias parameter; the convolutional kernel weight calculation formula is:

[0093]

[0094] where w k represents the k-th weight parameter in the temporal convolutional kernel, m represents the index in the summation process, M represents the total number of basis vectors, γ k,m represents the coefficient of the k-th convolutional kernel on the m-th basis vector, and u m represents the m-th basis vector;

[0095] In step S3, the steps of using a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data further include:

[0096] After obtaining the temporal features, a gating fusion strategy is adopted to fuse the temporal features with the original spatial features, and its gating function is defined as:

[0097]

[0098] Among them, G(i, j, t) represents the fusion gating coefficient at position (i, j) at time t, σ(·) represents the Sigmoid activation function, and W g represents the fusion weight matrix,

[0099] represents the vector obtained by concatenating T(i, j, t) and F S (i, j, t) along the channel dimension, and b g represents the fusion bias parameter;

[0100] The expression of the fusion weight matrix is: W g =[W g1 W g2 , where W g1 represents the fusion weight parameter corresponding to the temporal feature, and W g2 represents the fusion weight parameter corresponding to the spatial feature;

[0101] The two features are fused into a composite feature vector through a gating mechanism, and the expression is:

[0102] F fusion (i, j, t) = G(i, j, t) ⊙ T(i, j, t) + (1 - G(i, j, t)) ⊙ F S (i, j, t),

[0103] where F fusion (i, j, t) represents the composite feature vector at position (i, j) at time t, ⊙ represents the element-wise product operation, and G(i, j, t), T(i, j, t), and F S (i, j, t) represent the fusion gating coefficient, temporal feature, and spatial feature respectively;

[0104] Specifically, a temporal convolutional network is constructed here to model the temporal dynamic changes of consecutive frames, which can capture the temporal features in the process of leaking gas diffusion. Dilated convolutions are used to widen the temporal receptive field and enhance the ability to capture long-range dependence information. The basis vector decomposition strategy decomposes the weights of the convolutional kernels, effectively reducing parameter redundancy and improving expression flexibility. The fusion process automatically regulates the weights between the temporal feature and the spatial feature through a gating mechanism, making the advantages of both complementary and improving the discrimination ability of the overall feature representation;

[0105] Step S4, an embedding branch network is used to map the composite feature vector obtained in step S3 to the embedding space, and the pixel distribution is described using a two-dimensional Gaussian model to perform instance segmentation of the leaking gas region:

[0106] The offset and covariance parameters of each pixel are output to obtain its coordinates in the embedding space,

[0107] Cluster the pixels within the same leakage instance in an oblique ellipse manner within a two-dimensional Gaussian distribution region, and cluster the pixels in the embedding space based on the clustering loss function to obtain the segmentation results of each leakage instance;

[0108] In step S4, use the embedding branch network to perform instance segmentation on the leakage gas region. By learning the two-dimensional Gaussian model parameters and designing the clustering loss function, the pixels within the same instance are gathered into an oblique elliptical region in the embedding space, thereby achieving high-precision instance segmentation;

[0109] In step S4, the embedding branch network maps the composite feature vector F fusion (i, j, t) obtained in step S3 to obtain the offset of each pixel in the embedding space. The formula for the mapping process is:

[0110] δ(i, j, t) = f δ (F fusion (i, j, t)),

[0111] where δ(i, j, t) represents the offset of pixel (i, j) at time t, and f δ (·) represents the mapping function used to predict the offset in the embedding branch network, F fusion (i, j, t) represents the composite feature vector obtained in step S3, i and j are the horizontal and vertical spatial indices respectively, and t is the time index;

[0112] According to the predicted offset, superimpose the original pixel coordinates p(i, j) and the offset to obtain the pixel coordinates in the embedding space:

[0113] c(i, j, t) = p(i, j) + δ(i, j, t),

[0114] where c(i, j, t) represents the embedded coordinates of pixel (i, j) after offset at time t, and p(i, j) represents the spatial coordinates of pixel (i, j) in the original image;

[0115] In order to describe the distribution of pixels in the embedding space using a two-dimensional Gaussian model, the embedding branch network simultaneously outputs covariance parameters, which are described using an oblique ellipse model. The covariance matrix of each pixel is parameterized using a rotation matrix, and the expression is:

[0116]

[0117] where Σ(i, j, t) represents the covariance matrix of pixel (i, j) at time t, R(θ(i, j, t)) is the rotation matrix, θ(i, j, t) represents the predicted rotation angle used to determine the direction of the oblique ellipse, and σ1(i, j, t) and σ2(i, j, t) respectively represent the standard deviations of the major and minor axes of the ellipse, denotes the transpose of the rotation matrix;

[0118] The rotation matrix is defined as:

[0119]

[0120] where cosθ(i,j,t) and sinθ(i,j,t) denote the cosine and sine values of the angle θ(i,j,t), respectively;

[0121] For the learning of the pixel distribution in the embedding space, a clustering loss function is adopted to promote the pixels within the same leakage instance to be closely clustered while maintaining a gap between different instances. The clustering loss function consists of three parts. One is the internal compactness loss, which uses a metric with Mahalanobis distance, and the formula is:

[0122]

[0123] where denotes the internal compactness loss, is the set of pixels within the k-th leakage instance, N k is the number of pixels within the k-th instance, is the mean of the embedding coordinates of the k-th instance, Σ(i,j,t) -1 is the inverse of the covariance matrix;

[0124] Another part is the inter-instance separation loss, which is defined as:

[0125]

[0126] where denotes the separation loss between different instances, and are the embedding centers of the k-th and l-th instances respectively, Δ is a preset separation interval threshold, and ||·|| is the Euclidean norm;

[0127] Add a regularization term:

[0128] where denotes the covariance regularization loss, Ω is the set of all pixels, I is the identity matrix, and ||·|| F denotes the Frobenius norm;

[0129] The overall clustering loss function is:

[0130]

[0131] where λ intra 、λ inter and λ covThey are the weight coefficients of internal compactness, separation between instances, and covariance regularization loss, respectively;

[0132] Specifically, the composite features are mapped to the embedding space through the embedding branch network, and at the same time, the offset and covariance parameters are output. The two-dimensional Gaussian model is used to characterize the pixel distribution, parameterized by the rotation matrix, effectively capturing the oblique elliptical distribution characteristics of the leakage instances. The pixels within the same instance gather into directional clusters. In the design of the clustering loss function, the internal compactness loss uses the Mahalanobis distance to measure the similarity between pixels and the cluster center, while the separation loss between instances ensures sufficient differentiation between the embedding centers of different instances. The covariance regularization term prevents covariance degradation, achieving high-sensitivity and robust leakage gas region segmentation;

[0133] Step S5: According to the original sensor data obtained in step S1 and the processing and segmentation results of steps S2 - S4, the installation position and orientation of the sensor are adjusted through an online calibration algorithm to achieve leakage positioning based on multi-point data fusion;

[0134] In step S5, the multi-modal data of each sensor is fused with the segmentation result; the layout position and installation angle of the sensor are corrected in real time according to the fusion result to maintain high sensitivity and fast response ability to gas leakage in different leakage stages;

[0135] In step S5, the steps of fusing the multi-modal data of each sensor with the segmentation result are as follows:

[0136] The multi-modal data fusion part integrates the infrared spectral imaging data, thermal imaging data, visible light data, and the segmentation result into global features. Let the original data of each sensor be D IR , D T , D V , where D IR represents the infrared spectral imaging data, D T represents the thermal imaging data, and D V represents the visible light data;

[0137] Let the segmentation result be denoted as S seg , and the fusion process adopts weighted summation and feature splicing, and the fusion function is defined as:

[0138]

[0139] where F fuse represents the global feature obtained after fusion, represents the multi-modal data fusion function;

[0140] An online calibration algorithm is constructed. Let the initial installation position and orientation of the sensor be p and θ respectively, where p ∈ R 2 , θ ∈ R;

[0141] In step S5, the steps of correcting the layout position and installation angle of the sensor in real time according to the fusion result are as follows:

[0142] Introduce adjustment amounts for online calibration, denoted as Δp and Δθ. The sensor parameter calibration uses a transformation function, and the formula is:

[0143] Among them, the rotation matrix R(θ) is defined as:

[0144]

[0145] The goal of online calibration is to make the fused feature F fuse (i, j, t) match the predicted area of the sensor after calibration;

[0146] Construct the calibration loss function as:

[0147]

[0148] Among them, p(i, j) represents the coordinates of the pixel (i, j) in the original image, Ω is the set of pixels to be calibrated, Δp and Δθ respectively represent the adjustment amounts of the sensor position and orientation, ||·|| represents the Euclidean distance, and the calibration loss function is used to measure the deviation between the fused feature and the predicted area after calibration;

[0149] Minimize through the gradient descent method to obtain the optimal adjustment amounts Δp * and Δθ * ;

[0150] Specifically, here, the multi-modal data and the instance segmentation result are efficiently fused, and based on this, an online calibration loss is constructed. Through the fusion function, the advantages of infrared, thermal imaging, and visible light data are complementary to form a global feature expression. The calibration algorithm uses a transformation model composed of the initial parameters of the sensor and the adjustment amounts. By minimizing the calibration loss, the sensor position and orientation are corrected in real time. This online optimization mechanism can dynamically update the layout scheme according to real-time data at different development stages of the leaked gas, thereby reducing the impact brought by the initial layout deviation of the sensor and improving the overall detection sensitivity.

[0151] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An artificial intelligence-based harmful gas leakage detection method, characterized in that: including Step S1: Obtain image data within a target area using a sensor; Step S2: Perform spatial filtering, multi-frame superposition, and temporal smoothing on the image data to obtain preprocessed data; Step S3: Based on the preprocessed data, use a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data, and form a composite feature vector; Step S4: Adopt an embedding branch network to map the composite feature vector obtained in Step S3 to an embedding space, use a two-dimensional Gaussian model to describe the pixel distribution, and perform instance segmentation of the leakage gas area: Output the offset and covariance parameters of each pixel to obtain its coordinates in the embedding space, Cluster the pixels within the same leakage instance in an oblique ellipse manner within the two-dimensional Gaussian distribution area, Perform clustering on the pixels in the embedding space based on the clustering loss function to obtain the segmentation results of each leakage instance; Step S5: According to the original sensor data obtained in Step S1 and the processing and segmentation results of Steps S2 - S4, adjust the installation position and orientation of the sensor through an online calibration algorithm.

2. The method for detecting harmful gas leakage based on artificial intelligence according to claim 1, characterized in that: The sensor includes an infrared spectral imaging sensor, a thermal imaging sensor, and a visible light sensor; The infrared spectral imaging sensor detects the light absorption in a specific band based on the principle of non-dispersive infrared absorption and performs quantitative analysis based on Lambert-Beer's law; the thermal imaging sensor obtains a temperature distribution image; the visible light sensor is used to provide auxiliary visual information.

3. The method for detecting harmful gas leakage based on artificial intelligence according to claim 1, characterized in that: In Step S3, a temporal convolutional network is used to model the consecutive frame data to extract temporal features; Fuse the temporal features with the spatial features to form a composite feature vector.

4. The method for detecting harmful gas leakage based on artificial intelligence according to claim 3, characterized in that: In Step S3, the step of using a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data is as follows: Let the feature tensor extracted by the spatial convolutional neural network after preprocessing be denoted as F S ∈R T×H'×W'×D , where F S represents the feature tensor extracted after preprocessing, T represents the number of consecutive frames, H′ represents the height of the feature map, W′ represents the width of the feature map, and D represents the number of feature channels; For the features of consecutive frames at each spatial position (i, j), a temporal convolutional network is used for temporal modeling, and the features are represented as: Among them, T(i, j, t) represents the temporal feature at position (i, j) at time t, i represents the index in the spatial horizontal direction, j represents the index in the spatial vertical direction, t represents the time index, k represents the convolution kernel index in the summation process, K represents the size of the temporal convolution kernel, w k represents the k-th weight parameter in the temporal convolution kernel, d represents the dilation coefficient, and b represents the bias parameter; the convolution kernel weight calculation formula is: Among them, w k represents the k-th weight parameter in the temporal convolution kernel, m represents the index in the summation process, M represents the total number of basis vectors, γ k,m represents the coefficient of the k-th convolution kernel on the m-th basis vector, u m represents the m-th basis vector.

5. The method for detecting harmful gas leakage based on artificial intelligence according to claim 4, characterized in that: In Step S3, the step of using a deep neural network to extract the spatio-temporal dynamic features of the leakage gas diffusion in consecutive frames of the data further includes: After obtaining the temporal features, adopt a gating fusion strategy to fuse the temporal features with the original spatial features, and its gating function is defined as: Among them, G(i, j, t) represents the fusion gating coefficient at position (i, j) at time t, σ(·) represents the Sigmoid activation function, and W g represents the fusion weight matrix, Denote concatenating T(i, j, t) with F S The vector after concatenating (i, j, t) in the channel dimension, b g Denote the fusion bias parameter; The expression of the fusion weight matrix is: W g = [W g1 W g2 , where W g1 represents the fusion weight parameter corresponding to the temporal feature, and W g2 represents the fusion weight parameter corresponding to the spatial feature; Fuse the two types of features into a composite feature vector through a gating mechanism, and the expression is: F fusion (i, j, t) = G(i, j, t) ⊙ T(i, j, t) + (1 - G(i, j, t)) ⊙ F S (i, j, t), Among them, F fusion (i,j,t) represents the composite feature vector at position (i,j) at time t. ⊙ represents the element-wise product operation. G(i,j,t), T(i,j,t), and F S (i,j,t) represent the fusion gating coefficient, the temporal feature, and the spatial feature respectively.

6. The method for detecting harmful gas leakage based on artificial intelligence according to claim 1, characterized in that: In Step S4, use an embedding branch network to perform instance segmentation of the leakage gas area. By learning the two-dimensional Gaussian model parameters and designing a clustering loss function, the pixels within the same instance are gathered into an oblique elliptical area in the embedding space.

7. The method for detecting harmful gas leakage based on artificial intelligence according to claim 6, characterized in that: In step S4, the embedding branch network maps the composite feature vector F fusion (i, j, t) obtained in step S3 to obtain the offset of each pixel in the embedding space. The formula for the mapping process is: δ(i,j,t) = f δ (F fusion (i,j,t)), Among them, δ(i,j,t) represents the offset of pixel (i,j) at time t, and f δ (·) represents the mapping function used to predict the offset in the embedding branch network, and F fusion (i,j,t) represents the composite feature vector obtained in step S3, where i and j are the horizontal and vertical spatial indices respectively, and t is the time index; According to the predicted offset, superimpose the original pixel coordinates p(i, j) with the offset to obtain the pixel coordinates in the embedding space: c(i, j, t) = p(i, j) + δ(i, j, t), where c(i, j, t) represents the embedded coordinates of pixel (i, j) after offset at time t, and p(i, j) represents the spatial coordinates of pixel (i, j) in the original image; The embedding branch network simultaneously outputs covariance parameters, which are described by an oblique ellipse model. The covariance matrix of each pixel is parameterized by a rotation matrix, and the expression is: Among them, Σ(i,j,t) represents the covariance matrix of pixel (i,j) at time t, R(θ(i,j,t)) is the rotation matrix, and θ(i,j,t) represents the predicted rotation angle, which is used to determine the direction of the oblique ellipse. σ1(i,j,t) and σ2(i,j,t) respectively represent the standard deviations of the major and minor axes of the ellipse. represents the transpose of the rotation matrix; The rotation matrix is defined as: where cosθ(i,j,t) and sinθ(i,j,t) represent the cosine and sine values of the angle θ(i,j,t), respectively; For the learning of the pixel distribution in the embedding space, a clustering loss function is adopted to promote the pixels within the same leakage instance to be closely clustered while maintaining a gap between different instances. The clustering loss function consists of three parts. One is the internal compactness loss, which uses a metric with Mahalanobis distance, and the formula is: Among them, represents the internal compactness loss, is the set of pixels within the k-th leakage instance, N k is the number of pixels within the k-th instance, is the mean of the embedding coordinates of the k-th instance, Σ(i,j,t) -1 is the inverse of the covariance matrix; Another part is the inter-instance separation loss, which is defined as: Among them, represents the separation loss between different instances, and are the embedding centers of the k-th and l-th instances respectively, Δ is a preset separation interval threshold, and ||·|| is the Euclidean norm; Add a regularization term: Among them, represents the covariance regularization loss, Ω is the set of all pixels, I is the identity matrix, and ||·|| F represents the Frobenius norm; The overall clustering loss function is: Among them, λ intra , λ inter and λ cov are the weight coefficients of internal compactness, separation between instances, and covariance regularization loss, respectively.

8. The method for detecting harmful gas leakage based on artificial intelligence according to claim 1, characterized in that: In step S5, the multi-modal data of each sensor is fused with the segmentation result; the layout position and installation angle of the sensor are corrected in real time according to the fusion result.

9. The method for detecting harmful gas leakage based on artificial intelligence according to claim 8, characterized in that: In step S5, the step of fusing the multi-modal data of each sensor with the segmentation result is: The multi-modal data fusion part integrates infrared spectral imaging data, thermal imaging data, visible light data, and segmentation results into global features. Let the original data of each sensor be D IR , D T , D V , where D IR represents infrared spectral imaging data, D T represents thermal imaging data, and D V represents visible light data; Let the segmentation result be denoted as S seg , and the fusion process uses weighted summation and feature concatenation. The fusion function is defined as: Among them, F fuse represents the global feature obtained after fusion, represents the multi-modal data fusion function; Construct an online calibration algorithm. Let the initial installation position and orientation of the sensor be \(p\) and \(\theta\) respectively, where \(p\in R\) 2 , \(\theta\in R\).

10. The method for detecting harmful gas leakage based on artificial intelligence according to claim 9, wherein: In step S5, the step of correcting the layout position and installation angle of the sensor in real time according to the fusion result is: Introduce adjustment amounts for online calibration, denoted as Δp and Δθ. The sensor parameter calibration uses a transformation function, and the formula is: where the rotation matrix R(θ) is defined as: The goal of online calibration is to make the fused feature F fuse (i, j, t) match the sensor prediction area after calibration; Construct the calibration loss function as: where p(i,j) represents the coordinates of the pixel (i,j) in the original image, Ω is the set of pixels to be calibrated, Δp and Δθ represent the adjustment amounts of the sensor position and orientation respectively, ||·|| represents the Euclidean distance, and the calibration loss function is used to measure the deviation between the fused feature and the predicted region after calibration; Minimize by gradient descent method to obtain the optimal adjustment amount Δp * and Δθ * .

Images