Infrared gas detection model training method and device based on self-supervised learning

Abstract

The application relates to the field of gas detection, in particular to an infrared gas detection model training method and device based on self-supervised learning. The method comprises the following steps: acquiring a gas diffusion infrared video, acquiring a training image pair comprising adjacent current frame images and next frame images from the gas diffusion infrared video, acquiring a background image based on the gas diffusion infrared video, obtaining a current frame supervision signal corresponding to a current frame difference image and a next frame supervision signal corresponding to a next frame difference image according to the background image, constructing a training data set according to the current frame images, the next frame images, the current frame supervision signal and the next frame supervision signal, and training to obtain an infrared gas detection model based on the training data set. The infrared gas detection model training method and device based on self-supervised learning, the electronic equipment and the computer readable storage medium provided by the application realize the technical effect of reducing the training cost of the infrared gas detection model.

Description

Technical Field

[0001] This application relates to the field of gas detection, specifically to a method and apparatus for training an infrared gas detection model based on self-supervised learning, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Infrared gas detection technology has significant application value in fields such as petrochemicals, environmental monitoring, and industrial safety. Because gases (such as methane and sulfur hexafluoride) exhibit significant absorption characteristics to specific wavelengths of infrared radiation, mid-wave infrared thermal imagers can be used to achieve remote, non-contact, and visual detection of gas leaks. However, gases in infrared images typically appear as targets with weak contrast, blurred boundaries, and irregular shapes; their grayscale difference from the background is often only a few grayscale levels, posing a significant challenge to automated detection.

[0003] Existing infrared gas detection methods mainly include traditional image processing methods and methods based on deep learning models. Traditional image processing methods rely on background subtraction, optical flow estimation, or temporal filtering, which are parameter-sensitive and have weak noise resistance, making them difficult to adapt to complex industrial environments. While deep learning model segmentation methods have strong noise resistance and are suitable for complex industrial environments, the training process of deep learning models relies on large-scale pixel-level annotation of the training data. Due to the blurred gas boundaries and their rapid changes over time, accurate annotation is difficult, costly, and inconsistent, resulting in high model training costs. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and apparatus, electronic device and computer-readable storage medium for training infrared gas detection models based on self-supervised learning, so as to achieve the technical effect of reducing the training cost of infrared gas detection models.

[0005] To address the aforementioned technical problems, firstly, this application provides a method for training an infrared gas detection model based on self-supervised learning, comprising: Acquire a gas diffusion infrared video, and acquire multiple training image pairs from the gas diffusion infrared video, wherein the training image pairs include adjacent current frame images and next frame images; Background images are obtained based on the gas diffusion infrared video; For any of the training image pairs, calculate the current frame difference image between the current frame image and the background image, calculate the next frame difference image between the next frame image and the background image, and stitch the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image; A first quantile threshold is obtained by performing quantile mapping on the connected difference image using a first set quantile operator, and a second quantile threshold is obtained by performing quantile mapping on the connected difference image using a second set quantile operator. The current frame supervision signal corresponding to the current frame difference image is calculated based on the first quantile threshold, the second quantile threshold, and the current frame difference image; the next frame supervision signal corresponding to the next frame difference image is calculated based on the first quantile threshold, the second quantile threshold, and the next frame difference image. Construct a training dataset, which includes multiple training data, each of which includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal; The basic model is trained based on the training dataset to obtain the infrared gas detection model.

[0006] In one possible embodiment, the step of calculating the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculating the next frame supervision signal corresponding to the next frame difference image based on the first quantile threshold, the second quantile threshold, and the next frame difference image, includes: Based on formula , The current frame monitoring signal and the next frame monitoring signal are calculated. Where t is the time identifier. The current frame monitoring signal, This is the monitoring signal for the next frame. To prevent division by zero constant, To crop the results to Within the range, For the current frame differential image, For the next frame of the differential image, The first quantile threshold, The second quantile threshold is used, and the first quantile threshold is not greater than the second quantile threshold.

[0007] In one possible embodiment, acquiring the background image based on the gas diffusion infrared video includes: Based on a set sliding window, the gas diffusion infrared video is sampled to obtain multiple sampled infrared images; Acquire pixel data for each pixel in each of the sampled infrared images to form a pixel dataset corresponding to each pixel; The sampling quantile operator is used to perform quantile mapping on the pixel dataset to obtain sampled pixel data, and the background image corresponding to the sampled infrared image is constructed based on the sampled pixel data.

[0008] In one possible embodiment, training the base model based on the training dataset includes: Construct a loss function, which includes a physical consistency term L. phys The base model is trained based on the training dataset until the loss function converges; The physical consistency item includes: , , ; in, The height of the current frame image. The width of the current frame image. For Huber's robust penalty function, As a preset constant, These are the pixel coordinates. For discrete residuals, For spatial gradient operators, For divergence operators, The base model outputs a two-dimensional motion velocity field corresponding to the current frame image. The base model outputs the gas diffusion coefficient corresponding to the current frame image. The base model outputs the source term corresponding to the current frame image. The base model outputs a normalized gas concentration image corresponding to the current frame image. The base model outputs a normalized gas concentration image corresponding to the next frame image. For setting values, it means and The mean.

[0009] In one possible embodiment, it further includes: Provides a maximum speed setting, and sets the two-dimensional motion velocity field. The absolute value of each velocity component is less than or equal to the set maximum velocity; Provides a setting for the maximum diffusion coefficient, and sets the gas diffusion coefficient. It is less than or equal to the set maximum diffusion coefficient.

[0010] In one possible embodiment, the loss function further includes a data fidelity term. The data fidelity item include: ; in, The current frame monitoring signal, This is the monitoring signal for the next frame.

[0011] In one possible embodiment, the loss function further includes a smoothing regularization term. The smoothing regularization term include: ,in, , The gradient of velocity in two-dimensional motion. This represents the gradient of the gas diffusion coefficient output by the base model corresponding to the current frame image.

[0012] Secondly, this application provides a training device for an infrared gas detection model based on self-supervised learning, comprising: An image acquisition module is used to acquire gas diffusion infrared video and acquire multiple training image pairs from the gas diffusion infrared video. The training image pairs include adjacent current frame images and next frame images. Background separation module, the background separation module is used to acquire a background image based on the gas diffusion infrared video; The self-supervised module, for any training image pair, calculates the current frame difference image between the current frame image and the background image, calculates the next frame difference image between the next frame image and the background image, and concatenates the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image; performs quantile mapping on the connected difference image using a first set quantile operator to obtain a first quantile threshold, and performs quantile mapping on the connected difference image using a second set quantile operator to obtain a second quantile threshold; calculates the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculates the next frame supervision signal corresponding to the next frame difference image based on the first quantile threshold, the second quantile threshold, and the next frame difference image. A training data construction module is used to construct a training dataset, which includes multiple training data, each of which includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal. The model training module is used to train a base model based on the training dataset to obtain the infrared gas detection model.

[0013] Thirdly, this application also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the self-supervised learning-based infrared gas detection model training method described in any of the above implementations.

[0014] Fourthly, this application also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps in the self-supervised learning-based infrared gas detection model training method described in any of the above implementations.

[0015] The beneficial effects of this application are: Compared with related technologies, the infrared gas detection model training method and apparatus, electronic device, and computer-readable storage medium based on self-supervised learning provided in this application acquire a background image based on gas diffusion infrared video, then calculate the current frame difference image between the current frame image and the background image, calculate the next frame difference image between the next frame image and the background image, stitch the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image, use a first set quantile operator to perform quantile mapping on the connected difference image to obtain a first quantile threshold, use a second set quantile operator to perform quantile mapping on the connected difference image to obtain a second quantile threshold; calculate the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculate the current frame supervision signal based on the first quantile threshold. The next frame supervision signal corresponding to the next frame difference image is obtained by calculating the second quantile threshold and the next frame difference image. In this way, the current frame supervision signal and the next frame supervision signal corresponding to the current frame image and the next frame image can be directly calculated from the gas diffusion infrared video. Subsequently, the current frame image, the next frame image, the current frame supervision signal and the next frame supervision signal can be integrated to obtain a complete training data. The training dataset can be directly generated from the gas diffusion infrared video to train the basic model without labeling, and the infrared gas detection model is obtained. By separating the background image and performing differential normalization, the corresponding supervision signal is obtained, realizing label-free self-supervised learning of the infrared gas detection model, thereby reducing the cost of obtaining the training dataset and achieving the technical effect of reducing the training cost of the infrared gas detection model. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart illustrating the training method for an infrared gas detection model based on self-supervised learning provided in an embodiment of this application. Figure 2 This is a schematic diagram illustrating the process of obtaining a background image in the training method for an infrared gas detection model based on self-supervised learning provided in the embodiments of this application. Figure 3 A schematic diagram of the structure of the infrared gas detection model training device based on self-supervised learning provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0019] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] The terms "first," "second," etc., used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] This application provides a method and apparatus for training an infrared gas detection model based on self-supervised learning, as well as an electronic device and a computer-readable storage medium, which are described below.

[0023] Please refer to Figure 1The infrared gas detection model training method based on self-supervised learning provided in this application includes: Step S101: Acquire gas diffusion infrared video, and acquire multiple training image pairs from the gas diffusion infrared video. The training image pairs include adjacent current frame images and next frame images.

[0024] In this step, the gas diffusion infrared video comes from a publicly available infrared gas detection dataset. The gas diffusion infrared video consists of multiple frames of temporally consecutive infrared images, with each frame recording the infrared radiation intensity distribution at the corresponding moment. Alternatively, in some other embodiments of this application, the gas diffusion infrared video can also be obtained from actual infrared images of a scene containing gas diffusion acquired by a local infrared camera device such as a mid-wave infrared thermal imager.

[0025] Furthermore, in the embodiments of this application, the gas diffusion infrared video can be a single, complete, continuous video, or it can be an infrared video sampled and extracted from a long infrared video segment that includes the gas diffusion process. For example, a sliding window of a certain length is set, and the length of the gas diffusion infrared video is set to several times the length of the sliding window. Based on this sliding window, window sampling is performed from the long infrared video segment to obtain a sampled video covering different leakage intensities, different background textures, and different wind field conditions, while simultaneously incorporating images without gas, as the gas diffusion infrared video.

[0026] Based on infrared video of gas diffusion, select any moment from it. The two adjacent frames are respectively denoted as the current frame image. With the next frame image Current frame image With the next frame image Common composition and time The corresponding training image pairs. The same image selection is performed at multiple time points to obtain multiple training image pairs corresponding to different time points.

[0027] Step S102: Obtain background image based on gas diffusion infrared video.

[0028] In this step, please refer to Figure 2 Obtaining the background image specifically includes the following steps: Step S201: Perform window sampling on the gas diffusion infrared video based on the set sliding window to obtain multiple sampled infrared images.

[0029] In this step, the length is pre-set based on the length of the gas diffusion infrared video. Frame, and the setting sliding window corresponding to the sampling time. Based on this setting, the sliding window Window sampling was performed on the gas diffusion infrared video to obtain multiple sampled infrared images.

[0030] Among them, setting a sliding window Corresponding to the sampling time, multiple sampling times are preset. A sliding window is then used based on this setting. Window sampling is performed on the gas diffusion infrared video, that is, when the time sequence of the gas diffusion infrared video reaches any sampling time, a set sliding window is used. Window sampling is performed based on the video frames corresponding to the sampling time.

[0031] Step S202: Obtain the pixel data of each pixel in each sampled infrared image to form a pixel dataset corresponding to each pixel.

[0032] In this step, each sampled infrared image includes multiple pixels. Since all sampled infrared images are obtained from gas diffusion infrared video, the pixel distribution in all sampled infrared images is identical. Based on this, for each pixel, it corresponds to one pixel data point in each sampled infrared image, and all the pixel data points corresponding to each pixel constitute the pixel dataset corresponding to that pixel.

[0033] Step S203: Use the sampling quantile operator to perform quantile mapping on the pixel dataset to obtain sampled pixel data, and construct a background image corresponding to the sampled infrared image based on the sampled pixel data.

[0034] In this step, for each pixel, the sampling quantile operator is used to perform quantile mapping on the pixel dataset for that pixel, obtaining the sampled pixel data for that pixel. Further, the sampled pixel data is used to fill in the corresponding pixels to obtain the background image.

[0035] It is understood that the aforementioned method of obtaining the same background image based on all sampled infrared images is merely a specific example in this embodiment. In some other embodiments of this application, the background image may also be updated based on the sampled infrared images corresponding to each sampling time. That is, for the first sampling time, according to the set sliding window... The first sampling is performed to obtain the corresponding first-sample infrared image. Based on the aforementioned steps, the corresponding first-sample background image is calculated. For the subsequent second sampling time, the corresponding second-sampled background image is calculated using the same method. Based on the set fusion coefficient, the first sampled background image and the second sampled background image A weighted calculation is performed, and the result is used as the background image corresponding to the second sampling time. The specific formula is as follows: , .in, Sampling time, To set the fusion coefficient, These are the pixel coordinates. The number of the sampled infrared image. For the number Sampling infrared images, For pixels The corresponding pixel dataset, For the first Sampling quantile operator, For pixels At sampling time The corresponding sampled pixel data.

[0036] Among them, the sampling quantile operator The specific selection depends on the apparent characteristics of the gas imaging: when the gas appears as a bright target above the background, a low quantile (e.g., ...) is used. When the gas appears as a dark target below the background, a high quantile (e.g., Optionally, when a significant change is detected in the sampled infrared image corresponding to consecutive sampling times, the update of the fusion coefficient can be paused or reduced. This is to reduce data contamination of the background image.

[0037] Step S103: For any training image pair, calculate the current frame difference image between the current frame image and the background image, calculate the next frame difference image between the next frame image and the background image, and stitch the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image.

[0038] In this step, the pixel data of the current frame image and the background image are directly subtracted, and the difference image of the current frame is constructed based on the difference result. Similarly, the pixel data of the next frame image and the background image are directly subtracted, and the difference image of the next frame is constructed based on the difference result.

[0039] Pixel data subtraction involves calculating the difference between the pixel data of the same pixel in the current frame / next frame and the background image. The formula is expressed as: .in, Background image, For the current frame image, For the next frame image, For the current frame difference image, This is the next frame of the differential image.

[0040] The concatenated difference image is obtained by stitching the current frame difference image and the next frame difference image along the spatial dimension. This is achieved by stitching the current frame difference image and the next frame difference image together using the concat function, expressed as follows: .in, To connect the difference images.

[0041] Step S104: Use the first set quantile operator to perform quantile mapping on the connected difference image to obtain the first quantile threshold, and use the second set quantile operator to perform quantile mapping on the connected difference image to obtain the second quantile threshold.

[0042] In this step, the specific formula is expressed as follows: .in, The first quantile operator is defined. For the second quantile operator, The first quantile threshold, The second quantile threshold, and the first quantile threshold Not greater than the second quantile threshold .

[0043] Step S105: Calculate the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image; calculate the next frame supervision signal corresponding to the next frame difference image based on the first quantile threshold, the second quantile threshold, and the next frame difference image.

[0044] In this step, the specific formula is used. , The current frame monitoring signal and the next frame monitoring signal are calculated.

[0045] in, This is the current frame monitoring signal. For the monitoring signal of the next frame, To prevent division by zero constant, To crop the results to Within the range.

[0046] In this embodiment, < Percentiles are set according to specific scenarios. for .

[0047] Step S106: Construct a training dataset. The training dataset includes multiple training data, each of which includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal.

[0048] In this step, for any time t, the current frame image Next frame image Current frame monitoring signal and the monitoring signal for the next frame . The current frame image Next frame image Current frame monitoring signal and the monitoring signal for the next frame Corresponding storage, formation time The corresponding training data, all time points The corresponding training data constitute the training dataset.

[0049] Step S107: Train the basic model based on the training dataset to obtain the infrared gas detection model.

[0050] In this step, the base model is an initialized model that only has the model architecture and has not been trained. The base model may specifically include one or more of the shared encoder module, the segmentation decoding branch module, and the physical parameter decoding branch module.

[0051] In this embodiment, the shared encoder module adopts a layered architecture, including an infrared image embedding layer, a tile merging layer, and several boundary-aware state space blocks.

[0052] The infrared image embedding layer is used to map the input infrared image to a high-dimensional feature space to obtain a feature map. In this application, the infrared image embedding layer can be configured in two ways: when selecting native single-channel, a convolutional kernel size of [size missing] is used. A convolutional layer with a stride of 4 embeds a single-channel infrared image into a high-dimensional feature representation, and the number of output channels is equal to the initial embedding dimension. (In this embodiment, it is 96 for example), and after embedding, it undergoes layer normalization. When selecting pre-trained weight transfer, the number of input channels of the convolution in the infrared image embedding layer is set to 3, thereby accelerating model convergence without changing the main structure of the subsequent encoder.

[0053] The tile merging layer is used for spatial downsampling and channel expansion. The feature map output from the infrared image embedding layer is input into the tile merging layer. The tile merging layer divides the input feature map into 2×2 non-overlapping neighborhood blocks. The 4 pixel features in each block are spliced ​​along the channel dimension to make the number of channels 4 times the original. Then, the number of channels is reduced to twice the original through a linear layer. Thus, the spatial resolution is halved while channel information fusion and compression are achieved.

[0054] The boundary-aware state space block adopts a two-branch structure. The main branch sequentially passes through layer normalization, linear projection (channel expansion), and depthwise convolutional layers (…). The main branch performs an activation function, then enters the boundary enhancement multi-directional scanning unit, followed by layer normalization; the auxiliary branch undergoes layer normalization, linear projection, and activation function. The outputs of the main branch and auxiliary branch are multiplied element-wise to form a gating mechanism, projected back to the original number of channels through the output linear layer, and connected with the input residual to obtain the shared coded feature map of the infrared image for output. The boundary enhancement multi-directional scanning unit transforms two-dimensional features into a one-dimensional sequence for state space modeling through four scanning directions and introduces a boundary gradient modulation mechanism to enhance the response to weak boundaries. The boundary-aware state space block combines long-range dependency modeling and local gradient enhancement, effectively improving the detection rate of gases with blurred boundaries.

[0055] The segmentation and decoding branch module adopts a U-shaped structure. It recovers spatial resolution by upsampling the shared coded feature map output from the infrared image embedding layer layer by layer, and then fuses the shared coded feature map output from the infrared image embedding layer through skip connections (element-wise summation) to preserve multi-scale details. Finally, it... Convolution yields an unconstrained scalar field The output range after activation by the Sigmoid function is single-channel output .in For the Sigmoid function, This is the output of the segmented decoding branch module, which is also the output of the basic model.

[0056] The physical parameter decoding branch module is used to output physical parameters describing the inter-frame evolution. Specifically, the output physical parameters include a two-dimensional motion velocity field. diffusion coefficient With source item The physical parameter decoding branch module shares the encoder's underlying features with the segmentation decoding branch. At the input end, a differential concatenation and fusion operator is used, and the concatenated features are processed through two layers... Convolution is used to compress the channels (e.g., from 1536 channels to 384 and then to 192), and then the result is split into three parallel prediction heads, which output the original velocity field predictions respectively. Original prediction of diffusion coefficient With the original source item And interpolate and upsample to the input image resolution.

[0057] Furthermore, the original prediction of the output velocity field Original prediction of diffusion coefficient With the original source item Then, the two-dimensional velocity field at time t is obtained through source term structured constraints. The formula is expressed as: .in, To set the maximum velocity, i.e., the two-dimensional velocity field. The absolute value of any velocity component in the data must not exceed the set maximum speed. The diffusion coefficient is obtained by using a soft addition function and applying an upper bound truncation. The formula is expressed as: .in, To set the maximum diffusion coefficient, so that the obtained satisfy To improve training stability. Original source terms The unconstrained real-valued field (positive and negative values ​​are allowed) represents the net generation / net consumption intensity of the surrogate concentration field per unit time step (frame) (positive values ​​correspond to sources, negative values ​​correspond to sinks). In this application, it is sparsified and masked to obtain... This is to prevent the source terms from absorbing all inter-frame differences. By using structured constraints on the source terms and upper bound constraints on the diffusion coefficient, we prevent the network from excessively absorbing errors using physical parameters, ensuring that the model learns features that conform to physical laws, guaranteeing the rationality of physical parameters, and preventing degradation.

[0058] Based on the above structural model, the current frame image After inputting into the shared encoder module, the shared encoder module outputs multi-scale features. Multiscale features After processing by the segmentation and decoding branch module, the normalized gas concentration image corresponding to the current frame image is obtained. Next frame image Similarly, inputting the shared encoder module and the segmentation / decoding branch module yields multi-scale features. and the normalized gas concentration image corresponding to the next frame. Based on this, the physical parameter decoding branch module utilizes features and Construct differential splicing features, input physical parameters, decode the branch, and output physical parameters. And further based on the original source item The source term is calculated The specific formula is expressed as follows: .in, ; The preset soft threshold (0.1 in this embodiment) is used. This represents element-wise product.

[0059] Furthermore, for the aforementioned basic model, this embodiment also constructs a corresponding loss function. The basic model is trained based on the training dataset until the loss function converges, which means the basic model training is complete, and the infrared gas detection model is obtained.

[0060] Among them, L phys The physical consistency term is expressed by the following formula: , , In this application, a physical consistency term is introduced to the infrared gas detection model to introduce physical consistency constraints, thereby filtering out random noise in the normalized gas concentration image output by the model, reducing flicker and false responses, and improving the spatiotemporal continuity of the detection results.

[0061] For data fidelity items, the specific formula is expressed as follows: ;in, This is the current frame monitoring signal. This serves as the monitoring signal for the next frame.

[0062] The specific formula for the smoothing regularization term is as follows: .

[0063] in, The height of the current frame image. The width of the current frame image. For Huber's robust penalty function, As a preset constant, These are the pixel coordinates. For discrete residuals, For spatial gradient operators, For divergence operators, The basic model outputs a two-dimensional motion velocity field corresponding to the current frame image; The basic model outputs the gas diffusion coefficient corresponding to the current frame image. The base model outputs the source term corresponding to the current frame image. The basic model outputs a normalized gas concentration image corresponding to the current frame image. The basic model outputs a normalized gas concentration image corresponding to the next frame. For setting values, it means and The mean, , The gradient of velocity in two-dimensional motion. This represents the gradient of the gas diffusion coefficient output by the base model corresponding to the current frame image.

[0064] Furthermore, in this embodiment, a two-stage training strategy is employed to optimize the network by training the base model based on the training dataset. The first stage is the data fidelity fitting stage, which uses only data fidelity terms. The network is trained to obtain stable initial parameters; this phase lasts for [number] rounds. Total training rounds 10%–30%, and during this stage , Therefore, the total loss function is The second phase is the physical consistency enhancement phase, which involves further optimization. At the same time, a physical consistency term and a smoothing regularization term are introduced, and Incrementing from 0 to the upper limit according to preset rules. (For example, using linear increment), while also Set to a fixed positive value and enable it in the second stage to jointly optimize all loss terms. Through a two-stage training approach, first perform data-fidelity fitting, and then gradually introduce physical constraints to alleviate training instability and gradient conflicts.

[0065] Understandably, in this application, the physical consistency constraint serves as a regularization term in the training objective, guiding the optimization direction of the neural network parameters to make the model output more consistent with the continuous spatiotemporal evolution of gas leakage. Once training is complete, this prior is solidified into the network mapping relationship through the model parameters. Therefore, during the inference phase, there is no need to explicitly calculate the convection-diffusion residual or physical loss, and more stable, lower-noise detection results can still be obtained. In other words, the physical consistency constraint is only enabled during the model training phase and does not participate in the calculation during the model inference phase.

[0066] During the model inference phase, after the infrared gas detection model is trained through the aforementioned steps, when infrared gas monitoring is required, simply input the infrared image containing the gas image into the infrared gas detection model. The shared encoder module in the infrared gas detection model can then extract features from the infrared image and further output the corresponding infrared gas detection results via the segmentation and decoding branch module.

[0067] Compared with related technologies, the infrared gas detection model training method based on self-supervised learning provided in this embodiment acquires a background image based on gas diffusion infrared video, then calculates the current frame difference image between the current frame image and the background image, calculates the next frame difference image between the next frame image and the background image, stitches the current frame difference image and the next frame difference image together along the spatial dimension to obtain a connected difference image, uses a first set quantile operator to perform quantile mapping on the connected difference image to obtain a first quantile threshold, uses a second set quantile operator to perform quantile mapping on the connected difference image to obtain a second quantile threshold; calculates the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculates the signal based on the first quantile threshold and the second quantile threshold. The next frame supervision signal is calculated from the next frame difference image. In this way, the current frame supervision signal and the next frame supervision signal corresponding to the current frame image and the next frame image can be directly calculated from the gas diffusion infrared video. Subsequently, the current frame image, the next frame image, the current frame supervision signal and the next frame supervision signal can be integrated to obtain a complete training data. The training dataset can be directly generated from the gas diffusion infrared video to train the basic model without labeling, and the infrared gas detection model is obtained. Through background image separation and differential normalization processing, the corresponding supervision signal is obtained, realizing label-free self-supervised learning of the infrared gas detection model, thereby reducing the cost of obtaining the training dataset and achieving the technical effect of reducing the training cost of the infrared gas detection model.

[0068] To better implement the self-supervised learning-based infrared gas detection model training method in the embodiments of this application, based on the self-supervised learning-based infrared gas detection model training method, the corresponding method is as follows: Figure 3 As shown in the embodiments of this application, an infrared gas detection model training device based on self-supervised learning is also provided. The infrared gas detection model training device based on self-supervised learning includes: The image acquisition module 301 is used to acquire gas diffusion infrared video and acquire multiple training image pairs from the gas diffusion infrared video. The training image pairs include adjacent current frame images and next frame images. Background separation module 302 is used to acquire background images based on gas diffusion infrared video; For any training image pair, the self-supervised module 303 calculates the current frame difference image between the current frame image and the background image, calculates the next frame difference image between the next frame image and the background image, and concatenates the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image; it uses a first set quantile operator to perform quantile mapping on the connected difference image to obtain a first quantile threshold, and uses a second set quantile operator to perform quantile mapping on the connected difference image to obtain a second quantile threshold; it calculates the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculates the next frame supervision signal corresponding to the next frame difference image based on the first quantile threshold, the second quantile threshold, and the next frame difference image. The training data construction module 304 is used to construct the training dataset, which includes multiple training data, each of which includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal. The model training module 305 is used to train the base model based on the training dataset to obtain the infrared gas detection model.

[0069] The infrared gas detection model training device based on self-supervised learning provided in the above embodiments can realize the technical solutions described in the above embodiments of the infrared gas detection model training method based on self-supervised learning. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the infrared gas detection model training method based on self-supervised learning, and will not be repeated here.

[0070] Please refer to Figure 4 This application also provides an electronic device 400. The electronic device 400 includes a processor 401, a memory 402, and a display 403. Figure 4 Only some components of the electronic device 400 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0071] In some embodiments, processor 401 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 402 or process data, such as the infrared gas detection model training method based on self-supervised learning in this application.

[0072] In some embodiments, processor 401 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 401 may be local or remote. In some embodiments, processor 401 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0073] In some embodiments, memory 402 may be an internal storage unit of electronic device 400, such as a hard disk or memory of electronic device 400. In other embodiments, memory 402 may also be an external storage device of electronic device 400, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 400.

[0074] Furthermore, the memory 402 may include both internal storage units of the electronic device 400 and external storage devices. The memory 402 is used to store application software and various types of data installed on the electronic device 400.

[0075] In some embodiments, display 403 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 403 is used to display information from electronic device 400 and to display a visual user interface. Components 401-403 of electronic device 400 communicate with each other via a system bus.

[0076] In one embodiment, when processor 401 executes the training program for the infrared gas detection model based on self-supervised learning in memory 402, the following steps can be implemented: Acquire gas diffusion infrared video, and obtain multiple training image pairs from the gas diffusion infrared video. The training image pairs include adjacent current frame images and next frame images. Background image acquisition based on gas diffusion infrared video; For any pair of training images, calculate the current frame difference image between the current frame image and the background image, calculate the next frame difference image between the next frame image and the background image, and stitch the current frame difference image and the next frame difference image together along the spatial dimension to obtain the connected difference image. The first quantile threshold is obtained by performing quantile mapping on the connected difference image using the first set quantile operator, and the second quantile threshold is obtained by performing quantile mapping on the connected difference image using the second set quantile operator. The current frame supervision signal corresponding to the current frame difference image is calculated based on the first quantile threshold, the second quantile threshold, and the current frame difference image; the next frame supervision signal corresponding to the next frame difference image is calculated based on the first quantile threshold, the second quantile threshold, and the next frame difference image. Construct a training dataset, which includes multiple training data sets. Each training data set includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal. The basic model is trained based on the training dataset to obtain the infrared gas detection model.

[0077] It should be understood that when the processor 401 executes the training program for the infrared gas detection model based on self-supervised learning in the memory 402, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0078] Furthermore, this application does not specifically limit the type of electronic device 400 mentioned in the embodiments. Electronic device 400 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of this application, electronic device 400 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0079] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions in the self-supervised learning-based infrared gas detection model training method provided in the above-described method embodiments.

[0080] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0081] The above provides a detailed description of the training method, apparatus, electronic device, and storage medium for the infrared gas detection model based on self-supervised learning provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A training method for an infrared gas detection model based on self-supervised learning, characterized in that, include: Acquire a gas diffusion infrared video, and acquire multiple training image pairs from the gas diffusion infrared video, wherein the training image pairs include adjacent current frame images and next frame images; Background images are obtained based on the gas diffusion infrared video; For any of the training image pairs, calculate the current frame difference image between the current frame image and the background image, calculate the next frame difference image between the next frame image and the background image, and stitch the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image; A first quantile threshold is obtained by performing quantile mapping on the connected difference image using a first set quantile operator, and a second quantile threshold is obtained by performing quantile mapping on the connected difference image using a second set quantile operator. The current frame supervision signal corresponding to the current frame difference image is calculated based on the first quantile threshold, the second quantile threshold, and the current frame difference image; the next frame supervision signal corresponding to the next frame difference image is calculated based on the first quantile threshold, the second quantile threshold, and the next frame difference image. Construct a training dataset, which includes multiple training data, each of which includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal; The basic model is trained based on the training dataset to obtain the infrared gas detection model.

2. The method for training an infrared gas detection model based on self-supervised learning according to claim 1, characterized in that, The step of calculating the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculating the next frame supervision signal corresponding to the next frame difference image based on the first quantile threshold, the second quantile threshold, and the next frame difference image, includes: Based on formula , The current frame monitoring signal and the next frame monitoring signal are calculated. Where t is the time identifier. The current frame monitoring signal, This is the monitoring signal for the next frame. To prevent division by zero constant, To crop the results to Within the range, For the current frame differential image, For the next frame of the differential image, The first quantile threshold, The first quantile threshold is not greater than the second quantile threshold.

3. The method for training an infrared gas detection model based on self-supervised learning according to claim 1, characterized in that, The process of acquiring a background image based on the gas diffusion infrared video includes: Based on a set sliding window, the gas diffusion infrared video is sampled to obtain multiple sampled infrared images; The pixel data of each pixel in each of the sampled infrared images is used to form a pixel dataset corresponding to each pixel. The sampling quantile operator is used to perform quantile mapping on the pixel dataset to obtain sampled pixel data, and the background image corresponding to the sampled infrared image is constructed based on the sampled pixel data.

4. The method for training an infrared gas detection model based on self-supervised learning according to claim 1, characterized in that, The training of the base model based on the training dataset includes: Construct a loss function, which includes a physical consistency term L. phys The base model is trained based on the training dataset until the loss function converges; The physical consistency item includes: ， ， ； in, The height of the current frame image. The width of the current frame image. For Huber's robust penalty function, As a preset constant, These are the pixel coordinates. For discrete residuals, For spatial gradient operators, For divergence operators, The basic model outputs a two-dimensional motion velocity field corresponding to the current frame image. The basic model outputs the gas diffusion coefficient corresponding to the current frame image. The base model outputs the source term corresponding to the current frame image. The base model outputs a normalized gas concentration image corresponding to the current frame image. The base model outputs a normalized gas concentration image corresponding to the next frame image. For setting values, it means and The mean.

5. The method for training an infrared gas detection model based on self-supervised learning according to claim 4, characterized in that, Also includes: Provides a maximum speed setting, and sets the two-dimensional motion velocity field. The absolute value of each velocity component is less than or equal to the set maximum speed; Provides a setting for the maximum diffusion coefficient, and sets the gas diffusion coefficient. It is less than or equal to the set maximum diffusion coefficient.

6. The method for training an infrared gas detection model based on self-supervised learning according to claim 4, characterized in that, The loss function also includes a data fidelity term. The data fidelity item include: ； in, The current frame monitoring signal, This is the monitoring signal for the next frame.

7. The method for training an infrared gas detection model based on self-supervised learning according to claim 4, characterized in that, The loss function also includes a smoothing regularization term. The smoothing regularization term include: ,in, , The gradient of velocity in two-dimensional motion. This represents the gradient of the gas diffusion coefficient output by the base model corresponding to the current frame image.

8. A training device for an infrared gas detection model based on self-supervised learning, characterized in that, include: An image acquisition module is used to acquire gas diffusion infrared video and acquire multiple training image pairs from the gas diffusion infrared video. The training image pairs include adjacent current frame images and next frame images. Background separation module, the background separation module is used to acquire a background image based on the gas diffusion infrared video; The self-supervised module, for any training image pair, calculates the current frame difference image between the current frame image and the background image, calculates the next frame difference image between the next frame image and the background image, and concatenates the current frame difference image and the next frame difference image along the spatial dimension to obtain a connected difference image; performs quantile mapping on the connected difference image using a first set quantile operator to obtain a first quantile threshold, and performs quantile mapping on the connected difference image using a second set quantile operator to obtain a second quantile threshold; calculates the current frame supervision signal corresponding to the current frame difference image based on the first quantile threshold, the second quantile threshold, and the current frame difference image, and calculates the next frame supervision signal corresponding to the next frame difference image based on the first quantile threshold, the second quantile threshold, and the next frame difference image. A training data construction module is used to construct a training dataset, which includes multiple training data, each of which includes the current frame image, the next frame image, the current frame supervision signal, and the next frame supervision signal. The model training module is used to train a base model based on the training dataset to obtain the infrared gas detection model.

9. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store a program; the processor is coupled to the memory and is used to execute the program stored in the memory to implement the steps in the training method for an infrared gas detection model based on self-supervised learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the training method for an infrared gas detection model based on self-supervised learning as described in any one of claims 1 to 7.

Images