Mine leakage gas detection method, device and equipment based on Fourier transform and storage medium

Abstract

The invention discloses a mine leakage gas detection method based on Fourier transform, which enhances the detection capability of a mine gas leakage area by combining semantic information of a visible light image and temperature information of an infrared image. The LoFTR is used for carrying out multi-modal image matching, deep features extracted by a CNN neural network, frequency domain features converted by FFT, FMLP neural network weighting processing, inverse Fourier transform and other technologies, and rich data support is provided for image reconstruction and fusion. On the basis, through model training, verification and optimization, an intelligent model capable of accurately identifying and analyzing the potential safety hazards of the mining area is constructed. Finally, the trained model is deployed in practical application, real-time monitoring and analysis of the mining area are realized, and occurrence of mine safety accidents is effectively prevented and reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of mine gas leakage detection, and in particular to a mine gas leakage detection method, device, equipment and storage medium based on Fourier transform. Background Art

[0002] In mining operations, the necessity of gas leakage detection is not only related to the life safety of miners, but also the key to ensuring the stable operation of mines and environmental protection. The composition of underground environmental gases in mines is complex. With the acceleration of the industrialization process, mining activities are becoming more and more frequent, and the risk of gas leakage is also increasing. The leakage of harmful gases such as methane and carbon monoxide may not only cause explosions and poisoning accidents, but also cause serious pollution to the environment. Therefore, in order to improve the level of mine safety management, reduce accidents, protect the lives of miners, and maintain the sustainable development of the environment, it is necessary to detect the leakage of mine gases.

[0003] However, existing methods for detecting gas leakage in mines have many limitations. Existing methods for detecting gas leakage in mines, such as catalytic combustion, thermal conductivity, and optical interferometry, are often affected by environmental factors such as coal dust, temperature, humidity, and electromagnetic interference, resulting in inaccurate measurement results. In addition, these methods have limited capabilities in real-time monitoring and early warning, and cannot meet the needs of modern mines for rapid response and high sensitivity. Therefore, it is particularly urgent to develop a more efficient and accurate method for detecting gas leakage in mines. Summary of the invention

[0004] The purpose of the present invention is to solve the above technical problems and provide a mine gas leakage detection method based on Fourier transform, which can improve the accuracy and real-time performance of mine gas detection.

[0005] In order to solve the above problems, the present invention is implemented according to the following technical solutions:

[0006] In a first aspect, the present invention provides a method for detecting gas leakage in a mine based on Fourier transform, the method comprising:

[0007] S100, obtaining a visible light image and an infrared image of the mine to be inspected;

[0008] S200, extracting local feature data from the visible light image and the infrared image respectively, wherein the local feature data includes coarse-grained features and fine-grained features;

[0009] S300, extracting image feature points from the local feature data and performing multimodal image matching to obtain spatial position information data of matching between the visible light image and the infrared image;

[0010] S400, based on the spatial position information data, extracting respective deep feature data from the visible light image and the infrared image, wherein the deep feature data includes deep semantic information of the image;

[0011] S500, converting the deep feature data from the spatial domain to the frequency domain by means of a fast Fourier transform algorithm to obtain complex feature data;

[0012] S600, weighting the complex feature data by using a Fourier multilayer perceptron;

[0013] S700, converting the weighted complex feature data from the frequency domain back to the spatial domain by using an inverse fast Fourier transform algorithm;

[0014] S800, constructing a mine gas leakage detection model, and training the mine gas leakage detection model with the data obtained in step S700;

[0015] S900: Perform model verification on the trained mine gas leakage detection model, and adjust the trained mine gas leakage detection model according to the verification result to obtain a detection model that can be used to detect mine gas leakage.

[0016] Preferably, in step S200, local feature data of each is extracted from the visible light image and the infrared image respectively by using a convolutional neural network.

[0017] Preferably, in step S300, image feature points are extracted from the local features by using a LoFTR local feature matching algorithm.

[0018] Preferably, in step S400, based on the spatial position information data, respective deep feature data are extracted from the visible light image and the infrared image through a convolutional neural network.

[0019] Preferably, step S700 includes the following specific steps: S710, converting the weighted complex feature data from the frequency domain back to the spatial domain through an inverse fast Fourier transform algorithm to obtain fused time domain feature data; S720, performing image reconstruction through the fused time domain feature data to obtain a fused image, wherein the fused image includes the semantic information data of the visible light image and the temperature information data of the infrared image.

[0020] Preferably, step S700 further includes: S730, post-processing the fused image to improve the quality of the fused image.

[0021] Preferably, in step S900, the trained mine gas leakage detection model is validated by a cross-validation method.

[0022] In a second aspect, the present invention further provides a mine gas leakage detection device based on Fourier transform, wherein the mine gas leakage detection device based on Fourier transform is configured to execute the mine gas leakage detection method based on Fourier transform, and the mine gas leakage detection device based on Fourier transform comprises:

[0023] An image acquisition module, which is used to acquire visible light images and infrared images of the mine to be inspected;

[0024] A local feature extraction module, the local feature extraction module is used to extract local feature data from the visible light image and the infrared image respectively, the local feature data including coarse-grained features and fine-grained features;

[0025] A multimodal image matching module, which is used to extract image feature points from the local feature data and perform multimodal image matching to obtain spatial position information data of matching between the visible light image and the infrared image;

[0026] A deep feature extraction module, the deep feature extraction module is used to extract respective deep feature data from the visible light image and the infrared image based on the spatial position information data, the deep feature data including deep semantic information of the image;

[0027] A feature fusion module, the feature fusion module is used to convert the deep feature data from the spatial domain to the frequency domain through a fast Fourier transform algorithm to obtain complex feature data; weight the complex feature data through a Fourier multilayer perceptron; and convert the weighted complex feature data from the frequency domain back to the spatial domain through an inverse fast Fourier transform algorithm;

[0028] A model building and training module, the model building and training module is used to build a mine gas leakage detection model, and train the mine gas leakage detection model through the data obtained in step S700;

[0029] The model verification module is used to perform model verification on the trained mine gas leakage detection model, and adjust the trained mine gas leakage detection model according to the verification result to obtain a detection model that can be used to detect mine gas leakage.

[0030] In a third aspect, the present invention provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a Fourier transform-based mine gas leakage detection method as described in any one of claims 1 to 7.

[0031] In a fourth aspect, the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program is used to enable a processor to implement a Fourier transform-based mine gas leakage detection method as described in any one of claims 1 to 7 when executed.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention provides a method for detecting gas leakage in mines based on Fourier transform. First, through multimodal image acquisition, combined with visible light and infrared imaging technology, the semantic information of visible light images and the temperature information of infrared images are fully utilized to enhance the detection capability of leakage areas. Secondly, multimodal image matching is performed using LoFTR to accurately find the correspondence between the two images, laying a solid foundation for subsequent feature extraction and fusion. Then, the deep features extracted by the CNN neural network and the frequency domain features converted by FFT are used to realize the multi-dimensional analysis of image features, and the weighted processing of the FMLP neural network further enhances the fusion effect of cross-domain features. The inverse Fourier transform converts the fused frequency domain information back to the time domain, providing rich data support for image reconstruction and fusion. On this basis, through model training, verification and optimization, an intelligent model that can accurately identify and analyze safety hazards in mining areas is constructed. Finally, the trained model is deployed in practical applications to realize real-time monitoring and analysis of mining areas, effectively preventing and reducing the occurrence of mine safety accidents.

[0034] Overall, the mine leakage gas detection method based on Fourier transform described in the present invention not only improves the accuracy and real-time performance of mine gas detection, but also enhances the generalization ability and adaptability of the model through the application of deep learning technology, providing strong technical support for mine safety management, and has important practical application value and broad market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings, wherein:

[0036] Figure 1 It is a principle structure diagram of a mine gas leakage detection method based on Fourier transform of the present invention;

[0037] Figure 2 It is a technical flow chart of a preferred embodiment of a mine gas leakage detection method based on Fourier transform of the present invention;

[0038] Figure 3 It is a module diagram of a mine gas leakage detection device based on Fourier transform of the present invention;

[0039] Figure 4 It is a structural schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0041] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. Unless otherwise defined, the technical terms or scientific terms used in this specification should be understood by people with ordinary skills in the field to which this application belongs. The words "first", "second" and similar words used in this specification and claims do not indicate any order, quantity or importance, but are only used to distinguish different technical features.

[0042] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0043] like Figure 2 As shown, the mine gas leakage detection method based on Fourier transform described in the present invention comprises the following steps:

[0044] S100: Obtain a visible light image and an infrared image of a mine to be inspected.

[0045] Preferably, a visible light image of the mine to be inspected can be acquired by installing a camera on a mining intelligent vehicle or by using a drone equipped with a camera, and a single-channel infrared image can be acquired by an infrared night vision camera on a mining intelligent vehicle.

[0046] S200 , extracting local feature data from the visible light image and the infrared image respectively, where the local feature data includes coarse-grained features and fine-grained features.

[0047] It should be noted that local features refer to the features of specific areas or points of interest (such as corners, edges, textures, etc.) in an image. These features are usually extracted by detecting specific patterns in the image, such as SIFT (Scale Invariant Feature Transform), SURF (Speeded Robust Features) or ORB (Oriented FAST and Rotated BRIEF) algorithms. Preferably, local feature data of each is extracted from the visible light image and the infrared image respectively by a convolutional neural network. Coarse-grained features usually refer to more abstract and global information extracted from the image, which captures the overall structure and layout of the image at a larger scale. These features include but are not limited to the global color distribution, texture, shape, etc. of the image, which can provide an overall description of the image, but may not be rich enough in detail information. Coarse-grained features are very useful for processing tasks that require global understanding, such as scene classification, object recognition, etc. Fine-grained features refer to local features with smaller scales in the image, which usually include details and texture information in the image, such as edges, corners, local textures, etc. These features can capture more detailed parts of the image, and are particularly important for tasks that require fine classification or recognition, such as object part segmentation in images, face key point detection in face recognition, etc. In the present invention, coarse-grained features provide contextual information, while fine-grained features provide key local and detail information. The combination of the two can form complementary coarse-grained and fine-grained representations, enhancing the recognition ability of the model.

[0048] Preferably, local feature data of each is extracted from the visible light image and the infrared image respectively through a convolutional neural network.

[0049] It should be noted that convolutional neural network (CNN) is a deep learning model that performs well in signal data processing in the fields of images and videos, especially in classification and recognition tasks. In the present invention, local feature data of each is extracted from visible light images and infrared images respectively by convolutional neural network, which has the following advantages: first, CNN can automatically learn and extract features from image data without manually designing or selecting feature extractors, which greatly reduces the workload of feature engineering and improves the efficiency and accuracy of feature extraction. Secondly, CNN reduces the amount of parameters and the amount of calculation of the network by parameter sharing and sparse connection, making the network more efficient and reducing the risk of overfitting. Furthermore, CNN has the characteristic of translation invariance of input data, that is, for small changes in input data, the model can maintain a stable output, which has good robustness for small movements or deformations of images in practical applications. In addition, CNN can automatically extract the essential features of input data, and by increasing the number of convolutional layers and pooling layers, the network can learn more complex feature expressions, thereby improving the expressive power and generalization ability of the model. CNN has strong generalization ability and can show good classification performance on unseen data, thanks to its extraction of more representative features through multi-layer convolution and pooling operations. Finally, CNN has good spatial perception ability and can automatically learn local features of images or other grid structure data, which is crucial for tasks such as image matching and fusion. Through the above advantages, the present invention can effectively extract local feature data from visible light and infrared images, providing a solid foundation for subsequent image fusion and analysis, thereby improving the accuracy and efficiency of mine gas leakage detection.

[0050] S300 , extracting image feature points from local feature data and performing multimodal image matching to obtain spatial position information data of matching between the visible light image and the infrared image.

[0051] It should be noted that multimodal image matching refers to the process of establishing correspondence between overlapping areas of images due to differences in scale, rotation, affine and radiation between remote sensing images of different sensor types, different imaging times or viewpoints. This process is crucial for photogrammetry and remote sensing applications such as image registration, image fusion, and change detection. The challenge of multimodal image matching lies in dealing with significant geometric deformations and nonlinear grayscale differences between images of different modalities. The present invention particularly relates to feature point matching in multimodal image matching technology. Feature point matching is an important part of multimodal image matching, which involves extracting key feature points from images and finding matching point pairs between different images. Feature point matching can be based on methods such as graph matching and point set registration, which find the optimal solution by optimizing the overall correspondence matrix.

[0052] In this embodiment, the purpose of extracting image feature points from local feature data and performing multimodal image matching is to find the correspondence between the visible light image and the infrared image, that is, feature point matching. These feature points can be corner points, edge points or other significant image structures. After the matching process is completed, we get the matching relationship between the corresponding feature points of the two images. These matching relationships contain spatial position information data, which is the basis for subsequent image fusion work, and provides accurate spatial correspondence for subsequent feature fusion.

[0053] Preferably, image feature points are extracted from local features by using a LoFTR local feature matching algorithm.

[0054] It should be noted that the LoFTR local feature matching algorithm is an advanced feature matching technology that can directly extract feature points from images and establish corresponding relationships by utilizing the global receptive field and self-attention mechanism of the Transformer architecture without relying on traditional feature detectors. The advantage of this method in the present invention is that it can generate dense feature point matches, maintain high robustness even in areas with low texture or repeated texture, and simplify the feature matching process because it eliminates the independent steps of feature description and matching. LoFTR can also learn globally consistent matching priors so that the transformed feature representation is related to the context and position, thereby improving the matching accuracy. In addition, LoFTR refines the matching items at the sub-pixel level through a correlation refinement method, further improving the matching accuracy. Therefore, LoFTR provides a powerful feature point matching tool for the mine gas leakage detection technology in the present invention, which can achieve efficient and accurate multimodal image matching in complex environments, which is crucial to improving the efficiency and accuracy of mine safety monitoring.

[0055] In a preferred embodiment, the specific steps of extracting image feature points from local features by the LoFTR local feature matching algorithm include: S210, the local feature CNN extracts a coarse feature map (e.g., 1 / 8 of the image size) and a fine feature map (e.g., 1 / 2 of the image size) from the image pair. These feature maps contain local feature information of the image, where the coarse feature map provides more global information and the fine feature map provides more detailed information; S220, the coarse feature map is flattened into a one-dimensional vector and position encoding is added. These vectors are then processed by the LoFTR module, which includes self-attention and cross-attention layers. This step makes the feature representation relevant to the context and position, and enhances the global consistency of the features; S230, through the differentiable matching layer, LoFTR can match the transformed features and generate a confidence matrix Pc. This matrix contains the confidence scores of all possible matches. Based on the confidence threshold and mutual neighbor condition, matching items are selected from Pc to generate rough matching predictions Mc; S240, for each selected rough prediction, local windows are cropped from the fine-level feature map, and the matching is refined to the sub-pixel level in these local windows as the final matching predictions Mf.

[0056] S400: Based on the spatial position information data, extract respective deep feature data from the visible light image and the infrared image, where the deep feature data includes deep semantic information of the image.

[0057] In this embodiment, the purpose of extracting the respective deep feature data from the visible light image and the infrared image is to capture the deep semantic information of the image and use this information to perform effective image analysis and processing.

[0058] It should be noted that deep semantic information refers to the high-level, abstract features contained in the image, which include not only basic low-level features such as pixel values, colors, and textures, but also more complex patterns and structures composed of these low-level features, such as the shape, outline, and semantic content of objects. These deep features are crucial for tasks such as understanding image content, image classification, and object detection and recognition.

[0059] Preferably, based on the spatial position information data, respective deep feature data are extracted from the visible light image and the infrared image through a convolutional neural network.

[0060] It is understandable that the convolutional neural network (CNN) can automatically learn feature representations from simple to complex through its multi-layer structure. At a lower level, CNN may learn primary features such as edges, corners, and basic textures; at a higher level, it can recognize more complex shapes, object parts, and even semantic information of the entire object. This hierarchical feature extraction capability enables CNN to perform well in image recognition and classification tasks because it can capture the essential features of the image, thereby improving the accuracy and robustness of recognition. In the present invention, by extracting these deep features, safety hazards in mines, such as gas leaks, can be more accurately analyzed and identified, thereby achieving efficient monitoring and timely response to mining areas.

[0061] S500, converting the deep feature data from the spatial domain to the frequency domain through a fast Fourier transform algorithm to obtain complex feature data.

[0062] In this embodiment, "complex feature data" refers to the features obtained after converting the image from the spatial domain to the frequency domain through the fast Fourier transform. These features are in complex form because the result of the Fourier transform includes amplitude and phase information, which together constitute the real and imaginary parts of the complex number. In the present invention, the deep feature data is converted from the spatial domain to the frequency domain through the fast Fourier transform algorithm to obtain complex feature data in order to better fuse information from different modalities. Specifically, in the process of converting the deep feature data from the spatial domain to the frequency domain through the fast Fourier transform algorithm, each feature point in the spatial domain is converted into a complex representation in the frequency domain, which contains the frequency component, amplitude and phase information of the feature point.

[0063] It should be noted that the fast Fourier transform algorithm (FFT) is an efficient algorithm for calculating the discrete Fourier transform (DFT) and its inverse transform. It decomposes the large-scale DFT calculation into multiple small-scale DFT calculations through recursion and divide-and-conquer methods, thereby significantly reducing the amount of calculation and reducing the complexity from O(N^2) to O(NlogN). In the present invention, the deep feature data is converted from the spatial domain to the frequency domain through FFT. This process has multiple advantages: first, FFT greatly improves the computational efficiency, making the frequency domain analysis of image features possible, especially when processing large-scale data. Secondly, FFT can reveal the frequency components of the image, which is very useful for analyzing the texture, edges and other periodic patterns of the image. In addition, the application of FFT in spectrum analysis can help extract the spectral features of the image, which provides important data support for subsequent image processing and analysis tasks, such as image fusion, feature enhancement, etc. FFT can also extract the spectrum of the signal, which is very important for the fields of spectrum analysis and signal processing. Finally, the application of FFT can improve the accuracy of mine safety monitoring. By analyzing the spectral characteristics of microseismic signals, the separation of multi-source signal components can be achieved, providing reliable data and analysis method support for the classification and identification of mine seismic signals and the establishment of a feature sample library. In summary, the application of FFT in the present invention not only improves the efficiency of data processing, but also enhances the accuracy and real-time performance of mine gas leakage detection, providing strong technical support for mine safety.

[0064] S600: weighting the complex feature data by using a Fourier multilayer perceptron.

[0065] It should be noted that the Fourier Multilayer Perceptron (FMLP) is a neural network structure that combines Fourier transform and multilayer perceptron. It maps input data to a higher dimensional space through Fourier feature mapping, so that the network can learn high-frequency functions in the data. In the present invention, the advantage of weighting complex feature data through Fourier multilayer perceptron is that FMLP can process complex data in the frequency domain, and by adjusting the parameters of Fourier feature mapping, the frequency range of neural network learning can be controlled, thereby improving the network's ability to express image features. This method makes FMLP more efficient when processing data with periodic patterns because it operates directly in the frequency domain and utilizes the sensitivity of Fourier transform to periodic signals. In addition, FMLP enhances the network's ability to learn high-frequency details through Fourier feature mapping, which is particularly important for capturing subtle features and texture information in images, thereby improving the accuracy of image fusion and analysis. In this way, FMLP not only improves the processing efficiency of deep feature data, but also enhances the model's ability to understand and analyze complex image content, which is a key technological advancement for the mine gas leakage detection technology in the present invention.

[0066] S700, converting the weighted complex feature data from the frequency domain back to the spatial domain through an inverse fast Fourier transform algorithm.

[0067] Step S700 includes the following specific steps:

[0068] S710 . Convert the weighted complex feature data from the frequency domain back to the spatial domain by using an inverse fast Fourier transform algorithm to obtain fused time domain feature data.

[0069] It should be noted that the inverse fast Fourier transform algorithm (IFFT) is the inverse process of the fast Fourier transform (FFT), which converts the frequency domain signal back to the time domain signal. In the present invention, the purpose of converting the weighted complex feature data from the frequency domain back to the spatial domain by the inverse fast Fourier transform algorithm is to restore the original spatial information of the image for further image processing and analysis. This conversion is an indispensable step in image processing, especially in frequency domain filtering, image enhancement and feature extraction. The application of IFFT enables complex image processing operations performed in the frequency domain to be converted back to the spatial domain, thereby obtaining an improved image. Through IFFT, the frequency components of the image can be adjusted in the frequency domain, such as enhancing the details in the image or removing noise, and then the restored image is obtained by inverse transformation. In addition, IFFT is also used in image compression technology, such as using DCT (closely related to FFT) in the JPEG image compression standard to convert image data from the spatial domain to the frequency domain, and then quantizing and encoding to achieve the purpose of compression. Therefore, the advantage of IFFT in the present invention is that it can accurately restore the image data processed in the frequency domain to the spatial domain, providing clear image data for subsequent image reconstruction and analysis, which is crucial to improving the accuracy and efficiency of mine safety monitoring.

[0070] S720 , reconstructing an image using the fused time domain feature data to obtain a fused image, where the fused image includes semantic information data of the visible light image and temperature information data of the infrared image.

[0071] S730: Post-process the fused image to improve the quality of the fused image.

[0072] It is understandable that post-processing the fused image, including contrast adjustment, color correction, etc., can improve the visual effect and application performance of the fused image.

[0073] S800, constructing a mine gas leakage detection model, and training the mine gas leakage detection model with the data obtained in step S700.

[0074] In this embodiment, the weights and biases of the model are adjusted to minimize the loss function through the data obtained in step S700 through optimization techniques such as back propagation algorithm and gradient descent, thereby training the model. During the training process, hyperparameters such as learning rate, batch size, network architecture, etc. may need to be adjusted to improve the performance of the model.

[0075] S900: perform model verification on the trained mine gas leakage detection model, adjust the trained mine gas leakage detection model according to the verification result, and obtain a detection model that can be used to detect mine gas leakage.

[0076] Preferably, the trained mine gas leakage detection model is validated by a cross-validation method.

[0077] Preferably, a test set is constructed using the data obtained in step S700, and the model is verified using the test set to evaluate the performance of the model, such as accuracy, recall, etc. Based on the verification results, the model structure or hyperparameters may need to be further adjusted to improve the generalization ability of the model.

[0078] In an application scenario, by deploying the detection model obtained in step S900 for detecting gas leakage in a mine into an actual application environment, it can be used to monitor safety hazards in a mine area, such as gas leakage, in real time.

[0079] Compared with the prior art, the present invention has the following beneficial effects:

[0080] The present invention provides a method for detecting gas leakage in mines based on Fourier transform. First, through multimodal image acquisition, combined with visible light and infrared imaging technology, the semantic information of visible light images and the temperature information of infrared images are fully utilized to enhance the detection capability of leakage areas. Secondly, multimodal image matching is performed using LoFTR to accurately find the correspondence between the two images, laying a solid foundation for subsequent feature extraction and fusion. Then, the deep features extracted by the CNN neural network and the frequency domain features converted by FFT are used to realize the multi-dimensional analysis of image features, and the weighted processing of the FMLP neural network further enhances the fusion effect of cross-domain features. The inverse Fourier transform converts the fused frequency domain information back to the time domain, providing rich data support for image reconstruction and fusion. On this basis, through model training, verification and optimization, an intelligent model that can accurately identify and analyze safety hazards in mining areas is constructed. Finally, the trained model is deployed in practical applications to realize real-time monitoring and analysis of mining areas, effectively preventing and reducing the occurrence of mine safety accidents.

[0081] Overall, the mine leakage gas detection method based on Fourier transform described in the present invention not only improves the accuracy and real-time performance of mine gas detection, but also enhances the generalization ability and adaptability of the model through the application of deep learning technology, providing strong technical support for mine safety management, and has important practical application value and broad market prospects.

[0082] like Figure 3 As shown, the present invention also provides a mine gas leakage detection device based on Fourier transform, the mine gas leakage detection device based on Fourier transform is configured to execute the mine gas leakage detection method based on Fourier transform, and the mine gas leakage detection device based on Fourier transform includes:

[0083] (1) Image acquisition module.

[0084] The image acquisition module is used to obtain visible light images and infrared images of the mine to be inspected.

[0085] (2) Local feature extraction module.

[0086] The local feature extraction module is used to extract local feature data from the visible light image and the infrared image respectively, and the local feature data includes coarse-grained features and fine-grained features.

[0087] (3) Multimodal image matching module.

[0088] The multimodal image matching module is used to extract image feature points from local feature data and perform multimodal image matching to obtain spatial position information data of the match between the visible light image and the infrared image.

[0089] (4) Deep feature extraction module.

[0090] The deep feature extraction module is used to extract respective deep feature data from the visible light image and the infrared image based on the spatial position information data, and the deep feature data includes the deep semantic information of the image.

[0091] (5) Feature fusion module.

[0092] The feature fusion module is used to convert deep feature data from the spatial domain to the frequency domain through the fast Fourier transform algorithm to obtain complex feature data; weight the complex feature data through the Fourier multilayer perceptron; and convert the weighted complex feature data from the frequency domain back to the spatial domain through the inverse fast Fourier transform algorithm.

[0093] (6) Model construction and training module.

[0094] The model building and training module is used to build a mine gas leakage detection model, and train the mine gas leakage detection model through the data obtained in step S700.

[0095] (7) Model verification module.

[0096] The model verification module is used to perform model verification on the trained mine gas leakage detection model, and adjust the trained mine gas leakage detection model according to the verification result to obtain a detection model that can be used to detect mine gas leakage.

[0097] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0098] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0099] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0100] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as solving a mine gas leakage detection method based on Fourier transform.

[0101] In some embodiments, a method for detecting gas leakage in a mine based on Fourier transform may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for detecting gas leakage in a mine based on Fourier transform described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform a method for detecting gas leakage in a mine based on Fourier transform in any other appropriate manner (e.g., by means of firmware).

[0102] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0104] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0107] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0108] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements a mine gas leakage detection method based on Fourier transform as provided in the embodiment of the present invention.

[0109] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0110] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0111] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A mine gas leakage detection method based on Fourier transform, characterized in that: The method comprises: S100, obtaining a visible light image and an infrared image of the mine to be inspected; S200, extracting local feature data from the visible light image and the infrared image respectively, wherein the local feature data includes coarse-grained features and fine-grained features; S300, extracting image feature points from the local feature data and performing multimodal image matching to obtain spatial position information data of matching between the visible light image and the infrared image; S400, based on the spatial position information data, extracting respective deep feature data from the visible light image and the infrared image, wherein the deep feature data includes deep semantic information of the image; S500, converting the deep feature data from the spatial domain to the frequency domain by means of a fast Fourier transform algorithm to obtain complex feature data; S600, weighting the complex feature data by using a Fourier multilayer perceptron; S700, converting the weighted complex feature data from the frequency domain back to the spatial domain by using an inverse fast Fourier transform algorithm; S800, constructing a mine gas leakage detection model, and training the mine gas leakage detection model with the data obtained in step S700; S900: Performing model verification on the trained mine gas leakage detection model, and adjusting the trained mine gas leakage detection model according to the verification result to obtain a detection model that can be used to detect mine gas leakage.

2. A mine gas leakage detection method based on Fourier transform according to claim 1, characterized in that: In step S200, local feature data of each is extracted from the visible light image and the infrared image respectively through a convolutional neural network.

3. The method for detecting gas leakage in a mine based on Fourier transform according to claim 1, characterized in that: In step S300, image feature points are extracted from the local features using the LoFTR local feature matching algorithm.

4. The method for detecting gas leakage in a mine based on Fourier transform according to claim 1, characterized in that: In step S400, based on the spatial position information data, respective deep feature data are extracted from the visible light image and the infrared image through a convolutional neural network.

5. The method for detecting gas leakage in a mine based on Fourier transform according to claim 1, characterized in that: Step S700 includes the following specific steps: S710, converting the weighted complex feature data from the frequency domain back to the spatial domain through an inverse fast Fourier transform algorithm to obtain fused time domain feature data; S720: Reconstruct an image using the fused time-domain feature data to obtain a fused image, where the fused image includes the semantic information data of the visible light image and the temperature information data of the infrared image.

6. A mine gas leakage detection method based on Fourier transform according to claim 5, characterized in that: Step S700 also includes: S730: Post-process the fused image to improve the quality of the fused image.

7. The method for detecting gas leakage in a mine based on Fourier transform according to claim 1, characterized in that: In step S900, the trained mine gas leakage detection model is validated by a cross-validation method.

8. A mine gas leakage detection device based on Fourier transform, characterized in that: The mine gas leakage detection device based on Fourier transform is configured to execute the mine gas leakage detection method based on Fourier transform, and the mine gas leakage detection device based on Fourier transform includes: An image acquisition module, which is used to acquire visible light images and infrared images of the mine to be inspected; A local feature extraction module, the local feature extraction module is used to extract local feature data from the visible light image and the infrared image respectively, the local feature data including coarse-grained features and fine-grained features; A multimodal image matching module, which is used to extract image feature points from the local feature data and perform multimodal image matching to obtain spatial position information data of matching between the visible light image and the infrared image; A deep feature extraction module, the deep feature extraction module is used to extract respective deep feature data from the visible light image and the infrared image based on the spatial position information data, the deep feature data including deep semantic information of the image; A feature fusion module, the feature fusion module is used to convert the deep feature data from the spatial domain to the frequency domain through a fast Fourier transform algorithm to obtain complex feature data; weight the complex feature data through a Fourier multilayer perceptron; and convert the weighted complex feature data from the frequency domain back to the spatial domain through an inverse fast Fourier transform algorithm; A model building and training module, the model building and training module is used to build a mine gas leakage detection model, and train the mine gas leakage detection model through the data obtained in step S700; The model verification module is used to perform model verification on the trained mine gas leakage detection model, and adjust the trained mine gas leakage detection model according to the verification result to obtain a detection model that can be used to detect mine gas leakage.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the Fourier transform-based mine leakage gas detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, The computer program is used to enable a processor to implement a Fourier transform-based mine gas leakage detection method as described in any one of claims 1 to 7 when executed.

Images