A deep learning-based mine image processing method

Abstract

The present invention is a mine image processing method based on deep learning. The method includes: collecting mine images as sample data and dividing them into clean images and original physical noise images; constructing a generative adversarial network based on physical information, using the generative adversarial network for data augmentation to obtain an enhanced training set; constructing a deep operator denoising network to perform denoising processing on the enhanced training set; training the deep operator denoising network, using the L2 norm scoring method to perform pruning processing on the denoising network, testing the denoising network to obtain a lightweight deep operator denoising network model; and using the lightweight deep operator denoising network model to perform real-time denoising processing on mine images. The present invention effectively expands the data scale, improves the generalization ability of the deep operator denoising network model, improves the clarity and recognizability of mine images, and improves their image quality; improves the processing speed and realizes the real-time processing of mine images.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a mine image processing method based on deep learning. Background Art

[0002] The safe mining of coal mines is of great significance to the personal safety of mine workers. With the rapid development of the computer industry, great progress has been made in using intelligent devices to monitor the mining of coal mines underground and the safety of employees' lives. However, due to the poor lighting conditions underground in coal mines, the illumination of the images collected by monitoring devices is generally low, and there is also significant noise interference. Due to the low illumination and noise pollution, the image quality is reduced, and a large amount of information cannot be perceived and understood by humans in a timely manner, thus affecting subsequent image analysis and decision-making. With the development of artificial intelligence technology, machine learning algorithms and deep learning algorithms have gradually been applied to the field of mine image processing.

[0003] The invention patent with the application number CN202410877834.4 discloses an adaptive image defogging and enhancement method in a mine dust and fog environment. The specific steps are as follows: Step 1, establish an image processing model including an image grading preprocessing module, an adaptive processing module, and an image optimization module; Step 2, input the image processed by the image grading preprocessing module into the adaptive processing module; Step 3, further optimize the image processed by the adaptive processing module through the image optimization module; Step 4, output the optimized image.

[0004] The invention patent with the application number CN202410094224.7 discloses an underground environment image processing system for mine locomotives, which specifically includes:

[0005] Image acquisition module: Obtain an image acquisition instruction through the user control module, and obtain the original image through the acquisition device connected thereto;

[0006] Central processing module: Obtain the original image, generate a negative image and multiple channel images based on the original image; obtain enhanced image one and enhanced image two based on the channel images; obtain a monitoring image based on enhanced image one, enhanced image two, and the negative image; input the monitoring image into an anomaly monitoring model to obtain a marked anomaly image and an anomaly level, and generate an alarm signal according to the anomaly level; among them, the anomaly monitoring model is obtained through training of an artificial intelligence model;

[0007] User control panel: Obtain the marked anomaly image and the alarm signal, and display the marked anomaly image to the user.

[0008] However, the above related technologies have the following limitations:

[0009] (1) The actual environment in the mine was not fully considered. Due to the poor lighting conditions in the coal mine, the illuminance of the images collected by the monitoring equipment was generally low, which directly affected the clarity and recognizability of the images and led to low image quality.

[0010] (2) The computational efficiency of processing mine images is low and it is difficult to meet the real-time requirements of mine monitoring.

[0011] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0012] It should be noted that this part aims to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art just because it is included in this part. Summary of the Invention

[0013] The purpose of the present invention is to provide a mine image processing method based on deep learning, thereby at least to a certain extent overcoming one or more problems caused by the limitations and defects of the related technologies.

[0014] The present invention provides a mine image processing method based on deep learning, including:

[0015] S100, collecting mine images as sample data, adjusting the resolution of the sample data, dividing the sample data into a clean image data set and an original physical noise image data set, using the clean image data set as a training set, and using the original physical noise image data set as a test set;

[0016] S200, constructing a generative adversarial network based on physical information, using the generative adversarial network to perform data augmentation on the training set and the training set with applied physical noise to improve image quality, and obtaining an enhanced training set;

[0017] S300, constructing a deep operator denoising network to perform denoising processing on the enhanced training set, and obtaining a denoised enhanced training set;

[0018] S400, using the denoised enhanced training set to train the deep operator denoising network. During the training process, using the L2 norm scoring method to perform pruning processing on the deep operator denoising network, and using the test set to test the trained deep operator denoising network to obtain a lightweight deep operator denoising network model;

[0019] S500, using the lightweight deep operator denoising network model to perform real-time denoising processing on mine images.

[0020] In the present invention, S100 includes the following processes:

[0021] S101, Collect mine images as sample data and adjust the resolution of the sample data to be consistent;

[0022] S102, Perform two-dimensional Fourier transform on the sample data with consistent resolution to obtain a mine image frequency spectrum diagram, and divide the sample data into a clean image data set and an original physical noise image data set according to the frequency spectrum diagram;

[0023] S103, Use the clean image data set as the training set and the original physical noise image data set as the test set.

[0024] In the present invention, S200 includes the following processes:

[0025] S201, Apply physical noise to the training set to obtain a physical noise image data set, where the physical noise includes: read noise, shot noise, row noise, and quantization noise;

[0026] S202, Construct a generative adversarial network based on physical information;

[0027] S203, Use the generative adversarial network to perform data augmentation on the training set and the physical noise image data set respectively, and merge them after augmentation to obtain an augmented training set.

[0028] In the present invention, in S300, the depth operator denoising network includes: a backbone network, a branch network, a dot product layer, and a decoding network;

[0029] The backbone network is used to learn the global feature mapping of mine images;

[0030] The branch network is used to learn the local detail feature mapping of mine images;

[0031] The dot product layer is used to perform a dot product operation after receiving the global feature mapping and the local detail feature mapping;

[0032] The decoding network is used to process the data output by the dot product layer to obtain a reconstructed clean image.

[0033] In the present invention, the pruning process in S400 includes the following processes:

[0034] S401, Randomly freeze some elements in the initial weight matrix of the k-th layer of the depth operator denoising network model to obtain a lightweight pruning weight matrix;

[0035] S402. Calculate the L2 norm score between the initial weight matrix and the lightweight pruning weight matrix. If the calculated L2 norm score is greater than the preset lightweight pruning threshold, use the lightweight pruning weight matrix to replace the initial weight matrix. If the L2 norm score is not greater than the preset lightweight pruning threshold, repeat S401 - S402.

[0036] In the present invention, in S400, the loss function L1 during the training process of the depth operator denoising network is as follows:

[0037]

[0038] Where H and W are the height and width of the mine image input to the depth operator denoising network respectively. is the label corresponding to the i th data sample in the enhanced training set after denoising. is the modulo calculation symbol. represents the reconstructed clean image output by the decoding network at the pixel value at ( ). represents the pixel value of the image at ( ). ( ) represents the pixel coordinates of the two - dimensional image.

[0039] In the present invention, the backbone network and the branch network adopt a parallel architecture.

[0040] In the present invention, the backbone network includes 5 2D encoder units.

[0041] In the present invention, the branch network includes an image segmentation layer, a linear mapping layer, a self - attention layer, a cross - attention layer, and a feed - forward neural network layer.

[0042] In the present invention, the decoding network includes 5 2D decoder units, 5 downsampling units, and 5 feature splicing units.

[0043] The technical solution provided by the present invention may include the following beneficial effects:

[0044] The mine image processing method based on deep learning in the present invention constructs a generative adversarial network based on physical information, uses the generative adversarial network for data augmentation, thereby simulates the physical noise in the mine, effectively expands the data scale, improves the generalization ability of the deep operator denoising network model, improves the clarity and recognizability of the mine image, and improves its image quality; pruning the deep operator denoising network by the L2 norm scoring method makes the deep operator denoising network model lightweight, improves the processing speed of the network model, reduces the storage pressure and memory occupancy pressure of the network model, and thus realizes the real-time processing of mine images. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings based on these drawings without creative efforts.

[0046] Figure 1 A flowchart showing the method for processing mine images based on deep learning in an exemplary embodiment of the present disclosure;

[0047] Figure 2 A schematic diagram showing the specific steps of S100 in an exemplary embodiment of the present disclosure;

[0048] Figure 3 A schematic diagram showing the specific steps of S200 in an exemplary embodiment of the present disclosure;

[0049] Figure 4 A schematic diagram showing the specific steps of S400 in an exemplary embodiment of the present disclosure;

[0050] Figure 5 A structural diagram showing the generative adversarial network based on physical information in an exemplary embodiment of the present disclosure;

[0051] Figure 6 A structural schematic diagram showing the deep operator denoising network in an exemplary embodiment of the present disclosure;

[0052] Figure 7 A comparison test diagram showing an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments.

[0054] In addition, the accompanying drawings are only schematic illustrations of the embodiments of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0055] Please refer to Figure 1 , in this example embodiment, a mine image processing method based on deep learning is provided, and this method includes: S100 - S500, specifically as follows:

[0056] S100, Collect mine images as sample data, adjust the resolution of the sample data, divide the sample data into a clean image data set and an original physical noise image data set, use the clean image data set as the training set, and use the original physical noise image data set as the test set.

[0057] S200, Construct a generative adversarial network based on physical information, and use the generative adversarial network to perform data augmentation on the training set and the training set with applied physical noise to improve the image quality, obtaining an enhanced training set.

[0058] S300, Construct a deep operator denoising network to perform denoising processing on the enhanced training set, obtaining a denoised enhanced training set.

[0059] S400, Use the denoised enhanced training set to train the deep operator denoising network. During the training process, use the L2 norm scoring method to perform pruning processing on the deep operator denoising network, and use the test set to test the trained deep operator denoising network, obtaining a lightweight deep operator denoising network model.

[0060] S500, Use the lightweight deep operator denoising network model to perform real - time denoising processing on mine images.

[0061] It should be understood that by constructing a physics-informed generative adversarial network and using the generative adversarial network for data augmentation, the physical noise in the mine is simulated, effectively expanding the data scale, improving the generalization ability of the depth operator denoising network, enhancing the clarity and recognizability of the mine images, and improving their image quality. By pruning the depth operator denoising network using the L2 norm scoring method, a lightweight depth operator denoising network model is obtained, which improves the processing speed of the network model, reduces the storage pressure and memory occupancy pressure of the network model, thereby realizing the real-time processing of mine images.

[0062] The following combines Figures 2 to 6 to provide a more detailed description of the above implementation steps of this application.

[0063] Please refer to Figure 2 , and S100 specifically includes the following steps:

[0064] S101, Collect mine images as sample data, which is represented by X, , and the data scale is N. The mine images can be collected by a camera.

[0065] Adjust the resolution of the sample data to be consistent. For example, adjust the resolution to 512*512.

[0066] S102, Perform a two-dimensional Fourier transform on the sample data with the adjusted resolution to obtain a mine image spectrogram, and divide the sample data into a clean image data set and an original physical noise image data set according to the spectrogram. The mine image spectrogram among them is , when dividing the sample data, set the label of the original physical noise image data set to label = 0, and set the label of the clean image data set to label = 1.

[0067] S103, Use the clean image data set as the training set and the original physical noise image data set as the test set. Among them, the training set is represented as: , and the test set is represented as: , where N1 and N2 are the scales of the training set and the test set respectively, and N1 + N2 = N.

[0068] It should be understood that dividing the mine images into original physical noise images and clean images according to the mine image spectrogram and setting the corresponding labels provides basic data for subsequent image processing.

[0069] Please refer to Figure 3 , and the process of S200 specifically includes the following steps:

[0070] S201. Apply physical noise to the training set to obtain a physical noise image data set, where the physical noise includes: read noise, shot noise, row noise, and quantization noise;

[0071] S202. Construct a generative adversarial network based on physical information;

[0072] S203. Use the generative adversarial network to perform data augmentation on the training set and the physical noise image data set respectively, and merge them after augmentation to obtain an augmented training set. The specific process is as follows: Use the generative adversarial network to perform data augmentation on the training set (clean image data set) and the physical noise image data set respectively, obtain the augmented clean images and augmented physical noise images, and then merge the augmented clean images and augmented physical noise images with the clean images and physical noise images respectively to finally obtain the augmented training set .

[0073] It should be understood that applying physical noises such as read noise, shot noise, row noise, and quantization noise to the training set of mine images, constructing a generative adversarial network based on physical information, and then performing data augmentation to obtain an augmented training set effectively expands the data scale and improves the generalization ability of the subsequent deep operator denoising network model.

[0074] Specifically, the generative adversarial network constructed based on physical information includes: an underground mine physical noise synthesis module and a generative adversarial network module. The structure of the generative adversarial network is as Figure 5 shown.

[0075] Furthermore, the specific implementation process of S200 is as follows:

[0076] (1) Use the underground mine physical noise synthesis module to apply physical noise; for the i th image

[0077]

[0078] in , the specific calculation formula is as follows: i where is the th image in the physical noise image data set, and is a fixed pattern noise that exists when the camera captures an image and is independent of the image content; shot noise is related to the intensity of light hitting the camera sensor and is a random noise. In the present invention, and are all simulated as Gaussian noise; row noise is usually caused by circuit differences in each row of the camera sensor, and in the present invention, row noise is simulated as Gaussian noise; quantization noise is introduced when the analog-to-digital converter of the camera sensor converts an analog signal into a digital signal, that is, due to the limited precision of the digital representation, the continuous change of the analog signal will be discretized when converted into a digital signal, thus introducing noise. Therefore, in the present invention, quantization noise is simulated as uniformly distributed random noise. This method takes into account the basic characteristics of quantization noise, and approximates quantization noise through a uniform distribution, which can capture two statistical characteristics of the noise in the image, namely, uniformity and amplitude. After all the mine images in .

[0079] (2) Construct a generative adversarial network to and respectively perform data augmentation; the generative adversarial network module consists of a generator and a discriminator.

[0080] The generator consists of a convolutional layer, a feature pyramid generation layer, and a fully connected layer; the convolutional layer is used to encode the information of the input mine image and convert the image into a feature map in a high-dimensional space; a feature pyramid structure is used to capture the feature information of the mine image at different scales. The feature pyramid network is an effective multi-scale feature fusion method that can extract and fuse features at multiple scales to make full use of the context information and improve the quality of the generated image; the fully connected layer is used to convert the features output by the feature pyramid generation layer into the generated image; in the i th image and i in the th

[0081] image are respectively input into the generator, and the specific steps are as follows: and are respectively input into the convolutional layer to obtain the high-dimensional features of the clean image and the high-dimensional features of the physical noise image

[0082]

[0083]

[0084] where is the pixel value of the clean image at , is the element value of the convolutional kernel at , is the index of the convolutional kernel element in the vertical direction, is the index of the convolutional kernel element in the horizontal direction. In the present invention, the size of the convolutional kernel is , so as to fully capture the high-level abstract features in the image.

[0085] Input the high-dimensional features of the clean image and the high-dimensional features of the physical noise image into the feature pyramid generation layer to obtain the multi-scale features of the clean image and the multi-scale features of the physical noise image .

[0086] Build the feature pyramid from the bottom layer to the top layer. For the input high-dimensional features of the clean image and the high-dimensional features of the physical noise image , first extract the feature maps of different scales through the ResNet-101 (deep residual neural network) unit respectively and . The corresponding feature maps of different scales are , The corresponding feature maps of different scales are . Then build the feature pyramid according to the following steps:

[0087] First, input and into the convolutional layer respectively to adjust the number of channels. The specific calculation formula is as follows:

[0088]

[0089]

[0090] Among them, and are the top-layer features of the feature pyramid;

[0091] Then input and into the upsampling layer and the convolutional layer in sequence to obtain the intermediate-layer features and respectively. The specific calculation formula is as follows:

[0092]

[0093]

[0094]

[0095]

[0096] Among them,​ and are the intermediate layer features of the clean image and the physical noise image respectively. M2 and are the intermediate layer fusion features of the clean image and the physical noise image respectively. is an upsampling pooling layer implemented based on deconvolution, which is used to reduce the dimension of the feature map and reduce the risk of model overfitting.

[0097] Finally, and are successively input into the upsampling layer and the convolutional layer to obtain the underlying features and respectively. The specific calculation formulas are as follows:

[0098]

[0099]

[0100]

[0101]

[0102] Among them, and are the intermediate layer features of the clean image and the underlying features of the physical noise image respectively; M1 and are the underlying fusion features of the clean image and the underlying fusion features of the physical noise image respectively. is The input feature pyramid generation layer obtains the multi-scale features of the clean image , is The input feature pyramid generation layer obtains the multi-scale features of the physical noise image ;

[0103] Input and into the fully connected layer respectively to obtain the enhanced clean image and the enhanced physical noise image synthesized by the generator. The specific calculation formulas are as follows:

[0104]

[0105]

[0106] Among them, represents the fully connected layer, is the enhanced clean image obtained after inputting the i th image into the generator, is the enhanced clean image obtained after inputting the i th image The enhanced physical noise image obtained after the input generator.

[0107] The discriminator adopts the discriminator of the traditional generative adversarial network, aiming to distinguish whether the image input to the discriminator is a synthetic image by the generator or a real image, so as to improve the performance of the generator.

[0108] The generator is used to generate enhanced clean images and enhanced physical noise images ; and are respectively combined with the enhanced clean images and the enhanced physical noise images to construct an enhanced training set .

[0109] As Figure 6 shown, the deep operator denoising network consists of a backbone network, a branch network, a dot product layer and a decoding network. The backbone network and the branch network adopt a parallel architecture. The backbone network is used to learn the global feature mapping of the mine image, and the branch network is used to learn the local detail feature mapping of the mine image. After the global feature mapping of the mine image output by the backbone network and the local detail feature mapping of the mine image output by the branch network are input to the dot product layer for dot product operation, they are input to the decoding network for layer-by-layer reduction of features to obtain the reconstructed clean image.

[0110] The backbone network consists of 5 2D encoder units. The th i data sample in the obtained enhanced training set ( here is a different symbol from the th image in the physical noise image dataset in the previous text) is input to the backbone network to obtain the global feature mapping

[0111]

[0112] of the mine image. The specific calculation formula is as follows: , , , and respectively represent the first 2D encoder unit, the second 2D encoder unit, the third 2D encoder unit, the fourth 2D encoder unit and the fifth 2D encoder unit; , , and The output features of the first 2D encoder unit, the second 2D encoder unit, the third 2D encoder unit, and the fourth 2D encoder unit, respectively.

[0113] The branch network includes an image segmentation layer, a linear mapping layer, a self-attention layer, a cross-attention layer, and a feed-forward neural network layer; the enhanced training set The i th data sample is input into the branch network to obtain a local detail feature map of the mine image , and the specific steps are as follows:

[0114] First, the input is fed into the image segmentation layer, and the is randomly segmented into image patches, , where β1, ···, β α represent the image patches;

[0115] Second, each image patch is input into the linear mapping layer to be transformed into a feature map , and the feature maps are combined into a feature matrix ;

[0116] Third, the is input into the self-attention layer, and the relationship between the input features and themselves is calculated through the self-attention mechanism to update the representation of each position to obtain a self-attention feature matrix ;

[0117] Then, the self-attention feature matrix is input into the cross-attention layer for information interaction and weighted fusion between the image patch features (weighted fusion is to assign a weight to each image patch feature map and perform weighted summation on the image patch feature maps), so as to capture the correlation between the image patch features. The present invention adopts layers of cross-attention mechanisms. For the j th cross-attention mechanism, is used as the query and key, and the remaining elements are used as the values. After that, the outputs of cross-attention mechanisms are stacked as the output of the cross-attention layer ;

[0118] Finally, in order to further enhance the fused information between the image patches, the is input into the feed-forward neural network layer to further extract deep features and obtain the output of the branch network;

[0119] The and are input into the dot product layer to obtain the fused feature , the specific calculation formula is as follows:

[0120]

[0121] Among them, γ is the abscissa of LC and TC, δ is the ordinate of LC and TC, and D is the dimension of LC and TC.

[0122] The decoding network includes 5 2D decoder units, 5 downsampling units, and 5 feature concatenation units; Input the decoding network for decoding to obtain the denoised mine image, and the specific calculation formula is as follows:

[0123]

[0124]

[0125]

[0126]

[0127]

[0128] Among them, , , , and respectively represent the first 2D decoder unit, the second 2D decoder unit, the third 2D decoder unit, the fourth 2D decoder unit, and the fifth 2D decoder unit; , , and are the output features of the first 2D decoder unit, the second 2D decoder unit, the third 2D decoder unit, and the fourth 2D decoder unit respectively; is the denoised mine image output by the decoding network; represents the downsampling unit, which is responsible for downsampling the features to reduce the computational amount, prevent overfitting, and improve the generalization ability of the model; represents the feature concatenation unit, which directly merges two tensors with the same height and width in the channel dimension.

[0129] The deep operator denoising network constructed by the present invention adopts a parallel architecture of a backbone network and a branch network, which can simultaneously learn the global feature mapping and local detail feature mapping of the mine image. Through the fusion and restoration of the dot product layer and the decoding network, the mine image can be effectively denoised, and the image quality is improved.

[0130] The specific process of S400 is described below. In S400, a pixel-level lightweight training strategy is adopted to train the depth operator denoising network, ensuring that the backbone network, branch network, and decoding network in the depth operator denoising network can fully learn the pixel-level differences between the original physical noise image and the clean image, and the model parameters are relatively few, meeting the requirements for deploying real-time mine image processing tasks on the background server.

[0131] The loss function during the training process of the depth operator denoising network is as follows:

[0132]

[0133] Among them, H and W are the height and width of the mine image input into the depth operator denoising network respectively, is the label corresponding to the i th data sample in the enhanced training set after denoising, is the modulo calculation symbol, represents the reconstructed clean image output by the decoding network at the pixel value at ( ), represents the pixel value of the image at ( ), ( ) represents the coordinates of the pixel in the two-dimensional image, h represents the abscissa, and w represents the ordinate.

[0134] In S400, lightweight pruning adopts the method of L2 norm scoring to prune the weight matrix of each layer of the depth operator denoising network, freezing some elements in the weight matrix (setting some elements to 0).

[0135] Please refer to Figure 4 , the pruning process in S400 includes the following steps:

[0136] S401, randomly freeze some elements in the initial weight matrix of the kth layer of the depth operator denoising network model to obtain a lightweight pruning weight matrix.

[0137] S402, calculate the L2 norm score between the initial weight matrix and the lightweight pruning weight matrix. If the calculated L2 norm score is greater than the preset lightweight pruning threshold, use the lightweight pruning weight matrix to replace the initial weight matrix. If the L2 norm score is not greater than the preset lightweight pruning threshold, repeat S401 - S402.

[0138] Specifically, the initial weight matrix of the kth layer of the depth operator denoising network model is , randomly freeze Some elements in it to obtain a lightweight pruning weight matrix . Calculate and The L2 norm score between them. If the L2 norm score is greater than the preset lightweight pruning threshold, then use to replace . Otherwise, repeat steps S401 - S402.

[0139] According to the indexes of the initial weight matrix, rearrange the non - zero weights in the lightweight pruning weight matrix to obtain a reconstructed lightweight pruning weight matrix. Adjust the structure of the neural network by screening out the unfrozen neural network parameters. By removing redundant weights and branches in the model, the overall size of the model is greatly reduced, saving limited storage space and memory resources, thus greatly improving the real - time performance of image processing and meeting the actual application requirements of mine image processing.

[0140] After obtaining the lightweight depth operator denoising network model using the steps of S100 - S400, then use the lightweight depth operator denoising network model constructed by the above method to process mine images in real - time.

[0141] In summary, the present application has the following beneficial effects:

[0142] (1) Improve image quality: By constructing a generative adversarial network based on physical information to perform data augmentation on the training set and the training set with added physical noise, and then combining with the depth operator denoising network to perform denoising processing on the image, the quality of mine images can be significantly improved, making them clearer and easier to identify.

[0143] (2) Enhance data diversity: Using the generative adversarial network effectively expands the diversity of the mine image dataset, provides more training samples for deep learning algorithms, and thus improves the accuracy and generalization ability of the algorithms.

[0144] (3) Improve computational efficiency: Through the lightweight pruning step, the present invention optimizes the depth operator denoising network model, reduces the computational complexity of the model, and enables it to maintain high accuracy while meeting the real - time requirements of mine monitoring.

[0145] (4) Improve the level of mine safety monitoring: The mine image processing method of the present invention can process the images collected by monitoring devices in real - time, provide more accurate and reliable information support for mine safety monitoring, help improve the level of mine safety monitoring, and ensure the safety of employees' lives.

[0146] To verify the effectiveness of the method of the present invention, a comparative experiment was conducted on the denoising method of this application and the SSR filtering (Single Scale Retinex, single scale filtering) and denoising autoencoder. The experimental results can be seen in Figure 7 .

[0147] Figure 7 In (a) of Figure 7 is the original image without denoising, Figure 7 in (b) is the image after denoising by SSR, Figure 7 in (c) is the image after denoising by the denoising autoencoder, Figure 7 and in (d) is the image after denoising by the method of the present invention. It can be seen from

[0148] that the image after denoising by SSR filtering is still very blurred and has the worst denoising ability. Although the autoencoder can make the image clear, it still cannot remove the physical noise caused by light, while the method of the present invention can effectively remove the physical noise and obtain a relatively clear image.

[0149] Table 1 Evaluation results of the image processing effects of SSR, the method of the present invention, and the denoising autoencoder

[0150]

[0151] From the information entropy and peak signal-to-noise ratio, it can be seen that the method of the present invention can significantly remove the interference caused by uneven illumination, and has a good strengthening effect on the light illumination, contrast, edge details, etc. The peak signal-to-noise ratio is relatively high and the details of the image are maintained without blurring, and the overall processing effect is the best.

[0152] It should be noted that although several modules of the system for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present invention, the features and functions of two or more of the above-described modules can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules for embodiment. The components shown as modules can be either physical units or not, that is, they can be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present invention. Those of ordinary skill in the art can understand and implement it without creative work.

[0153] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these are within the protection scope of the present invention.

Claims

1. A mine image processing method based on deep learning, characterized in that: include: S100, collecting mine images as sample data, adjusting the resolution of the sample data, dividing the sample data into a clean image data set and an original physical noise image data set, using the clean image data set as a training set, and using the original physical noise image data set as a test set; S200, constructing a generative adversarial network based on physical information, and using the generative adversarial network to perform data enhancement on the training set and the training set with physical noise applied to improve image quality, thereby obtaining an enhanced training set; S300, constructing a deep operator denoising network to denoise the enhanced training set, and obtaining a denoised enhanced training set; S400, using the denoised enhanced training set to train the deep operator denoising network, during the training process, using the L2 norm scoring method to prune the deep operator denoising network, using the test set to test the trained deep operator denoising network, and obtaining a lightweight deep operator denoising network model; S500, performing real-time denoising processing on the mine image using the lightweight deep operator denoising network model; Among them, S200 includes the following processes: S201, applying physical noise to the training set to obtain a physical noise image dataset, wherein the physical noise includes: read noise, shot noise, line noise and quantization noise; S202, constructing a generative adversarial network based on physical information; S203, using the generative adversarial network to perform data enhancement on the training set and the physical noise image dataset respectively, and merging the enhanced data to obtain an enhanced training set; In S300, the deep operator denoising network includes: a backbone network, a branch network, a dot product layer and a decoding network; The backbone network is used to learn the global feature map of the mine image; The branch network is used to learn the local detail feature map of the mine image; The dot product layer is used to receive the global feature map and the local detail feature map and then perform a dot product operation; The decoding network is used to process the data output by the dot product layer to obtain a reconstructed clean image; The trunk network and the branch network adopt a parallel architecture; The step of using the branch network to learn the local detail feature map of the mine image includes: The branch network randomly divides the images in the augmented training set into image patches; Convert each image block into a feature map, and then combine the feature maps into a feature matrix; Use the self-attention mechanism to obtain the self-attention feature matrix; The self-attention feature matrix is ​​used to perform information interaction and weighted fusion between image block features; The fusion information between the weighted fused image blocks is used for deep feature extraction to obtain the local detail feature map of the mine image.

2. The mine image processing method based on deep learning according to claim 1, characterized in that: S100 includes the following processes: S101, collecting mine images as sample data, and adjusting the resolution of the sample data to be consistent; S102, performing a two-dimensional Fourier transform on the sample data after the resolution is adjusted to be consistent to obtain a mine image spectrum diagram, and dividing the sample data into a clean image data set and an original physical noise image data set according to the spectrum diagram; S103: Use the clean image dataset as a training set, and use the original physical noise image dataset as a test set.

3. The mine image processing method based on deep learning according to claim 1, characterized in that: The pruning process in S400 includes the following steps: S401, randomly freezing some elements in the k-th layer initial weight matrix of the deep operator denoising network model to obtain a lightweight pruning weight matrix; S402, calculate the L2 norm score between the initial weight matrix and the lightweight pruning weight matrix. If the calculated L2 norm score is greater than the preset lightweight pruning threshold, use the lightweight pruning weight matrix to replace the initial weight matrix. If the L2 norm score is not greater than the preset lightweight pruning threshold, repeat S401~S402.

4. The mine image processing method based on deep learning according to claim 3 is characterized in that: In S400, the loss function L1 in the deep operator denoising network training process is as follows: in, H and W are the height and width of the mine image input to the deep operator denoising network, is the first i The labels corresponding to the data samples are is the symbol for modulo calculation, Represents the reconstructed clean image output by the decoding network exist( ), express The image is in ( ), ( ) represents the pixel coordinates of a two-dimensional image.

5. The mine image processing method based on deep learning according to claim 3 is characterized in that: The backbone network includes 5 2D encoder units.

6. The mine image processing method based on deep learning according to claim 3, characterized in that: The branch network includes an image segmentation layer, a linear mapping layer, a self-attention layer, a cross-attention layer and a feedforward neural network layer.

7. The mine image processing method based on deep learning according to claim 5, characterized in that: The decoding network includes 5 2D decoder units, 5 downsampling units and 5 feature splicing units.

Images