Mine image processing method based on deep learning

Abstract

The invention relates to a mine image processing method based on deep learning, and the method comprises the steps: collecting a mine image as sample data, and dividing the mine image into a clean image and an original physical noise image; constructing a physical information-based generative adversarial network, and performing data enhancement by using the generative adversarial network to obtain an enhanced training set; constructing a depth operator denoising network to carry out denoising processing on the enhanced training set; training the depth operator denoising network, pruning the denoising network by using an L2 norm scoring method, and testing the denoising network to obtain a lightweight depth operator denoising network model; and performing real-time denoising processing on the mine image by using the lightweight depth operator denoising network model. According to the method, the data scale is effectively expanded, the generalization ability of the depth operator denoising network model is improved, the definition and the identifiability of the mine image are improved, and the image quality is improved; the processing speed is improved, and real-time processing of mine images is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular to a mine image processing method based on deep learning. Background Art

[0002] The safe mining of coal mines is of great significance to the personal safety of mine workers. With the rapid development of the computer industry, great progress has been made in the use of intelligent devices to monitor the mining of coal mines and the life safety of employees. However, due to the poor lighting conditions in coal mines, the illumination of images collected by monitoring equipment is generally low and is subject to large noise interference. Due to the low illumination and noise pollution, the image quality is reduced, and a large amount of information cannot be perceived and understood in time, which affects the subsequent image analysis and decision-making. With the development of artificial intelligence technology, machine learning algorithms and deep learning algorithms are gradually applied to the field of mine image processing.

[0003] The invention patent with application number CN202410877834.4 discloses an adaptive image dehazing and enhancement method in a mine dust and fog environment. The specific steps are as follows: Step 1, establish an image processing model including an image grading preprocessing module, an adaptive processing module and an image optimization module; Step 2, input the image processed by the image grading preprocessing module into the adaptive processing module; Step 3, further optimize the image processed by the adaptive processing module through the image optimization module; Step 4, output the optimized image.

[0004] The invention patent with application number CN202410094224.7 discloses an underground environment image processing system for a mining electric locomotive, which specifically includes: Image acquisition module: obtains image acquisition instructions through the user control module, and obtains the original image through the acquisition device connected to it; Central processing module: obtains the original image, generates a negative image and multiple channel images based on the original image; obtains enhanced image 1 and enhanced image 2 based on the channel image; obtains a monitoring image based on enhanced image 1, enhanced image 2 and the negative image; inputs the monitoring image into the abnormal monitoring model to obtain a marked abnormal image and abnormal level, and generates an alarm signal according to the abnormal level; wherein the abnormal monitoring model is obtained through training of an artificial intelligence model; User control panel: obtains marked abnormal images and alarm signals, and displays marked abnormal images to users.

[0005] However, the above-mentioned related technologies have the following limitations: (1) Failure to fully consider the actual environment in the mine. Due to the poor lighting conditions in the coal mine, the illumination of the images collected by the monitoring equipment is generally low, which directly affects the clarity and recognizability of the image, resulting in low image quality; (2) The computational efficiency of processing mine images is low, which makes it difficult to meet the real-time requirements of mine monitoring.

[0006] Therefore, it is necessary to improve one or more problems existing in the above-mentioned related technical solutions.

[0007] It should be noted that this section is intended to provide background or context for the technical solutions of the present disclosure stated in the claims. The description herein is not admitted to be prior art by virtue of being included in this section. Summary of the invention

[0008] The purpose of the present invention is to provide a mine image processing method based on deep learning, thereby overcoming one or more problems caused by the limitations and defects of the relevant technology at least to a certain extent.

[0009] The present invention provides a mine image processing method based on deep learning, comprising: S100, collecting mine images as sample data, adjusting the resolution of the sample data, dividing the sample data into a clean image data set and an original physical noise image data set, using the clean image data set as a training set, and using the original physical noise image data set as a test set; S200, constructing a generative adversarial network based on physical information, and using the generative adversarial network to perform data enhancement on the training set and the training set with physical noise applied to improve image quality, thereby obtaining an enhanced training set; S300, constructing a deep operator denoising network to denoise the enhanced training set, and obtaining a denoised enhanced training set; S400, using the denoised enhanced training set to train the deep operator denoising network, during the training process, using the L2 norm scoring method to prune the deep operator denoising network, using the test set to test the trained deep operator denoising network, and obtaining a lightweight deep operator denoising network model; S500, performing real-time denoising processing on the mine image using the lightweight deep operator denoising network model.

[0010] In the present invention, S100 includes the following processes: S101, collecting mine images as sample data, and adjusting the resolution of the sample data to be consistent; S102, performing a two-dimensional Fourier transform on the sample data after the resolution is adjusted to be consistent to obtain a mine image spectrum diagram, and dividing the sample data into a clean image data set and an original physical noise image data set according to the spectrum diagram; S103: Use the clean image dataset as a training set, and use the original physical noise image dataset as a test set.

[0011] In the present invention, S200 includes the following process: S201, applying physical noise to the training set to obtain a physical noise image dataset, wherein the physical noise includes: read noise, shot noise, line noise and quantization noise; S202, constructing a generative adversarial network based on physical information; S203, using the generative adversarial network to perform data enhancement on the training set and the physical noise image dataset respectively, and merging the enhanced data to obtain an enhanced training set.

[0012] In the present invention, in S300, the deep operator denoising network includes: a backbone network, a branch network, a dot product layer and a decoding network; The backbone network is used to learn the global feature map of the mine image; The branch network is used to learn the local detail feature map of the mine image; The dot product layer is used to receive the global feature map and the local detail feature map and then perform a dot product operation; The decoding network is used to process the data output by the dot product layer to obtain a reconstructed clean image.

[0013] In the present invention, the pruning process in S400 includes the following processes: S401, randomly freezing some elements in the k-th layer initial weight matrix of the deep operator denoising network model to obtain a lightweight pruning weight matrix; S402, calculate the L2 norm score between the initial weight matrix and the lightweight pruning weight matrix. If the calculated L2 norm score is greater than the preset lightweight pruning threshold, use the lightweight pruning weight matrix to replace the initial weight matrix. If the L2 norm score is not greater than the preset lightweight pruning threshold, repeat S401~S402.

[0014] In the present invention, in S400, the loss function L1 in the training process of the deep operator denoising network is as follows: in, H and W are the height and width of the mine image input to the deep operator denoising network, is the first i The labels corresponding to the data samples are is the symbol for modulo calculation, Represents the reconstructed clean image output by the decoding network exist( ), express The image is in ( ), ( ) represents the pixel coordinates of a two-dimensional image.

[0015] In the present invention, the backbone network and the branch network adopt a parallel architecture.

[0016] In the present invention, the backbone network includes 5 2D encoder units.

[0017] In the present invention, the branch network includes an image segmentation layer, a linear mapping layer, a self-attention layer, a cross-attention layer and a feedforward neural network layer.

[0018] In the present invention, the decoding network includes 5 2D decoder units, 5 downsampling units and 5 feature splicing units.

[0019] The technical solution provided by the present invention may include the following beneficial effects: The mine image processing method based on deep learning in the present invention constructs a generative adversarial network based on physical information, and uses the generative adversarial network to perform data enhancement, thereby simulating the physical noise of the mine, effectively expanding the data scale, improving the generalization ability of the deep operator denoising network model, and improving the clarity and recognizability of the mine image, thereby improving its image quality; the deep operator denoising network is pruned by the L2 norm scoring method, so that the deep operator denoising network model is lightweight, the processing speed of the network model is improved, and the storage pressure and memory occupancy pressure of the network model are reduced, thereby realizing real-time processing of mine images. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0021] Figure 1 A schematic diagram showing a flow chart of a mine image processing method based on deep learning in an exemplary embodiment of the present disclosure; Figure 2 A schematic diagram showing specific steps of S100 in an exemplary embodiment of the present disclosure is shown; Figure 3 A schematic diagram showing specific steps of S200 in an exemplary embodiment of the present disclosure is shown; Figure 4A schematic diagram showing specific steps of S400 in an exemplary embodiment of the present disclosure is shown; Figure 5 A structural diagram showing a physical information-based generative adversarial network in an exemplary embodiment of the present disclosure; Figure 6 A schematic diagram showing the structure of a deep operator denoising network in an exemplary embodiment of the present disclosure is shown; Figure 7 A comparative test diagram in an exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0023] In addition, the accompanying drawings are only schematic illustrations of the embodiments of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated descriptions will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0024] Please refer to Figure 1 In this example implementation, a mine image processing method based on deep learning is provided, and the method includes: S100-S500, which are as follows: S100, collecting mine images as sample data, adjusting the resolution of the sample data, dividing the sample data into a clean image data set and an original physical noise image data set, using the clean image data set as a training set, and using the original physical noise image data set as a test set.

[0025] S200, constructing a generative adversarial network based on physical information, and using the generative adversarial network to perform data enhancement on the training set and the training set with physical noise applied to improve image quality, thereby obtaining an enhanced training set.

[0026] S300, constructing a deep operator denoising network to denoise the enhanced training set to obtain a denoised enhanced training set.

[0027] S400, using the denoised enhanced training set to train the deep operator denoising network, during the training process, using the L2 norm scoring method to prune the deep operator denoising network, using the test set to test the trained deep operator denoising network, and obtaining a lightweight deep operator denoising network model.

[0028] S500, performing real-time denoising processing on the mine image using the lightweight deep operator denoising network model.

[0029] It should be understood that by constructing a generative adversarial network based on physical information and using the generative adversarial network for data enhancement, the physical noise of the mine is simulated, which effectively expands the data scale, improves the generalization ability of the deep operator denoising network, improves the clarity and recognizability of the mine image, and improves its image quality; the deep operator denoising network is pruned by the L2 norm scoring method to obtain a lightweight deep operator denoising network model, which improves the processing speed of the network model and reduces the storage pressure and memory occupancy pressure of the network model, thereby realizing real-time processing of mine images.

[0030] Combine the following Figure 2~Figure 6 , the above implementation steps of this application are explained in more detail.

[0031] Please refer to Figure 2 , S100 specifically includes the following steps: S101, collecting mine images as sample data, the sample data is represented by X, , the data scale is N. The mine images can be captured and collected by cameras.

[0032] The resolution of the sample data is adjusted to be consistent, for example, the resolution is adjusted to 512*512.

[0033] S102, performing a two-dimensional Fourier transform on the sample data after the resolution is adjusted to be consistent to obtain a mine image spectrum, and dividing the sample data into a clean image data set and an original physical noise image data set according to the spectrum. The mine image spectrum is , when dividing the sample data, the label of the original physical noise image dataset is set to label=0, and the label of the clean image dataset is set to label=1.

[0034] S103, using the clean image dataset as a training set and the original physical noise image dataset as a test set. The training set is represented as: , the test set is represented as: , where N1 and N2 are the sizes of the training set and test set respectively, and N1+N2=N.

[0035] It should be understood that the mine image is divided into original physical noise image and clean image according to the mine image spectrum diagram, and corresponding labels are set to provide basic data for subsequent image processing.

[0036] Please refer to Figure 3 The process of S200 specifically includes the following steps: S201, applying physical noise to the training set to obtain a physical noise image dataset, wherein the physical noise includes: read noise, shot noise, line noise and quantization noise; S202, constructing a generative adversarial network based on physical information; S203, using the generative adversarial network to perform data enhancement on the training set and the physical noise image dataset respectively, and merging them after enhancement to obtain an enhanced training set. The specific process is: using the generative adversarial network to perform data enhancement on the training set (clean image dataset) and the physical noise image dataset respectively, and correspondingly obtain enhanced clean images and enhanced physical noise images, and then merging the enhanced clean images and enhanced physical noise images with the clean images and physical noise images respectively, and finally obtaining an enhanced training set. .

[0037] It is important to understand that physical noises such as read noise, shot noise, row noise, and quantization noise are applied to the training set of mine images, a generative adversarial network is constructed based on physical information, and then data enhancement is performed to obtain an enhanced training set, which effectively expands the data scale and improves the generalization ability of the subsequent deep operator denoising network model.

[0038] Specifically, the generative adversarial network constructed based on physical information includes: a mine physical noise synthesis module and a generative adversarial network module. The structure of the generative adversarial network is as follows: Figure 5 shown.

[0039] Furthermore, the specific implementation process of S200 is as follows: (1) Using the underground physical noise synthesis module to Apply physical noise; for The i images , the specific calculation formula is as follows: in, is the first i images, , , and They are read noise, shot noise, line noise and quantization noise respectively. It is a fixed pattern noise, which exists when the camera captures the image and has nothing to do with the image content; shot noise is related to the intensity of light irradiating the camera sensor and is a random noise. In the present invention, and are all simulated as Gaussian noise; row noise is usually caused by the circuit differences of each row in the camera sensor, and in the present invention, row noise is simulated as Gaussian noise; quantization noise is introduced by the analog-to-digital converter of the camera sensor when converting the analog signal to a digital signal, that is, due to the limited precision of the digital representation, the continuous changes of the analog signal will be discretized when converted to a digital signal, thereby introducing noise. Therefore, in the present invention, quantization noise is simulated as uniformly distributed random noise. This method takes into account the basic characteristics of quantization noise and approximates quantization noise through uniform distribution, which can capture the two statistical characteristics of uniformity and amplitude of noise in the image. All mine images in the physical noise synthesis module are input into the mine to obtain the physical noise image dataset .

[0040] (2) Constructing a Generative Adversarial Network and Data enhancement is performed separately; the generative adversarial network module consists of a generator and a discriminator.

[0041] The generator consists of a convolutional layer, a feature pyramid generation layer, and a fully connected layer. The convolutional layer is used to encode the input mine image and convert the image into a feature map in a high-dimensional space. The feature pyramid structure is used to capture the feature information of mine images at different scales. The feature pyramid network is an effective multi-scale feature fusion method that can extract and fuse features at multiple scales to make full use of contextual information and improve the quality of generated images. The fully connected layer is used to convert the features output by the feature pyramid generation layer into the generated image. The i images and The i images Enter the generator separately. The specific steps are as follows: Will and Input the convolutional layer separately to obtain high-dimensional features of clean images and high-dimensional features of physical noise images , the calculation formula is as follows: in, Is a clean image in The pixel value at The convolution kernel is The element value at is the index of the convolution kernel element in the vertical direction, is the index of the convolution kernel element in the horizontal direction. In the present invention, the convolution kernel size is , to fully capture the high-level abstract features in the image.

[0042] High-dimensional features of clean images and high-dimensional features of physical noise images Input feature pyramid generation layer to obtain clean image multi-scale features and multi-scale features of physical noise images .

[0043] Build a feature pyramid from the bottom to the top layer, and for the high-dimensional features of the input clean image and high-dimensional features of physical noise images , firstly extract the and Feature maps of different scales, The corresponding feature maps of different scales are , The corresponding feature maps of different scales are , and then construct the feature pyramid according to the following steps: First, and Input the convolutional layer to adjust the number of channels respectively. The specific calculation formula is as follows: in, and It is the top feature of the feature pyramid; Then and Input the upsampling layer and convolution layer in sequence to obtain the intermediate layer features respectively and , the specific calculation formula is as follows: in, and are the intermediate layer features of clean images and physical noise images, M2 and They are the intermediate layer fusion features of clean images and the intermediate layer fusion features of physical noise images, It is an upsampling pooling layer based on deconvolution, which is used to reduce the dimension of the feature map and reduce the risk of model overfitting.

[0044] Finally and Input the upsampling layer and convolution layer in sequence to obtain the underlying features respectively and , the specific calculation formula is as follows: in, and are the middle-layer features of clean images and the bottom-layer features of physical noise images; M1 and They are the bottom-level fusion features of clean images and the bottom-level fusion features of physical noise images, That is Input feature pyramid generation layer to obtain clean image multi-scale features , That is Input feature pyramid generation layer to obtain multi-scale features of physical noise image ; Will and The fully connected layers are input respectively to obtain the enhanced clean image and enhanced physical noise image synthesized by the generator. The specific calculation formula is as follows: in, represents the fully connected layer, To be the first i images The enhanced clean image obtained after inputting into the generator, To be the first i images The enhanced physical noise image obtained after being fed into the generator.

[0045] The discriminator uses the discriminator of the traditional generative adversarial network, and its purpose is to identify whether the image input to the discriminator is an image synthesized by the generator or a real image, so as to improve the performance of the generator.

[0046] Use the generator to generate Enhanced clean image and Zhang enhanced physical noise image ;Will and Respectively and enhanced clean image and enhanced physical noise images Merge and construct an enhanced training set .

[0047] like Figure 6 As shown, the deep operator denoising network consists of a backbone network, a branch network, a dot product layer and a decoding network. The backbone network and the branch network adopt a parallel architecture. The backbone network is used to learn the global feature map of the mine image, and the branch network is used to learn the local detail feature map of the mine image. The global feature map of the mine image output by the backbone network and the local detail feature map of the mine image output by the branch network are input into the dot product layer for dot product operation, and then input into the decoding network for layer-by-layer feature restoration to obtain a reconstructed clean image.

[0048] The backbone network consists of 5 2D encoder units. The i Data samples (Here Compared with the i-th image in the physical noise image dataset mentioned above (with different symbols) input into the backbone network to obtain the global feature map of the mine image , the specific calculation formula is as follows: in, , , , and represent a first 2D encoder unit, a second 2D encoder unit, a third 2D encoder unit, a fourth 2D encoder unit, and a fifth 2D encoder unit, respectively; , , and are the output features of the first 2D encoder unit, the second 2D encoder unit, the third 2D encoder unit and the fourth 2D encoder unit respectively.

[0049] The branch network includes an image segmentation layer, a linear mapping layer, a self-attention layer, a cross-attention layer and a feedforward neural network layer; the training set is enhanced The i Data samples Input branch network to obtain local detail feature map of mine image , the specific steps are as follows: First, Input image segmentation layer, Randomly split into image blocks, , β1, ···β α Represents an image block; Secondly, each image block is input into the linear mapping layer to transform it into a feature map , and the feature maps are combined into a feature matrix ; Secondly, Input the self-attention layer, and calculate the relationship between the input feature and itself through the self-attention mechanism to update the representation of each position to obtain the self-attention feature matrix ; Then, the self-attention feature matrix The cross attention layer is input to perform information interaction and weighted fusion between image block features (weighted fusion is to assign a weight to each image block feature map and perform weighted summation on the image block feature maps), thereby capturing the association between image block features. Layer cross attention mechanism, for the j Layer cross attention mechanism, using As query, key, and the remaining elements As a value, then The output of the cross attention mechanism is superimposed as the output of the cross attention layer ; Finally, in order to further enhance the fusion information between image blocks, Input the feedforward neural network layer to further extract deep features and obtain the output of the branch network ; Will and Input dot product layer to obtain fusion features , the specific calculation formula is as follows: Among them, γ is the horizontal coordinate of LC and TC, δ is the vertical coordinate of LC and TC, and D is the dimension of LC and TC.

[0050] The decoding network includes 5 2D decoder units, 5 downsampling units and 5 feature concatenation units. Input the decoding network for decoding to obtain the denoised mine image. The specific calculation formula is as follows: in, , , , and represent a first 2D decoder unit, a second 2D decoder unit, a third 2D decoder unit, a fourth 2D decoder unit and a fifth 2D decoder unit respectively; , , and are output characteristics of the first 2D decoder unit, the second 2D decoder unit, the third 2D decoder unit, and the fourth 2D decoder unit, respectively; The denoised mine image output by the decoding network; Represents the downsampling unit, which is responsible for downsampling the features to reduce the amount of calculation, prevent overfitting and improve the generalization ability of the model; Represents a feature concatenation unit that directly merges two tensors of the same height and width in the channel dimension.

[0051] The deep operator-based denoising network constructed by the present invention adopts a parallel architecture of a trunk network and a branch network, and can simultaneously learn the global feature map and local detail feature map of the mine image. Through the fusion and restoration of the dot product layer and the decoding network, the mine image can be effectively denoised, thereby improving the image quality.

[0052] The specific process of S400 is described below. In S400, a pixel-level lightweight training strategy is used to train the deep operator denoising network, ensuring that the backbone network, branch network and decoding network in the deep operator denoising network can fully learn the pixel-level differences between the original physical noise image and the clean image, and the model parameters are relatively small, meeting the requirements for real-time mine image processing tasks deployed on the background server.

[0053] The loss function during the training of the deep operator denoising network is as follows: in, H and W are the height and width of the mine image input to the deep operator denoising network, is the first i The labels corresponding to the data samples are is the symbol for modulo calculation, Represents the reconstructed clean image output by the decoding network exist( ), express The image is in ( ), ( ) represents the coordinates of the pixel in the two-dimensional image, h represents the horizontal coordinate, and w represents the vertical coordinate.

[0054] In S400, lightweight pruning uses the L2 norm scoring method to prune the weight matrix of each layer of the deep operator denoising network and freeze some elements in the weight matrix (set some elements to 0).

[0055] Please refer to Figure 4 , the pruning process in S400 includes the following processes: S401, randomly freeze some elements in the initial weight matrix of the kth layer of the deep operator denoising network model to obtain a lightweight pruning weight matrix.

[0056] S402, calculate the L2 norm score between the initial weight matrix and the lightweight pruning weight matrix. If the calculated L2 norm score is greater than the preset lightweight pruning threshold, use the lightweight pruning weight matrix to replace the initial weight matrix. If the L2 norm score is not greater than the preset lightweight pruning threshold, repeat S401~S402.

[0057] Specifically, the initial weight matrix of the kth layer of the deep operator denoising network model is , random freezing Some elements in the , get the lightweight pruning weight matrix .calculate and If the L2 norm score is greater than the preset lightweight pruning threshold, use Alternative Otherwise, steps S401 to S402 are repeated.

[0058] According to the index of the initial weight matrix, the non-zero weights in the lightweight pruning weight matrix are rearranged to obtain a reconstructed lightweight pruning weight matrix. The structure of the neural network is adjusted by screening out the unfrozen neural network parameters. By removing redundant weights and branches in the model, the overall size of the model is greatly reduced, saving limited storage space and memory resources, thereby greatly improving the real-time performance of image processing and meeting the actual application needs of mine image processing.

[0059] After obtaining the lightweight deep operator denoising network model using steps S100 to S400, the lightweight deep operator denoising network model constructed by the above method is then used to perform real-time processing on the mine image.

[0060] In summary, this application has the following beneficial effects: (1) Improving image quality: By constructing a generative adversarial network based on physical information to perform data enhancement on the training set and the training set with physical noise, and then combining it with a deep operator denoising network to denoise the image, the quality of mine images can be significantly improved, making them clearer and easier to identify.

[0061] (2) Enhancing data diversity: The use of generative adversarial networks effectively expands the diversity of mine image datasets, providing more training samples for deep learning algorithms, thereby improving the accuracy and generalization ability of the algorithms.

[0062] (3) Improving computational efficiency: Through the lightweight pruning step, the present invention optimizes the deep operator denoising network model and reduces the computational complexity of the model, so that it can still maintain a high accuracy while meeting the real-time requirements of mine monitoring.

[0063] (4) Improving the level of mine safety monitoring: The mine image processing method of the present invention can process the images collected by the monitoring equipment in real time, provide more accurate and reliable information support for mine safety monitoring, help improve the level of mine safety monitoring, and ensure the safety of employees.

[0064] In order to verify the effectiveness of the method of the present invention, the denoising method of the present application is compared with the SSR filter (Single Scale Retinex) and the denoising autoencoder. The experimental results are shown in Figure 7 .

[0065] Figure 7 (a) in the figure is the original image without denoising. Figure 7 (b) is the image after SSR denoising. Figure 7 (c) in the figure is the image denoised by the denoising autoencoder. Figure 7 (d) in FIG. 1 is the image after denoising using the method of the present invention. Figure 7 It can be seen that the image after SSR filtering and denoising is still very blurry, and the denoising ability is the worst. Although the autoencoder can make the image clear, it still cannot remove the physical noise caused by light. The method of the present invention can effectively remove the physical noise and obtain a clearer image.

[0066] The image processing effects of the above three methods are objectively evaluated below, and the evaluation results are shown in Table 1.

[0067] Table 1 Evaluation results of image processing effects of SSR, the method of the present invention, and denoising autoencoder From the information entropy and peak signal-to-noise ratio, it can be seen that the method of the present invention can significantly remove the interference caused by uneven lighting, and has a good enhancement effect on brightness, contrast, edge details, etc. The peak signal-to-noise ratio is high and the details of the image are maintained without blurring, and the overall processing effect is the best.

[0068] It should be noted that although several modules of the system for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules described above can be concretized in one module. Conversely, the features and functions of a module described above can be further divided into multiple modules for concretization. The components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Those of ordinary skill in the art can understand and implement it without paying creative work.

[0069] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the enlightenment of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the purpose of the present invention and the claims, which all fall within the protection of the present invention.

Claims

1. A mine image processing method based on deep learning, characterized in that: include: S100, collecting mine images as sample data, adjusting the resolution of the sample data, dividing the sample data into a clean image data set and an original physical noise image data set, using the clean image data set as a training set, and using the original physical noise image data set as a test set; S200, constructing a generative adversarial network based on physical information, and using the generative adversarial network to perform data enhancement on the training set and the training set with physical noise applied to improve image quality, thereby obtaining an enhanced training set; S300, constructing a deep operator denoising network to denoise the enhanced training set, and obtaining a denoised enhanced training set; S400, using the denoised enhanced training set to train the deep operator denoising network, during the training process, using the L2 norm scoring method to prune the deep operator denoising network, using the test set to test the trained deep operator denoising network, and obtaining a lightweight deep operator denoising network model; S500, performing real-time denoising processing on the mine image using the lightweight deep operator denoising network model.

2. The mine image processing method based on deep learning according to claim 1, characterized in that: S100 includes the following processes: S101, collecting mine images as sample data, and adjusting the resolution of the sample data to be consistent; S102, performing a two-dimensional Fourier transform on the sample data after the resolution is adjusted to be consistent to obtain a mine image spectrum diagram, and dividing the sample data into a clean image data set and an original physical noise image data set according to the spectrum diagram; S103: Use the clean image dataset as a training set, and use the original physical noise image dataset as a test set.

3. The mine image processing method based on deep learning according to claim 1, characterized in that: S200 includes the following processes: S201, applying physical noise to the training set to obtain a physical noise image dataset, wherein the physical noise includes: read noise, shot noise, line noise and quantization noise; S202, constructing a generative adversarial network based on physical information; S203, using the generative adversarial network to perform data enhancement on the training set and the physical noise image dataset respectively, and merging the enhanced data to obtain an enhanced training set.

4. The mine image processing method based on deep learning according to claim 1, characterized in that: In S300, the deep operator denoising network includes: a backbone network, a branch network, a dot product layer and a decoding network; The backbone network is used to learn the global feature map of the mine image; The branch network is used to learn the local detail feature map of the mine image; The dot product layer is used to receive the global feature map and the local detail feature map and then perform a dot product operation; The decoding network is used to process the data output by the dot product layer to obtain a reconstructed clean image.

5. The mine image processing method based on deep learning according to claim 4 is characterized in that: The pruning process in S400 includes the following steps: S401, randomly freezing some elements in the k-th layer initial weight matrix of the deep operator denoising network model to obtain a lightweight pruning weight matrix; S402, calculate the L2 norm score between the initial weight matrix and the lightweight pruning weight matrix. If the calculated L2 norm score is greater than the preset lightweight pruning threshold, use the lightweight pruning weight matrix to replace the initial weight matrix. If the L2 norm score is not greater than the preset lightweight pruning threshold, repeat S401~S402.

6. The mine image processing method based on deep learning according to claim 5, characterized in that: In S400, the loss function L1 in the deep operator denoising network training process is as follows: in, H and W are the height and width of the mine image input to the deep operator denoising network, is the first i The labels corresponding to the data samples are is the symbol for modulo calculation, Represents the reconstructed clean image output by the decoding network exist( ), express The image is in ( ), ( ) represents the pixel coordinates of a two-dimensional image.

7. The mine image processing method based on deep learning according to claim 5, characterized in that: The trunk network and the branch network adopt a parallel architecture.

8. The mine image processing method based on deep learning according to claim 5, characterized in that: The backbone network includes 5 2D encoder units.

9. The mine image processing method based on deep learning according to claim 5, characterized in that: The branch network includes an image segmentation layer, a linear mapping layer, a self-attention layer, a cross-attention layer and a feedforward neural network layer.

10. The mine image processing method based on deep learning according to claim 8, characterized in that: The decoding network includes 5 2D decoder units, 5 downsampling units and 5 feature splicing units.

Images