Coal mine feature image mask generation method based on learnable features and convolution kernel

Abstract

The present disclosure proposes a method for generating a coal mine feature image mask based on learnable features and convolution kernels, and relates to the field of coal mine technology. The method includes: using deep separable convolution to process the feature image of the coal mine image, generating a learnable mask, and obtaining a mask feature; applying an iterative average pooling operation to generate a learnable convolution kernel; using the learnable convolution kernel to perform a convolution operation on the feature image, and performing element-by-element multiplication using the mask feature to obtain a mask image corresponding to the coal mine image. The present disclosure can greatly suppress the influence of background interference by generating mask features for background perception. In addition, convolution kernels related to underground scenes can be learned to generate multiple learnable convolution kernels. While realizing the locality of the convolution kernel, underground scene priors are introduced to improve the feature extraction effect. The mask feature is aggregated with the first feature map processed by the convolution kernel, and the background of the coal mine image can be efficiently suppressed.

Description

Technical Field

[0001] The present disclosure relates to the field of coal mine technology, and in particular to a method for generating a coal mine feature image mask based on learnable features and convolution kernels. Background Art

[0002] Applying visual AI algorithms to detect targets underground in coal mines is an important part of coal mine safety management. However, target detection in underground coal mine environments is susceptible to serious interference from dust, water mist, and other factors, and the underground environment has problems such as complex backgrounds, changing illumination, weak textures, and repeated textures. These factors make underground target detection data appear to have low clarity and overall dark colors, which makes underground target detection susceptible to background interference, misleading the designed neural network to detect areas with themselves as backgrounds as specific targets, or even detect them as abnormalities. In actual work scenarios, the number of false alarms in safety management increases dramatically, reducing the effectiveness of safety management.

[0003] When applying the designed target detection algorithm, the feature extraction backbone network mainly performs high-dimensional feature representation of the input image. Due to the above-mentioned unfavorable factors in the underground environment, the high-dimensional feature representation output by the deep backbone network is easily interfered by the background, and it is difficult for the detection head to effectively integrate specific target areas in the subsequent links of the target detection algorithm. For target detection in coal mines, traditional algorithms can manually set masks in areas of no concern, but this method cannot cope with situations such as camera zoom, rotation, and movement, and is also relatively inefficient. Summary of the invention

[0004] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first embodiment of the present disclosure proposes a method for generating a coal mine feature image mask based on learnable features and convolution kernels, comprising:

[0006] Acquire a feature image of a coal mine image to be processed;

[0007] Performing a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception;

[0008] Performing iterative average pooling and point convolution operations on the feature image in sequence to generate m learnable convolution kernels; wherein m is a positive integer and m is a preset value;

[0009] Using the m learnable convolution kernels to perform convolution operations on the feature images respectively, to obtain m first feature maps;

[0010] Performing element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps;

[0011] The m second feature maps are processed to obtain a mask image corresponding to the coal mine image to be processed.

[0012] In some embodiments of the present disclosure, the use of deep separable convolution to perform a convolution operation on the feature image to obtain a single-channel mask feature for background perception includes: performing a first deep separable convolution operation on the feature image to extract the mask area of ​​the feature image to obtain an intermediate feature; performing a second deep separable convolution operation on the intermediate feature, and combining it with spatial convolution to expand the receptive field to obtain the single-channel mask feature for background perception.

[0013] In some embodiments of the present disclosure, the feature image is iteratively averaged pooled and point convolutionally performed in sequence according to the following formula to generate m learnable convolution kernels:

[0014]

[0015] in, is the convolution kernel set, , AAP represents iterative average pooling operation, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism, and the number of groups is set to m.

[0016] In some embodiments of the present disclosure, the element-by-element product operation is a Hadamard product operation.

[0017] The second aspect of the present disclosure provides a target detection method, including:

[0018] Acquire coal mine images to be processed;

[0019] Processing the coal mine image to be processed to obtain a mask image corresponding to the coal mine image to be processed; the mask image is obtained by the method described in the first aspect above;

[0020] Object detection is performed based on the mask image.

[0021] The third aspect of the present disclosure proposes a coal mine feature image mask generation device based on learnable features and convolution kernels, comprising:

[0022] An acquisition module, used for acquiring a characteristic image of a coal mine image to be processed;

[0023] A first processing module, configured to perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception;

[0024] The second processing module is used to perform iterative average pooling operations and point convolution operations on the feature image in sequence to generate m learnable convolution kernels; wherein m is a positive integer and m is a preset value;

[0025] A third processing module, used to use the m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps;

[0026] A fourth processing module is used to perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps;

[0027] The fifth processing module is used to process the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

[0028] In some embodiments of the present disclosure, the first processing module is specifically used to: perform a first depth-separable convolution operation on the feature image to extract the mask area of ​​the feature image to obtain intermediate features; perform a second depth-separable convolution operation on the intermediate features, and combine spatial convolution to expand the receptive field to obtain the single-channel mask features for background perception.

[0029] In some embodiments of the present disclosure, the second processing module sequentially performs an iterative average pooling operation and a point convolution operation on the feature image through the following formula to generate m learnable convolution kernels:

[0030]

[0031] in, is the convolution kernel set, , AAP represents iterative average pooling operation, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism, and the number of groups is set to m.

[0032] In some embodiments of the present disclosure, the element-by-element product operation is a Hadamard product operation.

[0033] A fourth aspect of the present disclosure provides an object detection device, including:

[0034] An acquisition module, used for acquiring the coal mine image to be processed;

[0035] A processing module, used for processing the coal mine image to be processed to obtain a mask image corresponding to the coal mine image to be processed;

[0036] A target detection module, used for performing target detection based on the mask image;

[0037] Wherein, the processing module includes an acquisition unit, a first processing unit, a second processing unit, a third processing unit, a fourth processing unit and a fifth processing unit;

[0038] The acquisition unit is used to acquire a characteristic image of the coal mine image to be processed;

[0039] The first processing unit is used to perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception;

[0040] The second processing unit is used to perform iterative average pooling operations and point convolution operations on the feature image in sequence to generate m learnable convolution kernels; wherein m is a positive integer and m is a preset value;

[0041] The third processing unit is used to use the m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps;

[0042] The fourth processing unit is used to perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps;

[0043] The fifth processing unit is used to process the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

[0044] The coal mine feature image mask generation method based on learnable features and convolution kernels provided by the present disclosure can greatly suppress the influence of background interference by generating mask features for background perception. It can also learn convolution kernels related to underground scenes and generate multiple learnable convolution kernels. While realizing the locality of the convolution kernel, it introduces underground scene priors to improve the feature extraction effect. Aggregating the mask features with the first feature map processed by the convolution kernel can effectively suppress the background of the coal mine image.

[0045] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0047] Figure 1A schematic diagram of a flow chart of a method for generating a coal mine feature image mask based on learnable features and convolution kernels provided in an embodiment of the present disclosure;

[0048] Figure 2 A schematic diagram of a coal mine feature image mask generation network module provided by an embodiment of the present disclosure;

[0049] Figure 3 An effect diagram of mask generation provided by an embodiment of the present disclosure;

[0050] Figure 4 A schematic diagram of a coal mine feature image mask generation device based on learnable features and convolution kernels provided in an embodiment of the present disclosure;

[0051] Figure 5 A schematic diagram of a target detection device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0053] Dynamic region-aware convolution (DRConv) is an efficient convolution method that can be effectively applied to visual processing tasks. Dynamic region-aware convolution DRConv uses argmax to obtain background region perception, and divides the input data into different regions. At the same time, it uses a convolution kernel generation module to generate convolution kernels, and applies different convolution kernels to different background regions for feature extraction, thereby obtaining a high-dimensional feature representation of the module output. For example, in the field of face recognition, DRConv can effectively extract different regions of the face, such as eyes, nose and other regions. However, in the target detection task in the underground environment of a coal mine, due to the complexity of the underground environment of a coal mine, it is difficult for the dynamic region-aware convolution DRConv to distinguish the actual meaning of the underground background region. In addition, DRConv uses argmax to obtain a region-aware mask. Due to the non-differentiable characteristics of the operation itself, there are non-differentiable links in the calculation of the backward process, which requires a complex calculation structure for approximation. Based on this, the present disclosure provides a coal mine feature image mask generation method based on learnable features and convolution kernels, which performs efficient background suppression on coal mine images in target detection tasks. Specifically, the coal mine feature image mask generation method based on learnable features and convolution kernels according to an embodiment of the present disclosure is described below with reference to the accompanying drawings.

[0054] Figure 1The flowchart of a method for generating a coal mine feature image mask based on learnable features and convolution kernels provided by the embodiment of the present disclosure is as follows. Figure 1 As shown, the coal mine feature image mask generation method based on learnable features and convolution kernels may include the following steps:

[0055] Step 101, obtaining a feature image of a coal mine image to be processed.

[0056] Step 102: Perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception.

[0057] Given the feature image of the coal mine image to be processed, , where B represents the batch size and C represents the number of channels. Considering the efficiency of the solution, in order to reduce the number of parameters and the scale of calculation, this paper uses a deep separable convolution to perform convolution operations on the feature image to generate a learnable mask and obtain a single-channel mask feature for background perception. , no need to manually set masks in areas of no concern. To keep the mask size the same as the input feature image size, set stride=1, and the relationship between the convolution kernel size and the padding size is k=2×p+1, where p is the padding size and k is the convolution kernel size. This satisfies the output size constraint without upsampling or deconvolution calculations.

[0058] On the basis of ensuring the solution efficiency, in order to further improve the solution accuracy, in some embodiments of the present disclosure, two convolution operations can be performed on the feature image. The first convolution mainly extracts the mask area, and the second convolution expands the receptive field through spatial convolution to form a mask for perceiving the general area of ​​the background. Optionally, the second convolution can also be calculated by combining an enlarged convolution kernel with a spatial convolution. Figure 2 A schematic diagram of a coal mine feature image mask generation network module provided in an embodiment of the present disclosure. Figure 2 The input image feature tensor X in is the feature image of the coal mine image in this embodiment, and the output feature tensor Y is the mask image corresponding to the coal mine image in this embodiment. Figure 2 As shown in the figure, in the mask generation path of this module, the first depth-separable convolution operation is performed on the feature image to extract the mask area of ​​the feature image and obtain the intermediate feature. The second depth-separable convolution operation is performed on the intermediate feature, and combined with the spatial convolution to expand the receptive field, a mask that perceives the general area of ​​the background is formed, and finally the mask feature of feature fusion dynamic background perception is obtained.

[0059] Step 103, iterative average pooling operation and point convolution operation are performed on the feature image in sequence to generate m learnable convolution kernels, where m is a positive integer and m is a preset value.

[0060] This step corresponds to Figure 2 The convolution kernel generation path in , through which multiple learnable convolution kernels are generated, each convolution kernel size is , where O is the number of channels of the coal mine feature image mask generation network module, and k is the convolution kernel size. In some embodiments of the present disclosure, the feature image can be iteratively averaged pooled and point convolutionally operated in sequence to generate m learnable convolution kernels using the following formula:

[0061]

[0062] in, is the convolution kernel set, , AAP stands for Adaptive Average Pooling, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism. The number of groups is set to m, and the size is The convolution kernel is then chunked and the final convolution kernel size is obtained through the Chunk operation.

[0063] Step 104: Use m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps.

[0064] Use m learnable convolution kernels to perform convolution operations on the feature image respectively, apply the convolution kernel to each channel of the feature image, and after channel aggregation, obtain the first feature map corresponding to each of the m convolution kernels.

[0065] Step 105: perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps.

[0066] Optionally, in some embodiments of the present disclosure, the element-by-element product operation may use a Hadamard product operation. The Hadamard product operation can effectively increase the spatial dimension of the feature representation, thereby making the features more distinguishable. On the other hand, the coal mine feature image mask generation method based on learnable features and convolution kernels proposed in the present disclosure uses convolution for efficient feature extraction, aggregation, and representation, rather than using an attention mechanism for implementation, because feature extraction in the backbone network is more likely to retain more high-frequency information, and experiments show that the attention mechanism is better at capturing global features rather than local high-frequency features, that is, the attention mechanism calculation can be regarded as a low-pass filter.

[0067] Step 106, processing the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

[0068] As an example, Figure 3 This is a mask generation effect diagram provided by an embodiment of the present disclosure. Figure 3 (a) is the collected coal mine image to be processed. Figure 3 (b) is a mask image generated after the coal mine image to be processed is processed using the coal mine feature image mask generation method based on learnable features and convolution kernels provided by the embodiment of the present disclosure. Figure 3 As can be seen from (b), the mask generation method of the embodiment of the present disclosure can effectively suppress the background to adapt to the harsh environment underground.

[0069] Optionally, in some embodiments of the present disclosure, the method described in the embodiments of the present disclosure may be applied to the backbone network of the target detection model (or Figure 2 The coal mine feature image mask generation network module shown in the figure is applied to the backbone network of the target detection model), and the parameters are updated based on back propagation. For example, the coal mine feature image mask generation network module is applied to the deep model backbone network of YOLOv10, and the background area mask is dynamically generated through the module, and the mask features are further aggregated in combination with the Hadamard product, thereby improving the scale of the module feature space. At the same time, the learnable convolution kernel is used in the module to generate features, avoiding the problem that the forward operation is different from the backward operation and the backward operation is not differentiable due to the use of argmax operation in DRConv.

[0070] By implementing the disclosed embodiments, combined with deep separable convolution to generate mask features for background perception, the influence of background interference can be greatly suppressed. In addition, convolution kernels related to underground scenes can be learned to generate multiple learnable convolution kernels. While achieving the locality of the convolution kernel, underground scene priors are introduced to improve the feature extraction effect. Aggregating the mask features with the first feature map processed by the convolution kernel can effectively suppress the background of coal mine images.

[0071] The present disclosure also provides a target detection method, including: obtaining a coal mine image to be processed; processing the coal mine image to be processed using the coal mine feature image mask generation method based on learnable features and convolution kernels in any of the aforementioned embodiments to obtain a mask image corresponding to the coal mine image to be processed; and performing target detection based on the mask image. The target detection method can accurately focus on the non-background area in the mask image, avoid background environment interference to a certain extent, and improve the accuracy of target detection under background interference in coal mine underground scenes.

[0072] Figure 4 A schematic diagram of a coal mine feature image mask generation device based on learnable features and convolution kernels provided in an embodiment of the present disclosure. Figure 4As shown, the coal mine feature image mask generation device based on learnable features and convolution kernels includes: an acquisition module 401, a first processing module 402, a second processing module 403, a third processing module 404, a fourth processing module 405 and a fifth processing module 406.

[0073] The acquisition module 401 is used to acquire the characteristic image of the coal mine image to be processed.

[0074] The first processing module 402 is used to perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception.

[0075] In some embodiments of the present disclosure, the first processing module 402 is specifically used to: perform a first depth-separable convolution operation on the feature image to extract the mask area of ​​the feature image to obtain intermediate features; perform a second depth-separable convolution operation on the intermediate features, and combine them with spatial convolution to expand the receptive field to obtain a single-channel mask feature for background perception.

[0076] The second processing module 403 is used to perform iterative average pooling operations and point convolution operations on the feature image in sequence to generate m learnable convolution kernels, where m is a positive integer and m is a preset value.

[0077] In some embodiments of the present disclosure, the second processing module 403 performs iterative average pooling operations and point convolution operations on the feature image in sequence through the following formula to generate m learnable convolution kernels:

[0078]

[0079] in, is the convolution kernel set, , AAP represents iterative average pooling operation, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism, and the number of groups is set to m.

[0080] The third processing module 404 is used to use m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps.

[0081] The fourth processing module 405 is used to perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps.

[0082] In some embodiments of the present disclosure, the element-by-element product operation is a Hadamard product operation.

[0083] The fifth processing module 406 is used to process the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

[0084] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0085] Figure 5 Schematic diagram of a target detection device provided by an embodiment of the present disclosure. Figure 5 As shown, the target detection device includes: an acquisition module 501, a processing module 502 and a target detection module 503.

[0086] The acquisition module 501 is used to acquire the coal mine image to be processed.

[0087] The processing module 502 is used to process the coal mine image to be processed to obtain a mask image corresponding to the coal mine image to be processed.

[0088] The object detection module 503 is used to perform object detection based on the mask image.

[0089] The processing module 502 includes an acquisition unit 511 , a first processing unit 512 , a second processing unit 513 , a third processing unit 514 , a fourth processing unit 515 and a fifth processing unit 516 .

[0090] The acquisition unit 511 is used to acquire the characteristic image of the coal mine image to be processed.

[0091] The first processing unit 512 is used to perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception.

[0092] The second processing unit 513 is used to perform iterative average pooling operations and point convolution operations on the feature image in sequence to generate m learnable convolution kernels, where m is a positive integer and m is a preset value.

[0093] The third processing unit 514 is used to use m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps.

[0094] The fourth processing unit 515 is used to perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps.

[0095] The fifth processing unit 516 is used to process the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

[0096] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0097] In order to implement the above embodiments, the present disclosure also proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the coal mine feature image mask generation method based on learnable features and convolution kernels provided in the above embodiments, or to implement the target detection method provided in the above embodiments.

[0098] In order to implement the above embodiments, the present disclosure also proposes a computer-readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement the coal mine feature image mask generation method based on learnable features and convolution kernels provided in the above embodiments, or to implement the target detection method provided in the above embodiments.

[0099] In order to implement the above-mentioned embodiments, the present disclosure also proposes a computer program product, including a computer program, which, when executed by a processor, implements the coal mine feature image mask generation method based on learnable features and convolution kernels provided in the above-mentioned embodiments, or implements the target detection method provided in the above-mentioned embodiments.

[0100] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they contradict each other.

[0101] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0102] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0103] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0104] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0105] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0106] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0107] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. A method for generating a coal mine feature image mask based on learnable features and convolution kernels, characterized in that: The following steps are involved: Acquire a feature image of a coal mine image to be processed; A depth-wise separable convolution is used to perform a convolution operation on the feature image to obtain a single-channel mask feature for background perception; wherein the stride is set to 1, and the relationship between the convolution kernel size and the padding size is k=2×p+1, so as to keep the mask size the same as the input feature image size, wherein p is the size of the padding, and k is the size of the convolution kernel; The feature image is sequentially subjected to an iterative average pooling operation and a point convolution operation to generate m learnable convolution kernels; wherein m is a positive integer and m is a preset value; wherein the feature image is sequentially subjected to an iterative average pooling operation and a point convolution operation to generate the m learnable convolution kernels according to the following formula: in, is the convolution kernel set, , AAP represents iterative average pooling operation, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism, and the number of groups is set to m. Using the m learnable convolution kernels to perform convolution operations on the feature images respectively, to obtain m first feature maps; Performing element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps; The m second feature maps are processed to obtain a mask image corresponding to the coal mine image to be processed.

2. The method according to claim 1, characterized in that: The method of performing a convolution operation on the feature image by using a depthwise separable convolution to obtain a single-channel mask feature for background perception includes: Performing a first depth-wise separable convolution operation on the feature image to extract a mask region of the feature image to obtain an intermediate feature; A second depth-wise separable convolution operation is performed on the intermediate features, and combined with spatial convolution to expand the receptive field, so as to obtain the single-channel mask feature for background perception.

3. The method according to claim 1, characterized in that The element-by-element product operation is a Hadamard product operation.

4. A target detection method, characterized in that: include: Acquire coal mine images to be processed; Processing the coal mine image to be processed to obtain a mask image corresponding to the coal mine image to be processed; The mask image is obtained by using the method according to any one of claims 1 to 3; Object detection is performed based on the mask image.

5. A coal mine feature image mask generation device based on learnable features and convolution kernels, characterized in that: include: An acquisition module, used for acquiring a characteristic image of a coal mine image to be processed; The first processing module is used to perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception; wherein the stride is set to 1, and the relationship between the convolution kernel size and the padding size is k=2×p+1, so as to keep the mask size the same as the input feature image size, wherein p is the size of the padding, and k is the convolution kernel size; The second processing module is used to perform iterative average pooling operations and point convolution operations on the feature image in sequence to generate m learnable convolution kernels; wherein m is a positive integer and m is a preset value; wherein the second processing module performs iterative average pooling operations and point convolution operations on the feature image in sequence according to the following formula to generate the m learnable convolution kernels: in, is the convolution kernel set, , AAP represents iterative average pooling operation, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism, and the number of groups is set to m. A third processing module, used to use the m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps; A fourth processing module is used to perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps; The fifth processing module is used to process the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

6. The device according to claim 5, characterized in that The first processing module is specifically used for: Performing a first depth-wise separable convolution operation on the feature image to extract a mask region of the feature image to obtain an intermediate feature; A second depth-wise separable convolution operation is performed on the intermediate features, and combined with spatial convolution to expand the receptive field, so as to obtain the single-channel mask feature for background perception.

7. The device according to claim 5, characterized in that The element-by-element product operation is a Hadamard product operation.

8. A target detection device, characterized in that: include: An acquisition module, used for acquiring the coal mine image to be processed; A processing module, used for processing the coal mine image to be processed to obtain a mask image corresponding to the coal mine image to be processed; A target detection module, used for performing target detection based on the mask image; Wherein, the processing module includes an acquisition unit, a first processing unit, a second processing unit, a third processing unit, a fourth processing unit and a fifth processing unit; The acquisition unit is used to acquire a characteristic image of the coal mine image to be processed; The first processing unit is used to perform a convolution operation on the feature image using a depthwise separable convolution to obtain a single-channel mask feature for background perception; wherein the stride is set to 1, and the relationship between the convolution kernel size and the padding size is k=2×p+1, so as to keep the mask size the same as the input feature image size, wherein p is the size of the padding, and k is the convolution kernel size; The second processing unit is used to perform iterative average pooling operations and point convolution operations on the feature image in sequence to generate m learnable convolution kernels; wherein m is a positive integer and m is a preset value; wherein the m learnable convolution kernels are generated by performing iterative average pooling operations and point convolution operations on the feature image in sequence according to the following formula: in, is the convolution kernel set, , AAP represents iterative average pooling operation, It represents the point wise convolution operation. The outermost PW is calculated using the group mechanism, and the number of groups is set to m. The third processing unit is used to use the m learnable convolution kernels to perform convolution operations on the feature images respectively to obtain m first feature maps; The fourth processing unit is used to perform element-by-element product operations on the m first feature maps and the mask features respectively to obtain m second feature maps; The fifth processing unit is used to process the m second feature maps to obtain a mask image corresponding to the coal mine image to be processed.

Images