A coal mine conveying belt foreign matter detection method, a storage medium and an electronic device

Abstract

This application discloses a method, storage medium, and electronic device for detecting foreign objects on coal mine conveyor belts. The method includes acquiring historical video images of coal mine conveyor belts and constructing a foreign object dataset with annotation information based on these historical video images. Further, a foreign object detection model is constructed based on the dataset. The model includes an adaptive perception backbone network, a dual-domain enhanced Transformer module, and a dynamic multi-scale fusion module. Through multi-scale feature extraction, cross-domain feature modeling, and feature fusion, the model can express the features of foreign objects in complex scenarios. After training the model using the conveyor belt foreign object dataset, real-time video images of coal mine conveyor belts are input into the trained model for detection, and the foreign object recognition result is output. This method enables real-time detection of foreign objects on conveyor belts even in interference environments, significantly improving detection robustness, reducing false positives and false negatives, thereby reducing accidents such as conveyor belt tearing and blockages, ensuring equipment operation safety, and improving coal mine production efficiency.

Description

Technical Field

[0001] This application relates to the technical field of coal mine production, and in particular to a method for detecting foreign objects on a coal mine conveyor belt, a storage medium, and an electronic device. Background Technology

[0002] In modern coal mine production, belt conveyors are core equipment for coal mining and transportation, and their operating status directly affects production efficiency and operational safety. However, due to the complex and variable underground environment, large pieces of coal gangue, anchor bolts, and metal parts are easily mixed in during the conveying process. These foreign objects may cause problems such as belt misalignment, tearing, or even blockage during operation, which not only reduces the service life of the equipment but may also cause production interruptions. In severe cases, it can threaten the safety of workers and cause significant economic losses.

[0003] Existing detection methods mostly rely on manual inspection or traditional vision algorithms, which suffer from poor real-time performance, low recognition accuracy, and insufficient adaptability to complex working conditions, making it difficult to meet the high-precision and automated detection requirements of intelligent mines. Therefore, there is an urgent need for a detection method that can achieve efficient and accurate identification of foreign objects on conveyor belts in complex environments. Summary of the Invention

[0004] The purpose of this application is to overcome the above-mentioned problems and provide a foreign object detection, storage medium and electronic device for coal mine conveyor belts.

[0005] The technical solution of this application provides a method for detecting foreign objects on a coal mine conveyor belt, including: Acquire historical video images of coal mine conveyor belts; A dataset of foreign objects on conveyor belts was constructed based on the historical video images of coal mine conveyor belts. A conveyor belt foreign object detection model is constructed based on the conveyor belt foreign object dataset. The conveyor belt foreign object detection model includes an adaptive perception backbone network, a dual-domain enhanced Transformer module, and a dynamic multi-scale fusion module. The conveyor belt foreign object detection model is trained based on the aforementioned conveyor belt foreign object dataset. Real-time video images of coal mine conveyor belts are input into the trained conveyor belt foreign object detection model to obtain detection results.

[0006] Furthermore, the conveyor belt foreign object dataset includes historical coal mine conveyor belt images after data preprocessing; When the conveyor belt foreign object detection model includes an adaptive sensing backbone network, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: Based on the preprocessed historical coal mine conveyor belt images, multiple convolutional downsampling operations are performed to construct a multi-scale feature pyramid; the multi-scale feature pyramid includes input features. A bidirectional feature fusion module is embedded in each scale layer of the multi-scale feature pyramid; the bidirectional feature fusion module is used to adjust and split the input features by channels, and then splice and fuse the split input features. An adaptive gated Transformer is constructed inside the bidirectional feature fusion module; the adaptive gated Transformer is used to adaptively enhance the input features through a dual-path residual structure; A multi-receptive-field feature integrator is constructed inside the adaptive gated Transformer; the multi-receptive-field feature integrator is used to dynamically integrate input features at multiple scales; A morphology-guided convolutional unit is constructed inside the multi-receptive-field feature integrator; the morphology-guided convolutional unit is used to adaptively adjust the convolutional kernel weights according to the input features through multiple convolutional branches of different shapes and a dynamic weighting mechanism. The bidirectional feature fusion module, the adaptive gated Transformer, the multi-receptive field feature integrator, and the morphology-guided convolutional unit are integrated hierarchically to form an adaptive perceptual backbone network.

[0007] Furthermore, the embedding of a bidirectional feature fusion module in each scale layer of the multi-scale feature pyramid specifically includes: The input features of each scale layer are adjusted and split by 1×1 convolution to divide the input features into first-path features and second-path features. The first feature path is fed into a cascaded adaptive gated Transformer for enhancement processing; The second feature path is used to extract basic features through 3×3 ordinary convolution combined with batch normalization and the first preset activation function; The enhanced first-path features and the extracted second-path features are concatenated and fused according to the channel dimension to obtain the output features of the bidirectional feature fusion module. The construction of an adaptive gated Transformer within the bidirectional feature fusion module specifically includes: The first path features are normalized for the first time, and the normalized first path features are sent to the multi-receptive field feature integrator for dynamic integration of multi-scale features. The output of the multi-receptive-field feature integrator is added to the first path feature element by element to obtain the first path residual feature; The residual features of the first path are normalized a second time, and the normalized residual features of the first path are fed into a convolutional gated linear unit for nonlinear transformation and adaptive gating of the features. The output of the convolutional gated linear unit is added to the residual feature of the first path element by element to obtain the output feature of the adaptive gated Transformer.

[0008] Furthermore, the construction of a multi-receptive-field feature integrator within the adaptive gated Transformer specifically includes: The first feature after the first normalization process is split into two parallel feature sub-maps by channel dimension. The two sets of feature maps are fed into two parallel morphology-guided convolutional units to obtain multimorphic features. The multi-morphological features are concatenated according to the channel dimension to obtain the complete feature dimension; By performing channel fusion and dimension adjustment on the complete feature dimension through 1×1 convolution, a feature map with a unified dimension is obtained; The construction of morphology-guided convolutional units within the multi-receptive-field feature integrator specifically includes: Square convolution, horizontal strip convolution, and vertical strip convolution are applied to the feature map respectively to extract multimorphic features and obtain multimorphic features; Adaptive average pooling is performed on the feature map, and dynamic weight vectors corresponding to square convolution, horizontal strip convolution and vertical strip convolution are generated by 1×1 convolution and the second preset activation function, respectively. The output of the morphological guided convolution unit is obtained by multiplying the multimorphic features of square convolution, horizontal strip convolution, and vertical strip convolution with the corresponding dynamic weight vectors element by element, and then adding the weighted features according to the channel dimension.

[0009] Furthermore, the process of performing multiple convolutional downsampling operations on the preprocessed historical coal mine conveyor video images to construct a multi-scale feature pyramid specifically includes: The historical coal mine conveyor belt video images after data preprocessing were subjected to four consecutive convolutional downsampling operations to obtain a four-layer multi-scale feature pyramid. The downsampling operation is implemented through a 3×3 convolution with a kernel stride of 2, and combined with batch normalization and a preset activation function to perform nonlinear transformation of features.

[0010] Furthermore, when the conveyor belt foreign object detection model includes a dual-domain enhanced Transformer module, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: A bilinear attention module is constructed based on the multi-scale features output by the adaptive perception backbone network, and spatial domain features are calculated based on the bilinear attention module. The bilinear attention is used to split the multi-scale features into positive and negative polarity branches and to calculate attention. The spatial domain features are fused with the multi-scale features through residual connections and then normalized to obtain normalized spatial domain features. A frequency domain modulation network is constructed, and a fast Fourier transform is performed on the normalized spatial domain features to convert them to the frequency domain. After modulating different frequency components with a learnable frequency domain gate mask, the network is inversely transformed back to the time domain to obtain frequency domain enhancement features. The normalized spatial domain features and the frequency domain enhanced features are fused together and normalized again to obtain the output features of the dual-domain enhanced Transformer module.

[0011] Furthermore, when the conveyor belt foreign object detection model includes a dynamic multi-scale fusion module, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: Based on the output features of the dual-domain enhanced Transformer module, a multi-path parallel processing architecture is used to extract and fuse multi-scale features to obtain adaptation features, deep convolutional extracted features, and enhanced sub-features. The multi-path parallel processing architecture includes a direct-connected adaptation path, a deep convolutional feature extraction path, and an omnidirectional feature enhancement perceptual block cascaded enhancement path. The adaptation features, depthwise convolutional extracted features, and enhancement sub-features are concatenated along the channel dimension and then integrated through a 1×1 convolution to obtain the output features of the dynamic multi-scale fusion module.

[0012] The technical solution of this application also provides a storage medium that stores computer instructions. When the computer executes the computer instructions, it is used to perform the foreign object detection method for coal mine conveyor belts as described above.

[0013] The technical solution of this application also provides an electronic device, including at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed, enable the at least one processor to perform the aforementioned method for detecting foreign objects on a coal mine conveyor belt.

[0014] The technical solution of this application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the foreign object detection method for coal mine conveyor belts as described above.

[0015] The above technical solution has the following beneficial effects: This application discloses a method, storage medium, and electronic device for detecting foreign objects on coal mine conveyor belts. The method includes acquiring historical video images of coal mine conveyor belts and constructing a foreign object dataset with annotation information based on these historical video images. Further, a foreign object detection model is constructed based on the dataset. The model includes an adaptive perceptual backbone network, a dual-domain enhanced Transformer module, and a dynamic multi-scale fusion module. Through multi-scale feature extraction, cross-domain feature modeling, and feature fusion, the model can express the features of foreign objects in complex scenarios. After training the model using the conveyor belt foreign object dataset, real-time video images of coal mine conveyor belts are input into the trained model for detection, and the foreign object recognition result is output. This method enables real-time detection of foreign objects on conveyor belts even in interference environments, significantly improving detection robustness, reducing false positives and false negatives, thereby reducing accidents such as conveyor belt tearing and blockages, ensuring equipment operation safety, and improving coal mine production efficiency. Attached Figure Description

[0016] The disclosure of this application will become more readily understood with reference to the accompanying drawings. It should be understood that these drawings are for illustrative purposes only and are not intended to limit the scope of protection of this application. In the drawings: Figure 1 This is a flowchart of a foreign object detection method for coal mine conveyor belts in one embodiment of this application; Figure 2 This is a model overall framework diagram of a foreign object detection method for coal mine conveyor belts according to one embodiment of this application; Figure 3 This is a flowchart of the adaptive sensing backbone network in one embodiment of this application; Figure 4 This is a flowchart of the morphology-guided convolution unit in one embodiment of this application; Figure 5 This is a flowchart of the dual-domain enhanced Transformer module in one embodiment of this application; Figure 6 This is a flowchart of the dynamic multi-scale fusion module in one embodiment of this application; Figure 7 This is a flowchart of a foreign object detection method for a coal mine conveyor belt in one embodiment of this application; Figure 8 This is a flowchart of a foreign object detection method for coal mine conveyor belts in another embodiment of this application; Figure 9 This is a flowchart of the bidirectional feature fusion module in one embodiment of this application; Figure 10 This is a flowchart of the adaptive gated Transformer in one embodiment of this application; Figure 11This is a flowchart of the multi-receptive field feature integrator in one embodiment of this application; Figure 12 This is a flowchart of the morphology-guided convolution unit in one embodiment of this application; Figure 13 This is a schematic diagram of the hardware structure of an electronic device in one embodiment of this application. Detailed Implementation

[0017] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0018] It is readily understood that, based on the technical solution of this application, various structural and implementation methods can be interchanged by those skilled in the art without altering the essential spirit of this application. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this application and should not be considered as the entirety of this application or as limitations or restrictions on the technical solution of the application.

[0019] The directional terms such as up, down, left, right, front, back, front, back, top, and bottom mentioned or possibly used in this specification are defined relative to the structures shown in the accompanying drawings. These are relative concepts and may therefore vary depending on their location and usage. Therefore, these or other directional terms should not be interpreted as restrictive. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0020] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meanings of the above in this application according to the specific circumstances.

[0021] like Figure 1 As shown, a flowchart of a foreign object detection method for a coal mine conveyor belt according to an embodiment of this application includes: Step S101: Acquire historical video images of coal mine conveyor belts; Step S102: Construct a foreign object dataset for the conveyor belt based on the historical coal mine conveyor belt video images; Step S103: Construct a conveyor belt foreign object detection model based on the conveyor belt foreign object dataset. The conveyor belt foreign object detection model includes an adaptive sensing backbone network, a dual-domain enhanced Transformer module, and a dynamic multi-scale fusion module. Step S104: Train the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset; Step S105: Input the real-time video image of the coal mine conveyor belt into the trained conveyor belt foreign object detection model to obtain the detection result.

[0022] Specifically, in step S101, video images of the coal mine conveyor belt are acquired using industrial cameras or monitoring equipment deployed along the conveyor belt and stored as historical coal mine conveyor belt video images. In step S102, the historical coal mine conveyor belt video images are first preprocessed, including keyframe extraction and noise reduction, and foreign objects in the images are labeled. The labeling information includes the category and location information of the foreign objects, thereby constructing a conveyor belt foreign object dataset. The foreign object categories include targets such as coal gangue, anchor bolts, or metal foreign objects. The labeling accuracy is improved through manual labeling or semi-automatic labeling.

[0023] In step S103, a conveyor belt foreign object detection model is constructed based on the conveyor belt foreign object dataset. The model includes an adaptive perception backbone network, a dual-domain enhancement Transformer module, and a dynamic multi-scale fusion module. The adaptive perception backbone network is used to extract feature information at different scales and enhance the perception capability of foreign objects of different shapes through a multi-branch structure. The dual-domain enhancement Transformer module is used to globally model the features of the spatial domain and the channel domain to improve the integrity of feature representation. The dynamic multi-scale fusion module is used to adaptively weight and fuse features at different levels, thereby enhancing the model's ability to identify small-sized foreign objects and complex background interference.

[0024] In step S104, the constructed conveyor belt foreign object detection model is trained based on the conveyor belt foreign object dataset. By setting a loss function and using iterative optimization, the model parameters are updated so that the model can gradually converge and accurately learn the feature representations of different types of foreign objects. The training samples can be expanded by data augmentation, including operations such as rotation, random cropping, and brightness adjustment, to further improve the robustness and generalization ability of the model.

[0025] Data augmentation can include adding noise, adjusting brightness, rotating, randomly cropping, and color equalization to images. Specifically, adding noise can introduce random noise into images to simulate noise interference generated during imaging at night or in low-light environments, making the training data closer to the complex lighting environment of actual coal mines, thereby improving the robustness of the model under low signal-to-noise ratio conditions. It can also randomly superimpose occluded regions or objects into images to construct scenes where some targets are occluded, thereby enhancing the model's ability to identify incomplete targets and improving the accuracy of detecting occluded foreign objects under actual working conditions.

[0026] In step S105, the real-time video images of the coal mine conveyor belt are input into the trained conveyor belt foreign object detection model for processing, thereby outputting the type and location information of the foreign object; furthermore, the alarm or linkage control system can be triggered according to the detection results to decelerate or stop the conveyor belt, thereby realizing real-time detection and early warning of coal gangue, anchor bolts and metal foreign objects, effectively reducing the risk of conveyor belt tearing and blockage, and improving the safety and stability of equipment operation.

[0027] In one embodiment, a foreign object dataset for the conveyor belt is constructed based on the historical coal mine conveyor belt video images, specifically including: The historical coal mine conveyor belt video images are preprocessed to obtain historical coal mine conveyor belt images; the preprocessing operations include keyframe extraction and noise reduction. Foreign object targets are labeled on the historical coal mine conveyor belt images to obtain labeling information; the labeling information includes foreign object category and location information.

[0028] In this embodiment, by extracting keyframes from historical coal mine conveyor belt video images, coal mine conveyor belt images can be obtained, and noise reduction processing can be performed on them, which can effectively reduce the interference caused by sensor noise and other factors in the coal mine environment.

[0029] After preprocessing, foreign objects in the image are labeled. The labeling information includes the type of foreign object and its location in the image. Typical foreign object types include coal gangue, anchor bolts, and metallic foreign objects. Labeling can be done manually or semi-automatically using auxiliary labeling tools to improve efficiency and accuracy. This labeling process provides the data with clear supervisory information, which is beneficial for the model to learn the characteristic differences of different types of foreign objects.

[0030] In one embodiment, the conveyor belt foreign object dataset includes preprocessed historical coal mine conveyor belt images; When the conveyor belt foreign object detection model includes an adaptive sensing backbone network, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: Based on the preprocessed historical coal mine conveyor belt images, multiple convolutional downsampling operations are performed to construct a multi-scale feature pyramid; the multi-scale feature pyramid includes input features. A bidirectional feature fusion module is embedded in each scale layer of the multi-scale feature pyramid; the bidirectional feature fusion module is used to adjust and split the input features by channels, and then splice and fuse the split input features. An adaptive gated Transformer is constructed inside the bidirectional feature fusion module; the adaptive gated Transformer is used to adaptively enhance the input features through a dual-path residual structure; A multi-receptive-field feature integrator is constructed inside the adaptive gated Transformer; the multi-receptive-field feature integrator is used to dynamically integrate input features at multiple scales; A morphology-guided convolutional unit is constructed inside the multi-receptive-field feature integrator; the morphology-guided convolutional unit is used to adaptively adjust the convolutional kernel weights according to the input features through multiple convolutional branches of different shapes and a dynamic weighting mechanism. The bidirectional feature fusion module, the adaptive gated Transformer, the multi-receptive field feature integrator, and the morphology-guided convolutional unit are integrated hierarchically to form an adaptive perceptual backbone network.

[0031] In this embodiment, the conveyor belt foreign object dataset includes preprocessed historical coal mine conveyor belt images. When the conveyor belt foreign object detection model uses an adaptive perceptual backbone network, the model construction first performs multiple convolutional downsampling operations based on the processed conveyor belt images to form a multi-scale feature pyramid containing shallow details and deep semantics. A bidirectional feature fusion module is embedded in each scale layer to adjust and split the input features through channels, and enhances the texture and morphological features of the foreign objects while preserving the original information through splicing and fusion. The bidirectional feature fusion module further constructs an adaptive gated Transformer, which adaptively enhances the features through a dual-path residual structure. The adaptive gated Transformer integrates a multi-receptive field feature integrator to dynamically integrate features at multiple scales. The multi-receptive field feature integrator is equipped with a morphology-guided convolutional unit, which adaptively adjusts the convolutional kernel weights according to the input features through multi-morphological convolutional branches and dynamic weighting mechanisms to achieve accurate capture of diverse foreign object morphologies. By hierarchically integrating a bidirectional feature fusion module, an adaptive gated Transformer, a multi-receptive field feature integrator, and a morphology-guided convolutional unit, a complete adaptive perception backbone network is formed, which effectively improves the model's ability to extract features from foreign objects of different scales and shapes, and enhances the robustness and accuracy of the detection model.

[0032] This embodiment utilizes a structured, multi-level feature extraction network to achieve high-precision, real-time detection of foreign objects on conveyor belts in complex coal mine environments. The adaptive perception backbone network, while preserving original feature information, enhances the perception of foreign object texture and morphology, effectively mitigating the feature degradation and detail loss problems that may occur in traditional convolutional networks during deep feature extraction. This provides highly recognizable feature maps for subsequent foreign object detection modules, significantly improving the reliability and practical value of the entire detection system.

[0033] In one embodiment, embedding a bidirectional feature fusion module at each scale layer of the multi-scale feature pyramid specifically includes: The input features of each scale layer are adjusted and split by 1×1 convolution to divide the input features into first-path features and second-path features. The first feature path is fed into a cascaded adaptive gated Transformer for enhancement processing; The second feature path is used to extract basic features through 3×3 ordinary convolution combined with batch normalization and the first preset activation function; The enhanced first-path features and the extracted second-path features are concatenated and fused according to the channel dimension to obtain the output features of the bidirectional feature fusion module.

[0034] In this embodiment, a bidirectional feature fusion module is embedded in each scale layer of the multi-scale feature pyramid. Specifically, the input features at each scale layer are first adjusted and split using 1×1 convolutions, dividing the input features into two paths: the first path serves as the enhancement feature processing path, and the second path serves as the basic feature processing path. The first path of the enhancement path is fed into a cascaded adaptive gated Transformer. Through a multi-layer residual structure and a multi-receptive field feature integrator, adaptive enhancement of the input features is achieved, effectively capturing the texture details and morphological features of foreign objects while mitigating the problem of deep feature degradation. The second path of the basic path extracts basic features through 3×3 ordinary convolutions, batch normalization, and a first preset activation function, preserving the low-level information and detailed features of the original image, providing a stable foundation for subsequent feature fusion. The first preset activation function can be the SiLU activation function.

[0035] The specific formula is as follows: ; in, X As input features, This represents the two-way features obtained by adjusting and splitting the input features through a 1×1 convolution; M i This represents the i-th adaptive gated Transformer, used to enhance the multi-scale texture and morphological information of the first path features; C 2 indicates the output convolutional layer, which is used to fuse the dual-path features and the enhanced features after cascaded AGT processing to form the output feature map of the final bidirectional feature fusion module.

[0036] The expression for the SiLU activation function is as follows: ; Where x is the input feature, This represents the Sigmoid function. The SiLU function achieves a smooth, non-linear mapping of features by multiplying the input by its Sigmoid weights. Compared to the traditional ReLU function, it retains some response in the negative region and has smoother gradient changes, which is beneficial for the stable training of deep networks. Applying the SiLU activation function to the basic feature extraction path can enhance the continuity and non-linearity of feature representation while preserving detailed information.

[0037] The bidirectional feature fusion module can propagate information through multiple paths, enhancing the expressive power of features and making it suitable for capturing the complex geometric shapes and diverse texture features of foreign objects on coal mine conveyor belts. Simultaneously, through the dual-path design, it maintains the basic features of the original image, ensuring that small targets and detailed foreign objects are not lost; on the other hand, the enhanced path strengthens the semantic and morphological features of the foreign objects, significantly improving the detection model's accuracy in recognizing foreign objects at multiple scales and in various forms, further enhancing the robustness and generalization ability of conveyor belt foreign object detection.

[0038] In one embodiment, the construction of an adaptive gated Transformer within the bidirectional feature fusion module specifically includes: The first path features are normalized for the first time, and the normalized first path features are sent to the multi-receptive field feature integrator for dynamic integration of multi-scale features. The output of the multi-receptive-field feature integrator is added to the first path feature element by element to obtain the first path residual feature; The residual features of the first path are normalized a second time, and the normalized residual features of the first path are fed into a convolutional gated linear unit for nonlinear transformation and adaptive gating of the features. The output of the convolutional gated linear unit is added to the residual feature of the first path element by element to obtain the output feature of the adaptive gated Transformer.

[0039] In this embodiment, the adaptive gated Transformer's internal structure achieves adaptive enhancement of foreign object features through multi-level feature processing. First, the first-path features output by the bidirectional feature fusion module are normalized to stabilize the feature distribution and mitigate the vanishing or exploding gradient problem. The normalized features are then input into the multi-receptive-field feature integrator, which integrates features of different scales and shapes through parallel convolutional branches and dynamic weighting mechanisms, achieving dynamic extraction of multi-scale foreign object features.

[0040] Then, the output of the multi-receptive-field feature integrator is added element-wise with the original first-path features to form the first-path residual features. This operation preserves the original feature information while introducing enhanced features after multi-receptive-field processing. Next, the first-path residual features undergo a second normalization process and are input into a convolutional gated linear unit for nonlinear transformation and adaptive gating. The convolutional gated linear unit adaptively adjusts its output based on the input features, selectively enhancing important features and suppressing irrelevant information. Finally, the output of the convolutional gated linear unit is added element-wise with the first-path residual features again to obtain the final output features of the adaptively gated Transformer module.

[0041] This embodiment effectively alleviates the feature degradation problem in deep networks through a dual residual and adaptive enhancement mechanism, while improving the model's ability to perceive the complex texture and shape of foreign objects on the conveyor belt, providing highly recognizable, multi-scale foreign object features for subsequent detection modules.

[0042] The specific formula is as follows: ; ; in, The residual features of the first path; X The input features are the first-path features derived from the bidirectional feature fusion module. N 1 (X) Indicates input features X The first normalization operation performed; This indicates element-wise multiplication; F MFFI (N 1 (X)) λ represents the enhanced output of the multi-receptive-field feature integrator on the normalized features; λ1 is a learnable layer scaling parameter used to adjust the contribution weight of the multi-receptive-field feature integrator output in the residual connections.

[0043] N 2 ( ) This indicates the second normalization operation performed on the residual features of the first path; F CGLU (N 2 ( )) This represents the output of the convolutional gated linear unit, which performs nonlinear transformation and adaptive gating on the normalized features. λ 2 represents the learnable layer scaling parameter for the second residual path; ZThis represents the final output feature of the adaptive gated Transformer, which achieves adaptive enhancement and nonlinear transformation of the input features through dual residual connections.

[0044] This embodiment utilizes hierarchical residuals and adaptive enhancement mechanisms to enable the network to effectively extract multi-scale and multi-morphological foreign object features while preserving the original feature information, thereby significantly improving the accuracy and robustness of foreign object detection on conveyor belts.

[0045] In one embodiment, the construction of a multi-receptive-field feature integrator within the adaptive gated Transformer specifically includes: The first feature after the first normalization process is split into two parallel feature sub-maps by channel dimension. The two sets of feature maps are fed into two parallel morphology-guided convolutional units to obtain multimorphic features. The multi-morphological features are concatenated according to the channel dimension to obtain the complete feature dimension; By performing channel fusion and dimension adjustment on the complete feature dimensions through 1×1 convolution, a feature map with unified dimensions is obtained.

[0046] In this embodiment, a multi-receptive-field feature integrator is constructed within the adaptive gated Transformer to enhance the multi-scale and multi-morphological representation capabilities of the input features. Specifically, the first-path features, after initial normalization, are first split along the channel dimension, dividing them into two parallel feature sub-maps to achieve independent processing of features across different channel dimensions. Subsequently, these two sets of feature sub-maps are input into two parallel morphology-guided convolutional units. Each morphology-guided convolutional unit extracts multi-morphological, multi-receptive-field features from the input features through various convolutional branches, such as square convolution, horizontal strip convolution, and vertical strip convolution, as well as a dynamic weighting mechanism. This allows for adaptive adjustment of the contributions of different convolutional branches based on the geometric shape of the foreign object, achieving accurate capture of the complex texture and morphological features of foreign objects on coal mine conveyor belts.

[0047] Then, the multi-morphological features output from the two morphologically guided convolutional units are concatenated according to the channel dimension to restore the complete feature dimension. Next, a 1×1 convolution is used for channel fusion and dimension adjustment, converting the concatenated feature map into a feature map of a unified dimension, forming an output that integrates multi-scale and multi-morphological features. Technically, through the multi-receptive field feature integrator module, the adaptive gated Transformer can fully integrate features from different receptive fields and morphologies while preserving the original information of the input features. This enhances the network's ability to recognize foreign object textures, edges, and shapes, effectively improving the robustness and accuracy of the coal mine conveyor belt foreign object detection model in complex environments. It also provides high-quality, multi-scale basic feature input for the subsequent adaptive enhancement of convolutionally gated linear units.

[0048] In one embodiment, constructing a morphology-guided convolutional unit within the multi-receptive-field feature integrator specifically includes: Square convolution, horizontal strip convolution, and vertical strip convolution are applied to the feature map respectively to extract multimorphic features and obtain multimorphic features; Adaptive average pooling is performed on the feature map, and dynamic weight vectors corresponding to square convolution, horizontal strip convolution and vertical strip convolution are generated by 1×1 convolution and the second preset activation function, respectively. The output of the morphological guided convolution unit is obtained by multiplying the multimorphic features of square convolution, horizontal strip convolution, and vertical strip convolution with the corresponding dynamic weight vectors element by element, and then adding the weighted features according to the channel dimension.

[0049] In this embodiment, a morphology-guided convolutional unit is constructed within the multi-receptive-field feature integrator to further enhance the adaptability of features to the morphology of foreign objects. Specifically, the input feature map is first divided into several feature sub-maps by channel, and three different types of convolutional branches are set for each sub-map, such as square convolution, horizontal strip convolution, and vertical strip convolution. Through these convolutional branches, feature information can be extracted from different geometric shapes and directions, achieving multi-morphological capture of foreign object textures, edges, and structures, thereby improving the model's ability to identify complex-shaped foreign objects.

[0050] The morphology-guided convolutional unit also performs adaptive average pooling on the feature map and generates dynamic weight vectors for each convolutional branch through 1×1 convolution and a second preset activation function. These dynamic weights are used to weight the multimorphic features of square convolution, horizontal strip convolution, and vertical strip convolution element-wise, and then summed along the channel dimension to obtain the final output of the morphology-guided convolutional unit. This structure, by combining multimorphic convolutional branches with a dynamic weighting mechanism, achieves adaptive perception of the geometric shape of foreign objects, enabling the model to extract foreign object features more accurately in complex coal mine environments, retain detailed information, and improve detection robustness and accuracy, while alleviating the feature degradation problem in deep network training.

[0051] Adaptive average pooling is a pooling operation that adaptively adjusts the output size based on the size of the input feature map. It compresses a feature map of arbitrary size into a fixed-size representation by averaging the input feature map across spatial regions. Specifically, adaptive average pooling extracts global or local statistical feature information by averaging the spatial regions within each channel, thereby obtaining a descriptive vector that reflects the overall feature distribution.

[0052] This embodiment uses adaptive average pooling to compress input features into global semantic information, effectively extracting global information from the input features and providing a basis for dynamic weight generation. This allows different convolutional branches to adaptively adjust their contribution ratios according to the shape of foreign objects, thereby improving the model's ability to perceive complex-shaped foreign objects. At the same time, combining multimorphic convolution and residual connection structures not only enhances feature representation capabilities but also ensures the effective transmission of original information, significantly improving the accuracy and robustness of conveyor belt foreign object detection. This is used to generate dynamic weights for different convolutional branches, making weight allocation more dependent on the actual content of the current input features.

[0053] The specific calculation formula is as follows: ; in, G(X) Indicates based on input features X The generated global guiding weight, α i (X) This represents the dynamic weight coefficient of the i-th branch, where the dynamic weight is determined by the input features. X Adaptive average pooling combined with an attention mechanism is used to generate the data, and the data is normalized using the Softmax function to characterize the importance of different convolutional branches under the current input features. K i (X) This indicates that the i-th convolutional branch is related to the input features. X The feature response or mapping result obtained after processing; This indicates element-wise multiplication; B() This indicates batch normalization, which is used to stabilize feature distribution and accelerate model convergence. σ() This represents the activation function, used to introduce a nonlinear mapping and constrain the output range; it can be the SiLU activation function.

[0054] Dynamic weights α i (X) Generated through an attention mechanism, its calculation formula is as follows: ; in, A(X) Indicates input features X Global obtained by performing adaptive average pooling operation feature Vectors are used to extract global statistical information from input features; P() The weight mapping function is used to map the global feature vector to the weight response values ​​corresponding to each branch. The weight mapping function can be implemented by a fully connected layer or a 1×1 convolution. softmax () This represents the normalization function, used to normalize the weight response values ​​of each branch, thereby obtaining dynamic weight coefficients that satisfy the condition that the sum of the weights equals 1. a 1 (X) , a 2 (X) and a 3 (X) .

[0055] This embodiment utilizes adaptive average pooling to extract global statistical information from input features, enabling the model to perceive the distribution characteristics and semantic information of foreign objects in conveyor belt images from a holistic perspective. Secondly, it performs nonlinear transformation on global features through a weight mapping function and combines this with a Softmax normalization mechanism to establish a contribution competition relationship among multiple convolutional branches, achieving adaptive allocation of feature contributions for each branch. On the one hand, it can dynamically highlight feature branches more sensitive to foreign object recognition based on different input scenarios, effectively enhancing the representation ability of foreign objects with complex shapes, different scales, and diverse textures. On the other hand, it can suppress the influence of redundant or interfering features, improving the effectiveness and stability of feature fusion. It exhibits higher detection accuracy and robustness in foreign object detection on coal mine conveyor belts, especially under complex working conditions, lighting changes, and strong background interference, still achieving stable and reliable detection results.

[0056] In one embodiment, the bidirectional feature fusion module, the adaptive gated Transformer, the multi-receptive field feature integrator, and the morphology-guided convolutional unit are integrated hierarchically to form an adaptive perceptual backbone network. Specifically, this includes inputting features from each scale of the multi-scale feature pyramid into the bidirectional feature fusion module. Through channel splitting and dual-path processing, one path is fed into a cascaded adaptive gated Transformer to enhance texture and morphological features, while the other path performs basic convolutional feature extraction. The two paths are then concatenated and fused along the channel dimension. Inside the adaptive gated Transformer, the first path features are normalized and then fed into the multi-receptive field feature integrator for dynamic integration of multi-scale features. The multi-receptive field feature integrator extracts multi-morphological and multi-receptive field features through parallel convolutional gated linear units and dynamically weights each convolutional branch. Finally, the weighted fused features are residually connected to the original input features to achieve adaptive enhancement. Through hierarchical integration, the modules work collaboratively, enabling the network to enhance the texture, morphology, and multi-scale information of foreign objects while preserving the original image details.

[0057] In one embodiment, the process of performing multiple convolutional downsampling operations on the preprocessed historical coal mine conveyor belt image to construct a multi-scale feature pyramid specifically includes: The historical coal mine conveyor belt image after data preprocessing was subjected to four consecutive convolutional downsampling operations to obtain a four-layer multi-scale feature pyramid. The downsampling operation is implemented through a 3×3 convolution with a kernel stride of 2, and combined with batch normalization and a preset activation function to perform nonlinear transformation of features.

[0058] In this embodiment, multiple convolutional downsampling operations are performed on the preprocessed historical coal mine conveyor belt image to construct a multi-scale feature pyramid. Specifically, this involves performing four consecutive convolutional downsampling operations on the preprocessed image. Each downsampling operation uses a 3×3 convolution with a kernel stride of 2, combined with batch normalization and activation functions such as SiLU, to progressively reduce the spatial resolution of the feature map and increase the number of channels, thereby forming a multi-scale feature representation containing both shallow detail information and deep semantic information. This multi-level feature pyramid can effectively cover foreign objects of different scales, enabling the model to capture the texture features of small foreign objects as well as understand the overall shape of larger foreign objects.

[0059] This embodiment utilizes a multi-scale feature pyramid design to fully leverage multi-layered image information, enabling comprehensive feature extraction of foreign objects on coal mine conveyor belts. Shallow features preserve details and edge information, aiding in the accurate localization of small foreign objects; deep features enhance semantic understanding, improving the model's ability to recognize complex-shaped foreign objects. Combining convolutional downsampling with batch normalization and nonlinear transformations of activation functions makes the feature extraction process more efficient, thereby improving the accuracy, robustness, and generalization ability of the detection model in multi-scale, multi-shaped foreign object recognition.

[0060] In another embodiment, when the conveyor belt foreign object detection model includes a dual-domain enhanced Transformer module, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: A bilinear attention module is constructed based on the multi-scale features output by the adaptive perception backbone network, and spatial domain features are calculated based on the bilinear attention module. The bilinear attention is used to split the multi-scale features into positive and negative polarity branches and to calculate attention. The spatial domain features are fused with the multi-scale features through residual connections and then normalized to obtain normalized spatial domain features. A frequency domain modulation network is constructed, and a fast Fourier transform is performed on the normalized spatial domain features to convert them to the frequency domain. After modulating different frequency components with a learnable frequency domain gate mask, the network is inversely transformed back to the time domain to obtain frequency domain enhancement features. The normalized spatial domain features and the frequency domain enhanced features are fused together and normalized again to obtain the output features of the dual-domain enhanced Transformer module.

[0061] In this embodiment, when the conveyor belt foreign object detection model includes a dual-domain enhanced Transformer module, based on the multi-scale features output by the adaptive sensing backbone network, a bilinear attention module is first constructed. The multi-scale features are then split into positive and negative polarity branches according to the bilinear attention module, and attention weights are calculated separately to obtain spatial domain features. These features are then fused with the multi-scale features through residual connections to obtain normalized spatial domain features. This effectively highlights the salient areas of foreign objects in space while suppressing background interference. During the calculation of normalized spatial domain features, the bilinear attention module combines local and global information from the multi-scale features to achieve adaptive sensing of foreign objects at different scales, improving the spatial localization capability for small targets and complex-shaped foreign objects.

[0062] Then, based on the normalized spatial domain features, a frequency domain modulation network is constructed. The normalized spatial domain features are transformed to the frequency domain using a Fast Fourier Transform (FFT), and different frequency components are analyzed. High-frequency, low-frequency, and mid-frequency features are adaptively modulated using a learnable frequency domain gating mask. Subsequently, the modulated frequency domain features are restored to the time domain using an Inverse Fourier Transform (IFT), resulting in enhanced frequency domain features. This enhances the texture details and edge information of foreign objects in the frequency domain while suppressing noise and periodic background interference, supplementing the deficiencies of spatial domain features, and achieving multi-domain feature complementarity.

[0063] Finally, the normalized spatial domain features and frequency domain enhanced features are fused and then passed through a normalization layer again to obtain the output features of the dual-domain enhanced Transformer module. This approach balances spatial saliency and frequency texture information, making the detection model more accurate in recognizing multi-scale and multi-morphological foreign objects. It can fully exploit the spatial and frequency domain information of foreign objects on the conveyor belt, improve the model's detection accuracy for complex foreign objects, and enhance the ability to recognize foreign objects under different lighting, noise, and occlusion conditions. This significantly improves the reliability and generalization performance of the coal mine conveyor belt foreign object detection system in actual industrial environments.

[0064] The specific formulas for the dual-domain enhanced Transformer module are as follows: ; in, Xt Represents the input features of the t-th layer; LN() The representation layer normalization operation is used to normalize the input features along the channel dimension, ensuring stable feature distribution and accelerating model convergence. This represents the bilinear attention module, which is used to perform positive and negative polarity branching and spatial attention calculation on multi-scale features, thereby enhancing the saliency of foreign objects in the spatial domain. This represents a frequency domain modulation network that analyzes the frequency components of the input features through fast Fourier transform, performs adaptive modulation using a learnable frequency domain gated mask, and then inversely transforms back to the time domain to enhance the texture and detail features of foreign objects. This indicates a residual connection, used to preserve the original input information and alleviate the problem of deep feature decay.

[0065] This embodiment achieves multi-path fusion and enhancement of spatial and frequency domain features. First, the bilinear attention module highlights the saliency and morphological features of foreign objects in the spatial domain, enabling the model to accurately locate small targets and complex foreign objects. Subsequently, the frequency domain modulation network enhances high-frequency texture information in the frequency domain, suppresses noise and periodic background interference, and achieves complementarity between spatial and frequency domain features. The combination of residual connections and layer normalization ensures that deep feature information is not lost, while accelerating model convergence and improving stability. Overall, this design can significantly improve the recognition accuracy, robustness, and generalization ability of the foreign object detection model for coal mine conveyor belts for multi-scale and multi-morphological foreign objects, and is especially suitable for industrial scenarios with varying lighting, occlusion, or complex noise.

[0066] In one embodiment, the construction of a bilinear attention module based on the multi-scale features output by the adaptive perceptual backbone network, and the calculation of spatial domain features based on the bilinear attention module, specifically includes: Linear projection is performed on the multi-scale features to generate query feature vector, key feature vector, and value feature vector; The query feature vector and key feature vector are divided into positive and negative polarity branches, and the positive and negative polarity attention scores are calculated respectively. The positive and negative attention scores are weighted and fused with the value feature vector to obtain the spatial domain features.

[0067] In this embodiment, a bilinear attention module is constructed based on the multi-scale features output by the adaptive sensing backbone network, and spatial domain features are calculated according to the bilinear attention module. Specifically, this includes: linearly projecting the multi-scale features to generate query feature vectors, key feature vectors, and value feature vectors; dividing the query feature vectors and key feature vectors into positive and negative polarity branches, and calculating the positive and negative polarity attention scores respectively; subsequently, the attention scores are weighted and fused with the corresponding value feature vectors to obtain spatial domain features. These spatial domain features can reflect the local texture, geometric shape, and spatial distribution information of foreign objects on the coal mine conveyor belt, providing high-quality input for subsequent frequency domain modulation.

[0068] The bilinear attention module enhances the salient areas of foreign objects by splitting and weighting positive and negative polarity branches, while effectively suppressing background clutter and interference information. This improves the model's ability to detect low-contrast, small-sized, and irregularly shaped foreign objects. Combined with the multi-scale features provided by the adaptive perception backbone network, the spatial domain features retain the detailed information of the original image while enhancing key texture and morphological features, thus improving the accuracy, robustness, and generalization ability of the entire conveyor belt foreign object detection model.

[0069] The calculation formula for the bilinear attention module is as follows: ; in, =[ , ]; =[ , ];K=[ , ]; It is a ReLU activation function; X Represents the multi-scale features of the input; , V i They represent the first The query feature vectors, key feature vectors, and value feature vectors for each positive / negative polarity branch; This represents the expectation of the key feature vector, used to normalize the attention score; T This represents the matrix transpose operation; n The sequence length; It is a numerically stable constant; G This is the gate vector; F DWC This represents a depthwise separable convolution operation; The processed value is the feature vector; This represents an element-wise multiplication operation; Used to enhance the response of key feature regions, making foreign object features more prominent.

[0070] Subsequently, the spatial domain features are fused with multi-scale features through residual connections and then normalized to obtain normalized spatial domain features. Specifically, the spatial domain features processed by the bipolar linear attention module are added element-wise with the corresponding scale features output from the adaptive perceptual backbone network to achieve residual fusion, thereby introducing enhanced spatial semantic information while preserving the original feature information. Then, the fused features are processed through a normalization operation to stabilize the feature distribution and alleviate the numerical differences between features at different scales, thus obtaining normalized spatial domain features.

[0071] In one embodiment, the construction of the frequency domain modulation network involves performing a Fast Fourier Transform (FFT) to convert the normalized spatial domain features to the frequency domain, and then modulating different frequency components using a learnable frequency domain gating mask before inversely transforming them back to the time domain to obtain frequency domain enhancement features. Specifically, this includes: The normalized spatial domain features are sequentially subjected to channel transformation and nonlinear mapping through 1×1 convolution, GELU activation function, and 1×1 convolution. The transformed normalized spatial domain features are divided into blocks to obtain multiple feature blocks of equal size. Perform a Fast Fourier Transform on each feature block to transform it to the frequency domain, thus obtaining the frequency domain features; A learnable frequency-domain gated mask is applied to the frequency domain features, and the feature weights are adaptively adjusted according to the importance of the frequency components. The modulated frequency domain features are transformed back to the time domain using the inverse fast Fourier transform to obtain the inverse transformed time domain features. The time-domain features are block-merged to obtain the original graph structure, which is then used as the frequency-domain enhancement feature.

[0072] In this embodiment, constructing a frequency-domain modulation network can fully utilize the complementary characteristics of the spatial and frequency domains to enhance the foreign object features in video images of coal mine conveyor belts. Specifically, firstly, the normalized spatial domain features output from the adaptive perception backbone network are sequentially processed through 1×1 convolution, GELU activation function, and another 1×1 convolution to perform channel transformation and nonlinear mapping, achieving feature dimensionality adjustment and nonlinear enhancement, providing a more suitable feature representation for subsequent frequency domain operations. Subsequently, the transformed normalized spatial domain features are divided into blocks, resulting in multiple feature blocks of equal size. Each feature block independently undergoes a fast Fourier transform, mapping the spatial features to the frequency domain to obtain frequency domain features. The frequency domain features are then modulated using a learnable frequency-domain gating mask, adaptively adjusting the feature weights according to the importance of different frequency components, thereby highlighting the key texture and morphological information of the foreign object and suppressing background interference and irrelevant frequency information. The features after frequency-domain gating modulation are transformed back to the time domain through an inverse fast Fourier transform to obtain the inverse-transformed time domain features.

[0073] Subsequently, in order to maintain the consistency of spatial structure, the temporal feature blocks after inverse transformation are merged, that is, merged according to the original spatial position to restore the same spatial structure as the input features, and finally form frequency domain enhanced features; effectively realizing the fusion of spatial domain features and frequency domain features, enabling the model to capture local detail information and global frequency information at the same time, and improving the ability to identify complex foreign objects.

[0074] This embodiment uses a frequency domain modulation network to enhance the key texture and morphological features of foreign objects in video images of coal mine conveyor belts, improve the distinguishability of foreign objects, and suppress background noise and interference features. This significantly improves the accuracy, robustness, and generalization ability of the conveyor belt foreign object detection model, providing more reliable and richer feature inputs for the subsequent dual-domain enhancement Transformer module.

[0075] The calculation formula for frequency domain modulation networks is as follows: ; in, R This indicates a feature rearrangement operation, used to partition feature blocks; R -1This indicates a feature inverse rearrangement operation, used to restore the spatial structure of feature blocks; F 2D () and F 2D -1 () These represent the two-dimensional fast Fourier transform and the two-dimensional inverse fast Fourier transform, respectively. W 1 and W 2 are learnable weight matrices used to implement channel transformation; a and b Constructing a weighted matrix a+ib , used to modulate different frequency components in the frequency domain; represents element-wise multiplication operation; s This represents the output feature size of the inverse Fourier transform, used to recover the frequency domain features to the corresponding spatial dimension; w This represents the spatial dimensions of the feature blocks, specifically the dimensions of each feature block in the width and height directions after feature rearrangement. s=(w,w) Used to constrain the output of the inverse Fourier transform to a spatial feature block of the corresponding size.

[0076] In one embodiment, fusing the normalized spatial domain features and the frequency domain enhancement features to obtain the output features of the dual-domain enhanced Transformer module specifically includes: The element-wise residuals of the normalized spatial domain features and the frequency domain enhanced features are fused to obtain dual-domain fused features. The dual-domain fusion features are normalized to obtain the output features of the dual-domain enhanced Transformer module.

[0077] In this embodiment, the normalized spatial domain features and the frequency domain enhanced features are fused to obtain the output features of the dual-domain enhanced Transformer module. Specifically, the normalized spatial domain features and the frequency domain enhanced features are first fused element-wise using residual fusion. Information from the two types of features is superimposed through element-wise addition, thereby introducing texture and frequency response information enhanced by the frequency domain while preserving the original structural information of the spatial domain features, resulting in dual-domain fused features. This eliminates the need for additional complex parameters and achieves coordinated expression of spatial and frequency domain information while maintaining computational efficiency. Then, the dual-domain fused features are normalized to standardize the feature distribution, suppress scale differences between features from different sources, and improve feature stability, resulting in the output features of the dual-domain enhanced Transformer module for subsequent recognition tasks.

[0078] This embodiment fuses normalized spatial domain features and frequency domain enhanced features element-wise, enabling the model to simultaneously utilize spatial structure information and frequency distribution information to achieve multi-dimensional feature modeling of foreign objects on coal mine conveyor belts. Normalized spatial domain features effectively characterize the contour, shape, and location information of foreign objects, while frequency domain enhanced features highlight the texture details and periodic features of the foreign objects. The fusion of the two significantly improves the model's ability to represent complex foreign objects. Subsequent normalization of the fused features effectively alleviates the training instability caused by the inconsistency between spatial and frequency domain feature distributions, making the model more likely to converge during training and improving overall robustness, thereby significantly enhancing the accuracy and generalization ability of conveyor belt foreign object detection.

[0079] like Figure 2 As shown, the overall framework of a conveyor belt foreign object detection model in one embodiment of this application includes an adaptive perception network, a dual-domain enhanced Transformer, and a dynamic multi-scale fusion module. In the adaptive perception network, historical coal mine conveyor belt video images are used as input data for preliminary feature extraction. The images are then processed alternately through convolutional layers and multiple bidirectional feature fusion modules. Through multi-level feature extraction and fusion operations, the feature expression capability is gradually enhanced to obtain a basic feature representation with multi-scale information.

[0080] The features processed by the adaptive perceptual network are input into a hybrid encoder for feature encoding. The hybrid encoder includes a dual-branch structure: one branch directly maps the input features, outputting high-level semantic features F5; the other branch downsamples the input features step-by-step, obtaining feature layers S3, S4, and S5 sequentially. The S5 features are further input into a dual-domain enhancement Transformer module for enhancement processing. The dual-domain enhancement Transformer module internally includes an input embedding layer, a bipolar linear attention module, a normalization layer, and a frequency domain modulation network, enhancing the features through spatial and frequency domain co-modeling. The processed features, along with the F5 features, participate in subsequent encoding processes.

[0081] Then, the multi-scale features output by the hybrid encoder, including F5 features and S3 and S4 related features, are input into the encoding branch for multi-scale fusion processing. The encoding branch constructs a feature pyramid structure through convolution, upsampling, downsampling and feature fusion operations. A dynamic multi-scale fusion module is introduced in the feature fusion process. Through multi-path parallel processing and feature enhancement mechanism, efficient interaction and fusion between features of different scales are achieved, thereby obtaining the fused multi-scale feature representation.

[0082] The fused features are then input into a query selection module based on the IoU loss function to filter and optimize candidate queries, thereby improving the localization accuracy and matching quality of subsequent object detection. The filtered query features are then input into the decoder and output head module for decoding and prediction processing. Finally, after the decoder and output head module decode and calculate the input features, the final foreign object detection result is output, which includes foreign object category information and its location information in the image.

[0083] like Figure 3 As shown, the adaptive perception network in this application comprises convolutional units and multiple bidirectional feature fusion modules stacked alternately, used to achieve progressive extraction and multi-scale fusion of input features. Specifically, the input features are first sequentially input into multiple convolutional layers for preliminary feature extraction, and bidirectional feature fusion modules are inserted between different layers. Through the alternating structure of convolutional layers and bidirectional feature fusion modules, features are modeled and fused at multiple levels to gradually enhance feature representation capabilities and improve the ability to perceive foreign objects at different scales.

[0084] The structure of the bidirectional feature fusion module includes: first, the input features are transformed through a convolutional layer, and then the features are divided into two paths for parallel processing. One path, P1, is input into multiple adaptive gated Transformer modules for feature enhancement, while the other path, P2, is extracted through a convolutional layer. The two paths are then concatenated along the channel dimension and integrated through a convolutional layer to obtain the fused output features.

[0085] The processing flow of the adaptive gated Transformer module includes: first, normalizing the input features, then inputting them into a multi-receptive-field feature integrator for multi-scale feature extraction; then, adding the output of the multi-receptive-field feature integrator to the input features element-wise to form a residual connection; subsequently, inputting the features into a gated linear unit for nonlinear transformation and feature filtering, and then adding them to the output of the previous stage element-wise again to obtain the final output features.

[0086] The processing flow of the multi-receptive-field feature integrator includes: dividing the input features into two sub-features by channel division; inputting the two sub-features into the morphology-guided convolution module for feature extraction to obtain feature representations of different receptive fields and different morphological responses; concatenating the two processed features in the channel dimension and fusing them through a convolutional layer to obtain the output features.

[0087] like Figure 4The diagram shows the workflow of a morphology-guided convolutional unit in one embodiment of this application. Specifically, the morphology-guided convolutional unit takes an input feature map as input, which is a three-dimensional feature map with dimensions C×H×W, where C represents the number of channels, H represents the feature map height, and W represents the feature map width. The input feature map is first divided into multiple parallel processing paths, including a main feature extraction path and an attention weight generation path. In the main feature extraction path, the input features are fed into various convolutional operations of different shapes for feature extraction, including square convolution (k×k convolution), horizontal strip convolution (M×1 convolution), and vertical strip convolution (1×M), thereby obtaining multi-morphological feature representations sensitive to different spatial structures and orientations; the output feature dimension of each convolutional branch remains C×H×W.

[0088] Meanwhile, in the attention weight generation path, an adaptive average pooling operation is performed on the input features to compress the spatial dimension to 1×1, thereby obtaining global statistical features. Subsequently, through convolution transformation and activation functions, such as the Softmax function, the weight coefficients corresponding to each convolution branch are generated. The weight coefficients are used to characterize the importance of different morphological features under the current input conditions, and their output dimension is C×1×1.

[0089] Furthermore, the multimorphic features extracted from each convolutional branch are multiplied element-wise with their corresponding weight coefficients to achieve adaptive weighting of the feature responses, thereby highlighting key morphological features and suppressing irrelevant information. Subsequently, the weighted features are fused through residual connections to obtain fused features. Finally, the fused features are used as the output of the morphology-guided convolutional unit, with its feature dimensions remaining consistent with the input, still being C×H×W.

[0090] like Figure 5 As shown, the dual-domain enhancement Transformer module in this embodiment includes first inputting the input features into the input embedding layer for feature mapping; then inputting them into the bipolar linear attention module for feature relationship modeling, and performing a residual connection between its output and the input embedded features; after processing by the normalization layer, inputting them into the frequency domain modulation network for frequency domain feature enhancement, and performing a residual connection between the output of the frequency domain modulation network and the features from the previous stage again; and finally outputting the enhanced features after processing by the normalization layer.

[0091] The processing flow of the frequency domain modulation network includes: channel mapping of the input features through 1×1 convolutional layers, nonlinear transformation by the GELU activation function, and feature adjustment through another 1×1 convolutional layer; then, the features are divided into blocks, and a fast Fourier transform is performed on each feature block to convert it to the frequency domain representation; in the frequency domain, the transformed features are modulated element-wise with a learnable filter W, and the original frequency domain information is preserved through residual connections; then, an inverse fast Fourier transform is performed on the modulated features to restore them to the spatial domain, and the original feature structure is restored through block merging operations to obtain the output features.

[0092] The processing flow of the bipolar linear attention module includes: dividing the input features into two branches, one for modeling the feature relationship between regions with the same label and the other for modeling the feature relationship between regions with opposite labels; one branch is used for modeling regions with the same label, based on feature pairs. and Perform mapping operations and use a specific transformation function. () Feature projection and reconstruction are performed, followed by calculation of the correlation between features through matrix multiplication, and then compared with the weight terms. G s One approach performs element-wise multiplication to obtain features related to the same label; the other approach is used for modeling opposite labels, employing the same processing flow and weighting terms. G o Modulation is performed to obtain opposite label relationship features; finally, the two features are concatenated along the channel dimension to obtain the attention-enhanced output features.

[0093] in, X s Indicates the features of regions with the same label. X o Indicates the features of the opposite label region. X + Indicates positive regional characteristics. X - ⊕ represents the negative region feature, ⊙ represents the element-wise multiplication operation, ⊕ represents the residual connection operation, and W represents the learnable filter used to modulate different frequency components in the frequency domain.

[0094] In another embodiment, when the conveyor belt foreign object detection model includes a dynamic multi-scale fusion module, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: Based on the output features of the dual-domain enhanced Transformer module, a multi-path parallel processing architecture is used to extract and fuse multi-scale features to obtain adaptation features, deep convolutional extracted features, and enhanced sub-features. The multi-path parallel processing architecture includes a direct-connected adaptation path, a deep convolutional feature extraction path, and an omnidirectional feature enhancement perceptual block cascaded enhancement path. The adaptation features, depthwise convolutional extracted features, and enhancement sub-features are concatenated along the channel dimension and then integrated through a 1×1 convolution to obtain the output features of the dynamic multi-scale fusion module.

[0095] In this embodiment, in another embodiment, when the conveyor belt foreign object detection model includes a dynamic multi-scale fusion module, the process of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset further includes: constructing a multi-path parallel processing architecture based on the output features of the dual-domain enhanced Transformer module, and performing multi-scale feature extraction and fusion on the input features. Specifically, the multi-path parallel processing architecture includes a direct connection adaptation path, a deep convolutional feature extraction path, and an omnidirectional feature enhancement perception block cascade enhancement path. Specifically, the direct connection adaptation path is used to quickly map and lightweightly adjust the input features to retain the original semantic information and reduce information loss; the deep convolutional feature extraction path extracts features layer by layer through multi-layer convolutional operations, strengthening local structural information and fine-grained texture features to obtain deep convolutional extracted features; the omnidirectional feature enhancement perception block cascade enhancement path models the contextual information of the input features in multiple directions and at multiple scales through multi-level feature enhancement units to obtain enhanced sub-features. Subsequently, the adaptation features, depthwise convolutional extracted features, and enhancement sub-features are concatenated according to the channel dimension, and the concatenated features are channel-integrated and dimension-compressed by 1×1 convolution to obtain the output features of the dynamic multi-scale fusion module.

[0096] This embodiment introduces a multi-path parallel processing architecture, enabling the model to model the features of foreign objects on the conveyor belt from different scales and receptive fields. The direct-connect adaptation path ensures the complete transmission of basic semantic information, avoiding the feature attenuation problem caused by deep network structures; the deep convolutional feature extraction path enhances the ability to capture local details and edge information; and the omnidirectional feature enhancement perception path improves the model's ability to perceive complex backgrounds and the diversity of foreign objects through multi-directional information interaction. The synergistic effect of these three paths allows the model to simultaneously consider global semantic information and local detailed features, thereby significantly improving the detection capability of foreign objects of different sizes and shapes.

[0097] In one embodiment, based on the output features of the dual-domain enhanced Transformer module, a multi-path parallel processing architecture is used to perform multi-scale feature extraction and fusion to obtain adaptation features, depthwise convolutional extracted features, and enhanced sub-features, specifically including: The output features of the dual-domain enhanced Transformer module are respectively input to the direct connection adaptation path, the deep convolution feature extraction path, and the omnidirectional feature enhancement perceptual block cascade enhancement path; In the direct connection adaptation path, the input features are channel-mapped and dimension-adjusted through 1×1 convolution to obtain the adaptation features; In the deep convolution feature extraction path, the input features are extracted step by step through multiple convolutional units, which can be a series of 1×1 convolutions, deep convolutions, and 1×1 convolutions to obtain the deep convolution extracted features. In the cascaded enhancement path of the omnidirectional feature enhancement sensing block, channel adaptation is first completed through 1×1 convolution, and then channel splitting operation is performed on the features to divide the features into multiple groups of sub-features according to the channel dimension. Each group of sub-features is then cascaded with the omnidirectional feature enhancement sensing unit in sequence to obtain the enhanced sub-features.

[0098] In this embodiment, the output features of the dual-domain enhanced Transformer module are respectively input to the direct connection adaptation path, the deep convolution feature extraction path, and the omnidirectional feature enhancement perceptual block cascade enhancement path. In the direct adaptation path, the input features are mapped and their dimensions adjusted using 1×1 convolutions to obtain the adapted features, preserving the original semantic information while minimizing additional computational overhead and enabling rapid feature transfer. In the deep convolution feature extraction path, the input features are extracted step-by-step through multiple convolutional units, where the convolutional units can be sequentially connected 1×1 convolutions, deep convolutions, and 1×1 convolutions. By expanding the receptive field and enhancing nonlinear expressive power, deep convolution extracted features containing rich local details and texture information are obtained. In the omnidirectional feature enhancement perceptual block cascade enhancement path, channel adaptation is first completed using 1×1 convolutions, and then channel splitting is performed on the features, dividing them into multiple groups of sub-features according to the channel dimension. Each group of sub-features is then sequentially input into the omnidirectional feature enhancement perceptual unit for cascaded processing, thereby enhancing the features under multi-directional and multi-scale receptive fields and obtaining enhanced sub-features containing rich contextual information and global dependencies.

[0099] This embodiment constructs a multi-path parallel processing architecture, enabling the model to perform multi-scale modeling of input features from different perspectives. Specifically, the direct-connect adaptation path ensures the complete preservation of basic semantic information, avoiding the information decay problem in deep networks; the deep convolutional feature extraction path enhances the ability to capture local structure, edge details, and texture features through successive convolutional operations, resulting in higher accuracy in recognizing small or fine-grained foreign objects; and the cascaded enhancement path of omnidirectional feature enhancement perception blocks achieves multi-directional interaction and multi-scale perception of features through channel splitting and cascading of omnidirectional feature enhancement perception units.

[0100] In one embodiment, the step of sequentially concatenating each group of sub-features with an omnidirectional feature enhancement perception unit to obtain enhanced sub-features specifically includes: Perform depthwise convolution on the input sub-features; The output features of the depthwise convolution are split into two parallel 1×1 convolution branches. The output features of one branch are processed by the ReLU activation function to obtain non-linear feature labels. The output features of the two 1×1 convolutional branches are multiplied element-wise to obtain the product result; The product results are sequentially processed through 1×1 convolutions to integrate the channels, and then further processed through depthwise convolutions to obtain the output features of the current omnidirectional feature enhancement perception unit. The output features of the current omnidirectional feature enhancement sensing unit are used as the input of the next omnidirectional feature enhancement sensing unit for cascade processing. All cascaded sub-features are aggregated to obtain the enhanced sub-features.

[0101] In this embodiment, each group of sub-features is sequentially concatenated with an omnidirectional feature enhancement perception unit to obtain enhanced sub-features. Specifically, this includes: performing deep convolution on the input sub-features to enhance the ability to extract spatial local features while maintaining computational efficiency; splitting the output features of the deep convolution into two parallel 1×1 convolution branches, where the output features of one branch are processed by the ReLU activation function to obtain a nonlinear feature representation, and the other branch is used to generate a linear feature response; performing element-wise multiplication on the output features of the two 1×1 convolution branches to obtain a product result, thereby achieving adaptive gating modulation of features from different channels; sequentially integrating the product result through 1×1 convolution and further performing deep convolution to obtain the output features of the current omnidirectional feature enhancement perception unit; using the output features of the current omnidirectional feature enhancement perception unit as the input of the next omnidirectional feature enhancement perception unit for cascading processing to achieve multi-level feature progressive enhancement; and converging all cascaded sub-features to obtain the enhanced sub-features.

[0102] This embodiment uses depthwise convolution to preprocess the input sub-features, enabling the model to fully extract local spatial information. Then, a linear and non-linear feature representation path is constructed through a dual-branch 1×1 convolution structure, and a non-linear feature identifier is introduced with the help of the ReLU activation function. Then, a feature selection process similar to a gating mechanism is implemented through element-wise multiplication, which can adaptively enhance the response to key features and suppress redundant or noisy information, thereby improving the discriminativeness and robustness of feature representation.

[0103] Subsequently, by channel integration and deep convolution processing of the product result, a joint representation of spatial and channel information is achieved. Simultaneously, a cascaded structure of multi-level omnidirectional feature enhancement perception units allows features to be progressively enhanced at different levels, balancing the fusion of local detail information and global semantic information. Finally, by converging sub-features, enhanced sub-features with higher information density and expressive power are formed. This improves the recognition accuracy of the conveyor belt foreign object detection model for small targets, weak features, and multi-scale targets under complex working conditions, and enhances the model's adaptability to background interference, illumination changes, and noise disturbances, thereby improving detection accuracy and stability.

[0104] like Figure 6 The diagram shows the workflow of the dynamic multi-scale fusion module in one embodiment of this application. Specifically, the omnidirectional feature enhancement perceptual block in the dynamic multi-scale fusion module takes the input features as input and first performs preliminary extraction of the input features through deep convolution to enhance the local spatial expressive power of the features and expand the receptive field.

[0105] Subsequently, the processed features are divided into two parallel paths for further processing. In the first path, the features are directly output as the baseline features without any additional transformation, preserving the original semantic information. In the second path, the features are sequentially processed through 1×1 convolutions for channel mapping and then subjected to a nonlinear transformation using the ReLU activation function to obtain enhanced features with stronger expressive power. The output features of the second path are then multiplied element-wise with the baseline features of the first path to achieve adaptive weighting of important features, thereby highlighting responses in key regions and suppressing irrelevant information.

[0106] Furthermore, the features obtained after element-wise multiplication are channel-integrated through 1×1 convolution and then input into depthwise convolution for feature extraction to obtain the enhanced features of the current perceptual block. The enhanced features are used as the output of the perceptual block and are fed back to the dynamic multi-scale fusion module to be spliced ​​and fused with features from other paths or scales, thereby achieving the collaborative expression of multi-scale features.

[0107] In one embodiment, training the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: The foreign object dataset of the conveyor belt is input into the foreign object detection model of the conveyor belt, and the gradient is calculated based on the loss function and backpropagation; When the maximum number of iterations is reached or the loss function tends to stabilize, the conveyor belt foreign object detection model is determined based on the iteration results; The performance of the conveyor belt foreign object detection model is verified using the conveyor belt foreign object dataset. If the performance meets the preset accuracy threshold, the training of the conveyor belt foreign object detection model is complete.

[0108] In this embodiment, training the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: inputting the conveyor belt foreign object dataset into the conveyor belt foreign object detection model; performing forward propagation calculation based on the loss function during training, and calculating and updating the gradient through backpropagation; performing iterative training under preset training parameters, wherein the input image size is uniformly 640×640, the batch size is 16, the training period is 300, and the AdamW optimizer is used for parameter optimization, with an initial learning rate of 0.0001, a momentum parameter of 0.9, and a weight decay of 0.0001; when the maximum learning rate is reached... When the number of iterations is large or the loss function tends to stabilize, the conveyor belt foreign object detection model is determined based on the iteration results. The performance of the conveyor belt foreign object detection model is verified using the test set in the conveyor belt foreign object dataset. If the performance meets the preset accuracy threshold, the conveyor belt foreign object detection model training is completed. The preset accuracy threshold includes the mean accuracy mAP ≥ 0.5, precision P, recall R, and F1 score reaching the preset standard. At the same time, the model's frames per second (FPS) is not less than 30 to meet the real-time detection requirements, and the number of model parameters, the number of floating-point operations, and the weight file size are within the preset range.

[0109] This embodiment introduces explicit training parameter settings and optimization strategies during model training, enabling the model to exhibit good convergence characteristics in the early stages of training. The AdamW optimizer, combined with a weight decay mechanism, effectively suppresses overfitting and improves the model's generalization ability. Simultaneously, continuous optimization of model parameters is achieved through loss functions and backpropagation mechanisms, allowing the model to fully learn the feature distribution of conveyor belt foreign objects at different scales, shapes, and under complex backgrounds, thereby improving detection accuracy and robustness. Furthermore, by setting multi-dimensional performance evaluation indicators as the criteria for determining model training completion, not only can the model's performance in detection accuracy be comprehensively reflected, but its computational complexity and real-time processing capabilities can also be constrained, thus achieving a balance between accuracy, speed, and resource consumption. In particular, by constraining FPS to be no less than 30, the model ensures it can meet the real-time detection requirements of industrial sites. Combined with parameter quantity, FLOPs, and weight scale control, the model possesses good engineering deployment and operational stability, significantly improving the reliability and practical value of the conveyor belt foreign object detection system in real-world applications.

[0110] like Figure 7 As shown, a flowchart of a foreign object detection method for coal mine conveyor belts in one embodiment of this application includes: S1: Real-time video images of the coal mine conveyor belt are captured by a camera positioned diagonally above the conveyor belt. S2: Annotate and preprocess foreign objects in the images to construct a conveyor belt foreign object dataset; S3: Construct a foreign object detection model for coal mine conveyor belts, including an adaptive perceptual backbone network for enhanced feature extraction, a dual-domain enhanced Transformer module for spatiotemporal-frequency domain coordination, and a dynamic multi-scale fusion module for improved feature fusion. Train the foreign object detection model based on a coal mine conveyor belt foreign object detection dataset to obtain the trained coal mine conveyor belt foreign object detection model; S4: Use the conveyor belt foreign object detection model to train, validate, and test the foreign object dataset of coal mine conveyor belts to verify the model's performance; S5: Acquire video images of foreign objects on the conveyor belt to be detected, use the trained conveyor belt foreign object detection model to perform foreign object detection, obtain the detection results and output them.

[0111] Specifically, in step S1, the continuous video acquisition of the conveyor belt operation process is achieved by a camera installed diagonally above the conveyor belt. This enables full coverage monitoring of the conveyor belt surface and the materials it carries, thereby obtaining raw video data containing different working conditions, different lighting conditions, and different types of foreign objects, providing a data foundation for subsequent data processing and model training.

[0112] In step S2, by performing preprocessing operations such as keyframe extraction and noise reduction on the acquired video images, the image quality can be effectively improved and redundant data can be reduced. At the same time, foreign objects in the images are labeled to obtain labeled data containing foreign object category and location information, thereby constructing a high-quality conveyor belt foreign object dataset.

[0113] In step S3, a conveyor belt foreign object detection model is constructed, comprising an adaptive perception backbone network, a dual-domain enhanced Transformer module, and a dynamic multi-scale fusion module. This model enables efficient extraction and representation of foreign object features across multiple scales, shapes, and complex backgrounds. The adaptive perception backbone network enhances basic feature extraction capabilities, the dual-domain enhanced Transformer module enables collaborative modeling of spatial and frequency domain features, and the dynamic multi-scale fusion module enables dynamic fusion of multi-scale features, thereby significantly improving the model's ability to detect foreign objects in complex scenarios. Based on this, the model is trained using the conveyor belt foreign object dataset, enabling it to learn the feature distribution patterns of different types of foreign objects.

[0114] In step S4, by validating and testing the trained model, the model performance can be evaluated from multiple dimensions such as precision, recall, and average accuracy to determine whether the model meets the preset detection accuracy requirements, thereby ensuring the reliability and stability of the model in practical applications.

[0115] In step S5, the video image of the conveyor belt to be detected is input into the trained model for inference calculation, which can output the foreign object detection result in real time. By outputting the detection result, the automatic identification and positioning of foreign objects on the conveyor belt can be realized, providing technical support for the safe operation of coal mine conveyor belts.

[0116] like Figure 8 As shown, a flowchart of a foreign object detection method for a coal mine conveyor belt in one embodiment of this application is illustrated, including: S801: Acquire historical video images of coal mine conveyor belts; S802: Construct a foreign object dataset for the conveyor belt based on the historical coal mine conveyor belt video images; S803: Perform multiple convolutional downsampling operations on historical coal mine conveyor belt images after data preprocessing to construct a multi-scale feature pyramid; the multi-scale feature pyramid includes input features; S804: Embed a bidirectional feature fusion module in each scale layer of the multi-scale feature pyramid; the bidirectional feature fusion module is used to adjust and split the input features by channels, and then splice and fuse the split input features. S805: An adaptive gated Transformer is constructed inside the bidirectional feature fusion module; the adaptive gated Transformer is used to adaptively enhance the input features through a dual-path residual structure; S806: Construct a multi-receptive-field feature integrator inside the adaptive gated Transformer; the multi-receptive-field feature integrator is used to perform multi-scale dynamic integration of input features; S807: A morphology-guided convolutional unit is constructed inside the multi-receptive-field feature integrator; the morphology-guided convolutional unit is used to adaptively adjust the convolutional kernel weights according to the input features through multiple convolutional branches of different shapes and a dynamic weighting mechanism; S808: The bidirectional feature fusion module, the adaptive gated Transformer, the multi-receptive field feature integrator, and the morphology-guided convolutional unit are integrated hierarchically to form an adaptive perceptual backbone network; S809: Construct a bilinear attention module based on the multi-scale features output by the adaptive perception backbone network, and calculate spatial domain features according to the bilinear attention module; the bilinear attention is used to split the multi-scale features into positive and negative polarity branches and calculate attention; the spatial domain features are fused with the multi-scale features through residual connections, and then normalized to obtain normalized spatial domain features. S810: Construct a frequency domain modulation network, perform a fast Fourier transform to the frequency domain based on the normalized spatial domain features, and then modulate different frequency components through a learnable frequency domain gate mask before inversely transforming back to the time domain to obtain frequency domain enhancement features. S811: The normalized spatial domain features and the frequency domain enhancement features are fused to obtain the output features of the dual-domain enhanced Transformer module; S812: Based on the output features of the dual-domain enhanced Transformer module, a multi-path parallel processing architecture is used to perform multi-scale feature extraction and fusion to obtain adaptation features, deep convolutional feature extraction features, and enhanced sub-features; the multi-path parallel processing architecture includes a direct-connected adaptation path, a deep convolutional feature extraction path, and an omnidirectional feature enhancement perceptual block cascaded enhancement path. S813: The adaptation features, depth convolution extracted features, and enhancement sub-features are concatenated according to the channel dimension, and then integrated through 1×1 convolution to obtain the output features of the dynamic multi-scale fusion module; S814: Train the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset; S815: Input the real-time video image of the coal mine conveyor belt into the trained conveyor belt foreign object detection model to obtain the detection result.

[0117] In one embodiment, the workflow diagram of embedding a bidirectional feature fusion module in each scale layer of the multi-scale feature pyramid in one embodiment of this application includes: S901: The input features of each scale layer are adjusted and split by 1×1 convolution to split the input features into first-path features and second-path features. S902: The first feature path is fed into a cascaded adaptive gated Transformer for enhancement processing; S903: Extract basic features from the second feature path using a 3×3 ordinary convolution combined with batch normalization and a first preset activation function; S904: The enhanced first-path features and the extracted second-path features are concatenated and fused according to the channel dimension to obtain the output features of the bidirectional feature fusion module.

[0118] In one embodiment, the workflow diagram for constructing an adaptive gated Transformer in one embodiment of this application includes: S1001: Perform initial normalization on the first path features, and then send the normalized first path features into the multi-receptive field feature integrator for dynamic integration of multi-scale features. S1002: Add the output of the multi-receptive-field feature integrator to the first path feature element by element residual to obtain the first path residual feature; S1003: Perform a second normalization process on the first path residual features, and send the normalized first path residual features into a convolutional gated linear unit for nonlinear transformation and adaptive gating of the features; S1004: Add the output of the convolutional gated linear unit to the residual features of the first path element by element to obtain the output features of the adaptive gated Transformer.

[0119] In one embodiment, a flowchart of the workflow for constructing a multi-receptive-field feature integrator in one embodiment of this application includes: S1101: The first feature after the first normalization process is split into two parallel feature sub-maps by channel dimension. S1102: The two sets of feature maps are fed into two parallel morphology-guided convolutional units to obtain multimorphic features. S1103: The multi-morphological features are spliced ​​together according to the channel dimension to obtain the complete feature dimension; S1104: Channel fusion and dimension adjustment are performed on the complete feature dimensions through 1×1 convolution to obtain a feature map with unified dimensions.

[0120] In one embodiment, the flowchart of constructing a morphology-guided convolutional unit in one embodiment of this application includes: S1201: Apply square convolution, horizontal strip convolution, and vertical strip convolution to the feature map respectively to extract multimorphic features and obtain multimorphic features; S1202: Perform adaptive average pooling on the feature map, and generate dynamic weight vectors corresponding to square convolution, horizontal strip convolution and vertical strip convolution respectively through 1×1 convolution and the second preset activation function; S1203: Multiply the multimorphic features of square convolution, horizontal strip convolution and vertical strip convolution with the corresponding dynamic weight vector element by element, and then add the weighted features according to the channel dimension to obtain the output of the morphology-guided convolution unit.

[0121] Embodiments of this application also provide a storage medium that stores computer instructions, which, when executed by a computer, are used to perform a foreign object detection method for a coal mine conveyor belt as described in any of the preceding embodiments.

[0122] Figure 13 An electronic device according to this application is shown, comprising: At least one processor 1301; and, Memory 1302 is communicatively connected to at least one processor 1301; wherein, The memory 1302 stores instructions that can be executed by at least one processor 1301, which enables the at least one processor 1301 to perform all the steps of a foreign object detection method for a coal mine conveyor belt in any of the foregoing method embodiments.

[0123] Figure 13 Taking a processor 1301 as an example: The electronic device may also include an input device 1303 and an output device 1304.

[0124] The processor 1301, memory 1302, input device 1303 and output device 1304 can be connected by a bus or other means. The figure shows an example of connection by bus.

[0125] The memory 1302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to a foreign object detection method for a coal mine conveyor belt in this embodiment of the application. Figure 1 , Figures 7-12 The method flow is shown. The processor 1301 executes various functional applications and data processing by running non-volatile software programs, instructions, and modules stored in the memory 1302, thereby realizing a foreign object detection method for coal mine conveyor belts in the above embodiments.

[0126] The memory 1302 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created based on the use of a foreign object detection method for a coal mine conveyor belt. Furthermore, the memory 1302 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 1302 may optionally include memory remotely located relative to the processor 1301, and these remote memories may be connected via a network to the apparatus performing a foreign object detection method for a coal mine conveyor belt. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0127] The input device 1303 can receive user clicks and generate signal inputs related to user settings and function control of the real-time zone dimming method of the vehicle lighting system. The output device 1304 may include a display device such as a display screen.

[0128] One or more modules are stored in memory 1302, and when run by one or more processors 1301, they execute a foreign object detection method for coal mine conveyor belts in any of the above method embodiments.

[0129] Embodiments of this application also provide a computer program product, including a computer program / instructions that, when executed by a processor, implement a foreign object detection method for a coal mine conveyor belt as described in any of the preceding embodiments.

[0130] As needed, the above technical solutions can be combined to achieve the best technical effect.

[0131] The above description is merely the principle and preferred embodiment of this application. It should be noted that for those skilled in the art, implementation methods obtained by appropriately combining the technical solutions disclosed in different embodiments are also included within the technical scope of this invention. Based on the principle of this application, several other modifications can also be made, which should also be considered within the protection scope of this application.

Claims

1. A method for detecting foreign objects on a coal mine conveyor belt, characterized in that, include: Acquire historical video images of coal mine conveyor belts; A dataset of foreign objects on conveyor belts was constructed based on the historical video images of coal mine conveyor belts. A conveyor belt foreign object detection model is constructed based on the conveyor belt foreign object dataset. The conveyor belt foreign object detection model includes an adaptive perception backbone network, a dual-domain enhanced Transformer module, and a dynamic multi-scale fusion module. The conveyor belt foreign object detection model is trained based on the aforementioned conveyor belt foreign object dataset. Real-time video images of coal mine conveyor belts are input into the trained conveyor belt foreign object detection model to obtain detection results.

2. The method for detecting foreign objects on a coal mine conveyor belt according to claim 1, characterized in that, The conveyor belt foreign object dataset includes historical coal mine conveyor belt images after data preprocessing. When the conveyor belt foreign object detection model includes an adaptive sensing backbone network, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: Based on the preprocessed historical coal mine conveyor belt images, multiple convolutional downsampling operations are performed to construct a multi-scale feature pyramid; the multi-scale feature pyramid includes input features. A bidirectional feature fusion module is embedded in each scale layer of the multi-scale feature pyramid; the bidirectional feature fusion module is used to adjust and split the input features by channels, and then splice and fuse the split input features. An adaptive gated Transformer is constructed inside the bidirectional feature fusion module; the adaptive gated Transformer is used to adaptively enhance the input features through a dual-path residual structure; A multi-receptive-field feature integrator is constructed inside the adaptive gated Transformer; the multi-receptive-field feature integrator is used to dynamically integrate input features at multiple scales; A morphology-guided convolutional unit is constructed inside the multi-receptive-field feature integrator; the morphology-guided convolutional unit is used to adaptively adjust the convolutional kernel weights according to the input features through multiple convolutional branches of different shapes and a dynamic weighting mechanism. The bidirectional feature fusion module, the adaptive gated Transformer, the multi-receptive field feature integrator, and the morphology-guided convolutional unit are integrated hierarchically to form an adaptive perceptual backbone network.

3. The method for detecting foreign objects on a coal mine conveyor belt according to claim 2, characterized in that, The embedding of a bidirectional feature fusion module at each scale layer of the multi-scale feature pyramid specifically includes: The input features of each scale layer are adjusted and split by 1×1 convolution to divide the input features into first-path features and second-path features. The first feature path is fed into a cascaded adaptive gated Transformer for enhancement processing; The second feature path is used to extract basic features through 3×3 ordinary convolution combined with batch normalization and the first preset activation function; The enhanced first-path features and the extracted second-path features are concatenated and fused according to the channel dimension to obtain the output features of the bidirectional feature fusion module. The construction of an adaptive gated Transformer within the bidirectional feature fusion module specifically includes: The first path features are normalized for the first time, and the normalized first path features are sent to the multi-receptive field feature integrator for dynamic integration of multi-scale features. The output of the multi-receptive-field feature integrator is added to the first path feature element by element to obtain the first path residual feature; The residual features of the first path are normalized a second time, and the normalized residual features of the first path are fed into a convolutional gated linear unit for nonlinear transformation and adaptive gating of the features. The output of the convolutional gated linear unit is added to the residual feature of the first path element by element to obtain the output feature of the adaptive gated Transformer.

4. The method for detecting foreign objects on a coal mine conveyor belt according to claim 2, characterized in that, The construction of a multi-receptive-field feature integrator within the adaptive gated Transformer specifically includes: The first feature after the first normalization process is split into two parallel feature sub-maps by channel dimension. The two sets of feature maps are fed into two parallel morphology-guided convolutional units to obtain multimorphic features. The multi-morphological features are concatenated according to the channel dimension to obtain the complete feature dimension; By performing channel fusion and dimension adjustment on the complete feature dimension through 1×1 convolution, a feature map with a unified dimension is obtained; The construction of morphology-guided convolutional units within the multi-receptive-field feature integrator specifically includes: Square convolution, horizontal strip convolution, and vertical strip convolution are applied to the feature map respectively to extract multimorphic features and obtain multimorphic features; Adaptive average pooling is performed on the feature map, and dynamic weight vectors corresponding to square convolution, horizontal strip convolution and vertical strip convolution are generated by 1×1 convolution and the second preset activation function, respectively. The output of the morphological guided convolution unit is obtained by multiplying the multimorphic features of square convolution, horizontal strip convolution, and vertical strip convolution with the corresponding dynamic weight vectors element by element, and then adding the weighted features according to the channel dimension.

5. The method for detecting foreign objects on a coal mine conveyor belt according to claim 2, characterized in that, The process involves performing multiple convolutional downsampling operations on the preprocessed historical coal mine conveyor belt video images to construct a multi-scale feature pyramid, specifically including: The historical coal mine conveyor belt video images after data preprocessing were subjected to four consecutive convolutional downsampling operations to obtain a four-layer multi-scale feature pyramid. The downsampling operation is implemented through a 3×3 convolution with a kernel stride of 2, and combined with batch normalization and a preset activation function to perform nonlinear transformation of features.

6. The method for detecting foreign objects on a coal mine conveyor belt according to claim 1, characterized in that, When the conveyor belt foreign object detection model includes a dual-domain enhanced Transformer module, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: A bilinear attention module is constructed based on the multi-scale features output by the adaptive perception backbone network, and spatial domain features are calculated based on the bilinear attention module. The bilinear attention is used to split the multi-scale features into positive and negative polarity branches and to calculate attention. The spatial domain features are fused with the multi-scale features through residual connections and then normalized to obtain normalized spatial domain features. A frequency domain modulation network is constructed, and a fast Fourier transform is performed on the normalized spatial domain features to convert them to the frequency domain. After modulating different frequency components with a learnable frequency domain gate mask, the network is inversely transformed back to the time domain to obtain frequency domain enhancement features. The normalized spatial domain features and the frequency domain enhanced features are fused together and normalized again to obtain the output features of the dual-domain enhanced Transformer module.

7. The method for detecting foreign objects on a coal mine conveyor belt according to claim 1, characterized in that, When the conveyor belt foreign object detection model includes a dynamic multi-scale fusion module, the step of constructing the conveyor belt foreign object detection model based on the conveyor belt foreign object dataset specifically includes: Based on the output features of the dual-domain enhanced Transformer module, a multi-path parallel processing architecture is used to extract and fuse multi-scale features to obtain adaptation features, deep convolutional extracted features, and enhanced sub-features. The multi-path parallel processing architecture includes a direct-connected adaptation path, a deep convolutional feature extraction path, and an omnidirectional feature enhancement perceptual block cascaded enhancement path. The adaptation features, depthwise convolutional extracted features, and enhancement sub-features are concatenated along the channel dimension and then integrated through a 1×1 convolution to obtain the output features of the dynamic multi-scale fusion module.

8. A storage medium, characterized in that, The storage medium stores computer instructions, which, when executed by the computer, are used to perform a foreign object detection method for coal mine conveyor belts as described in any one of claims 1-7.

9. An electronic device, characterized in that, Includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform a method for detecting foreign objects on a coal mine conveyor belt as described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement a method for detecting foreign objects on a coal mine conveyor belt as described in any one of claims 1-7.

Images