Coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation

Abstract

The present invention belongs to the field of artificial intelligence and coal mine production technology, and specifically relates to a coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation. First, a real-time video image of a conveyor belt is obtained through a multi-angle camera device to fully cover all areas on the conveyor belt. Secondly, a mixed-scale Transformer model is used to extract and aggregate features of the video image, fully integrating feature information at different scales to enhance the model's recognition ability of coal gangue. Finally, the aggregated features are analyzed using a coal gangue detection algorithm to achieve high-precision coal gangue detection. The present invention obtains real-time video images of the conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt, thereby eliminating any blind spots and dead angles. Combined with advanced image preprocessing techniques, such as denoising and image enhancement, the clarity and detail of the image are significantly improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and coal mine production technology, and specifically relates to a coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation. Background Art

[0002] With the continuous expansion of coal mining, coal mine conveyor belts are important transportation equipment in the coal mine production process. Their operating efficiency and safety directly affect the overall production efficiency of coal mines. However, in the process of coal transportation, coal gangue is often mixed in, which not only affects the quality of coal, but also may damage the conveyor belt and other equipment, increasing production costs. Therefore, how to efficiently and accurately detect and separate coal gangue has become an important issue in coal mine production.

[0003] Traditional gangue detection methods mainly rely on manual detection and simple image processing technology. Manual detection is not only labor-intensive and inefficient, but also easily affected by human factors, resulting in low detection accuracy. Although simple image processing technology has improved detection efficiency to a certain extent, under complex working conditions, such as lighting changes, dust interference, and variable gangue shapes, the accuracy and stability of detection still cannot meet actual needs.

[0004] In recent years, with the rapid development of computer vision and deep learning technology, coal gangue detection methods based on video images have gradually become a research hotspot. Deep learning models, especially convolutional neural networks (CNNs), have performed well in image recognition and classification tasks and are widely used in various image processing tasks. However, traditional CNNs still face some challenges when processing conveyor belt video images. First, conveyor belt video images are dynamic and complex, and the features at different angles and scales vary greatly. Feature extraction at a single scale often cannot fully capture the important information in the image. Secondly, how to effectively fuse multi-scale features to improve the recognition ability and robustness of the model is also a problem that needs to be solved urgently.

[0005] In addition, the coal mine environment is complex and changeable. During operation, the conveyor belt will be disturbed by various factors such as vibration, dust and light changes, which have a great impact on the image quality and detection results. Therefore, a detection method that can operate stably in a complex environment is needed to ensure the accuracy and real-time performance of coal gangue detection. Summary of the invention

[0006] In response to the above technical problems, the present invention provides a coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation, aiming to solve the coal gangue detection task in coal mine conveyor belt videos by combining multi-angle perception and Transformer technology.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0008] The gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation includes the following steps:

[0009] S1. The overall framework of the coal mine conveyor belt video gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation; the framework mainly includes three parts: multi-angle video image acquisition and preprocessing module, mixed-scale Transformer feature aggregation module and coal gangue detection algorithm module;

[0010] S2, multi-angle video image acquisition and preprocessing module acquires real-time video images of the coal mine conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt, thereby eliminating any blind spots and dead angles, and ensuring the comprehensiveness and accuracy of monitoring; preprocessing the video images, using image denoising algorithms to remove various noise interferences, while enhancing image quality, improving image clarity and contrast; using image enhancement technology to optimize image details, providing an image basis for coal gangue detection;

[0011] S3, mixed-scale Transformer feature aggregation module uses a mixed-scale Transformer model to extract features from the preprocessed image, making full use of features of different scales to capture details and global information in the image, and obtain multi-scale feature representation; aggregate the multi-scale features, combine the advantages of features of each scale, and generate fusion features through weighted fusion to improve the recognition ability and robustness of the model;

[0012] S4, the gangue detection algorithm module analyzes the fusion features through a deep separable convolutional network combined with an attention mechanism algorithm to identify and detect gangue; outputs the detection results and marks the location of the gangue, including the location and bounding box coordinates of the detected gangue.

[0013] The method of the multi-angle video image acquisition and preprocessing module in S2 is:

[0014] S2.1, the multi-angle video image acquisition and pre-processing module acquires real-time video images of the coal mine conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt;

[0015] S2.2. Image I total (y) Preprocessing, using image denoising algorithm to remove various noise interferences;

[0016] S2.3. Use image enhancement technology to optimize image details and use the Laplacian operator for sharpening.

[0017] The method for obtaining the real-time video image of the coal mine conveyor belt by using the multi-angle camera device in S2.1 is: the camera device is C i (i=1,2,…,n), where each camera device C i The conveyor belt is photographed from different angles, and the resulting images are shown in Figure 1. i (t); stitch and fuse the images to generate a comprehensive image I covering the entire conveyor belt total where w i is the weight of each camera.

[0018] The method of using the image denoising algorithm to remove various noise interferences in S2.2 is: the noise model is N(t), and the denoised image is I denoised (t) = I total (t)―N(t); using contrast enhancement algorithm, the contrast of the original image is C original , the contrast after enhancement is C enhanced =f(C original ), where f is the contrast enhancement function.

[0019] The method of using the Laplace operator for sharpening processing in S2.3 is: the enhanced image is I enhanced (t), which is expressed as Where α is the adjustment coefficient, It is the Laplace operator; after the preprocessing steps, it provides the basis for subsequent image analysis.

[0020] The method of the mixed-scale Transformer feature aggregation module in S3 is:

[0021] S3.1, Mixed-scale Transformer feature aggregation module uses a mixed-scale Transformer model to extract features from the preprocessed image; the preprocessed image is I pre , extract features of different scales through multiple Transformer modules;

[0022] There are n scales, and the feature of each scale is represented by F i (i=1,2,…,n); For each scale i, the initial feature representation F is extracted through convolution operation i =Conv(I pre ,k i ), where k i is the size of the i-th convolution kernel; the initial feature representation is processed by Transformer to obtain a deeper feature representation:

[0023] F′ i = Transformer(F i )

[0024] Denote the features of all scales as F′ i Combine them to get the multi-scale feature representation F multi :

[0025] F multi ={F′1,F′2,…,F′ n}

[0026] S3.2. Use weighted fusion technology to aggregate multi-scale features; the weight of each scale feature is w i , the fusion feature representation F fused It is expressed as:

[0027]

[0028] Among them, the weight w i It can be learned adaptively through training data; through multi-scale feature extraction and weighted fusion methods, it can fully combine the advantages of each scale feature to generate fusion features, improve the recognition ability and robustness of the model; the processed fusion feature F fused Provide data support for subsequent coal gangue detection.

[0029] The gangue detection algorithm module method in S4 is:

[0030] S4.1. The gangue detection algorithm module first uses a deep separable convolutional network combined with an attention mechanism to process the fusion features, and then identifies and locates the gangue through a classifier and a locator; the deep separable convolutional network includes deep convolution and point-by-point convolution, and the input feature map is F fused , whose dimensions are H×W×C;

[0031] In the depth convolution, each channel performs convolution operation independently to obtain the intermediate feature map F d =DepthwiseConv(F fused ), where DepthwiseConv represents the deep convolution operation; point-by-point convolution is performed on the intermediate feature map F d Perform 1×1 convolution to fuse the information of each channel and obtain the output feature map F p =PointwiseConv(F d ), where PointwiseConv represents the point-wise convolution operation;

[0032] Introduce the attention mechanism channel attention and spatial attention; channel attention calculates the channel attention of the feature map through global average pooling and global maximum pooling:

[0033] F ca =σ(FC(GAP(F p ))+FC(GMP(F p )))

[0034] Among them, σ represents the Sigmoid function, FC represents the fully connected layer, GAP and GMP represent global average pooling and global maximum pooling respectively;

[0035] Spatial attention is calculated on the feature map through global average pooling and global maximum pooling:

[0036] F sa =σ(Conv(Concat(GAP(F p ),GMP(F p )))))

[0037] Among them, Concat represents the feature map concatenation operation;

[0038] Fusion attention multiplies channel attention and spatial attention to obtain the final weighted feature map F att :

[0039] F att =F p ·F ca ·F sa

[0040] Through the classifier, the feature map F att Processing is performed to generate a probability map P of whether each location contains gangue:

[0041] P = Softmax(Conv(F att ))

[0042] Among them, the Softmax function is used to convert the output after the convolution operation into a probability value, indicating the probability of each position belonging to coal gangue;

[0043] S4.2. Using the locator to map the feature map F att Processing is performed to generate the bounding box coordinates B of the gangue:

[0044] B=Conv(F att )

[0045] The bounding box coordinates B include the coordinates (x1, y1, x2, y2) of the upper left corner and the lower right corner of each detected gangue.

[0046] By combining the probability map P and the bounding box coordinates B, gangue can be identified and located; for each position (i, j), if the probability P(i, j) exceeds a certain threshold τ, it is considered that gangue is detected at the position, and its bounding box coordinates B(i, j) are recorded;

[0047] The final output detection result is expressed as:

[0048]

[0049] Among them, Results contains all detected gangue and its location and bounding box information.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention obtains real-time video images of the conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt, thereby eliminating any blind spots and dead angles. Combined with advanced image preprocessing techniques, such as denoising and image enhancement, the clarity and detail of the image are significantly improved. The use of a mixed-scale Transformer model for multi-scale feature extraction can fully capture various details and global information in the image and generate richer and more diverse feature representations. By weighted fusion of features at different scales, more representative fusion features are generated, further improving the recognition ability and robustness of the model. This series of steps work together to significantly improve the accuracy and efficiency of coal gangue detection.

[0052] 2. The method of the present invention performs well in dealing with complex working conditions and can effectively cope with challenges such as illumination changes, dust interference, and variable coal gangue morphology. Through mixed-scale feature extraction and weighted fusion technology, the model can make full use of feature information at different scales, enhancing the adaptability and robustness to complex working conditions. Even in harsh working environments, the system can still maintain efficient and stable detection performance. This feature ensures that coal gangue detection can achieve high accuracy and high reliability in various environments, providing stable technical support for coal mine production.

[0053] 3. Through the optimized deep learning detection algorithm, the system can quickly analyze the fusion features and identify and locate the gangue in real time. The detection algorithm combines the probability map and the bounding box coordinates to accurately mark the gangue at each location. The detection results are output in real time, including the location, quantity and size information of the gangue, which improves the real-time and automation level of the system. This feature not only greatly reduces the burden of manual detection, but also improves the automation level of coal mine production, improves production efficiency and safety, and provides strong technical support for intelligent management. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0055] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0056] Figure 1 is a flow chart of an embodiment of the present invention;

[0057] Figure 2 The overall framework diagram of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. These descriptions are only to further illustrate the features and advantages of the present invention, rather than to limit the claims of the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0059] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0060] like Figure 1 As shown, the framework of the present invention is mainly divided into the following four steps, which are connected layer by layer and finally integrated. The learning process mainly includes the following steps:

[0061] S1. The overall framework of the coal mine conveyor belt video gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation; the framework mainly includes three parts: multi-angle video image acquisition and preprocessing module, mixed-scale Transformer feature aggregation module and coal gangue detection algorithm module.

[0062] S2, multi-angle video image acquisition and preprocessing module acquires real-time video images of the coal mine conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt, thereby eliminating any blind spots and dead angles, and ensuring the comprehensiveness and accuracy of monitoring. The video image is preprocessed, and an image denoising algorithm is used to remove various noise interferences, while enhancing the image quality and improving the image clarity and contrast. Image enhancement technology is used to optimize image details to provide an image basis for coal gangue detection.

[0063] Figure 2 This is the overall framework diagram of the coal mine conveyor belt video gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation.

[0064] The multi-angle video image acquisition and preprocessing module in step S2 further comprises:

[0065] S2.1, multi-angle video image acquisition and preprocessing module acquires real-time video images of the coal mine conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt. i (i=1,2,…,n), where each camera device Ci shoots the conveyor belt from different angles, and the image obtained is I i (t). Stitch and fuse the images to generate a comprehensive image covering the entire conveyor belt. where w i is the weight of each camera.

[0066] S2.2. Image I total (t) is preprocessed and image denoising algorithm is used to remove various noise interferences. The noise model is N(t), and the denoised image is I denoised (t) = I total (t)―N(t). Using the contrast enhancement algorithm, the contrast of the original image is C original , the contrast after enhancement is C enhanced =f(C original ), where f is the contrast enhancement function.

[0067] S2.3, use image enhancement technology to optimize image details, and use Laplace operator for sharpening. The enhanced image is I enhanced (t), which is expressed as I denoised (t), where α is the adjustment coefficient, It is the Laplace operator. After the preprocessing steps, it provides the basis for the subsequent image analysis.

[0068] S3, mixed-scale Transformer feature aggregation module uses a mixed-scale Transformer model to extract features from the preprocessed image, making full use of features of different scales to capture details and global information in the image and obtain multi-scale feature representation. Multi-scale features are aggregated, combining the advantages of features at each scale, and fusion features are generated through weighted fusion to improve the recognition ability and robustness of the model.

[0069] The mixed-scale Transformer feature aggregation module in step S3 further comprises:

[0070] S3.1, the mixed-scale Transformer feature aggregation module uses the mixed-scale Transformer model to extract features from the preprocessed image. pre , features of different scales are extracted through multiple Transformer modules.

[0071] There are n scales, and the feature of each scale is represented by F i (i=1,2,…,n). For each scale i, the initial feature representation F is extracted through convolution operation. i =Conv(I pre ,k i ), where k i is the size of the i-th convolution kernel. The initial feature representation is processed by Transformer to obtain a deeper feature representation:

[0072] F′ i = Transformer(F i )

[0073] Denote the features of all scales as F′ i Combine them to get the multi-scale feature representation F multi :

[0074] F multi ={F′1,F′2,…,F′ n}

[0075] S3.2. Use weighted fusion technology to aggregate multi-scale features. The weight of each scale feature is w i , the fusion feature representation F fused It is expressed as:

[0076]

[0077] Among them, the weight w iIt can be learned adaptively through training data. Through the method of multi-scale feature extraction and weighted fusion, the advantages of each scale feature can be fully combined to generate fusion features, thereby improving the recognition ability and robustness of the model. The processed fusion feature F fused Provide data support for subsequent coal gangue detection.

[0078] S4, the gangue detection algorithm module analyzes the fusion features through the deep separable convolutional network combined with the attention mechanism algorithm to identify and detect gangue. Output the detection results and mark the location of the gangue, including the location and bounding box coordinates of the detected gangue.

[0079] The gangue detection algorithm module in step S4 further comprises:

[0080] S4.1. The gangue detection algorithm module first uses the deep separable convolutional network combined with the attention mechanism to process the fusion features, and then identifies and locates the gangue through the classifier and locator. The deep separable convolutional network includes deep convolution and point-by-point convolution. The input feature map is F fused , whose dimensions are H×W×C (height, width, and number of channels).

[0081] In the depth convolution, each channel performs convolution operation independently to obtain the intermediate feature map F d =DepthwiseConv(F fused ), where DepthwiseConv represents the depth convolution operation. Point-wise convolution is applied to the intermediate feature map F d Perform 1×1 convolution to fuse the information of each channel and obtain the output feature map F p =PointwiseConv(F d ), where PointwiseConv represents the point-wise convolution operation.

[0082] Introduce the attention mechanism channel attention and spatial attention. Channel attention calculates the channel attention of the feature map through global average pooling and global maximum pooling:

[0083] F ca =σ(FC(GAP(F p ))+FC(GMP(F p )))

[0084] Among them, σ represents the Sigmoid function, FC represents the fully connected layer, GAP and GMP represent global average pooling and global maximum pooling, respectively.

[0085] Spatial attention is calculated on the feature map through global average pooling and global maximum pooling:

[0086] Fsa =σ(Conv(Concat(GAP(F p ),GMP(F p )))))

[0087] Among them, Concat represents the feature map concatenation operation.

[0088] Fusion attention multiplies channel attention and spatial attention to obtain the final weighted feature map F att :

[0089] F att =F p ·F ca ·F sa

[0090] Through the classifier, the feature map F att Processing is performed to generate a probability map P of whether each location contains gangue:

[0091] P = Softmax(Conv(F att ))

[0092] Among them, the Softmax function is used to convert the output after the convolution operation into a probability value, which indicates the probability of belonging to coal gangue at each position.

[0093] S4.2. Using the locator to map the feature map F att Processing is performed to generate the bounding box coordinates B of the gangue:

[0094] B=Conv(F att )

[0095] The bounding box coordinates B include the coordinates (x1, y1, x2, y2) of the upper left corner and the lower right corner of each detected gangue.

[0096] Gangue can be identified and located by combining the probability map P and the bounding box coordinates B. For each position (i, j), if the probability P(i, j) exceeds a certain threshold τ, it is considered that gangue is detected at that position, and its bounding box coordinates B(i, j) are recorded.

[0097] The final output detection result is expressed as:

[0098]

[0099] Among them, Results contains all detected gangue and its location and bounding box information.

[0100] In order to verify the effectiveness of the proposed method in gangue identification, a gangue identification dataset target detection task was designed. The number of labeled categories is 2, and the labeled category names are coal (Coal) and gangue (Rock). The accuracy is used to measure the average precision of a single category, which is the average of the average precision of all categories, and is used to comprehensively evaluate the performance of the model on multiple categories. IoU (intersection over union) is used to measure the overlap between the predicted box and the true box, and loU ≥ 0.5 is defined as a correct detection.

[0101] Comparison methods include:

[0102] (1) LeNet-5, a classic and efficient convolutional neural network proposed by Yann Lecun, has one input layer, two convolutional layers, two pooling layers, and three fully connected layers.

[0103] (2) VGG-Net was proposed by the Visual Geometry Group of the Department of Science and Engineering at the University of Oxford. Compared with AlexNet, it has smaller convolution kernels and deeper layers. Using continuous 3x3 small convolution kernels, the parameters of VGG-Net mainly appear in the last two fully connected layers, each with 4096 neurons.

[0104] Table 1 Comparison of experimental results of different methods

[0105] Model Coal-test accuracy Rock-test accuracy Mean average precision LeNet-5 73.4％ 63.8％ 68.6％ VGG-Net 75.9％ 70.2％ 73.05％ This embodiment 85.4％ 80.3％ 82.85％

[0106] The detection accuracy of the method proposed in this embodiment on Coal-test and Rock-test is significantly higher than that of LeNet-5 and VGG-Net. The accuracy on Coal-test is 0.854, the accuracy on Rock-test is 0.803, and the average accuracy is 0.8285, which is much higher than 0.686 of LeNet-5 and 0.7305 of VGG-Net. This shows that the method proposed in this embodiment shows higher stability and accuracy in different test environments.

[0107] The method proposed in this embodiment has significant advantages in feature extraction and fusion through multi-angle perception and mixed-scale Transformer feature aggregation. The mixed-scale Transformer model can better capture the details and global information in the image, and combined with weighted fusion technology, it makes the feature representation richer and more diverse, thereby improving the accuracy and robustness of detection.

[0108] The experimental results show that the method proposed in this embodiment has significant performance advantages in coal gangue detection, can maintain high accuracy and stability under complex working conditions, and is significantly better than the traditional LeNet-5 and VGG-Net models. This result verifies the effectiveness and superiority of multi-angle perception and mixed-scale Transformer feature aggregation technology in practical applications.

[0109] Only the preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention, and various changes should be included in the protection scope of the present invention.

Claims

1. A coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation, characterized in that: The following steps are involved: S1. The overall framework of the coal mine conveyor belt video gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation; the framework mainly includes three parts: multi-angle video image acquisition and preprocessing module, mixed-scale Transformer feature aggregation module and coal gangue detection algorithm module; S2, multi-angle video image acquisition and pre-processing module acquires real-time video images of the coal mine conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt, thereby eliminating any blind spots and dead angles, and ensuring the comprehensiveness and accuracy of monitoring; Preprocess the video image, use image denoising algorithm to remove various noise interferences, enhance image quality, and improve image clarity and contrast; use image enhancement technology to optimize image details and provide an image basis for coal gangue detection; S3, mixed-scale Transformer feature aggregation module uses a mixed-scale Transformer model to extract features from the preprocessed image, making full use of features of different scales to capture details and global information in the image, and obtain multi-scale feature representation; aggregate the multi-scale features, combine the advantages of features of each scale, and generate fusion features through weighted fusion to improve the recognition ability and robustness of the model; The method of the mixed-scale Transformer feature aggregation module in S3 is: S3.1, Mixed-scale Transformer feature aggregation module uses a mixed-scale Transformer model to extract features from the preprocessed image; the preprocessed image is I pre , extract features of different scales through multiple Transformer modules; There are n scales, and the feature of each scale is represented by F i (i=1,2,…,n); For each scale i, the initial feature representation F is extracted through convolution operation i =Conv(I pre ,k i ), where k i is the size of the i-th convolution kernel; the initial feature representation is processed by Transformer to obtain a deeper feature representation: F i ′=Transformer(F i ) The features of all scales are represented as F i ′ is combined to obtain the multi-scale feature representation F multi : F multi ={F′1,F′2,…,F′ n } S3.

2. Use weighted fusion technology to aggregate multi-scale features; the weight of each scale feature is w i , the fusion feature representation F fused It is expressed as: Among them, the weight w i It can be learned adaptively through training data; through multi-scale feature extraction and weighted fusion methods, it can fully combine the advantages of each scale feature to generate fusion features, improve the recognition ability and robustness of the model; the processed fusion feature F fused Provide data support for subsequent coal gangue testing; S4, the gangue detection algorithm module analyzes the fusion features through the deep separable convolutional network combined with the attention mechanism algorithm to identify and detect gangue; Output the detection results and mark the location of the gangue, including the location and bounding box coordinates of the detected gangue.

2. The method for coal gangue detection based on multi-angle perception and mixed-scale Transformer feature aggregation according to claim 1 is characterized in that: The method of the multi-angle video image acquisition and preprocessing module in S2 is: S2.1, the multi-angle video image acquisition and pre-processing module acquires real-time video images of the coal mine conveyor belt through a multi-angle camera device to ensure comprehensive coverage of all areas of the conveyor belt; S2.

2. Image I total (t) Perform preprocessing and use image denoising algorithm to remove various noise interferences; S2.

3. Use image enhancement technology to optimize image details and use the Laplacian operator for sharpening.

3. The coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation according to claim 2 is characterized in that: The method for obtaining the real-time video image of the coal mine conveyor belt by using the multi-angle camera device in S2.1 is: the camera device is C i (i=1,2,…,n), where each camera device C i The conveyor belt is photographed from different angles, and the resulting images are shown in Figure 1. i (t); stitch and fuse the images to generate a comprehensive image covering the entire conveyor belt where w i is the weight of each camera.

4. The coal gangue detection method based on multi-angle perception and mixed-scale Transformer feature aggregation according to claim 2 is characterized in that: The method of using the image denoising algorithm to remove various noise interferences in S2.2 is: the noise model is N(t), and the denoised image is I denoised (t) = I total (t)-N(t); using contrast enhancement algorithm, the contrast of the original image is C original , the contrast after enhancement is C enhanced =f(C original ), where f is the contrast enhancement function.

5. The method for coal gangue detection based on multi-angle perception and mixed-scale Transformer feature aggregation according to claim 2 is characterized in that: The method of using the Laplace operator for sharpening processing in S2.3 is: the enhanced image is I enhanced (t), which is expressed as Where α is the adjustment coefficient, It is the Laplace operator; after the preprocessing steps, it provides the basis for subsequent image analysis.

6. The method for coal gangue detection based on multi-angle perception and mixed-scale Transformer feature aggregation according to claim 1 is characterized in that: The gangue detection algorithm module method in S4 is: S4.

1. The gangue detection algorithm module first uses a deep separable convolutional network combined with an attention mechanism to process the fusion features, and then identifies and locates the gangue through a classifier and a locator; the deep separable convolutional network includes deep convolution and point-by-point convolution, and the input feature map is F fused , whose dimensions are H×W×C; In the depth convolution, each channel performs convolution operation independently to obtain the intermediate feature map F d =DepthwiseConv(F fused ), where DepthwiseConv represents the deep convolution operation; point-by-point convolution is performed on the intermediate feature map F d Perform 1×1 convolution to fuse the information of each channel and obtain the output feature map F p =PointwiseConv(F d ), where PointwiseConv represents the point-wise convolution operation; Introduce the attention mechanism channel attention and spatial attention; channel attention calculates the channel attention of the feature map through global average pooling and global maximum pooling: F ca =σ(FC(GAP(F p ))+FC(GMP(F p ))) Among them, σ represents the Sigmoid function, FC represents the fully connected layer, GAP and GMP represent global average pooling and global maximum pooling respectively; Spatial attention is calculated on the feature map through global average pooling and global maximum pooling: F sa =σ(Conv(Concat(GAP(F p ),GMP(F p ))))) Among them, Concat represents the feature map concatenation operation; Fusion attention multiplies channel attention and spatial attention to obtain the final weighted feature map F att : F att =F p ·F ca ·F sa Through the classifier, the feature map F att Processing is performed to generate a probability map P of whether each location contains gangue: P=Softmax(Conv(F att )) Among them, the Softmax function is used to convert the output after the convolution operation into a probability value, indicating the probability of each position belonging to coal gangue; S4.

2. Using the locator to map the feature map F att Processing is performed to generate the bounding box coordinates B of the gangue: B=Conv(F att ) Wherein, the bounding box coordinates B include the coordinates (x1, y1, x2, y2) of the upper left corner and the lower right corner of each detected gangue; By combining the probability map P and the bounding box coordinates B, gangue can be identified and located; for each position (i, j), if the probability P(i, j) exceeds a certain threshold τ, it is considered that gangue is detected at the position, and its bounding box coordinates B(i, j) are recorded; The final output detection result is expressed as: Among them, Results contains all detected gangue and its location and bounding box information.

Images