A coal mine conveyor deviation detection method based on federated learning

Abstract

The application provides a coal mine conveyor deviation detection method based on federated learning, which comprises coal mine conveyor deviation state image dataset construction, conveyor deviation state detection model construction, cross-mine federated learning training strategy design, and federated deviation detection model deployment and intelligent operation and maintenance application, wherein the conveyor deviation state detection model comprises basic feature extraction, cross-scale feature extraction, multi-branch attention collaborative fusion and double prediction head modules. The detection method of the application realizes collaborative training of multiple mines without sharing original data, and combines deviation category and continuous offset amount for joint prediction, thereby improving the detection precision and generalization ability in a complex underground environment.

Description

Technical Field

[0001] This invention relates to the field of coal mine underground conveyor detection technology, specifically to a coal mine conveyor deviation detection method based on federated learning. Background Technology

[0002] As a crucial link in coal transportation, the operational stability of underground coal conveying systems directly impacts production efficiency and equipment safety. Conveyor belt misalignment is a common problem in actual operation, manifesting as a deviation of the conveyor belt from its centerline. In severe cases, this can lead to material spillage, accelerated equipment wear, and even shutdown accidents, posing a significant threat to mine safety. Due to the complex underground environment of coal mines, including high dust concentrations, unstable lighting conditions, and long-term equipment vibration, the conveyor belt's operating state exhibits significant nonlinear and uncertain characteristics. Existing methods for detecting conveyor belt misalignment include the following:

[0003] 1) Conveyor Misalignment Detection Method Based on Manual Inspection

[0004] In existing coal mine production systems, monitoring conveyor misalignment primarily relies on manual inspections or experience-based judgments by monitoring center personnel through video feeds. This includes observing whether the conveyor belt deviates from its center position, whether there is localized material buildup, or edge anomalies. While this method relies on the operator's accumulated experience and offers some flexibility, it is easily influenced by subjective factors in practical applications. Furthermore, due to the complex underground environment, the continuous operation of the conveyor system, and the wide distribution of equipment, manual methods are insufficient for long-term continuous monitoring of the conveyor's operating status. This leads to issues such as detection delays, missed detections, or misjudgments, failing to meet the real-time monitoring requirements of intelligent mines.

[0005] 2) Image Processing-Based Conveyor Misalignment Detection Method

[0006] Some existing technologies employ traditional image processing methods to detect conveyor belt misalignment, such as estimating the conveyor belt position through edge detection, Hough transform, or centerline fitting, and calculating its offset from the image center. While these methods can achieve a certain degree of misalignment detection in relatively clear and well-lit environments, in actual underground coal mines, dust interference, severe occlusion, and complex background variations lead to unstable edge extraction and significant centerline positioning errors. Furthermore, these methods typically rely on manual feature design, lacking the ability to model complex semantic structures and global spatial relationships, making them ill-suited to the complex distribution and variations across multiple mines and working conditions.

[0007] 3) Industrial vision inspection method based on centralized training

[0008] Some industrial visual inspection methods employ a centralized data training framework, aggregating multi-source mine data to a central server for unified training to improve model performance and generalization ability. However, in coal mining scenarios, data from different mines exhibits significant distribution differences, and issues of production safety and data privacy are involved. Centralized training carries risks of data leakage and compliance limitations. Furthermore, the high cost of transmitting and storing large-scale image data makes continuous updates and online optimization difficult in practical deployments, thus limiting its application in multi-mine collaborative scenarios.

[0009] Therefore, existing manual inspection methods for detecting conveyor misalignment are insufficient for accurate and continuous monitoring of misalignment, especially in the early stages of minor misalignment where missed detections are common. Furthermore, significant differences exist between coal mines in terms of geological conditions, equipment models, and operating conditions, resulting in limited data distribution from a single mine and weak generalization ability of the trained detection model. In addition, due to the sensitivity and security requirements of data among coal mining enterprises, centralized data aggregation poses a high risk of data leakage, thus limiting cross-mine collaborative modeling. Summary of the Invention

[0010] To address the technical problems of existing conveyor misalignment detection methods, such as heavy reliance on manual labor, weak model generalization ability, and difficulty in sharing data among multiple mines, this invention provides a coal mine conveyor misalignment detection method based on federated learning. This method enables collaborative training among multiple mines without sharing original data and integrates misalignment category and continuous offset joint prediction to improve detection accuracy and generalization ability in complex underground environments.

[0011] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0012] A method for detecting conveyor belt misalignment in coal mines based on federated learning includes the following steps:

[0013] S1. Construction of coal mine conveyor belt misalignment image dataset: By deploying industrial-grade image acquisition equipment in key operating areas of conveyors in multiple underground coal mines, images of the conveyor belt's operating status during operation are continuously collected. After standardizing the collected images, conveyor belt images are obtained. Professionals then label the conveyor belt misalignment status to construct a joint labeling system that includes misalignment status categories and continuous offsets, thereby forming a misalignment status image dataset covering multiple mines and multiple working conditions. At the same time, distributed data subsets are constructed on a mine-by-mine basis, and each data subset is stored independently on the local machine.

[0014] S2. Construction of Conveyor Deviation Detection Model: A network structure for the conveyor deviation detection model is constructed, including basic feature extraction, cross-scale structure modeling, multi-branch attention fusion, and dual prediction heads. Specifically, the model first uses a pre-trained ResNet50 model to extract multi-layer semantic features of the conveyor belt image, and achieves feature adaptation through partial parameter freezing and fine-tuning strategies. Then, a cross-scale feature extraction module is introduced to jointly model the local details, mesoscale structure, and global offset of the conveyor belt. On this basis, a multi-branch attention collaborative fusion module is constructed to enhance the feature expression capability from three dimensions: center position perception, edge structure enhancement, and channel semantic expression. Finally, the dual prediction head module outputs the deviation status category and continuous offset respectively.

[0015] S3. Cross-mine federated learning training strategy design: Construct a federated learning training framework for multi-mine distributed data environments. Each mine node trains the model independently based on local data, and the central server achieves secure aggregation of model parameters, completing global model optimization without transmitting the original image data. To address the differences in computing resources among different mines, a hierarchical training strategy based on computing power grouping is introduced. By grouping clients and assigning different parameter training ranges, the overall training efficiency is improved. At the same time, a masking mechanism based on parameter amplitude is introduced during federated communication to sparsely transmit model parameters, thereby reducing communication overhead and improving training stability.

[0016] S4. Deployment and Intelligent Operation and Maintenance Application of Federated Misalignment Detection Model: The conveyor misalignment status detection model obtained through federated learning is deployed in the edge computing equipment or monitoring server in the coal mine to realize real-time detection and analysis of the conveyor belt operation status. The equipment continuously collects images of the conveyor belt operation status and inputs them into the model for inference, outputs the misalignment status category and continuous offset results, and feeds back the detection results to the mine conveyor system monitoring platform or safety management system in real time. Based on the detection results, corresponding operation and maintenance strategies are triggered, thereby improving the safety of conveyor operation and the level of intelligent operation and maintenance.

[0017] Furthermore, the construction of the coal mine conveyor misalignment image dataset in step S1 includes:

[0018] S11. Deployment of the Vision Acquisition System for Conveyors: Industrial-grade image acquisition equipment is deployed in key operating areas of conveyors in multiple underground coal mines to continuously acquire images of the operating status of the conveyor belt during operation. The key operating areas of the conveyor include the head drive area, the tail tensioning area, and the middle operating area.

[0019] S12. Conveyor Belt Operating Status Image Acquisition and Standardization Processing: The industrial-grade image acquisition equipment deployed in step S11 is used to acquire images of the conveyor belt operating status. The acquired images are then normalized to obtain standardized conveyor belt images. ;

[0020] S13. Construction of joint labels for conveyor belt misalignment status: For each standardized conveyor belt image Construct a system that includes categories of deviation states. With continuous offset The joint labeling system, the deviation status category This indicates the current offset of the conveyor belt center relative to the ideal centerline, including normal, slight, and severe offset states; the continuous offset amount This is used to quantify the degree of deviation of the current center of the conveyor belt from the ideal center line, and is expressed in a normalized manner with a value range of [-1, 1]. Where 0 indicates that the conveyor belt is at the ideal center position, negative values ​​indicate left deviation, and positive values ​​indicate right deviation, thereby achieving a unified qualitative and quantitative expression of the deviation status.

[0021] S14. Consistent Labeling and Dataset Construction Across Multiple Mines: Professionals familiar with the operational characteristics of coal mine conveyor systems complete the image labeling work of step S13 locally at each mine, using standardized conveyor belt images. For input, the deviation state category With continuous offset As output, a set of deviation state image datasets is constructed, based on the method in the first... Extensive dataset collection was conducted in each mine to obtain a complete dataset for that mine. The dataset collection process was repeated in different mines, yielding a total of [number missing] datasets. The complete dataset of each mine is thus formed into a multi-mine distributed dataset, i.e. ,and Each mine's data subset is stored independently on its local machine, without sharing the original data, thus providing a distributed data foundation for subsequent model training under the federated learning framework.

[0022] Furthermore, the construction of the conveyor misalignment detection model in step S2 includes:

[0023] S21. Basic Feature Extraction Design: A pre-trained ResNet50 model is used as the basic feature extraction network. Specifically, the conveyor belt image to be detected... First, low-level features are extracted through Conv1 convolutional layers and max pooling layers to obtain shallow features containing edge and texture information. Then, it sequentially passes through Conv2, Conv3, Conv4, and Conv5 convolutional layers containing residuals to progressively extract higher-level semantic information, ultimately obtaining deep features with global structural representation capabilities. Furthermore, during model training, the model parameters from Conv1 to Conv3 are kept frozen, and only the parameters of Conv4 and Conv5 are fine-tuned during training.

[0024] S22. Cross-scale feature extraction module design: based on deep features For the input, construct three parallel convolutional branches:

[0025] (1) A 3×3 convolution kernel is used to extract local detail features, focusing on characterizing the conveyor belt edge and subtle offset information, to obtain local structural features. ;

[0026] (2) A 5×5 convolution kernel is used to extract mesoscale structural features to capture the local bending and regional offset trends of the conveyor belt, thus obtaining mesoscale features. ;

[0027] (3) A 7×7 convolution kernel is used to extract large receptive field features, which are used to model the overall positional shift and global structural changes of the conveyor belt, and to obtain global structural features. ;

[0028] Each of the above branches uses a convolutional layer followed by a ReLU activation function to enhance nonlinear expressive power. Subsequently, the three features are concatenated to obtain cross-scale fused features. ;

[0029] S23. Multi-branch attention collaborative fusion module design: based on cross-scale fusion features Using this as input, three attention branches are constructed, including coordinate attention, edge attention, and channel attention mechanisms, and center-aware features are obtained from each branch. Edge enhancement features and semantic enhancement features Subsequently, the three attention features are uniformly scaled through 1×1 convolution, and then nonlinearly fused using a fully connected mapping layer and the LeakyReLU activation function to finally obtain the attention-enhanced features. ;

[0030] S24. Dual Prediction Head Module Design: Constructing a dual-branch prediction structure and applying attention-enhanced features. Joint modeling is performed, specifically, attention-enhanced features. Each prediction branch follows one of the following two independent prediction branches:

[0031] (1) Deviation category prediction branch: The conveyor belt status is classified and determined by two fully connected layers and the Softmax classification function, and the deviation category result is output. ;

[0032] (2) Offset Regression Branch: Through two fully connected layers and using the Tanh activation function to constrain the output within the range of [-1,1], continuous estimation of the conveyor belt center offset is achieved, and the offset result is output. ;

[0033] The parameters of the two branches are independent of each other, thus enabling qualitative judgment and quantitative analysis;

[0034] S25. Components of Trainable Parameters: During the training of the conveyor misalignment detection model, the trainable parameters of the model include the following components:

[0035] (1) The parameters of the Conv4 and Conv5 convolutional layers in the pre-trained ResNet50 model are used as the first set of parameters. ;

[0036] (2) All parameters of the cross-scale feature extraction module are used as the second set of parameters. ;

[0037] (3) All parameters of the multi-branch attention collaborative fusion module are used as the third set of parameters. ;

[0038] (4) All parameters of the dual prediction head module are used as the fourth set of parameters. .

[0039] Furthermore, step S3, the design of the cross-mine federated learning training strategy, includes:

[0040] S31. System Initialization: For N mining servers, define them as N Clients, i.e. Meanwhile, a central server for coordinating training is set up as the Server side, i.e., S;

[0041] S32. Client Grouping: Based on the local server computing resources of N Clients, divide the N Clients into four sets, specifically as follows:

[0042] The local server computing resources for N clients are respectively Then, based on the computing resources of the N clients, the kmean algorithm is used to divide the N clients into the following four sets. ,in This indicates that the Client's local server has the worst computing resources within this set, and The set with the strongest computing resources;

[0043] S33. Training Mode Initialization: This invention designs a hierarchical training method, specifically:

[0044] (1) For the worst computing resources The client within the system only fine-tunes all parameters of the dual prediction head module during training, i.e., the parameters. ;

[0045] (2) For those with slightly better computing resources The client within the system fine-tunes the parameters of the dual-prediction-head module and the multi-branch attention collaborative fusion module during training, i.e., the parameters. and ;

[0046] (3) For those with superior computing resources The client within the system fine-tunes the parameters of the dual prediction head module, the multi-branch attention collaborative fusion module, and the cross-scale feature extraction module during training. , and ;

[0047] (4) For the best computing resources The client within the system fine-tunes all the parameters of the model during training, i.e., the parameters. , , and ;

[0048] S34. Cross-mine federated learning training strategy: After completing the initialization steps S31 to S33, the following process steps are used for cross-mine federated learning training:

[0049] (1) Initialize the conveyor misalignment detection model on the central server S, and initialize the four parameters of the model as follows: ;

[0050] (2) Calculate the Top-k parameter mask with reference to the parameter magnitude, and obtain the mask. and apply the mask to each Regarding parameters, the parameters obtained after masking Then, all masked parameters are sent simultaneously to all local clients in the four sets, thereby reducing the number of communication parameters.

[0051] (3) No. Client Parameters after receiving the mask Based on the set to which the Client belongs, determine which parameters are frozen and which are trainable in the Client according to the criteria in step S33;

[0052] (4) No. Client Based on its locally collected complete mine dataset Perform local fine-tuning training to obtain the trained parameters. For the parameters that remain frozen, their values ​​remain unchanged before and after training; subsequently, the... Client The trained parameters Send to the central server;

[0053] (5) All clients have gone through steps (3) and (4), so the central server has collected N sets of trained parameters. Then, the FedAvg algorithm is used to average and aggregate these parameters to obtain the parameters after the first round of training. ;

[0054] (6) Repeat steps (2) to (5) for a total of Finally, the trained conveyor deviation detection model is obtained on the central server, whereby... The maximum number of training rounds is preset.

[0055] Furthermore, step S4, the deployment of the federated deviation detection model and the application of intelligent operation and maintenance, includes:

[0056] S41. Conveyor Monitoring System and Model Deployment: Industrial-grade image acquisition equipment is deployed in key operating areas of conveyors in multiple underground coal mines to continuously collect images of the conveyor belt's operating status during operation. The key operating areas of the conveyor include the head drive area, the tail tensioning area, and the middle operating area. Simultaneously, the conveyor deviation detection model trained in step S3 is deployed in underground coal mine edge computing devices or monitoring servers, enabling the model to run independently in each mine and achieve localized real-time inference and analysis.

[0057] S42. Real-time acquisition of conveyor belt operating status images: The operating status of the conveyor belt is continuously acquired through deployed image acquisition equipment, and the acquired images are normalized to obtain standardized conveyor belt images. ;

[0058] S43. Real-time detection and output of conveyor belt misalignment: Standardized conveyor belt images... The input is fed into a locally deployed conveyor misalignment detection model. The model performs forward inference on the image and outputs the following two types of results:

[0059] (1) Results of deviation status It is used to determine whether the conveyor belt is currently in a normal state, slightly misaligned, or severely misaligned state.

[0060] (2) Continuous offset results This is used to quantify the degree of deviation of the current center of the conveyor belt from the ideal centerline;

[0061] S44. Intelligent operation and maintenance application driven by deviation status: The deviation detection results output by the model are fed back to the mine conveying system monitoring platform or safety management system in real time, and corresponding operation and maintenance strategies are triggered according to the detection results.

[0062] Furthermore, the corresponding operation and maintenance strategies in step S44 include, but are not limited to:

[0063] (1) When the deviation status category result When the vehicle deviates slightly or severely, it will automatically issue a warning signal to prompt on-site personnel to check or make adjustments.

[0064] (2) Based on the continuous offset results Dynamically assess the degree of belt misalignment to provide a basis for adjusting the conveyor belt tensioning or adjustment device;

[0065] (3) Record and analyze historical test results for conveyor operation status assessment and fault trend prediction, thereby improving the level of intelligent equipment operation and maintenance.

[0066] Compared with existing technologies, the coal mine conveyor deviation detection method based on federated learning provided by this invention has the following advantages: 1) High-precision detection of conveyor deviation: By constructing a joint prediction model of deviation status category and continuous offset, the qualitative identification and quantitative estimation of conveyor belt deviation status are integrated into a single model, improving the accuracy and robustness of detection in complex underground environments. 2) Ensuring data privacy across multiple mines and improving model generalization ability: Using a distributed federated learning training method, data from each mine is stored locally and used for training, eliminating the need for centralized uploading of original image data. Through a hierarchical client training strategy and a selective parameter update mechanism, cross-mine collaborative modeling is achieved while ensuring data security, improving the model's adaptability to different working conditions. 3) Improving model training efficiency and engineering deployment capability: Through hierarchical federated training and a differentiated parameter update mechanism, the training burden and communication overhead of different computing nodes are reduced, enabling efficient collaborative training in heterogeneous environments across multiple mines, and supporting real-time deployment and online application of the model at the edge. Attached Figure Description

[0067] Figure 1 This is an overall flowchart of the coal mine conveyor deviation detection method based on federated learning provided by the present invention.

[0068] Figure 2 This is a network structure diagram of the conveyor misalignment detection model provided by the present invention.

[0069] Figure 3 This is a flowchart of the cross-mine federated learning training strategy provided by the present invention. Detailed Implementation

[0070] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below with reference to specific illustrations.

[0071] Please refer to Figure 1 As shown, this invention provides a coal mine conveyor misalignment detection method based on federated learning, comprising the following steps:

[0072] S1. Construction of coal mine conveyor belt misalignment image dataset: By deploying industrial-grade image acquisition equipment in key operating areas of conveyors in multiple underground coal mines, images of the conveyor belt's operating status during operation are continuously collected. After standardizing the collected images, conveyor belt images are obtained. Professionals then label the conveyor belt misalignment status to construct a joint labeling system that includes misalignment status categories and continuous offsets, thereby forming a misalignment status image dataset covering multiple mines and multiple working conditions. At the same time, distributed data subsets are constructed on a mine-by-mine basis, and each data subset is stored independently on the local machine.

[0073] S2. Construction of Conveyor Deviation Detection Model: A network structure for the conveyor deviation detection model is constructed, including basic feature extraction, cross-scale structure modeling, multi-branch attention fusion, and dual prediction heads. Specifically, the model first uses a pre-trained ResNet50 model to extract multi-layer semantic features of the conveyor belt image, and achieves feature adaptation through partial parameter freezing and fine-tuning strategies. Then, a cross-scale feature extraction module is introduced to jointly model the local details, mesoscale structure, and global offset of the conveyor belt. On this basis, a multi-branch attention collaborative fusion module is constructed to enhance the feature expression capability from three dimensions: center position perception, edge structure enhancement, and channel semantic expression. Finally, the dual prediction head module outputs the deviation status category and continuous offset respectively.

[0074] S3. Cross-mine federated learning training strategy design: Construct a federated learning training framework for multi-mine distributed data environments. Each mine node trains the model independently based on local data, and the central server achieves secure aggregation of model parameters, completing global model optimization without transmitting the original image data. To address the differences in computing resources among different mines, a hierarchical training strategy based on computing power grouping is introduced. By grouping clients and assigning different parameter training ranges, the overall training efficiency is improved. At the same time, a masking mechanism based on parameter amplitude is introduced during federated communication to sparsely transmit model parameters, thereby reducing communication overhead and improving training stability.

[0075] S4. Deployment and Intelligent Operation and Maintenance Application of Federated Misalignment Detection Model: The conveyor misalignment status detection model obtained through federated learning is deployed in the edge computing equipment or monitoring server in the coal mine to realize real-time detection and analysis of the conveyor belt operation status. The equipment continuously collects images of the conveyor belt operation status and inputs them into the model for inference, outputs the misalignment status category and continuous offset results, and feeds back the detection results to the mine conveyor system monitoring platform or safety management system in real time. Based on the detection results, corresponding operation and maintenance strategies are triggered, thereby improving the safety of conveyor operation and the level of intelligent operation and maintenance.

[0076] As a specific implementation, due to the significant differences in underground environments among different coal mines, the data collected from a single mine has limited distribution, making it difficult to support the model's universality under complex working conditions. Furthermore, if a centralized approach is used to aggregate image data from multiple mines, there is a risk of data leakage within the mining area. Therefore, this invention employs a distributed method for collecting and labeling conveyor misalignment data. Data from each mine is stored and labeled locally only and participates in subsequent federated learning training. Specifically, step S1, the construction of the coal mine conveyor misalignment image dataset, includes:

[0077] S11. Deployment of the Vision Acquisition System for Conveyors: Industrial-grade image acquisition equipment is deployed in key operating areas of conveyors in multiple underground coal mines to continuously acquire images of the operating status of the conveyor belt during operation. The key operating areas of the conveyor include the head drive area, the tail tensioning area, and the middle operating area. The acquisition system is deployed independently in different mines, and each node is only responsible for local data acquisition and storage, without cross-mine data transmission.

[0078] S12. Conveyor Belt Operating Status Image Acquisition and Standardization Processing: The industrial-grade image acquisition equipment deployed in step S11 is used to acquire images of the conveyor belt operating status. The acquired images are then normalized to obtain standardized conveyor belt images. .

[0079] S13. Construction of joint labels for conveyor belt misalignment status: For each standardized conveyor belt image Construct a system that includes categories of deviation states. With continuous offset The joint labeling system, the deviation status category This indicates the current offset of the conveyor belt center relative to the ideal centerline, including normal, slight, and severe offset states; the continuous offset amount This is used to quantify the degree of deviation of the current center of the conveyor belt from the ideal center line, and is expressed in a normalized manner with a value range of [-1, 1]. Here, 0 indicates that the conveyor belt is at the ideal center position, negative values ​​indicate left deviation, and positive values ​​indicate right deviation, thereby achieving a unified qualitative and quantitative expression of the deviation status.

[0080] S14. Consistent Labeling and Dataset Construction Across Multiple Mines: Professionals familiar with the operational characteristics of coal mine conveyor systems complete the image labeling work of step S13 locally at each mine, using standardized conveyor belt images. For input, the deviation state category With continuous offset As output, a set of deviation state image datasets is constructed, based on the method in the first... Extensive dataset collection was conducted in each mine to obtain a complete dataset for that mine. The dataset collection process was repeated in different mines, yielding a total of [number missing] datasets. The complete dataset of each mine is thus formed into a multi-mine distributed dataset, i.e. ,and Each mine's data subset is stored independently on its local machine, without sharing the original data, thus providing a distributed data foundation for subsequent model training under the federated learning framework.

[0081] As a specific embodiment, to achieve accurate detection of conveyor misalignment in the complex environment of underground coal mines, this invention designs a conveyor misalignment detection model for distributed federated learning scenarios. This model can simultaneously output the misalignment status category and continuous offset, and is adaptable to multi-mine non-independent and identically distributed data environments. Please refer to [reference needed] for details. Figure 2 As shown, the construction of the conveyor misalignment detection model in step S2 includes:

[0082] S21. Basic Feature Extraction Design: To improve model training efficiency and fully utilize the general capabilities gained from large-scale data pre-training, this invention employs a pre-trained ResNet50 model as the basic feature extraction network. Specifically, the conveyor belt image to be detected... First, low-level features are extracted through Conv1 convolutional layers and max pooling layers to obtain shallow features containing edge and texture information. Then, it sequentially passes through Conv2, Conv3, Conv4, and Conv5 convolutional layers containing residuals to progressively extract higher-level semantic information, ultimately obtaining deep features with global structural representation capabilities. In this invention, considering the distinct scene specificity of conveyor belt images in underground coal mines, and to balance model training efficiency and feature adaptation capability, a strategy of partial freezing and partial fine-tuning is adopted: during model training, the model parameters in stages Conv1 to Conv3 are frozen to retain the ability to express general visual features, while only the parameters of Conv4 and Conv5 are fine-tuned and trained. This enables the model to adapt features to conveyor belt deviation, improving detection accuracy while reducing computational overhead.

[0083] S22. Design of a Cross-Scale Feature Extraction Module: Conveyor belt misalignment exhibits different characteristics at different spatial scales. Relying solely on single-scale features makes accurate modeling difficult. Therefore, this invention designs a cross-scale conveyor belt structure modeling module to enhance the perception of misalignment patterns at different scales. Specifically, it uses deep features... For the input, construct three parallel convolutional branches:

[0084] (1) A 3×3 convolution kernel is used to extract local detail features, focusing on characterizing the conveyor belt edge and subtle offset information, to obtain local structural features. ;

[0085] (2) A 5×5 convolution kernel is used to extract mesoscale structural features to capture the local bending and regional offset trends of the conveyor belt, thus obtaining mesoscale features. ;

[0086] (3) A 7×7 convolution kernel is used to extract large receptive field features, which are used to model the overall positional shift and global structural changes of the conveyor belt, and to obtain global structural features. ;

[0087] Each of the above branches uses a convolutional layer followed by a ReLU activation function to enhance nonlinear expressive power. Subsequently, the three features are concatenated to obtain cross-scale fused features. .

[0088] S23. Multi-branch attention collaborative fusion module design: To further enhance the robustness of deviation detection, this invention designs a multi-branch attention collaborative fusion module. Specifically, it uses cross-scale feature fusion... Using this as input, three attention branches are constructed, including coordinate attention, edge attention, and channel attention mechanisms, and center-aware features are obtained from each branch. Edge enhancement features and semantic enhancement features Subsequently, the three attention features are uniformly scaled through 1×1 convolution, and then nonlinearly fused using a fully connected mapping layer and the LeakyReLU activation function to finally obtain the attention-enhanced features. .

[0089] S24. Dual Prediction Head Module Design: To separately detect the conveyor belt deviation direction category and continuous offset, this invention constructs a dual-branch prediction structure and incorporates attention-enhanced features. Joint modeling is performed, specifically, attention-enhanced features. Each prediction branch follows one of the following two independent prediction branches:

[0090] (1) Deviation category prediction branch: The conveyor belt status is classified and determined by two fully connected layers and the Softmax classification function, and the deviation category result is output. ;

[0091] (2) Offset Regression Branch: Through two fully connected layers and using the Tanh activation function to constrain the output within the range of [-1,1], continuous estimation of the conveyor belt center offset is achieved, and the offset result is output. ;

[0092] The parameters of the two branches are independent of each other, thus enabling qualitative judgment and quantitative analysis.

[0093] S25. Components of Trainable Parameters: During the training of the conveyor misalignment detection model, the trainable parameters of the model include the following components:

[0094] (1) The parameters of the Conv4 and Conv5 convolutional layers in the pre-trained ResNet50 model are used as the first set of parameters. ;

[0095] (2) All parameters of the cross-scale feature extraction module are used as the second set of parameters. ;

[0096] (3) All parameters of the multi-branch attention collaborative fusion module are used as the third set of parameters. ;

[0097] (4) All parameters of the dual prediction head module are used as the fourth set of parameters. .

[0098] As a specific embodiment, in order to achieve effective training without centrally collecting local datasets from mines, thereby protecting data information from leakage, this invention designs a cross-mine federated learning training strategy. This strategy can simultaneously accelerate training speed and ensure model stability. Specifically, step S3, the design of the cross-mine federated learning training strategy, includes:

[0099] S31. System Initialization: For N mining servers, define them as N Clients, i.e. At the same time, a central server for coordinating training is set up as the Server side, i.e., S.

[0100] S32. Client Grouping: Based on the local server computing resources of N Clients, divide the N Clients into four sets, specifically as follows:

[0101] The local server computing resources for N clients are respectively This metric comprehensively considers factors such as CPU performance, GPU computing power, and memory capacity, with higher values ​​indicating stronger local computing capabilities and faster computation speeds. Then, based on the computing resources of N Clients, the k-means algorithm is used to divide the N Clients into the following four sets. ,in This indicates that the Client's local server has the worst computing resources within this set, and This is the set with the strongest computing resources.

[0102] S33. Training Mode Initialization: Due to significant differences in computational resources among different sets, in order to fully utilize computational potential and improve training speed, this invention designs a hierarchical training method, specifically:

[0103] (1) For the worst computing resources The client within the system only fine-tunes all parameters of the dual prediction head module during training, i.e., the parameters. ;

[0104] (2) For those with slightly better computing resources The client within the system fine-tunes the parameters of the dual-prediction-head module and the multi-branch attention collaborative fusion module during training, i.e., the parameters. and ;

[0105] (3) For those with superior computing resources The client within the system fine-tunes the parameters of the dual prediction head module, the multi-branch attention collaborative fusion module, and the cross-scale feature extraction module during training. , and ;

[0106] (4) For the best computing resources The client within the system fine-tunes all the parameters of the model during training, i.e., the parameters. , , and ;

[0107] This strategy ensures that clients in different sets can complete a local training session as synchronously as possible, thereby minimizing the need for clients with high computing resources to wait for clients with low resources after completing training.

[0108] S34. Cross-Mine Federated Learning Training Strategy: After completing the initialization steps S31 to S33, this invention designs the following... Figure 3 The cross-mine federated learning training strategy process shown includes the following steps:

[0109] (1) Initialize the conveyor misalignment detection model on the central server S, and initialize the four parameters of the model as follows: ;

[0110] (2) Calculate the Top-k parameter mask with reference to the parameter magnitude, and obtain the mask. and apply the mask to each Regarding parameters, the parameters obtained after masking Then, all masked parameters are sent simultaneously to all local clients in the four sets, thereby reducing the number of communication parameters.

[0111] (3) No. Client Parameters after receiving the mask Based on the set to which the Client belongs, determine which parameters are frozen and which are trainable for that Client according to the criteria in step S33; for example, if the Client belongs to ,but Keep frozen, only train and The same applies to other clients;

[0112] (4) No. Client Based on its locally collected complete mine dataset Perform local fine-tuning training to obtain the trained parameters. For the parameters that remain frozen, their values ​​remain unchanged before and after training; subsequently, the... Client The trained parameters Send to the central server;

[0113] (5) All clients have gone through steps (3) and (4), so the central server has collected N sets of trained parameters. Then, the FedAvg algorithm is used to average and aggregate these parameters to obtain the parameters after the first round of training. ;

[0114] (6) Repeat steps (2) to (5) for a total of Finally, the trained conveyor deviation detection model is obtained on the central server, whereby... The maximum number of training rounds is preset.

[0115] As a specific embodiment, in order to achieve real-time monitoring and intelligent operation and maintenance of conveyor misalignment in coal mines, and to improve equipment operation safety and inspection efficiency while ensuring data privacy, this invention further proposes a method for deploying and applying a federated misalignment detection model oriented towards the actual mine environment. Specifically, step S4, the deployment and intelligent operation and maintenance application of the federated misalignment detection model, includes:

[0116] S41. Conveyor Monitoring System and Model Deployment: Industrial-grade image acquisition equipment is deployed in key operating areas of conveyors in multiple underground coal mines to continuously collect images of the conveyor belt's operating status during operation. The key operating areas of the conveyor include the head drive area, the tail tensioning area, and the middle operating area. Simultaneously, the conveyor deviation detection model trained in step S3 is deployed in underground edge computing devices or monitoring servers in the coal mines, enabling the model to run independently in each mine and achieve localized real-time inference and analysis. Each mine node uses only the local model for inference, without relying on external data transmission, thereby ensuring data security.

[0117] S42. Real-time acquisition of conveyor belt operating status images: The operating status of the conveyor belt is continuously acquired through deployed image acquisition equipment, and the acquired images are normalized to obtain standardized conveyor belt images. .

[0118] S43. Real-time detection and output of conveyor belt misalignment: Standardized conveyor belt images... The input is fed into a locally deployed conveyor misalignment detection model. The model performs forward inference on the image and outputs the following two types of results:

[0119] (1) Results of deviation status It is used to determine whether the conveyor belt is currently in a normal state, slightly misaligned, or severely misaligned state.

[0120] (2) Continuous offset results It is used to quantify the degree of deviation of the current center of the conveyor belt from the ideal centerline.

[0121] S44. Intelligent operation and maintenance application driven by deviation status: The deviation detection results output by the model are fed back to the mine conveying system monitoring platform or safety management system in real time, and corresponding operation and maintenance strategies are triggered according to the detection results.

[0122] As a specific embodiment, the corresponding operation and maintenance strategies in step S44 include, but are not limited to:

[0123] (1) When the deviation status category result When the vehicle deviates slightly or severely, it will automatically issue a warning signal to prompt on-site personnel to check or make adjustments.

[0124] (2) Based on the continuous offset results Dynamically assess the degree of belt misalignment to provide a basis for adjusting the conveyor belt tensioning or adjustment device;

[0125] (3) Record and analyze historical test results for conveyor operation status assessment and fault trend prediction, thereby improving the level of intelligent equipment operation and maintenance.

[0126] Compared with existing technologies, the coal mine conveyor deviation detection method based on federated learning provided by this invention has the following advantages: 1) High-precision detection of conveyor deviation: By constructing a joint prediction model of deviation status category and continuous offset, the qualitative identification and quantitative estimation of conveyor belt deviation status are integrated into a single model, improving the accuracy and robustness of detection in complex underground environments. 2) Ensuring data privacy across multiple mines and improving model generalization ability: Using a distributed federated learning training method, data from each mine is stored locally and used for training, eliminating the need for centralized uploading of original image data. Through a hierarchical client training strategy and a selective parameter update mechanism, cross-mine collaborative modeling is achieved while ensuring data security, improving the model's adaptability to different working conditions. 3) Improving model training efficiency and engineering deployment capability: Through hierarchical federated training and a differentiated parameter update mechanism, the training burden and communication overhead of different computing nodes are reduced, enabling efficient collaborative training in heterogeneous environments across multiple mines, and supporting real-time deployment and online application of the model at the edge.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting conveyor belt misalignment in coal mines based on federated learning, characterized in that, Includes the following steps: S1. Construction of coal mine conveyor belt misalignment image dataset: By deploying industrial-grade image acquisition equipment in key operating areas of conveyors in multiple underground coal mines, images of the conveyor belt's operating status during operation are continuously collected. After standardizing the collected images, conveyor belt images are obtained. Professionals then label the conveyor belt misalignment status to construct a joint labeling system that includes misalignment status categories and continuous offsets, thereby forming a misalignment status image dataset covering multiple mines and multiple working conditions. At the same time, distributed data subsets are constructed on a mine-by-mine basis, and each data subset is stored independently on the local machine. S2. Construction of Conveyor Deviation Detection Model: A network structure for the conveyor deviation detection model is constructed, including basic feature extraction, cross-scale structure modeling, multi-branch attention fusion, and dual prediction heads. Specifically, the model first uses a pre-trained ResNet50 model to extract multi-layer semantic features of the conveyor belt image, and achieves feature adaptation through partial parameter freezing and fine-tuning strategies. Then, a cross-scale feature extraction module is introduced to jointly model the local details, mesoscale structure, and global offset of the conveyor belt. On this basis, a multi-branch attention collaborative fusion module is constructed to enhance the feature expression capability from three dimensions: center position perception, edge structure enhancement, and channel semantic expression. Finally, the dual prediction head module outputs the deviation status category and continuous offset respectively. S3. Cross-mine federated learning training strategy design: Construct a federated learning training framework for multi-mine distributed data environments. Each mine node trains the model independently based on local data, and the central server realizes the secure aggregation of model parameters, completing global model optimization without transmitting the original image data. In view of the differences in computing resources among different mines, a hierarchical training strategy based on computing power grouping is introduced. By grouping clients and assigning different parameter training ranges, the overall training efficiency is improved. Meanwhile, a masking mechanism based on parameter magnitude is introduced in the federated communication process to sparsely transmit model parameters, thereby reducing communication overhead and improving training stability. S4. Deployment and Intelligent Operation and Maintenance Application of Federated Misalignment Detection Model: The conveyor misalignment status detection model obtained through federated learning is deployed in the edge computing equipment or monitoring server in the coal mine to realize real-time detection and analysis of the conveyor belt operation status. The equipment continuously collects images of the conveyor belt operation status and inputs them into the model for inference, outputs the misalignment status category and continuous offset results, and feeds back the detection results to the mine conveyor system monitoring platform or safety management system in real time. Based on the detection results, corresponding operation and maintenance strategies are triggered, thereby improving the safety of conveyor operation and the level of intelligent operation and maintenance.

2. The coal mine conveyor misalignment detection method based on federated learning according to claim 1, characterized in that, The step S1, constructing the coal mine conveyor misalignment image dataset, includes: S11. Deployment of the Vision Acquisition System for Conveyors: Industrial-grade image acquisition equipment is deployed in key operating areas of conveyors in multiple underground coal mines to continuously acquire images of the operating status of the conveyor belt during operation. The key operating areas of the conveyor include the head drive area, the tail tensioning area, and the middle operating area. S12. Conveyor Belt Operating Status Image Acquisition and Standardization Processing: The industrial-grade image acquisition equipment deployed in step S11 is used to acquire images of the conveyor belt operating status. The acquired images are then normalized to obtain standardized conveyor belt images. ; S13. Construction of joint labels for conveyor belt misalignment status: For each standardized conveyor belt image Construct a system that includes categories of deviation states. With continuous offset The joint labeling system, the deviation status category This indicates the current offset of the conveyor belt center relative to the ideal centerline, including normal, slight, and severe offset states; the continuous offset amount This is used to quantify the degree of deviation of the current center of the conveyor belt from the ideal center line, and is expressed in a normalized manner with a value range of [-1, 1]. Where 0 indicates that the conveyor belt is at the ideal center position, negative values ​​indicate left deviation, and positive values ​​indicate right deviation, thereby achieving a unified qualitative and quantitative expression of the deviation status. S14. Consistent Labeling and Dataset Construction Across Multiple Mines: Professionals familiar with the operational characteristics of coal mine conveyor systems complete the image labeling work of step S13 locally at each mine, using standardized conveyor belt images. For input, the deviation state category With continuous offset As output, a set of deviation state image datasets is constructed, based on the method in the first... Extensive dataset collection was conducted in each mine to obtain a complete dataset for that mine. The dataset collection process was repeated in different mines, yielding a total of [number missing] datasets. The complete dataset of each mine is thus formed into a multi-mine distributed dataset, i.e. ,and Each mine's data subset is stored independently on its local machine, without sharing the original data, thus providing a distributed data foundation for subsequent model training under the federated learning framework.

3. The coal mine conveyor misalignment detection method based on federated learning according to claim 1, characterized in that, The construction of the conveyor misalignment detection model in step S2 includes: S21. Basic Feature Extraction Design: A pre-trained ResNet50 model is used as the basic feature extraction network. Specifically, the conveyor belt image to be detected... First, low-level features are extracted through Conv1 convolutional layers and max pooling layers to obtain shallow features containing edge and texture information. Then, it sequentially passes through Conv2, Conv3, Conv4, and Conv5 convolutional layers containing residuals to progressively extract higher-level semantic information, ultimately obtaining deep features with global structural representation capabilities. Furthermore, during model training, the model parameters from Conv1 to Conv3 are kept frozen, and only the parameters of Conv4 and Conv5 are fine-tuned during training. S22. Cross-scale feature extraction module design: based on deep features For the input, construct three parallel convolutional branches: (1) A 3×3 convolution kernel is used to extract local detail features, focusing on characterizing the conveyor belt edge and subtle offset information, to obtain local structural features. ; (2) A 5×5 convolution kernel is used to extract mesoscale structural features to capture the local bending and regional offset trends of the conveyor belt, thus obtaining mesoscale features. ; (3) A 7×7 convolution kernel is used to extract large receptive field features, which are used to model the overall positional shift and global structural changes of the conveyor belt, and to obtain global structural features. ; Each of the above branches uses a convolutional layer followed by a ReLU activation function to enhance nonlinear expressive power. Subsequently, the three features are concatenated to obtain cross-scale fused features. ; S23. Multi-branch attention collaborative fusion module design: based on cross-scale fusion features Using this as input, three attention branches are constructed, including coordinate attention, edge attention, and channel attention mechanisms, and center-aware features are obtained from each branch. Edge enhancement features and semantic enhancement features Subsequently, the three attention features are uniformly scaled through 1×1 convolution, and then nonlinearly fused using a fully connected mapping layer and the LeakyReLU activation function to finally obtain the attention-enhanced features. ; S24. Dual Prediction Head Module Design: Constructing a dual-branch prediction structure and applying attention-enhanced features. Joint modeling is performed, specifically, attention-enhanced features. Each prediction branch follows one of the following two independent prediction branches: (1) Deviation category prediction branch: The conveyor belt status is classified and determined by two fully connected layers and the Softmax classification function, and the deviation category result is output. ; (2) Offset Regression Branch: Through two fully connected layers and using the Tanh activation function to constrain the output within the range of [-1,1], continuous estimation of the conveyor belt center offset is achieved, and the offset result is output. ; The parameters of the two branches are independent of each other, thus enabling qualitative judgment and quantitative analysis; S25. Components of Trainable Parameters: During the training of the conveyor misalignment detection model, the trainable parameters of the model include the following components: (1) The parameters of the Conv4 and Conv5 convolutional layers in the pre-trained ResNet50 model are used as the first set of parameters. ; (2) All parameters of the cross-scale feature extraction module are used as the second set of parameters. ; (3) All parameters of the multi-branch attention collaborative fusion module are used as the third set of parameters. ; (4) All parameters of the dual prediction head module are used as the fourth set of parameters. .

4. The coal mine conveyor misalignment detection method based on federated learning according to claim 3, characterized in that, The design of the cross-mine federated learning training strategy in step S3 includes: S31. System Initialization: For N mining servers, define them as N Clients, i.e. Meanwhile, a central server for coordinating training is set up as the Server side, i.e., S; S32. Client Grouping: Based on the local server computing resources of N Clients, divide the N Clients into four sets, specifically as follows: The local server computing resources for N clients are respectively Then, based on the computing resources of the N clients, the kmean algorithm is used to divide the N clients into the following four sets. ,in This indicates that the Client's local server has the worst computing resources within this set, and The set with the strongest computing resources; S33. Training Mode Initialization: This invention designs a hierarchical training method, specifically: (1) For the worst computing resources The client within the system only fine-tunes all parameters of the dual prediction head module during training, i.e., the parameters. ; (2) For those with slightly better computing resources The client within the system fine-tunes the parameters of the dual-prediction-head module and the multi-branch attention collaborative fusion module during training, i.e., the parameters. and ; (3) For those with superior computing resources The client within the system fine-tunes the parameters of the dual prediction head module, the multi-branch attention collaborative fusion module, and the cross-scale feature extraction module during training. , and ; (4) For the best computing resources The client within the system fine-tunes all the parameters of the model during training, i.e., the parameters. , , and ; S34. Cross-mine federated learning training strategy: After completing the initialization steps S31 to S33, the following process steps are used for cross-mine federated learning training: (1) Initialize the conveyor misalignment detection model on the central server S, and initialize the four parameters of the model as follows: ; (2) Calculate the Top-k parameter mask with reference to the parameter magnitude, and obtain the mask. and apply the mask to each Regarding parameters, the parameters obtained after masking Then, all masked parameters are sent simultaneously to all local clients in the four sets, thereby reducing the number of communication parameters. (3) No. Client Parameters after receiving the mask Based on the set to which the Client belongs, determine which parameters are frozen and which are trainable in the Client according to the criteria in step S33; (4) No. Client Based on its locally collected complete mine dataset Perform local fine-tuning training to obtain the trained parameters. For the parameters that remain frozen, their values ​​remain unchanged before and after training; subsequently, the... Client The trained parameters Send to the central server; (5) All clients have gone through steps (3) and (4), so the central server has collected N sets of trained parameters. Then, the FedAvg algorithm is used to average and aggregate these parameters to obtain the parameters after the first round of training. ; (6) Repeat steps (2) to (5) for a total of Finally, the trained conveyor deviation detection model is obtained on the central server, whereby... The maximum number of training rounds is preset.

5. The coal mine conveyor misalignment detection method based on federated learning according to claim 1, characterized in that, Step S4, the deployment of the federated deviation detection model and the application of intelligent operation and maintenance, includes: S41. Conveyor Monitoring System and Model Deployment: Industrial-grade image acquisition equipment is deployed in key operating areas of conveyors in multiple underground coal mines to continuously collect images of the conveyor belt's operating status during operation. The key operating areas of the conveyor include the head drive area, the tail tensioning area, and the middle operating area. Simultaneously, the conveyor deviation detection model trained in step S3 is deployed in underground coal mine edge computing devices or monitoring servers, enabling the model to run independently in each mine and achieve localized real-time inference and analysis. S42. Real-time acquisition of conveyor belt operating status images: The operating status of the conveyor belt is continuously acquired through deployed image acquisition equipment, and the acquired images are normalized to obtain standardized conveyor belt images. ; S43. Real-time detection and output of conveyor belt misalignment: Standardized conveyor belt images... The input is fed into a locally deployed conveyor misalignment detection model. The model performs forward inference on the image and outputs the following two types of results: (1) Results of deviation status It is used to determine whether the conveyor belt is currently in a normal state, slightly misaligned, or severely misaligned state. (2) Continuous offset results This is used to quantify the degree of deviation of the current center of the conveyor belt from the ideal centerline; S44. Intelligent operation and maintenance application driven by deviation status: The deviation detection results output by the model are fed back to the mine conveying system monitoring platform or safety management system in real time, and corresponding operation and maintenance strategies are triggered according to the detection results.

6. The coal mine conveyor misalignment detection method based on federated learning according to claim 5, characterized in that, The corresponding operation and maintenance strategies in step S44 include, but are not limited to: (1) When the deviation status category result When the vehicle deviates slightly or severely, it will automatically issue a warning signal to prompt on-site personnel to check or make adjustments. (2) Based on the continuous offset results Dynamically assess the degree of belt misalignment to provide a basis for adjusting the conveyor belt tensioning or adjustment device; (3) Record and analyze historical test results for conveyor operation status assessment and fault trend prediction, thereby improving the level of intelligent equipment operation and maintenance.

Images