Belt damage detection method and detection system for belt conveyor based on improved centernet

Abstract

A belt damage detection method and detection system for a belt conveyor based on an improved CenterNet. The detection method comprises device deployment, data collection, edge denoising, local distribution and damage detection. The detection system comprises an image collection module, a data transmission module, a data processing module, a control module, an edge-end module and a cloud data processing module. On the basis of a deep learning model, tear damage, wear damage and rupture damage of a belt can be effectively detected by using a KDFA-CenterNet object detection loss algorithm; a real-time image denoising and enhancement model based on an invertible neural network can perform denoising and enhancement processing on collected images; and a cloud-edge-end collaborative load computing architecture based on a long short-term memory neural network is provided, and a cloud server, an edge computing node and a local server operate collaboratively, such that massive high-density data of belt images of a belt conveyor can be processed in real time.

Description

Belt conveyor belt damage detection method and detection system based on improved CenterNet Technical Field

[0001] The present invention relates to a belt damage detection method, in particular to a belt conveyor belt damage detection method and detection system based on an improved CenterNet, belonging to the technical field of coal mine transportation monitoring. Background Art

[0002] Belt conveyors are friction-driven, efficient, continuous cargo transport devices that combine traction and load-bearing mechanisms. They are crucial transportation equipment in coal mine production. However, conveyor belts are prone to tearing and breaking during transportation, causing accidents and seriously threatening production safety and economic benefits. The key to ensuring stable conveyor belt operation lies in real-time monitoring of the belt's operating status.

[0003] At present, the main methods for detecting belt damage in belt conveyors include manual visual inspection, detection based on tension, displacement, speed and other sensors, magnetic field changes, X-rays, hyperspectral imaging, and machine vision. For example, a conveying method and device for preventing conveyor belt damage disclosed in publication number CN116495428A uses an external monitoring component to inspect the belt.

[0004] Publication No. CN117945101A discloses a belt tear detection system and method based on dual-energy X-rays. The system transmits dual-energy X-rays to the belt to be detected while the belt is in operation, and obtains the high-energy transmission signal value and low-energy transmission signal value of the dual-energy X-rays in real time to determine whether the belt to be detected is torn.

[0005] Publication number CN116912210A discloses a method and system for detecting coal, foreign matter, and belt damage images under low-light dust and fog. An industrial camera is used to collect multi-scale target images of coal, foreign matter, and early belt damage on a coal mine conveyor belt under low-light dust and fog conditions. The model detection layer structure and feature extraction module are designed, and an efficient channel attention mechanism is introduced to optimize the target detection model.

[0006] The problems existing in the prior art are:

[0007] 1) Belt damage detection based on manual visual inspections is usually performed by experienced workers who inspect key locations such as the head, tail, and conveyor belt along the conveyor belt's transport direction. This method has significant drawbacks: it is difficult to detect belt damage such as punctures, tears, and excessive wear in a timely manner, which can easily lead to the premature scrapping of a large number of conveyor belts.

[0008] 2) The belt conveyor belt damage detection method based on various sensors is affected by the sensors and the complex underground environment, and the detection accuracy and stability are average;

[0009] 3) Belt damage detection methods based on hyperspectral cameras and X-rays have high equipment costs and are difficult to deploy, making them difficult to promote and apply on a large scale. Furthermore, X-rays are harmful to the human body, requiring the design of additional shielding equipment and resulting in a complex system structure.

[0010] 4) The mainstream target detection algorithm based on deep learning establishes feature vectors by extracting various spatial features or statistical features of the image, thereby detecting and identifying whether there is belt damage in the belt image. However, there are also problems such as the complex structure of the backbone feature extraction network and the large number of parameters, which is not conducive to deployment on embedded computing devices with limited computing power based on the improved CenterNet.

[0011] Summary of the Invention

[0012] The purpose of the present invention is to provide a belt conveyor belt damage detection method and detection system based on improved CenterNet in order to solve at least one of the above technical problems, which has the advantages of strong feature extraction capability, fast detection and reasoning speed, high accuracy and strong applicability.

[0013] The present invention achieves the above-mentioned object through the following technical solution: a belt conveyor belt damage detection method based on improved CenterNet, the detection method comprising the following steps:

[0014] S1. Equipment Deployment: Deploy the image acquisition module, namely the mining-grade lightweight explosion-proof camera and mining-grade explosion-proof light source, between the upper and lower belts of the belt conveyor. Set operating parameters. Deploy the industrial router in the control cabinet next to the belt conveyor.

[0015] S2. Data collection: Select the conveyor belt working scene and use the mining lightweight explosion-proof camera to collect belt images in real time;

[0016] S3, edge denoising: Input the image into the reversible neural network model to separate low-frequency structural features, high-frequency texture detail features and high-frequency noise. A trainable guided filter is used to deeply fuse the high-frequency features with the low-resolution structural features to effectively denoise and enhance the image.

[0017] S4. Load Distribution: The control center uses a long short-term memory network model to evaluate and predict the resource consumption of image processing tasks and distributes the tasks to cloud servers, edge computing nodes, and local servers. The three work together to process large amounts of high-density data from conveyor belt images in real time.

[0018] S5. Damage detection: The local server uses the KDFA-CenterNet target detection algorithm to detect each damaged target or its significant visual features in the real-time belt image.

[0019] As a further solution of the present invention: In S1, the image acquisition module deployment and parameter settings are as follows:

[0020] The image acquisition module is deployed between the upper and lower belts of the belt conveyor, and its working parameters are set to the optimal exposure of the camera and the brightness of the nearest mining explosion-proof light source.

[0021] As a further solution of the present invention: in S5, the damage detection result covers the following situations:

[0022] (1) The belt is in good condition;

[0023] (2) Belt tear damage;

[0024] (3) Belt wear and damage;

[0025] (4) Belt rupture and damage.

[0026] As a further solution of the present invention: In S5, after the original image is input into the KDFA-CenterNet target detection algorithm, the image is processed by a downsampling module, and the downsampling parameter R is set to 4. After downsampling, the image data enters the backbone network to extract features, and is processed by knowledge distillation. The channel-spatial attention transfer mechanism is integrated. The channel attention obtains the segmentation coefficient of the hidden layer feature map. The spatial attention weight represents the distribution of the feature along the channel. The spatial attention is applied to compress the feature map. The outputs of the two pooling layers of average pooling and maximum pooling are transmitted to the 2D convolution layer to achieve information fusion. The three sub-networks of thermal feature map, offset feature map and size feature map are processed separately. The final model outputs the target prediction result;

[0027] The channel-spatial fusion attention FCSA description formula is:

[0028] Among them, ω i ∈W A , the spatial-channel fusion attention contains high-dimensional information features of both spatial and channel parts; Spa i Represents operations on spatial channels.

[0029] As a further solution of the present invention: In S5, the total loss function of the KDFA-CenterNet target detection algorithm is: L total =L det +λ1L feat +λ2L resp L det =Lhm +μ1L hw +μ2L off

[0030] Among them, λ1, λ2, μ1 and μ2 are hyperparameters to adjust the proportion of each item; L feat is the knowledge distillation of the hidden layer feature map; L resp Knowledge distillation for response results; L hm is the heat map loss; L hw is the width and height regression loss; L off is the offset loss.

[0031] A belt conveyor belt damage detection system based on an improved CenterNet system includes an image acquisition module, a data transmission module, a data processing module, a control module, an edge module, and a cloud data processing module. The image acquisition module uses a lightweight explosion-proof mining camera deployed between the upper and lower belts of the belt conveyor to collect belt images in real time.

[0032] Among them, the data transmission module interacts with the image acquisition module, data processing module, and control module; the data processing module implements task scheduling for the collected belt images, and assigns the loading of the belt damage model for real-time detection tasks; the control module processes the control information in a timely manner; the edge computing node of the edge module performs inference operations by deploying a trained deep learning model; the cloud data processing module is a cloud server, which can assign and process some tasks when the vision controller cannot cope with high-intensity computing tasks.

[0033] As a further solution of the present invention: the data transmission module is an industrial gateway and an industrial router with data interaction function, which realizes data input and output of the image acquisition module, the data processing module, and the control module.

[0034] As a further solution of the present invention: the data processing module is a visual controller and a central server with task scheduling and damage detection functions. By loading the belt damage model, the denoised image of the edge computing node is subjected to belt damage detection, and the detection results are recorded in the central server, or the execution command is sent to the control module through the data transmission module.

[0035] As a further solution of the present invention: the edge module is an edge computing node with reasoning operation function, which can deploy trained deep learning models for calculation processing.

[0036] As a further solution of the present invention: the cloud data processing module is a cloud server capable of processing assigned tasks.

[0037] The beneficial effects of the present invention are:

[0038] 1) The belt damage detection method of the present invention is based on a deep learning model and adopts the KDFA-CenterNet target detection loss algorithm. It can effectively detect belt tear damage, wear damage, and rupture damage, and has strong detection feasibility, high accuracy, and good stability.

[0039] 2) The present invention adopts a real-time image denoising and enhancement model based on a reversible neural network, which can perform denoising and enhancement processing on the collected images, effectively improving the damage detection performance of the belt damage detection system;

[0040] 3) The present invention sets up a cloud-edge-end collaborative load computing architecture based on long short-term memory neural networks. The cloud server, edge computing nodes and local servers work together to process large amounts of high-density data of belt conveyor belt images in real time;

[0041] 4) The present invention takes into account the coordination of software and hardware. After completing the detection of belt damage of the belt conveyor, the alarm information can be executed by the equipment, thereby issuing an execution instruction of alarm or shutdown. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG1 is a schematic flow chart of the detection method of the present invention;

[0043] FIG2 is a schematic diagram of the belt damage detection model of KFDA-CenterNet of the present invention;

[0044] FIG3 is a schematic structural diagram of a detection system of the present invention;

[0045] FIG4 is a schematic diagram of the central server structure of the detection system of the present invention.

[0046] In the figure, 1. central server, 2. mining belt conveyor, 3. first image acquisition module, 4. second image acquisition module, 5. third image acquisition module. DETAILED DESCRIPTION

[0047] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] In the first embodiment, as shown in FIG1 and FIG2 , a belt conveyor belt damage detection method based on an improved CenterNet is provided. The detection method comprises the following steps:

[0049] S1. Equipment deployment:

[0050] The image acquisition module, consisting of a lightweight explosion-proof camera and a flameproof light source for mining, was deployed between the upper and lower belts of the belt conveyor. It was horizontally fixed to the aluminum alloy profile using an adapter plate. Operating parameters were set, and image acquisition nodes were configured to capture real-time images of the inner and outer sides of the belt conveyor. The industrial router was deployed in a control cabinet next to the belt conveyor.

[0051] S2. Data collection:

[0052] Select the conveyor belt working scene, and use the mining lightweight explosion-proof camera to collect belt images in real time;

[0053] S3, edge denoising:

[0054] The input image is transmitted from the industrial router of the data transmission module to the edge computing node, where a reversible neural network model is loaded to separate low-frequency structural features, high-frequency texture detail features, and high-frequency noise. A trainable guided filter is then used to deeply fuse the high-frequency features with the low-resolution structural features, effectively denoising and enhancing the image.

[0055] S4. Load distribution:

[0056] The control center uses a long-short-term memory network model to evaluate and predict the resource consumption of image processing tasks, and then distributes the tasks to cloud servers, edge computing nodes, and local servers. The three work together to process large amounts of high-density data from conveyor belt images in real time.

[0057] S5. Damage detection:

[0058] The local server uses the KDFA-CenterNet target detection loss algorithm to detect each damaged target or its significant visual features in the real-time belt image;

[0059] S6. Alarm processing:

[0060] After the system identifies the damage, the control end sends an alarm message, the alarm cause is displayed in the upper computer software visualization panel, and the belt conveyor alarms or shuts down.

[0061] Damage detection results cover the following situations:

[0062] (1) The belt is in good condition;

[0063] (2) Belt tear damage;

[0064] (3) Belt wear and damage;

[0065] (4) Belt rupture and damage.

[0066] After the original image is input into the KFDA-CenterNet belt damage detection model of the present invention, the image is processed by a downsampling module, and the downsampling parameter R is set to 4. After downsampling, the image data enters the backbone network to extract features, and knowledge distillation is applied. The channel-space attention transfer mechanism is integrated to compress the model and extract key information. The three sub-networks of thermal feature map, offset feature map and size feature map are processed separately. Finally, the model outputs the target prediction result. By comparing the target true value, the loss function is improved to better optimize the model.

[0067] After the original image is input into the KDFA-CenterNet target detection algorithm, the image is processed by the downsampling module, and the downsampling parameter R is set to 4. After downsampling, the image data enters the backbone network to extract features, and knowledge distillation is applied to integrate the channel-spatial attention transfer mechanism. The channel attention obtains the segmentation coefficient of the hidden layer feature map, and the spatial attention weight represents the distribution of features along the channel. Spatial attention is applied to compress the feature map. The outputs of the two pooling layers of average pooling and maximum pooling are sent to the 2D convolution layer to achieve information fusion. The three sub-networks of thermal feature map, offset feature map and size feature map are processed separately, and the final model outputs the target prediction result.

[0068] Channel-spatial fusion attention FCSA can be described as:

[0069] Among them, ω i ∈W A , the spatial-channel fusion attention contains high-dimensional information features of both spatial and channel parts; Spa i It represents the operation of spatial channels. Channel attention obtains the segmentation coefficient of the hidden layer feature map. The spatial attention weight represents the distribution of features along the channel. Spatial attention is applied to compress the feature map. The outputs of the two pooling layers, average pooling and maximum pooling, are sent to the 2D convolution layer to achieve information fusion.

[0070] The total loss function of the KDFA-CenterNet target detection algorithm is: L total =L det +λ1L feat +λ2L resp L det =L hm +μ1L hw +μ2L off

[0071] Among them, λ1, λ2, μ1 and μ2 are hyperparameters to adjust the proportion of each item; L feat is the knowledge distillation of the hidden layer feature map; L resp Knowledge distillation for response results; L hmis the heat map loss; L hw is the width and height regression loss; L off is the offset loss.

[0072] Embodiment 2, as shown in Figures 3 and 4, a belt conveyor belt damage detection system based on the improved CenterNet includes a central server 1, a mining belt conveyor 2, a first image acquisition module 3, a second image acquisition module 4, and a third image acquisition module 5. The central server 1 includes a data transmission module, a data processing module, a control module, an edge module, and a cloud data processing module. The first image acquisition module 3 includes a mining light-proof camera and a mining explosion-proof light source. The first image acquisition module 3 is deployed between the upper and lower belts of the belt conveyor 2. The second image acquisition module 4 includes a mining light-proof camera and a lens to capture images of the belt conveyor belt in real time. The strip-shaped mining explosion-proof light source is combined with a mining explosion-proof light source controller and is arranged on the side of the mining light-proof camera to provide fill light for the mining light-proof camera to capture real-time images of the belt, and to capture real-time images of the inner surface of the belt conveyor belt, mainly detecting two types of penetrating damage: tearing and rupture of the belt. The third image acquisition module 5 is installed horizontally to capture real-time images of the outer surface of the belt for detecting the wear of the belt bearing surface.

[0073] The data transmission module of the central server 1 is an industrial gateway and an industrial router, which realizes data interaction between the image acquisition module, the data processing module, and the control module. The data processing module of the central server 1 can perform real-time detection of the collected belt images. It includes a visual controller and a central server. The visual controller can load the belt damage model and use the KDFA-CenterNet target detection loss algorithm to detect belt damage on the denoised images of the edge computing nodes. The detection results are recorded in the central server or an execution command is sent to the control module through the data transmission module. The central server 1 runs a load balancing algorithm to detect possible belt damage in the image and dispatch sudden loads. When the visual controller cannot cope with high-intensity computing tasks, task scheduling and allocation can be performed.

[0074] The control module is a PLC controller that can process control information in a timely manner; the edge module of the central server 1 is an edge computing node, which has a high-performance GPU computing chip embedded in it, loads an image denoising model, and performs real-time denoising on the collected belt conveyor belt images to improve the performance of the subsequent KDFA-CenterNet belt damage detection model; the cloud data processing module of the central server 1 is a cloud server, which uses virtualization technology to realize physical resource segmentation and isolation, making each computing node an independent computing instance. When the visual controller cannot cope with high-intensity computing tasks, it can allocate and process some tasks.

[0075] Through the collaborative work of hardware and software, a cloud-edge-end collaborative load computing architecture is set up, which can process large amounts of high-density belt image data in real time, and can perform real-time detection of belt tear damage, wear damage, and rupture damage. The detection feasibility is strong, the accuracy is high, and the stability is good.

[0076] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0077] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A belt conveyor belt damage detection method based on improved CenterNet, characterized in that: The detection method comprises the following steps: S1. Equipment Deployment: Deploy the image acquisition module, namely the mining-grade lightweight explosion-proof camera and mining-grade explosion-proof light source, between the upper and lower belts of the belt conveyor. Set operating parameters. Deploy the industrial router in the control cabinet next to the belt conveyor. S2. Data collection: Select the conveyor belt working scene and use the mining lightweight explosion-proof camera to collect belt images in real time; S3, edge denoising: Input the image into the reversible neural network model to separate low-frequency structural features, high-frequency texture detail features and high-frequency noise. A trainable guided filter is used to deeply fuse the high-frequency features with the low-resolution structural features to effectively denoise and enhance the image. S4. Load Distribution: The control center uses a long short-term memory network model to evaluate and predict the resource consumption of image processing tasks and distributes the tasks to cloud servers, edge computing nodes, and local servers. The three work together to process large amounts of high-density data from conveyor belt images in real time. S5. Damage detection: The local server uses the KDFA-CenterNet target detection algorithm to detect each damaged target or its significant visual features in the real-time belt image.

2. The belt conveyor belt damage detection method based on the improved CenterNet according to claim 1 is characterized in that: In S1, the image acquisition module deployment and parameter settings are as follows: The image acquisition module is deployed between the upper and lower belts of the belt conveyor, and its working parameters are set to the optimal exposure of the camera and the brightness of the nearest mining explosion-proof light source.

3. The belt conveyor belt damage detection method based on the improved CenterNet according to claim 1 is characterized in that: In S5, the damage detection results cover the following situations: (1) The belt is in good condition; (2) Belt tear damage; (3) Belt wear and damage; (4) Belt rupture and damage.

4. The belt conveyor belt damage detection method based on the improved CenterNet according to claim 1 is characterized in that: In S5, after the KDFA-CenterNet target detection algorithm inputs the original image, the image is processed by the downsampling module, and the downsampling parameter R is set to 4. After downsampling, the image data enters the backbone network to extract features, and is processed by knowledge distillation. The channel-spatial attention transfer mechanism is integrated. The channel attention obtains the segmentation coefficient of the hidden layer feature map. The spatial attention weight represents the distribution of the feature along the channel. The spatial attention is applied to compress the feature map. The outputs of the two pooling layers of average pooling and maximum pooling are sent to the 2D convolution layer to achieve information fusion. The three sub-networks of thermal feature map, offset feature map and size feature map are processed separately. The final model outputs the target prediction result; The channel-spatial fusion attention FCSA description formula is: Among them, ω i ∈W A , the spatial-channel fusion attention contains high-dimensional information features of both spatial and channel parts; Spa i Represents operations on spatial channels.

5. The belt conveyor belt damage detection method based on the improved CenterNet according to claim 1 is characterized in that: In S5, the total loss function of the KDFA-CenterNet target detection algorithm is: L total =L det +λ1L feat +λ2L resp L det =L hm +μ1L hw +μ2L off Among them, λ1, λ2, μ1 and μ2 are hyperparameters to adjust the proportion of each item; L feat is the knowledge distillation of the hidden layer feature map; L resp Knowledge distillation for response results; L hm is the heat map loss; L hw is the width and height regression loss; L off is the offset loss.

6. A belt conveyor belt damage detection system based on improved CenterNet, characterized by: It includes an image acquisition module, a data transmission module, a data processing module, a control module, an edge module and a cloud data processing module. The image acquisition module uses a mining-used lightweight explosion-proof camera deployed between the upper and lower belts of the belt conveyor to collect belt images in real time. Among them, the data transmission module is used to transmit the image acquisition module, data processing module and control module. Data interaction; the data processing module implements task scheduling for the collected belt images, and assigns the loaded belt damage model to perform real-time detection tasks; the control module processes the control information in a timely manner; the edge computing node of the edge module performs inference operations by deploying a trained deep learning model; the cloud data processing module is a cloud server, which can assign and process some tasks when the visual controller cannot cope with high-intensity computing tasks.

7. The belt conveyor belt damage detection system based on the improved CenterNet according to claim 6 is characterized in that: The data transmission module is an industrial gateway and industrial router with data interaction function, which realizes data input and output of the image acquisition module, data processing module and control module.

8. The belt conveyor belt damage detection system based on the improved CenterNet according to claim 6 is characterized in that: The data processing module is a visual controller and central server with task scheduling and damage detection functions. By loading the belt damage model, it performs belt damage detection on the denoised image of the edge computing node, and records the detection results on the central server, or sends an execution command to the control module through the data transmission module.

9. The belt conveyor belt damage detection system based on the improved CenterNet according to claim 6 is characterized in that: The edge module is an edge computing node with inference computing capabilities, which can deploy trained deep learning models for calculation processing.

10. The belt conveyor belt damage detection system based on the improved CenterNet according to claim 6, characterized in that: The cloud data processing module is a cloud server that processes assigned tasks.

Images