Mine hoist steel wire rope damage detection method and system based on machine vision

Abstract

The present application relates to the technical field of mechanical component damage detection, and particularly relates to a mine hoist steel wire rope damage detection method and system based on machine vision, which comprises four steps of data set image acquisition, image preprocessing, design algorithm training model and system real-time detection; the present application proposes a motion blur restoration algorithm based on edge guidance to overcome the motion blur that occurs when the steel wire rope runs at high speed; in view of the problem that the small fracture wire feature information is weak and the detection is difficult, a steel wire rope wire breakage detection algorithm is proposed, which performs better than the mainstream target detection model and the existing steel wire rope wire breakage detection algorithm on the self-built data set; in combination with the actual application requirements of coal mines, the present application designs a steel wire rope wire breakage real-time detection system, which integrates the wire breakage detection algorithm into the online detection software and is deployed on the detection terminal, realizes high reliability real-time detection, and overcomes the problem of low detection efficiency.

Description

Technical Field

[0001] This invention relates to the field of mechanical component damage detection technology, and in particular to a method and system for detecting damage to steel wire ropes in mining hoists based on machine vision. Background Technology

[0002] With the continuous advancement of technology, numerous scholars both domestically and internationally have conducted extensive and in-depth research in the field of wire rope breakage detection, achieving many significant breakthroughs and accomplishments. Existing research mainly includes electromagnetic detection methods, acoustic emission detection methods, optical detection methods, manual detection methods, and machine vision detection methods.

[0003] Physical methods such as electromagnetic detection, acoustic emission detection, and optical detection still have significant limitations in mining applications. First, the harsh working environment in mines, with its large amounts of dust, water vapor, and corrosive gases, affects the performance and accuracy of detection equipment. Second, mining wire ropes are typically made of multiple strands of twisted wire, resulting in a complex structure that is subject to various forces such as tension, bending, and torsion during use, leading to relative displacement and deformation between the wires and affecting detection accuracy. Finally, the presence of numerous electrical devices and electromagnetic fields in mines generates electromagnetic interference that severely disrupts the normal operation of physical detection equipment.

[0004] With the development of artificial intelligence and machine vision, detecting the condition of steel ropes through visual image recognition shows promising prospects. Traditional machine vision mainly relies on manually designed feature extractors and algorithms to identify and understand objects and content in images through image processing and analysis. These algorithms typically require image preprocessing to extract features such as edges, textures, and colors, and then identification is performed through classifiers or matching algorithms. Although they have achieved good results in the field of steel rope broken wire detection, traditional machine vision has poor generalization ability for complex scenes and diverse objects. Furthermore, if the data quality is low or insufficient, it may lead to inaccurate model training, affecting the model's generalization ability.

[0005] With the rise of deep learning, the field of machine vision has witnessed a revolutionary breakthrough. Deep learning automatically learns and extracts high-level features from images by constructing deep neural network models, achieving a higher level of understanding of image content. Compared with traditional computer vision, deep learning has stronger expressive and generalization capabilities, and can handle more complex and fine-grained tasks. It not only improves the recognition accuracy and efficiency of traditional machine vision algorithms, but also expands the application scope of machine vision.

[0006] Because of the high precision, high generalization, and better adaptability to complex and ever-changing mining scenarios of the deep learning-based wire rope breakage detection method, this invention introduces deep learning and image processing technology into the practical application of wire rope breakage detection in mining, so as to improve the reliability of real-time wire rope detection. Summary of the Invention

[0007] The purpose of this invention is to provide a machine vision-based method and system for detecting damage to wire ropes in mining hoists. This system utilizes deep learning to detect broken wires in the wire rope in real time. The broken wire detection algorithm is integrated into online detection software and deployed on the detection terminal, achieving highly reliable real-time detection.

[0008] To achieve the above-mentioned objectives, the present invention employs the following technical solution: a machine vision-based method for detecting damage to wire ropes in mine hoists, comprising the following steps:

[0009] S1. Dataset Image Acquisition: Install industrial cameras in the hoisting section of the vertical shaft hoisting system in a coal mine to collect images of the wire rope during operation;

[0010] S2. Image preprocessing, as detailed below:

[0011] S21. Wire rope motion blur restoration: Based on Wiener filtering, edge weights are added to the filter to weaken the excessive smoothing of edges by the Wiener filter, thereby obtaining the image of the wire rope after motion blur removal.

[0012] S22. Image annotation: The collected images of broken wires in the wire rope are annotated with three types of broken wires: broken wires at the top of the strand, broken wires at the bottom of the strand, and broken wires that jump, in order to establish a wire rope damage dataset.

[0013] S3. Design the algorithm training model: Introduce the RFC3 lightweight feature extraction module, the skip connection of the feature fusion network, the CBAM-R attention mechanism, and the improved Focal EIoU loss function into the YOLOv5s backbone network, name it CTR-YOLO, set its structure and parameters, and train it using the dataset constructed in S22 to obtain a model that meets the accuracy and speed requirements of actual applications.

[0014] S4. Establish a real-time detection system: A distributed deployment scheme is adopted. The inspection device is installed at the hoist to obtain images of the wire rope. The wire rope motion blur restoration algorithm is deployed on the image processing workstation to receive the images from the inspection device and perform motion blur restoration. The algorithm model obtained in S3 is deployed on the motion blur restored images from the detection interruption receiving workstation to perform wire breakage detection and obtain the detection results.

[0015] By adopting the above technical solutions, this invention proposes an edge-guided motion fuzzing restoration algorithm to overcome motion fuzziness that occurs when steel wire ropes are running at high speeds. Addressing the problem of weak feature information and high detection difficulty for small-fracture wires, a steel wire rope broken wire detection algorithm (CTR-YOLO) is proposed, which outperforms mainstream target detection models and existing steel wire rope broken wire detection algorithms on a self-built dataset. Combining the actual application needs of coal mines, this invention designs a real-time steel wire rope broken wire detection system. This system integrates the broken wire detection algorithm into online detection software and deploys it on the detection terminal, achieving highly reliable real-time detection and overcoming the problems of low efficiency in manual detection methods.

[0016] Furthermore, the weight calculation method in S21 is as follows:

[0017] S211. Image gradient calculation: The Sobel operator is used for calculation. The Sobel operator calculates the horizontal gradient Gx and vertical gradient Gy through a convolution kernel. Based on the Euclidean norm, the gradient magnitude of the pixel can be calculated.

[0018] S212, Image gradient normalization: According to the calculation formula Obtain the normalized gradient result NG at the current position, where G is the gradient magnitude at the current position, and Gmax and Gmin are the maximum and minimum gradient values ​​in the current image, respectively.

[0019] S213. Weight Calculation: The normalization result reflects the rate of change in different regions of the image. Weight balancing can be achieved using NG (Not Given) parameters. In edge regions, NG is larger, and the corresponding weight is smaller, thus reducing the filter's effect and preserving edge details. In smooth regions, NG is smaller, and the corresponding weight is larger, making the filter more effective in smooth regions to reduce noise. The weight is set to Weight = 1 - NG, and this weight is applied to the original Wiener filter. The filter conversion formula of the present invention is obtained above:

[0020] Where W(u,v) is the introduced weight, H(u,v) is the motion degradation function, F(u,v) and G(u,v) represent the Fourier transforms of the original image f(x,y) and the blurred image g(x,y) respectively, and K is the adjustment factor of the filter.

[0021] By adopting the above technical solution, it is easier to calculate the weights.

[0022] Furthermore, in S2, after motion blur recovery of the acquired image, in order to improve the generalization ability of the model, data augmentation is performed on the acquired image by means of rotation, brightness adjustment, etc., to obtain the type of broken wire target through the image, and after software annotation, the broken wire targets are normalized to the same size as the dataset of the network model.

[0023] By adopting the above technical solution, it is easy to normalize different types of broken wires, and by establishing a dataset, it is easy to detect steel wire ropes.

[0024] Furthermore, in S3, the YOLOv5s model is improved to build the CTR-YOLO algorithm model. Specifically, the C3 module in the YOLOv5s feature extraction network is replaced with the RFC3 lightweight feature extraction module. The RFC3 module replaces the two 3×3 convolutions in the bottleneck of the C3 module with 3×3 partial convolutions and two 1×1 convolutions, which significantly reduces the number of model parameters and improves detection speed. Based on BiFPN, the feature fusion network of the original model is reconstructed by establishing skip connections between different layers, so that the model can make full use of the low-level feature information of the steel wire rope. The CBAM-R attention mechanism is added between the 4th and 16th layers and the 6th and 12th layers of YOLOv5s. The low-level feature information extracted by the feature extraction network is further enhanced by the improved CBAM-R attention mechanism and fed into the feature fusion network.

[0025] By adopting the above technical solution, it is easier to obtain a model that meets the accuracy and speed requirements of actual applications.

[0026] Furthermore, the specific improvements made by the CBAM-R attention mechanism based on the CBAM attention mechanism are as follows:

[0027] Random pooling: Random pooling is a special pooling operation that introduces randomness into the pooling process to address the potential risk of overfitting in the model. It can be seen as a form of implicit regularization, reducing the model's dependence on specific patterns in the input data.

[0028] Dilated Convolution: To avoid the loss of feature information caused by an excessively large receptive field and to reduce the computational cost of this module, the 7×7 convolution in the channel attention mechanism is replaced with a 3×3 ordinary convolution and a 3×3 dilated convolution with an expansion factor of X. Dilated convolution is a special type of convolution that increases the size of the receptive field without changing the computational cost by inserting holes in the convolution kernel.

[0029] Residual connection: In order to solve the problem of insufficient effective information acquisition of spatial attention mechanism caused by attention concatenation, residual connection can combine shallow features and deep features generated by CAM to generate new feature maps and send them into the spatial attention mechanism module, thereby improving SAM's utilization of global information.

[0030] Based on the above methods, the improved calculation formula for CBAM-R is determined as follows: Where Mc(F) represents the improved channel attention mechanism, Ms(F) represents the improved spatial attention mechanism, F represents the input feature map, and F' represents the output feature map. This indicates element-wise multiplication.

[0031] By adopting the above technical solutions, the detection accuracy of steel wire ropes can be improved.

[0032] Furthermore, in step S3, the original model's bounding box regression loss function CIoU is replaced with Focal EIoU. During backpropagation, the model can calculate a more accurate loss value. The formula for calculating Focal EIoU is L. Focal-EIoU =IOU β L EIoU Where β is a parameter controlling the degree of outlier suppression, and IOU is the intersection-union ratio, which calculates the overlap rate between the predicted bounding box and the ground truth bounding box. EIoU The EIoU loss function is calculated using the following formula:

[0033]

[0034] Where b is the center point of the predicted bounding box, b gt Let ρ be the center point of the ground truth bounding box, ρ be the formula for calculating its distance, c be the diagonal length of the minimum diplomatic rectangle of the ground truth bounding box and the predicted bounding box, and Cw and Ch be the width and height of the minimum bounding rectangle of the two bounding boxes, respectively.

[0035] By adopting the above technical solution, the loss function can be calculated, so as to obtain a model with the accuracy and speed required for actual application.

[0036] To better achieve the above-mentioned objectives, the present invention also provides a machine vision-based wire rope damage detection system for mine hoists, comprising:

[0037] The mobile inspection device includes an industrial camera, an artificial light source, a precision track, and a motion controller, used to capture images of steel wire ropes during transportation;

[0038] An image processing workstation is deployed to perform motion blur restoration on real-time acquired wire rope images using the edge-guided wire rope motion blur restoration algorithm proposed in this invention.

[0039] The engineer's workstation is equipped with the CTR-YOLO wire rope real-time detection algorithm. The images collected by the inspection device are restored by the image processing workstation and then transmitted at high speed to the detection terminal via industrial Ethernet for wire breakage detection.

[0040] By adopting the above technical solution, steel wire ropes can be tested.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] 1. This invention proposes an edge-guided motion fuzz restoration algorithm to overcome motion fuzziness that occurs when wire ropes are running at high speeds. Addressing the issues of weak feature information and high detection difficulty for small broken wires, a wire rope broken wire detection algorithm (CTR-YOLO) is proposed, which outperforms mainstream target detection models and existing wire rope broken wire detection algorithms on a self-built dataset. Furthermore, considering the practical application needs of coal mines, this invention designs a real-time wire rope broken wire detection system. This system integrates the broken wire detection algorithm into online detection software and deploys it on the detection terminal, achieving highly reliable real-time detection and overcoming the low efficiency of manual detection methods.

[0043] 2. The model of this invention effectively improves the detection accuracy of small-fracture wires by introducing the CBAM-R attention mechanism and the Focal EIoU loss function. The improvement of the algorithm of this invention not only improves the detection capability of small-fracture wires, but also improves the overall detection accuracy of the model, while the detection speed still meets the requirements of real-time detection. Attached Figure Description

[0044] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0045] Figure 1 This is a flowchart illustrating the machine vision-based method for detecting damage to steel wire ropes in mining hoists, as described in an embodiment of the present invention.

[0046] Figure 2 This paper compares the performance of the edge-guided wire rope motion fuzz restoration algorithm of this invention with the original image and other motion fuzz restoration methods. Among them, (a) original image, (b) inverse filtering, (c) Wiener filtering, and (d) algorithm of this invention.

[0047] Figure 3 This is a schematic diagram of the types of broken wire ropes labeled when constructing the dataset in an embodiment of the present invention, wherein (a) broken wire at the top of the strand, (b) broken wire at the valley of the strand, and (c) broken wire.

[0048] Figure 4 This is a network structure diagram of the CTR-YOLO model proposed in this embodiment of the invention.

[0049] Figure 5 This is a structural diagram of the improved CBAM-R attention mechanism proposed in the embodiments of the present invention.

[0050] Figure 6 This is a performance comparison result of the algorithm proposed in this embodiment of the invention with other classic algorithms.

[0051] Figure 7 The algorithm proposed in this embodiment of the invention is shown in the detection effect on images of broken wires in steel wire ropes.

[0052] Figure 8 This is a schematic diagram of the layout and installation of the wire rope damage detection system for mining hoists according to an embodiment of the present invention.

[0053] Figure 9 This is a hierarchical diagram of the software design for a mine hoist wire rope damage detection system according to an embodiment of the present invention.

[0054] Figure 10 This is the real-time detection interface of the wire rope damage detection system for mining hoists according to an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0056] Example:

[0057] See Figures 1 to 7 The present invention provides a technical solution for detecting damage to wire ropes in mining hoists based on machine vision, comprising the following steps:

[0058] S1. Dataset Image Acquisition: Industrial cameras are installed in the hoisting section of the vertical shaft hoisting system in coal mines to collect images of the wire rope during operation.

[0059] S2. Image preprocessing, as detailed below:

[0060] S21. Wire rope motion blur restoration: Based on Wiener filtering, edge weights are added to the filter to weaken the excessive smoothing of edges by the Wiener filter, thereby obtaining the image of the wire rope after motion blur removal.

[0061] S22. LabelImg Image Annotation: LabelImg software is used to annotate the collected wire rope broken wire images with three types of broken wires: broken wire at the top of the strand, broken wire at the bottom of the strand, and broken wire skipping, thereby establishing a wire rope damage dataset.

[0062] S3. Design the algorithm and train the model: Introduce the RFC3 lightweight feature extraction module, the skip connection of the feature fusion network, the CBAM-R attention mechanism, and the improved Focal EIoU loss function into the YOLOv5s backbone network. Name it CTR-YOLO and set its structure and parameters. Use the dataset constructed in S22 to train the model and obtain a model with the accuracy and speed required for practical applications.

[0063] S4. Establish a real-time detection system: A distributed deployment scheme is adopted. The inspection device is installed at the hoist to acquire images of the wire rope. The wire rope motion blur restoration algorithm is deployed on the image processing workstation to receive the images from the inspection device and perform motion blur restoration. The algorithm model obtained in S3 is deployed on the detection interrupt receiving workstation to receive the motion blur restored images and perform wire breakage detection to obtain the detection results.

[0064] Reference Figure 2 This paper compares the performance of the edge-guided wire rope motion blur restoration algorithm proposed in this invention with the original image and other motion blur restoration methods. (a) Original image, (b) Inverse filtering, (c) Wiener filtering, and (d) Algorithm of this invention. Based on Wiener filtering, the algorithm of this invention adds edge weights to the filter, thereby weakening the excessive smoothing of edges by the Wiener filter and preserving important structural information. The specific process of weight calculation is as follows:

[0065] S211, Image Gradient Calculation: The Sobel operator is a common gradient operator, often used to detect edges in images. The Sobel operator calculates the horizontal (x-direction) gradient Gx and the vertical (y-direction) gradient Gy using a convolution kernel. The gradient magnitude at a given pixel can be calculated using the Euclidean norm.

[0066] S212. Image Gradient Normalization: Image gradient normalization standardizes the image gradient, mapping its numerical range to a specific range, typically [0, 1]. According to the calculation formula... Obtain the normalized gradient result NG at the current position, where G is the gradient magnitude at the current position, and Gmax and Gmin are the maximum and minimum gradient values ​​in the current image, respectively.

[0067] S213. Weight Calculation: The normalized result NG reflects the rate of change in different regions of the image. NG can be used to balance the weights. In edge regions, NG is larger, and the corresponding weight is smaller, thus reducing the filter's effect and preserving edge details. In smooth regions, NG is smaller, and the corresponding weight is larger, making the filter more effective in smooth regions to reduce noise. The weight is set to Weight = 1 - NG, and this weight is applied to the original Wiener filter. The filter conversion formula of the present invention is obtained above:

[0068] Where W(u, v) is the introduced weight, H(u, v) is the motion degradation function, F(u, v) and G(u, v) represent the Fourier transforms of the original image f(x, y) and the blurred image g(x, y) respectively, and K is the adjustment factor of the filter. Comparing (b) inverse filtering and (c) Wiener filtering, it can be seen that when performing Wiener filtering, if noise is not suppressed, the noise in the image will be amplified, and the restored image will have obvious noise traces, resulting in poor overall image quality. From (c) Wiener filtering and (d) the algorithm of this invention, it can be seen that Wiener filtering and the algorithm of this invention have better effects on the restoration of the motion blur of the wire rope, and the texture features of the wire rope in the restored image are relatively clear, but the image restored by Wiener filtering still has some noise.

[0069] Reference Figure 3 This diagram illustrates the types of wire rope breakage labeled during the dataset construction process of this invention, where (a) top strand breakage, (b) valley strand breakage, and (c) skipped wire breakage. Top strand breakage occurs at the top of the strand of the wire rope, i.e., the outermost layer of wire. This type of breakage is usually caused by direct external forces such as collision, compression, or overload. Valley strand breakage refers to the breakage of wires at the valley where they contact each other between strands. This type of breakage is usually caused by friction between wires when the wire rope is under load. Skipped wire breakage refers to the phenomenon where, during actual use, inner layer broken wires jump out onto the surface of the wire rope due to uneven tension. In this embodiment, after motion blur recovery of the acquired images, data augmentation is performed on the acquired images by means of rotation, brightness adjustment, etc. After augmentation, there are a total of 3237 images, containing 3468 broken wire targets, including 1668 broken wires at the top of the stock, 1098 broken wires at the bottom of the stock, and 702 skipped wires. They are labeled with LabelImg as d1, d2, and d3 and then normalized to the same size as the dataset of the network model.

[0070] Reference Figure 4The diagram shows the network structure of the CTR-YOLO model proposed in this invention. It can be divided into a feature extraction network, a feature fusion network, and a detection network. The feature extraction network is responsible for extracting image features; the feature fusion network is responsible for fusing feature information extracted from different levels of the backbone network; and the detection network is responsible for predicting the detection results. Specifically, the C3 module in the YOLOv5s feature extraction network is replaced with the RFC3 lightweight feature extraction module. The RFC3 module replaces the two 3×3 convolutions that bottleneck the C3 module with 3×3 partial convolutions and two 1×1 convolutions, significantly reducing the module size. The model reduces the number of parameters, improving detection speed. Based on BiFPN, the feature fusion network of the original model is reconstructed by establishing skip connections between different layers, enabling the model to fully utilize the low-level feature information of the wire rope and improving the overall detection accuracy. A CBAM-R attention mechanism is added between layers 4 and 16, and between layers 6 and 12. The low-level feature information extracted by the feature extraction network is further enhanced by the CBAM-R attention mechanism and fed into the feature fusion network, improving the model's detection accuracy for small broken wires. The bounding box regression loss function CIoU of the original model is replaced with Focal EIoU. During backpropagation, the model can calculate a more accurate loss value, increasing its attention to small broken wires and further improving the detection accuracy and convergence speed of the model.

[0071] Reference Figure 5 This is a structural diagram of the improved CBAM-R attention mechanism proposed in this invention. Convlayer1 and Convlayer2 represent a 3×3 ordinary convolution and a 3×3 dilated convolution, respectively. Different dilation coefficients are used to select appropriate receptive fields: when enhancing low-level feature information, an excessively large receptive field can cause the details of small targets to be confused with the surrounding background information, leading to blurring or loss of target features. Therefore, a dilation coefficient of 1 is used, equivalent to a 3×3 ordinary convolution. The combined receptive field is then equivalent to a 5×5 ordinary convolution. When enhancing mid-to-high-level feature information, a larger receptive field can improve the model's feature extraction and representation capabilities. Therefore, a dilation coefficient of 2 is used, and the combined receptive field is equivalent to a 7×7 ordinary convolution.

[0072] Reference Figure 6The table shows the performance comparison results of the algorithm proposed in this invention with other classic algorithms. Here, d1, d2, and d3 represent the top break, the bottom break, and the skipped break, respectively, with small breakages all belonging to d1. The model of this invention has the highest detection accuracy for d1, with an AP value of 86.3%, indicating that the model effectively improves the detection accuracy for small breakages by introducing the CBAM-R attention mechanism and the Focal EIoU loss function. The model of this invention has the highest mAP@0.5 value among all models, at 87.4%, and a detection FPS of 75.3. Although slightly inferior to YOLOv7-tiny, this is perfectly acceptable considering the high performance of other metrics. Figure 6 This demonstrates that the improved algorithm of this invention enhances the detection capability of small-fracture wires while improving the overall detection accuracy of the model, and the detection speed still meets the requirements of real-time detection, showing superior performance compared to current mainstream target detection models.

[0073] Reference Figure 7 The algorithm proposed in this invention is used to detect broken wire images of steel wire ropes. The algorithm model of this invention did not have any missed detections or false detections. The predicted bounding box of this algorithm model fits the target very well and the confidence rate of the predicted bounding box is also very high.

[0074] Reference Figure 8 This is a schematic diagram illustrating the layout and installation of the wire rope damage detection system for mining hoists according to the present invention. Its components include:

[0075] The mobile inspection device includes an industrial camera, an artificial light source, a precision track, and a motion controller, used to capture images of steel wire ropes during transportation;

[0076] An image processing workstation is deployed to perform motion blur restoration on real-time acquired wire rope images using the edge-guided wire rope motion blur restoration algorithm proposed in this invention.

[0077] The engineer's operating station (inspection terminal) deploys the CTR-YOLO real-time detection algorithm for steel wire rope proposed in this invention. The images collected by the inspection device are restored by the image processing workstation and then transmitted to the inspection terminal at high speed via industrial Ethernet for wire breakage detection.

[0078] Reference Figure 9 This is a hierarchical diagram of the software design for the wire rope damage detection system for mining hoists of this invention. The interaction layer refers to the content displayed on the software interface, mainly including functions such as real-time detection, damage information statistics and viewing, and exporting detection records; the implementation layer refers to the technical support for implementing the interaction layer, mainly including file management, the algorithm of this invention, PyQt, etc.; the data layer provides data sources for the entire system, mainly including acquisition hardware, a MySQL database, and local files.

[0079] Reference Figure 10This invention also provides a machine vision-based wire rope damage detection system for mining hoists. This interface primarily displays the status of four line-scan cameras in the mobile inspection device and the real-time images of the wire rope collected. It also displays the hoist's operating status (direction of travel, speed, depth) and the most recent alarm information. Furthermore, this interface provides users with access to functions such as damage query and inspection report export.

[0080] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A machine vision-based method for detecting damage to wire ropes in mining hoists, characterized in that, Includes the following steps: S1. Dataset Image Acquisition: Install industrial cameras in the hoisting section of the vertical shaft hoisting system in a coal mine to collect images of the wire rope during operation; S2. Image preprocessing, as detailed below: S21. Wire rope motion blur restoration: Based on Wiener filtering, edge weights are added to the filter to weaken the excessive smoothing of edges by the Wiener filter, thereby obtaining the image of the wire rope after motion blur removal. S22. Image annotation: The collected images of broken wires in the wire rope are annotated with three types of broken wires: broken wires at the top of the strand, broken wires at the bottom of the strand, and broken wires that jump, in order to establish a wire rope damage dataset. S3. Design the algorithm training model: Introduce the RFC3 lightweight feature extraction module, the skip connection of the feature fusion network, the CBAM-R attention mechanism, and the improved Focal EIoU loss function strategy into the YOLOv5s backbone network, name it CTR-YOLO, set its structure and parameters, and train it using the dataset constructed in S22 to obtain a model that meets the accuracy and speed requirements of actual applications. Replace the C3 module in the YOLOv5s feature extraction network with the RFC3 lightweight feature extraction module. The RFC3 module replaces the two 3×3 convolutions that bottleneck the C3 module with 3×3 partial convolutions and two 1×1 convolutions. The specific improvements made by the CBAM-R attention mechanism based on the CBAM attention mechanism are as follows: Random pooling: Random pooling is a special pooling operation that introduces randomness into the pooling process to address the potential risk of overfitting in the model. It can be seen as a form of implicit regularization, reducing the model's dependence on specific patterns in the input data. Dilated Convolution: To avoid the loss of feature information caused by an excessively large receptive field and to reduce the computational cost of this module, the 7×7 convolution in the channel attention mechanism is replaced with a 3×3 ordinary convolution and a 3×3 dilated convolution with an expansion factor of X. Dilated convolution is a special type of convolution that increases the size of the receptive field without changing the computational cost by inserting holes in the convolution kernel. Residual connection: In order to solve the problem of insufficient effective information acquisition of spatial attention mechanism caused by attention concatenation, residual connection can combine shallow features and deep features generated by CAM to generate new feature maps and send them into the spatial attention mechanism module, thereby improving SAM's utilization of global information. Based on the above methods, the improved calculation formula for CBAM-R is determined as follows: ,in, For the improved channel attention mechanism, To improve the spatial attention mechanism, For the input feature map, To output the feature map, This represents element-wise multiplication; S4. Establish a real-time detection system: A distributed deployment scheme is adopted. The inspection device is installed at the hoist to obtain images of the wire rope. The wire rope motion blur restoration algorithm is deployed on the image processing workstation to receive the images from the inspection device and perform motion blur restoration. The algorithm model obtained in S3 is deployed on the motion blur restored images from the detection interruption receiving workstation to perform wire breakage detection and obtain the detection results.

2. The method for detecting damage to wire ropes in mining hoists based on machine vision according to claim 1, characterized in that, The weight calculation method in S21 is as follows: S211. Image gradient calculation: Calculated using the Sobel operator, which uses a convolution kernel to calculate the horizontal gradient. and vertical gradient The gradient magnitude of this pixel can be calculated using the Euclidean norm. ; S212, Image gradient normalization: According to the calculation formula Obtain the result of gradient normalization at the current position , The gradient magnitude at the current position. and These are the maximum and minimum values ​​of the gradient in the current image, respectively; S213. Weight Calculation: The normalization result reflects the rate of change in different regions of the image. Weight balancing can be achieved using NG (Not Given) parameters. In edge regions, a larger NG corresponds to a smaller weight, thus reducing the filter's effect and preserving edge details. In smooth regions, a smaller NG corresponds to a larger weight, making the filter more effective in smooth areas to reduce noise. The weights are set to... Apply this weight to the original Wiener filter The filter conversion formula of the present invention is obtained above: , in, For the introduced weights, Let be the motion degradation function. , Representing the original image respectively and blurry images Fourier transform, The value is the adjustment factor for this filter.

3. The method for detecting damage to wire ropes in mining hoists based on machine vision according to claim 1, characterized in that, In step S2, after motion blur recovery of the acquired images, data augmentation is performed on the acquired images by means of rotation and brightness adjustment to improve the generalization ability of the model. The type of broken wire target is obtained from the image, and the broken wire targets are normalized to the same size after software annotation as the dataset of the network model.

4. The method for detecting damage to wire ropes in mining hoists based on machine vision according to claim 1, characterized in that, In S3, the CTR-YOLO algorithm model is built based on the YOLOv5s model. Based on BiFPN, the feature fusion network of the original model is reconstructed by establishing skip connections between different layers, so that the model can make full use of the low-level feature information of the steel wire rope. CBAM-R attention mechanism is added between layers 4 and 16, and between layers 6 and 12 of YOLOv5s. The low-level feature information extracted by the feature extraction network is further enhanced by the improved CBAM-R attention mechanism and then fed into the feature fusion network.

5. The method for detecting damage to wire ropes in mining hoists based on machine vision according to claim 1, characterized in that, In step S3, the original model's bounding box regression loss function CIoU is replaced with Focal EIoU. During backpropagation, the model can calculate a more accurate loss value. The formula for calculating Focal EIoU is... ,in, The parameter IOU (Intersection over Union) controls the degree of outlier suppression; it calculates the overlap ratio between the predicted bounding box and the ground truth bounding box. for The loss function is calculated using the following formula: ； Where b is the center point of the predicted bounding box. The center point of the true bounding box. The formula for calculating the distance is denoted by c, where c is the diagonal length of the smallest diplomatic rectangle between the ground truth box and the predicted box. , These are the width and height of the minimum bounding rectangle of the two bounding boxes, respectively.

6. A machine vision-based damage detection system for steel wire ropes of mining hoists, comprising a machine vision-based method for detecting damage to steel wire ropes of mining hoists according to any one of claims 1-5, characterized in that, include: The mobile inspection device includes an industrial camera, an artificial light source, a precision track, and a motion controller, used to capture images of steel wire ropes during transportation; An image processing workstation is deployed to perform motion blur restoration on real-time acquired wire rope images using the edge-guided wire rope motion blur restoration algorithm proposed in this invention. The engineer's workstation is equipped with the CTR-YOLO wire rope real-time detection algorithm. The images collected by the inspection device are restored by the image processing workstation and then transmitted at high speed to the detection terminal via industrial Ethernet for wire breakage detection.

Images