A cableway steel wire rope defect detection method and system based on improved YOLO

Abstract

A method and system for detecting defects in cableway steel wire ropes based on an improved YOLO model is disclosed. The method includes the following steps: establishing a detection network model based on an improved YOLOv8 model, comprising a backbone network, a neck network, and a prediction layer; replacing standard convolutions with dynamic serpentine convolutions in the C2f and Bottleneck modules of the backbone network; controlling the continuous extension of the receptive field of the convolution kernel through a cumulative offset mechanism; adopting a Slim-neck structure in the neck network, replacing 3×3 ordinary convolutions with GSConv modules and C2f modules with VoV-GSCSP modules; the GSConv module consists of standard convolutional layers, depthwise separable convolutional layers, and channel rearrangement operations. This method and system, based on YOLOv8, systematically improves upon it in three dimensions: feature extraction, multi-scale fusion, and bounding box regression, forming a three-in-one approach of "dynamic serpentine feature extraction - implicit connection feature fusion - corner geometric constraint regression," balancing the needs of slender targets, accuracy, and efficiency.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to a method and system for detecting defects in cableway wire ropes based on an improved YOLO. Background Technology

[0002] As a core load-bearing component of critical equipment such as mine hoists, port loading and unloading systems, bridge cableways, and elevators, the safe operation of wire ropes is directly related to the safety of personnel and equipment. Therefore, achieving efficient and accurate detection of wire rope defects is of great significance.

[0003] In recent years, vision-based defect detection methods have gradually become a research hotspot due to their advantages such as non-contact operation, low cost, and real-time monitoring. Among them, deep learning object detection algorithms, represented by YOLO, have been widely used in industrial defect detection.

[0004] However, existing vision-based defect detection methods still face the following key technical challenges: 1. Difficulty in Feature Extraction for Slender Targets – Mismatch Between Receptive Field and Target Shape. Defects in wire ropes (such as broken wires, wear, and corrosion) are characterized by their slender shape, small size, and irregular orientation. Existing detection models (such as YOLOv8n) use standard convolutions with fixed sampling points, and their square receptive field is ill-suited to slender targets with significant aspect ratio differences. When the convolution kernel covers the defect area, it introduces a large amount of background noise, leading to a decrease in feature representation ability, and slender defects are easily overlooked or misjudged.

[0005] 2. Multi-scale feature transmission interruption – Slender features are prone to breakage during transmission. During the transmission of features in the feature pyramid, each spatial compression and channel expansion leads to the loss of some semantic information. For slender targets, their morphological features are inherently fragile, making them more susceptible to "feature breakage" during layer-by-layer transmission. This makes it difficult to retain complete slender structural information in higher-level feature maps, affecting subsequent detection accuracy.

[0006] 3. Insufficient accuracy in bounding box regression – the relative aspect ratio is insufficient for depicting slender targets. The CIOU loss function in YOLOv8 measures the difference between the predicted and ground truth bounding boxes using an aspect ratio penalty term. However, CIOU's aspect ratio penalty term relies on relative values ​​rather than absolute values. When the predicted and ground truth bounding boxes have the same aspect ratio but different dimensions (e.g., one slender box is completely contained within another), CIOU cannot effectively distinguish the differences, resulting in limited bounding box regression accuracy, especially noticeable with slender targets.

[0007] In summary, existing visual inspection methods suffer from three major bottlenecks in wire rope defect scenarios: difficulty in feature extraction, easy breakage of feature transfer, and low accuracy of bounding box regression. There is an urgent need for an inspection method that is suitable for slender targets and balances accuracy and efficiency. Summary of the Invention

[0008] The purpose of this invention is to propose a defect detection method and system for cableway steel wire ropes based on an improved YOLO. Based on YOLOv8, it systematically improves the method from three dimensions: feature extraction, multi-scale fusion, and bounding box regression, forming a three-in-one approach of "dynamic serpentine feature extraction - implicit connection feature fusion - corner point geometric constraint regression", which takes into account slender targets, accuracy, and efficiency.

[0009] The technical solution adopted in this invention is: a defect detection method for cableway steel wire ropes based on an improved YOLO, comprising the following steps: A detection network model based on the improved YOLOv8 model is established, which includes a backbone network, a neck network, and a prediction layer. The C2f and Bottleneck modules of the backbone network use dynamic serpentine convolution instead of standard convolution; the dynamic serpentine convolution controls the continuous extension of the receptive field of the convolution kernel through an accumulated offset mechanism. The neck network adopts a Slim-neck structure, replacing the 3×3 ordinary convolution with a GSConv module and the C2f module with a VoV-GSCSP module. The GSConv module consists of standard convolutional layers, depthwise separable convolutional layers, and channel rearrangement operations, which are used to reduce the amount of computation while preserving the implicit connections between channels. The VoV-GSCSP module achieves multi-level feature extraction and fusion by stacking multiple GSBottleneck modules. The prediction layer uses the minimum point distance intersection-to-union loss function instead of the full intersection-to-union loss function to optimize the bounding box regression from the perspective of corner geometric constraints. The image of the cableway wire rope to be inspected is input into the detection network model, and the detection results of the wire rope defects are output.

[0010] As a preferred embodiment, the cumulative offset mechanism of the dynamic serpentine convolution is as follows: with the center coordinates of the convolution kernel as the starting position, a learnable deformation offset is introduced, and the coordinates of subsequent positions in the convolution kernel are obtained by accumulating the coordinates of the previous position and the corresponding deformation offset, so that the convolution kernel is continuously distributed along the target direction.

[0011] As a preferred embodiment, the dynamic serpentine convolution further includes: using bilinear interpolation to achieve feature sampling for fractional-order deformation offsets.

[0012] As a preferred embodiment, the processing flow of the GSConv module is as follows: first, features are extracted through standard convolution, then lightweight processing is performed through depthwise separable convolution, and then the feature maps obtained by the standard convolution and the depthwise separable convolution are interleaved and rearranged to form implicit connections.

[0013] As a preferred embodiment, the minimum point distance intersection-union ratio loss function optimizes bounding box regression by minimizing the Euclidean distance between the top-left corner and the bottom-right corner of the predicted bounding box and the true bounding box.

[0014] As a preferred embodiment, the wire defects include at least one of broken wires, wear, and corrosion.

[0015] A system for implementing the cableway wire rope defect detection method includes: The image acquisition unit is used to acquire images of the cableway wire rope and transmit them to the processing unit; The processing unit receives images of the cableway wire rope and inputs them into a detection network model based on the improved YOLOv8 model, which is loaded onto the cableway wire rope, and outputs the defect location and category.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. Compared with ordinary deformable convolution, this invention ensures the continuity of convolution kernel movement through the cumulative offset mechanism, so that the receptive field is strictly distributed along the direction of the slender target, avoiding the receptive field from deviating from the target due to excessive offset. This fundamentally solves the problem of difficult feature extraction of slender defects.

[0017] 2. This invention reveals for the first time the fundamental problem of feature fragmentation in the original YOLO neck network when processing slender targets, and proposes a channel implicit connectivity fidelity mechanism for GSConv. Unlike solutions that simply pursue lightweight design, this solution achieves an optimal balance between lightweight design and feature integrity, ensuring that slender features can be transferred to the detection head without loss, and maximizing the utilization of multi-scale information while maintaining low computational cost.

[0018] 3. This invention uses MPDIoU to directly regress the Euclidean distance between the top-left and bottom-right corner points, transforming the bounding box regression problem into a corner position constraint problem. For slender targets, even a small offset of the corner points can cause drastic changes in IoU. Direct regression of the corner points is more accurate and efficient than indirect regression through the center point and aspect ratio. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a diagram of the YOLOv8n-NEW network structure in this invention; Figure 2This is a schematic diagram of the DySnakeConv adaptive process in this invention; Figure 3 This is a schematic diagram of the C2f module in this invention; Figure 4 This is a schematic diagram of the Slim-neck network module in this invention; Figure 5 This is a schematic diagram of the factors influencing MDPIOU in this invention. Detailed Implementation

[0021] The present invention will now be described in detail through exemplary embodiments. However, it should be understood that, without further description, elements, structures, and features in one embodiment may be advantageously incorporated into other embodiments.

[0022] It should be noted that, unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "a," "an," or "the," etc., used in the specification and claims of this patent application do not express a limitation on quantity, but rather indicate the presence of at least one; the terms "first," "second," and "third," as used herein, should not be considered as a limitation on the order of components, but are merely for distinguishing different components; the terms "comprising," "including," etc., indicate that the elements or objects preceding "comprising" or "including" encompass the elements or objects listed following "comprising" or "including" and their equivalents, but do not exclude other elements or objects having the same function.

[0023] To more clearly describe the cableway wire rope defect detection method and system based on the improved YOLO, in conjunction with the appendix... Figure 1-5 Description of the embodiment: like Figure 1-5 As shown, a defect detection method for cableway wire ropes based on an improved YOLO protocol includes the following steps: First, we establish a detection network model, YOLOv8n-NEW, which is an improvement on the YOLOv8 model. The detection network model includes a backbone network, a neck network, and a prediction layer. The following section provides a systematic explanation of the improvements from three dimensions: feature extraction, multi-scale fusion, and bounding box regression.

[0024] In the feature extraction dimension, since the receptive field of standard convolution is fixed, it is difficult to adapt to the direction and shape of slender targets. Therefore, dynamic serpentine convolution is used to replace standard convolution in the C2f and Bottleneck modules of the backbone network. Dynamic serpentine convolution controls the continuous extension of the receptive field of the convolution kernel through the cumulative offset mechanism. The cumulative offset mechanism of dynamic serpentine convolution is as follows: with the center coordinate of the convolution kernel as the starting position, a learnable deformation offset is introduced. The coordinates of subsequent positions in the convolution kernel are obtained by accumulating the coordinates of the previous position and the corresponding deformation offset, so that the convolution kernel is continuously distributed along the direction of the target.

[0025] Let the coordinates of the standard convolution kernel be... The center coordinates are After introducing the deformation offset Δ, the convolution kernel positions are determined sequentially according to the accumulation method: x-axis direction: (1) y-axis direction: (2) in, For learnable offsets, , These represent the x-axis and y-axis coordinates of the convolution kernel as it moves to the next position. By accumulating the offset, each position of the convolution kernel depends on the previous position, ensuring that the receptive field extends continuously along the target direction, rather than spreading out randomly.

[0026] Since the offset Δ is of fractional order, bilinear interpolation is used to achieve feature sampling. (3) in, Indicates the coordinate position of the target score; Represents an integer spatial location; A is the bilinear interpolation kernel.

[0027] The improved structures of Bottleneck_DySnakeConv and C2f_DySnakeConv are as follows: Figure 3 As shown.

[0028] In the multi-scale fusion dimension, traditional feature pyramids are prone to breakage of slender features during transmission. Therefore, the neck network adopts a Slim-neck structure and designs a customized fusion method of GSConv + VoV-GSCSP for slender targets. The GSConv module replaces the 3×3 ordinary convolution and the VoV-GSCSP module replaces the C2f module, maximizing the use of multi-scale information while maintaining low computational cost. See Figure 4The GSConv module consists of standard convolutional layers, depthwise separable convolutional layers, and channel shuffle operations. The processing flow of the GSConv module is as follows: first, features are extracted by standard convolution, then lightweight processing is performed by depthwise separable convolution (DWConv), and then the feature maps obtained by standard convolution and depthwise separable convolution are interleaved and rearranged through the channel shuffle operation. This design reduces the amount of computation while preserving the implicit connections between channels and preventing long and thin features from being cut off during the transmission process. The VoV-GSCSP module achieves multi-level feature extraction and fusion by stacking multiple GSBottleneck modules, gradually capturing multi-level information of the input features and generating more representative feature representations.

[0029] In the bounding box regression dimension, since the aspect ratio penalty term of the complete intersection-union ratio loss function CIOU depends on the relative value, the regression accuracy for slender targets is limited. Therefore, the prediction layer of this invention uses the minimum point distance intersection-union ratio loss function MPDIoU instead of CIOU to optimize the bounding box regression from the perspective of corner geometric constraints. YOLOv8 employs a decoupled-head structure to separate the classification and detection heads. Its loss function consists of two parts: classification loss and regression loss. The classification loss uses Binary Cross-Entropy Loss (BCELoss) for measurement; the regression loss incorporates Distribution Focal Loss (DFL) and Box Identification Loss (CIOU). The total loss is shown in Equation 4.

[0030] Loss tal =Loss BCE + Loss DFL + Loss CIOU (4) Binary cross-entropy loss (BCE) measures the difference between the probability distribution predicted by the model and the actual labels, thus measuring the accuracy of the model's predictions.

[0031] (5) Where N represents the total number of samples, Represents the partial derivative. The distributed focal loss (DFL) is achieved by explicitly amplifying y. i and y i+1 The probability of this forces the network to quickly focus on the vicinity of the label y-value. Since bounding box learning only targets positive samples, it fundamentally solves the problem of imbalanced samples. iThis refers to the predicted probability value, specifically, in the bounding box regression task, after discretizing continuous location coordinates into multiple integer intervals, the probability of the network predicting the i-th discrete location.

[0032] (6) (7) Complete IoU (CIOU) builds upon traditional IoU by considering the following three relationships between the predicted bounding box and the target bounding box: 1) Center Distance: CIOU introduces a minimum closure region, considering the distance between the center points of the target boxes, i.e., GIOU; 2) Diagonal Distance: Based on GIOU, CIOU adds a normalization term for diagonal distance, considering the distance between the diagonals of the target boxes, i.e., DIOU; 3) Aspect Ratio: CIOU introduces an aspect ratio penalty term based on DIOU. By considering the aspect ratio of the target boxes, it ensures that target boxes with different aspect ratios are treated more equally.

[0033] (8) (9) (10) (11) Where α is the weighting function; v is the aspect ratio similarity; b, b gt - Center points of predicted bounding boxes and ground truth bounding boxes; Euclidean distance between two center points; w, -Predict the width of the bounding box and the true bounding box; h, - Heights of the predicted bounding box and the ground truth bounding box; c - Diagonal distance between the minimum closure regions of the predicted and ground truth bounding boxes.

[0034] As can be seen from formulas (8)-(11), the aspect ratio of CIOU depends on the relative value rather than the absolute value. Therefore, it may lose its effectiveness when dealing with the same aspect ratio but different width and height values. Therefore, MPDIOU is used to replace CIOU.

[0035] MDPIOU focuses more on the similarity between the predicted bounding box and the ground truth bounding box. Considering horizontal geometric features, it simplifies the similarity comparison between the two bounding boxes by minimizing the distances between the top-left and bottom-right points, thereby improving the accuracy and efficiency of bounding box regression. Figure 5 As shown.

[0036] x erg ,y erg ,werg ,h erg These represent the x-coordinate of the top-left corner, y-coordinate of the top-left corner, width, and height of the predicted bounding box, respectively.

[0037] X sd ,y sd ,w sd ,h sd These represent the x-coordinate of the top-left corner, y-coordinate of the top-left corner, width, and height of the actual bounding box, respectively.

[0038] Model predicts bounding boxes: (12) Approximate the true bounding box by minimizing the loss function: (13) (14) Berg - Set of predicted bounding boxes; Bsd - Set of true bounding boxes; - Regression depth model parameters. D is defined as the smallest bounding rectangle that simultaneously encloses the ground truth bounding box and the predicted bounding box, and |D| is the area of ​​this rectangle: (15) |D| is the area of ​​the smallest enclosed rectangle covering Bsd and Berg. Formula (15) normalizes the distance between the corner points by using the area of ​​the smallest enclosed region |D|, making MPDIoU more stable on slender targets. Unlike CIoU, which depends on the relative value of the aspect ratio, MPDIoU directly constrains the absolute positions of the upper left and lower right corner points, solving the problem of inaccurate regression under the same aspect ratio and significantly improving the positioning accuracy of the wire rope defect frame.

[0039] As can be seen from formulas (12)-(15), the current loss function comprehensively considers many factors, such as the non-overlapping area, the distance between the center points, and the deviation between the width and height. These factors can all be determined by the coordinates of the upper left and lower right corners of the bounding box. When the predicted box has the same aspect ratio as the ground truth box and the predicted box is inside the ground truth box, the value of MDPioU is lower; while when the predicted box is outside the ground truth box, the value of MDPioU is higher. This characteristic ensures the accuracy of bounding box regression. MDPioU not only simplifies the calculation process but also reduces the redundancy of the predicted bounding box.

[0040] The main defects in the steel wire rope are broken wires, wear, and corrosion. The image of the cableway steel wire rope to be inspected is input into the detection network model, and the detection results of the steel wire rope defects are output.

[0041] This invention provides high-quality slender features for subsequent fusion through DySnakeConv, ensures that the features extracted by DySnakeConv are completely transferred to the detection head through Slim-neck, and fully utilizes the features provided by the first two modules to achieve accurate regression of the bounding box. All three are indispensable and together constitute a complete detection solution for slender targets.

[0042] The technical effects of the present invention will be explained below by combining comparative experiments, ablation experiments and specific analysis of slender defects.

[0043] Comparative experiment Table 1 lists the detection results of six detection models on the cableway wire rope dataset, including the one-stage algorithm SSD, the typical two-stage algorithm Faster RCNN, and successive YOLO series algorithms. The proposed YOLOv8n-NEW algorithm achieves the highest mAP@0.5 detection rate of 82.5%; furthermore, compared to these models, the YOLOv8n-NEW model has the fewest parameters and computational cost. These results demonstrate that the proposed algorithm outperforms other methods in terms of detection performance, parameter count, and computational cost, further validating the superiority of YOLOv8n-NEW.

[0044] Table 1 Comparison of Detection Method Results ablation experiment To verify the contribution of each innovative module, eight ablation experiments were designed, and the results are shown in Table 2.

[0045] Table 2 Ablation Experiment Ablation experiment conclusions: Introducing DySnakeConv (Group 2) alone improved mAP by 3.8%, demonstrating the effectiveness of cumulative offset snake convolution for extracting slender features; Introducing Slim-neck (Group 3) alone improved mAP by 2.4%, demonstrating the contribution of the channel implicit connectivity mechanism to feature fidelity. Introducing MPDIoU (Group 4) alone improved mAP by 1.5%, demonstrating the optimization effect of corner geometric constraints on bounding box regression; When paired (groups 5-7), mAP is further improved, indicating that there is a synergistic effect between modules; The complete combination of the three modules (Group 8) achieved an mAP of 82.5%, significantly higher than any pairwise combination, verifying the overall superiority of the "three-in-one" solution.

[0046] Special analysis of slender defects To verify the targeted improvement effect of this patent on slender targets, specific statistics were conducted on two typical slender defects: wire breakage and wear. The results are shown in Table 3. Table 3 Statistical results of slender defects Experimental results show that the method of this patent has a particularly significant improvement in recall rate for slender defects, proving the targeted effectiveness of this patent for slender targets.

[0047] The present invention also discloses a system for implementing a method for detecting defects in cableway wire ropes, including an image acquisition unit and a processing unit.

[0048] The image acquisition unit can be a high-speed industrial camera, used to acquire images of the cableway wire rope and transmit them to the processing unit; The processing unit receives images of the cableway wire rope and inputs them into a detection network model based on the improved YOLOv8 model, which is loaded onto the cableway wire rope, and outputs the defect location and category.

[0049] The technical effects of the present invention will be described below with reference to specific embodiments.

[0050] Example 1: Defect Detection of Cableway Wire Ropes This patented method was used to detect defects in the steel wire rope of a cableway in a scenic area. The acquired images included various defect types such as broken wires, wear, and corrosion, with an image resolution of 1920×1080. The model was trained using 2000 labeled images, and a test set of 500 images was used.

[0051] The test results show: The recall rate for detecting broken wire defects reached 89.2%, significantly higher than YOLOv8n's 78.5%; Under conditions of uneven lighting and complex backgrounds, the false detection rate is reduced by approximately 40% compared to YOLOv8n; The inference time for a single image is approximately 37ms, and the inference speed is stable at 27 FPS, meeting the requirements for real-time industrial detection.

[0052] Example 2: Online Monitoring of Mine Hoisting Wire Ropes This patented method was deployed in an online monitoring system for wire ropes in mine hoists. The system uses an industrial camera to acquire real-time images of the operating wire rope. Each frame is processed using the YOLOv8n-NEW model to output the defect location and category. Results from 72 hours of continuous operation show that: The system operates stably without any abnormal alarms caused by missed or false detections; The model inference speed is stable at 26-28 FPS, which matches the operating speed of the hoist. The test results showed a 94.7% consistency rate with manual verification, validating the engineering applicability of this method.

[0053] The parts not described in detail in the above embodiments are existing technologies.

[0054] It should be noted that although the present invention has been described through the above embodiments, the present invention may have many other embodiments. Without departing from the spirit and scope of the present invention, those skilled in the art can obviously make various corresponding changes and modifications to the present invention, but all such changes and modifications should fall within the scope of protection of the appended claims and their equivalents.

Claims

1. A method for detecting defects in cableway wire ropes based on an improved YOLO, characterized in that, Includes the following steps: A detection network model based on the improved YOLOv8 model is established, which includes a backbone network, a neck network, and a prediction layer. The C2f and Bottleneck modules of the backbone network use dynamic serpentine convolution instead of standard convolution; the dynamic serpentine convolution controls the continuous extension of the receptive field of the convolution kernel through an accumulated offset mechanism. The neck network adopts a Slim-neck structure, replacing the 3×3 ordinary convolution with a GSConv module and the C2f module with a VoV-GSCSP module. The GSConv module consists of standard convolutional layers, depthwise separable convolutional layers, and channel rearrangement operations, which are used to reduce the amount of computation while preserving the implicit connections between channels. The VoV-GSCSP module achieves multi-level feature extraction and fusion by stacking multiple GSBottleneck modules. The prediction layer uses the minimum point distance intersection-to-union loss function instead of the full intersection-to-union loss function to optimize the bounding box regression from the perspective of corner geometric constraints. The image of the cableway wire rope to be inspected is input into the detection network model, and the detection results of the wire rope defects are output.

2. The method for detecting defects in cableway wire ropes based on an improved YOLO as described in claim 1, characterized in that, The cumulative offset mechanism of the dynamic serpentine convolution is as follows: with the center coordinates of the convolution kernel as the starting position, a learnable deformation offset is introduced. The coordinates of subsequent positions in the convolution kernel are obtained by accumulating the coordinates of the previous position and the corresponding deformation offset, so that the convolution kernel is continuously distributed along the target direction.

3. The method for detecting defects in cableway wire ropes based on an improved YOLO as described in claim 2, characterized in that, The dynamic serpentine convolution also includes: for fractional-order deformation offsets, using bilinear interpolation to achieve feature sampling.

4. The method for detecting defects in cableway wire ropes based on an improved YOLO as described in claim 1, characterized in that, The processing flow of the GSConv module is as follows: first, features are extracted through standard convolution, then lightweight processing is performed through depthwise separable convolution, and then the feature maps obtained by the standard convolution and the depthwise separable convolution are interleaved and rearranged to form implicit connections.

5. The cableway wire rope defect detection method based on improved YOLO according to claim 1, characterized in that: The minimum point distance intersection-union loss function optimizes bounding box regression by minimizing the Euclidean distance between the top-left and bottom-right corners of the predicted bounding box and the true bounding box.

6. The method for detecting defects in cableway wire ropes based on an improved YOLO as described in claim 1, characterized in that: The defects in the steel wire include at least one of broken wire, wear, and corrosion.

7. A system for implementing the cableway wire rope defect detection method according to any one of claims 1-6, characterized in that, include: The image acquisition unit is used to acquire images of the cableway wire rope and transmit them to the processing unit; The processing unit receives images of the cableway wire rope and inputs them into a detection network model based on the improved YOLOv8 model, which is loaded onto the cableway wire rope, and outputs the defect location and category.

Images