Anti-vibration hammer defect detection method and system based on structural feature extraction module guidance

By introducing a structural feature extraction module in the YOLOv7 model, using the bar structural characteristics of the shock-proof hammer, the problem of low detection accuracy in the prior art is solved, and efficient identification and distinction of shock-proof hammer defects is achieved.

CN118096645BActive Publication Date: 2024-11-26STATE GRID SICHUAN ELECTRIC POWER CORP ELECTRIC POWER RES INST
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311770149.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-20
Publication Date
2024-11-26
Estimated Expiration
2043-12-20

AI Technical Summary

Technical Problem

In the prior art, the detection method of shock-proof hammer defects in transmission lines does not consider the structural characteristics of shock-proof hammers, resulting in low detection accuracy and it is difficult to effectively distinguish the characteristics of different defect samples.

Method used

A structural feature extraction module is designed to extract the structural features of the shock-proof hammer using a bar feature extractor and add it to the YOLOv7 model to enhance the structural feature extraction capability of the shock-proof hammer and its defects and improve the model detection accuracy.

Benefits of technology

Through the guidance of the structural feature extraction module, the model detection accuracy of shock-proof hammer defect detection is significantly improved, and it can effectively distinguish the characteristics of different defect samples, such as block drop, deformation, intersection, corrosion, etc.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118096645B_ABST
    Figure CN118096645B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for detecting a shock-proof hammer defect guided by a structural feature extraction module. The method comprises: obtaining historical shock-proof hammer images and forming a data set, dividing the data set into a training set, a verification set and a test set; and marking normal samples and defective samples of the shock-proof hammer in the training set; constructing a structural feature extraction module based on the strip structure characteristics and task requirements of the shock-proof hammer; and adding the structural feature extraction module to the YOLOv7 model to obtain an improved YOLOv7 model; using the training set, training the improved YOLOv7 model to obtain a training weight, and generating a shock-proof hammer defect detection model; and using the verification set, verifying the shock-proof hammer defect detection model; and based on the test set, using the trained shock-proof hammer defect detection model to perform shock-proof hammer defect detection to obtain a detection result. The present invention can distinguish different defect sample features and improve the model detection accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of power equipment defect detection, and in particular to a shock-absorbing hammer defect detection method and system guided by a structural feature extraction module. Background Art

[0002] The safe operation of transmission lines is essential to ensure the normal transmission of electric energy. With the rapid development of power systems and the accelerated advancement of urbanization, transmission lines, as key infrastructure for power transmission, play a vital role in modern society. Shock absorbers are components of transmission lines that maintain the balance of transmission lines. They usually appear in pairs and achieve vibration reduction by generating opposite forces to the conductors.

[0003] Traditional power transmission line defect detection is mainly based on manual inspection, which is time-consuming, low in safety and extremely costly. With the application of drone inspection technology in the power field, drone inspection has gradually replaced manual inspection with its advantages of high safety, low cost and no geographical restrictions.

[0004] With the repeated breakthroughs of convolutional neural networks (CNN) in the field of image recognition, it has driven the development of fields such as target detection, image segmentation, and target tracking. At present, mainstream algorithms in the field of target detection, such as the R-CNN series and the YOLO series, have greatly improved detection accuracy and efficiency. Target detection technology based on deep learning has begun to be combined with drone inspection technology to achieve high-precision and high-speed inspections.

[0005] Since the shock-proof hammer is exposed to the atmosphere in the outdoor environment for a long time, it is easily affected by the erosion of natural disasters such as wind, rain, thunder and lightning, and various defects are easily produced on the surface. Defects in the shock-proof hammer will have a negative impact on the safety of the transmission line. If the defective shock-proof hammer is not discovered and repaired in time during regular manual inspections, it may bring hidden dangers to the normal operation of the transmission line. In recent years, the inspection of transmission lines has transitioned from unmanned to intelligent operation and maintenance mode. Intelligent monitoring and intelligent detection robots have gradually become the mainstream methods of monitoring and detection. Therefore, it is very important to conduct regular inspections and monitoring through drones to determine whether they need maintenance and replacement.

[0006] However, the existing method for detecting defects of the power transmission line anti-vibration hammer does not take into account the structural characteristics of the anti-vibration hammer itself, cannot distinguish the characteristics of different defect samples well, and has low model detection accuracy. Summary of the invention

[0007] The technical problem to be solved by the present invention is that the existing method for detecting defects in the anti-vibration hammer of a power transmission line does not take into account the structural characteristics of the anti-vibration hammer itself, cannot distinguish the characteristics of different defect samples well, and has low model detection accuracy. The purpose of the present invention is to provide an anti-vibration hammer defect detection method and system guided by a structural feature extraction module. In order to distinguish the characteristics of different defect samples well, such as block drop, deformation, intersection, rust and other defects, the present invention takes into account that the anti-vibration hammer has a strip structure feature and the structure has symmetry, thereby designing a strip feature extractor (structural feature extraction module), through which the structural feature extraction of the anti-vibration hammer is improved; and then the structural feature extraction module is added to the YOLOv7 model to strengthen the structural feature extraction of the anti-vibration hammer and its defects; and then the prepared data set is put into the improved YOLOv7 model for training to obtain training weights. The present invention improves the model detection accuracy.

[0008] The present invention is achieved through the following technical solutions:

[0009] In a first aspect, the present invention provides a method for detecting defects of a shock-absorbing hammer guided by a structural feature extraction module, the method comprising:

[0010] Obtain historical anti-vibration hammer images and form a data set, which is divided into a training set, a validation set, and a test set; and annotate normal samples and defective samples of the anti-vibration hammer in the training set;

[0011] Based on the strip structure characteristics of the shock-absorbing hammer and the task requirements, a structural feature extraction module is constructed; and the structural feature extraction module is added to the YOLOv7 model to obtain an improved YOLOv7 model;

[0012] The improved YOLOv7 model is trained using the training set to obtain the training weights and generate a shock-absorbing hammer defect detection model. The shock-absorbing hammer defect detection model is verified using the verification set.

[0013] Based on the test set, the trained anti-shock hammer defect detection model is used to perform anti-shock hammer defect detection and obtain the detection results.

[0014] Furthermore, defective samples include samples of anti-vibration hammer falling off, samples of anti-vibration hammer deformation, samples of anti-vibration hammer intersection, and samples of anti-vibration hammer rust.

[0015] Furthermore, the length-to-width ratios of normal samples, rusted samples, and deformed samples are all 5:1, the length-to-width ratio of the fallen-off samples is 3:1, and the length-to-width ratio of the intersecting samples is 7:1.

[0016] Furthermore, the structural feature extraction module is used to capture and extract the structural features of the anti-vibration hammer and its defects. The anti-vibration hammer has a strip-shaped structural feature, and the structure is symmetrical.

[0017] Furthermore, the structural feature extraction module includes:

[0018] The first branch structure includes a convolution layer with a convolution kernel size of 1×1. The first branch structure adopts a residual structure, which can effectively retain global information;

[0019] The second branch structure includes a convolution layer with a convolution kernel size of 1×1, a first strip convolution layer, a second strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence;

[0020] The third branch structure includes a convolution layer with a convolution kernel size of 1×1, a third strip convolution layer, a fourth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence;

[0021] The fourth branch structure includes a convolution layer with a convolution kernel size of 1×1, a fifth strip convolution layer, a sixth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence;

[0022] The output ends of the first branch structure, the second branch structure, the third branch structure and the fourth branch structure are spliced ​​with the first branch structure after the first splicing, which can effectively extract the structural characteristics of the shock-absorbing hammer and reduce the amount of calculation.

[0023] Furthermore, the convolution kernel size of the first strip convolution layer is 1×3, and the convolution kernel size of the second strip convolution layer is 3×1;

[0024] The convolution kernel size of the third strip convolution layer is 1×5, and the convolution kernel size of the fourth strip convolution layer is 5×1;

[0025] The convolution kernel size of the fifth strip convolution layer is 1×7, and the convolution kernel size of the sixth strip convolution layer is 7×1.

[0026] Furthermore, the improved YOLOv7 model is based on the original YOLOv7 model, adding the structural feature extraction module to the backbone network, and correspondingly connecting the neck network through three structural feature extraction modules, and outputting three feature maps of different scales to the detection output layer through the neck network;

[0027] The three structural feature extraction modules are respectively recorded as a first structural feature extraction module, a second structural feature extraction module and a third structural feature extraction module.

[0028] Furthermore, the improved YOLOv7 model includes a backbone network, a neck network, and a detection output layer;

[0029] The backbone network is composed of two groups of sequentially arranged first structural feature extraction modules, first ELAN modules and first maximum pooling modules, second structural feature extraction modules, second ELAN modules and second maximum pooling modules, and a group of sequentially arranged SPPCSPC modules and third structural feature extraction modules;

[0030] The neck network includes an upsampling branch and a downsampling branch. The upsampling branch is sequentially provided with a first UP module, a first ELAN-H module, a second UP module, and a second ELAN-H module; the first UP module is connected to the third structural feature extraction module; the first structural feature extraction module is connected to the second ELAN-H module through the first CBS module and the first Concat module; the second UP module is also connected to the first Concat module; the second structural feature extraction module is connected to the first ELAN-H module through the second CBS module and the second Concat module; the first UP module is also connected to the second Concat module;

[0031] The down-sampling branch dimensions are sequentially arranged as a third maximum pooling module, a third Concat module, a third ELAN-H module, a fourth maximum pooling module, and a fourth Concat module, the second ELAN-H module is connected to the third maximum pooling module, the first ELAN-H module is connected to the third Concat module, and the fourth Concat module is connected to the fourth ELAN-H module;

[0032] The second ELAN-H module, the third Concat module, and the fourth ELAN-H module are respectively connected to the rearranged convolution module (REPConv) and output three feature maps of different sizes to the detection output layer.

[0033] Among them, Concat refers to concatenation, UP refers to upsampling, ELAN refers to a multi-branch stacking module, and ELAN-H is a multi-branch stacking module with half the number of channels.

[0034] In a second aspect, the present invention further provides a shock-absorbing hammer defect detection system guided by a structural feature extraction module, the system using the shock-absorbing hammer defect detection method guided by a structural feature extraction module; the system comprises:

[0035] An acquisition unit, used to acquire historical anti-vibration hammer images and form a data set;

[0036] The data set division and labeling unit is used to divide the data set into a training set, a validation set, and a test set; and to label the normal samples and defective samples of the shock-absorbing hammer in the training set;

[0037] The improved YOLOv7 model generation unit is used to construct a structural feature extraction module based on the strip-shaped structural feature of the anti-vibration hammer; and the structural feature extraction module is added to the YOLOv7 model to obtain the improved YOLOv7 model; wherein the structural feature extraction module is used to capture and extract the structural features of the anti-vibration hammer and its defects, and the anti-vibration hammer has a strip-shaped structural feature and a symmetrical structure;

[0038] The model training and verification unit is used to train the improved YOLOv7 model using the training set, obtain the training weight, and generate the anti-vibration hammer defect detection model; and verify the anti-vibration hammer defect detection model using the verification set;

[0039] The defect detection unit is used to perform anti-shock hammer defect detection based on the test set by using the trained anti-shock hammer defect detection model to obtain a detection result.

[0040] Furthermore, the structural feature extraction module includes:

[0041] The first branch structure includes a convolution layer with a convolution kernel size of 1×1. The first branch structure adopts a residual structure, which can effectively retain global information;

[0042] The second branch structure includes a convolution layer with a convolution kernel size of 1×1, a first strip convolution layer, a second strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the first strip convolution layer is 1×3, and the convolution kernel size of the second strip convolution layer is 3×1;

[0043] The third branch structure includes a convolution layer with a convolution kernel size of 1×1, a third strip convolution layer, a fourth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the third strip convolution layer is 1×5, and the convolution kernel size of the fourth strip convolution layer is 5×1;

[0044] The fourth branch structure includes a convolution layer with a convolution kernel size of 1×1, a fifth strip convolution layer, a sixth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the fifth strip convolution layer is 1×7, and the convolution kernel size of the sixth strip convolution layer is 7×1;

[0045] The output ends of the first branch structure, the second branch structure, the third branch structure and the fourth branch structure are spliced ​​with the first branch structure after the first splicing, which can effectively extract the structural characteristics of the shock-absorbing hammer and reduce the amount of calculation.

[0046] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0047] The present invention is based on a shock-absorbing hammer defect detection method and system guided by a structural feature extraction module. In order to distinguish the sample features of normal samples and different defects (such as falling blocks, deformation, intersection, rust and other defects), the present invention takes into account that the shock-absorbing hammer has a strip structure feature and the structure has symmetry, and thus designs a strip feature extractor (structural feature extraction module), through which the structural feature extraction of the shock-absorbing hammer is improved; and then the structural feature extraction module is added to the YOLOv7 model to strengthen the structural feature extraction of the shock-absorbing hammer and its defects; and then the prepared data set is put into the improved YOLOv7 model for training to obtain training weights. The present invention can distinguish the features of different defect samples and improve the model detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0049] Figure 1 It is a flow chart of the anti-vibration hammer defect detection method guided by the structural feature extraction module of the present invention;

[0050] Figure 2 A data set sample of an embodiment of the present invention;

[0051] Figure 3 This is the improved YOLOv7 model diagram of the present invention;

[0052] Figure 4 The specific structure of the structural feature extraction module of the present invention;

[0053] Figure 5 The first is the comparative test result of the baseline model (YOLOv7) and the improved model of the present invention (YOLOv7+structural feature extraction module);

[0054] Figure 6 The second comparison test result is between the baseline model (YOLOv7) and the improved model of the present invention (YOLOv7+structural feature extraction module);

[0055] Figure 7 The third comparison test result is between the baseline model (YOLOv7) and the improved model of the present invention (YOLOv7+structural feature extraction module);

[0056] Figure 8 The fourth comparative test result is between the baseline model (YOLOv7) and the improved model of the present invention (YOLOv7+structural feature extraction module);

[0057] Fig. 9This is a structural block diagram of the anti-vibration hammer defect detection system guided by the structural feature extraction module of the present invention. DETAILED DESCRIPTION

[0058] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary implementation modes of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0059] The existing method for detecting defects of the anti-vibration hammer of a power transmission line does not take into account the structural characteristics of the anti-vibration hammer itself, cannot distinguish the characteristics of different defect samples well, and has low model detection accuracy.

[0060] The present invention designs a shock-absorbing hammer defect detection method and system based on the guidance of a structural feature extraction module. In order to distinguish the sample features of normal samples and different defects (such as falling blocks, deformation, intersection, rust and other defects), the present invention takes into account that the shock-absorbing hammer has a strip structure feature and the structure has symmetry, and thus designs a strip feature extractor (structural feature extraction module), through which the structural feature extraction of the shock-absorbing hammer is improved; and then the structural feature extraction module is added to the YOLOv7 model to strengthen the structural feature extraction of the shock-absorbing hammer and its defects; and then the prepared data set is put into the improved YOLOv7 model for training to obtain training weights. The present invention can distinguish the features of different defect samples and improve the detection accuracy of the model.

[0061] Example 1

[0062] like Figure 1 As shown, the present invention is based on a shock-absorbing hammer defect detection method guided by a structural feature extraction module. First, aerial photos are used to construct a transmission line shock-absorbing hammer defect data set; then, a training set and a validation set in the data set are used to train a shock-absorbing hammer defect detection model; and a test set in the data set is input into the trained model to obtain a detection result index.

[0063] Specifically, the method includes:

[0064] Step 1: Obtain historical anti-vibration hammer images and form a data set, and divide the data set into a training set, a validation set, and a test set; and annotate normal samples and defective samples of the anti-vibration hammer in the training set;

[0065] Firstly, all aerial photos containing anti-vibration hammers are divided into training set, validation set and test set in a ratio of 4:1:1. Then, the normal samples of anti-vibration hammers and samples of falling blocks, deformation, intersection and rust in the pictures are annotated by labelimg annotation software, and the annotation files are saved. Finally, data enhancement is performed on the training set pictures. Figure 2are data set samples, where (a) is a normal sample, (b) is a rusted sample, (c) is a deformed sample, (d) is a fallen block sample, and (e) is an intersection sample.

[0066] Step 2: Based on the bar structure characteristics of the shock-absorbing hammer and the task requirements, a structural feature extraction module is constructed; and the structural feature extraction module is added to the YOLOv7 model to obtain an improved YOLOv7 model;

[0067] Based on the complexity of the task and the size of the dataset, YOLOv7 was chosen as the baseline model. YOLOv7 is a brand-new target detection algorithm proposed in 2022. At that time, the algorithm was in the leading position in the target detection network model in terms of speed and accuracy with a transmission rate of 5FPS to 160FPS.

[0068] The YOLOv7 model mainly proposes an extended efficient layer aggregation network architecture (Extended Efficient Layer Aggregation Networks, ELAN), which effectively improves the learning and convergence process of deep networks by controlling the shortest and longest gradient paths. At the same time, ELAN uses expansion, shuffling, and merging without breaking the original gradient path to continue to enhance the network feature extraction ability. The maximum pooling module (Max-Pooling, MP) in the backbone network downsamples the features to reduce the loss of feature details. The neck network uses the feature pyramid network and path aggregation network in YOLOv5 to fuse network features. Based on the cross stage partial (CSP) structure, the spatial pyramid pooling (Spatial PyramidPooling, SPP) is improved to SPPCSPC, and convolution operations and convolution branches are added before and after SPP, which reduces the amount of calculation and improves the accuracy. The re-arranged convolution (Receptive Field Enhanced Convolution, REPConv) module has two structures during training and inference. During inference, re-parameterization is used to reduce parameters without reducing accuracy.

[0069] In order to distinguish the characteristics of normal samples from samples with different defects (such as falling blocks, deformation, intersection, rust, etc.), the present invention takes into account the strip-shaped structural characteristics of the shock-absorbing hammer and the symmetry of the structure, and thus designs a strip-shaped feature extractor (structural feature extraction module), through which the structural feature extraction of the shock-absorbing hammer is improved; and then the structural feature extraction module is added to the backbone network of the YOLOv7 model, and the specific structure of the network is designed according to the characteristics of the shock-absorbing hammer data and the task requirements. Adding the structural feature extraction module to the backbone network can effectively capture the structural characteristics of the transmission line shock-absorbing hammer and its defects.

[0070] The shock-absorbing hammer has a strip-shaped structural feature and a symmetrical structure. The strip-shaped convolution kernel has a good feature extraction effect on the strip-shaped feature. The aspect ratio of normal samples, rusted samples and deformed samples is close to 5:1, the aspect ratio of fallen samples is close to 3:1, and the aspect ratio of intersecting samples is close to 7:1. Therefore, a strip-shaped feature extractor is set to improve the extraction of structural features of the shock-absorbing hammer.

[0071] like Figure 4 As shown in the figure, the structural feature extraction module consists of four branches, namely:

[0072] The first branch structure includes a convolution layer with a convolution kernel size of 1×1. The first branch structure adopts a residual structure, which can effectively retain global information;

[0073] The second branch structure includes a convolution layer with a convolution kernel size of 1×1, a first strip convolution layer, a second strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the first strip convolution layer is 1×3, and the convolution kernel size of the second strip convolution layer is 3×1;

[0074] The third branch structure includes a convolution layer with a convolution kernel size of 1×1, a third strip convolution layer, a fourth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the third strip convolution layer is 1×5, and the convolution kernel size of the fourth strip convolution layer is 5×1;

[0075] The fourth branch structure includes a convolution layer with a convolution kernel size of 1×1, a fifth strip convolution layer, a sixth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the fifth strip convolution layer is 1×7, and the convolution kernel size of the sixth strip convolution layer is 7×1.

[0076] After the first splicing, the output ends of the first branch structure, the second branch structure, the third branch structure and the fourth branch structure are spliced ​​with the previous residual structure, and finally spliced ​​with the trunk features retained above, and finally output through the activation function. In the experiment, the aspect ratio of normal samples, rust samples and deformed samples in the statistical data set is close to 5:1, the aspect ratio of block samples is close to 3:1, and the aspect ratio of intersection samples is close to 7:1. The convolution kernels 3*3, 5*5, and 7*7 are decomposed into strip convolutions such as 1*3 and 3*1, 1*5 and 5*1, and 1*7 and 7*1, respectively, where 1*3 and 3*1 correspond to block defects, 1*5 and 5*1 correspond to deformation defects, and 1*7 and 7*1 correspond to intersection defects, in order to distinguish the feature differences between normal shockproof hammers and shockproof hammers with structural defects. The shockproof hammer, a hardware with a strip structure, can effectively extract its structural features through strip convolution, and can effectively reduce the amount of calculation.

[0077] This structural feature extraction module is added to the YOLOv7 model. This module is placed in the backbone network of the YOLOv7 model and the Neck connection part for feature fusion and splicing, which can effectively improve the model's perception of structural features. After the ELAN module, this efficient aggregation network effectively improves the deep network learning and convergence process by controlling the shortest and longest gradient path. After the ELAN module, the structural feature learning and convergence speed can be increased. Strengthen the structural feature extraction of the shock-absorbing hammer and its defects.

[0078] Specifically, the improved YOLOv7 model is based on the original YOLOv7 model, adding a structural feature extraction module to the backbone network, and correspondingly connecting the neck network through three structural feature extraction modules, and outputting three feature maps of different scales to the detection output layer through the neck network; the three structural feature extraction modules are respectively recorded as the first structural feature extraction module, the second structural feature extraction module and the third structural feature extraction module.

[0079] like Figure 3 As shown, the improved YOLOv7 model includes a backbone network, a neck network, and a detection output layer;

[0080] The backbone network is composed of two groups of sequentially arranged first structural feature extraction modules, first ELAN modules and first maximum pooling modules, second structural feature extraction modules, second ELAN modules and second maximum pooling modules, and a group of sequentially arranged SPPCSPC modules and third structural feature extraction modules;

[0081] The neck network includes an upsampling branch and a downsampling branch. The upsampling branch is sequentially provided with a first UP module, a first ELAN-H module, a second UP module, and a second ELAN-H module; the first UP module is connected to the third structural feature extraction module; the first structural feature extraction module is connected to the second ELAN-H module through the first CBS module and the first Concat module; the second UP module is also connected to the first Concat module; the second structural feature extraction module is connected to the first ELAN-H module through the second CBS module and the second Concat module; the first UP module is also connected to the second Concat module;

[0082] The down-sampling branch dimensions are sequentially arranged as a third maximum pooling module, a third Concat module, a third ELAN-H module, a fourth maximum pooling module, and a fourth Concat module, the second ELAN-H module is connected to the third maximum pooling module, the first ELAN-H module is connected to the third Concat module, and the fourth Concat module is connected to the fourth ELAN-H module;

[0083] The second ELAN-H module, the third Concat module, and the fourth ELAN-H module are respectively connected to the rearranged convolution module (REPConv) and output three feature maps of different sizes to the detection output layer.

[0084] Among them, Concat refers to concatenation, UP refers to upsampling, ELAN refers to a multi-branch stacking module, and ELAN-H is a multi-branch stacking module with half the number of channels.

[0085] Step 3: Use the training set to train the improved YOLOv7 model, obtain the training weight, and generate a shock-absorbing hammer defect detection model; and use the verification set to verify the shock-absorbing hammer defect detection model;

[0086] Specifically, the prepared test set is put into the improved YOLOv7 model for training to obtain the training weights.

[0087] Step 4: Based on the test set, the trained anti-shock hammer defect detection model is used to perform anti-shock hammer defect detection to obtain the detection result.

[0088] The test set in the data set is put into the trained shock-absorbing hammer defect detection model for shock-absorbing hammer defect detection to obtain various parameters of the model, such as parameter quantity, floating-point calculation amount and model inference speed. Then, the detection accuracy of normal samples, rust, deformation, chipping and intersection of the shock-absorbing hammer is obtained, and then statistics are performed to obtain the final result.

[0089] This embodiment uses aerial photos to construct a transmission line anti-vibration hammer defect dataset. The above steps are implemented and the experimental results are as follows Table 1: Figures 5 to 8 As shown in Table 1, mAP represents the average precision. After adding the structural feature extraction module, the accuracy of the samples with obvious structural defects such as block loss, intersection, and deformation is significantly improved. Specifically, the present invention uses multi-branch strip convolution and residual network to effectively distinguish normal samples from samples with defects such as block loss, deformation, intersection, and rust, thereby improving the detection accuracy of the model.

[0090] Figure 5 middle, Figure 5 (a) is the confidence of normal and corrosion detection of the baseline model (YOLOv7), Figure 5 (b) is the confidence of normal and corrosion detection of the improved model (YOLOv7+structural feature extraction module) of the present invention. Obviously, the confidence of normal and corrosion detection of the improved model of the present invention is higher than that of the baseline model.

[0091] Figure 6 middle, Figure 6(a) The baseline model (YOLOv7) misdetects the intersection sample as a corrosion sample. Figure 6 (b) The improved model of the present invention (YOLOv7+structural feature extraction module) correctly identifies intersection samples.

[0092] Figure 7 middle, Figure 7 (a) The baseline model (YOLOv7) does not accurately locate the deformation sample. Figure 7 (b) The improved model of the present invention (YOLOv7+structural feature extraction module) accurately locates the deformation.

[0093] Figure 8 middle, Figure 8 (a) The baseline model (YOLOv7) does not accurately locate the dropped samples. Figure 8 (b) The improved model of the present invention (YOLOv7+structural feature extraction module) accurately locates the dropped samples.

[0094] The present invention illustrates that the structural feature extraction module can effectively improve the extraction of structural features by the model.

[0095] Table 1 Experimental results

[0096]

[0097] Example 2

[0098] like Figure 6 As shown, the difference between this embodiment and embodiment 1 is that this embodiment provides an anti-vibration hammer defect detection system guided by a structural feature extraction module, and the system uses the anti-vibration hammer defect detection method guided by a structural feature extraction module of embodiment 1; the system includes:

[0099] An acquisition unit, used to acquire historical anti-vibration hammer images and form a data set;

[0100] The data set division and labeling unit is used to divide the data set into a training set, a validation set, and a test set; and to label the normal samples and defective samples of the shock-absorbing hammer in the training set;

[0101] The improved YOLOv7 model generation unit is used to construct a structural feature extraction module based on the strip-shaped structural feature of the anti-vibration hammer; and the structural feature extraction module is added to the YOLOv7 model to obtain the improved YOLOv7 model; wherein the structural feature extraction module is used to capture and extract the structural features of the anti-vibration hammer and its defects, and the anti-vibration hammer has a strip-shaped structural feature and a symmetrical structure;

[0102] The model training and verification unit is used to train the improved YOLOv7 model using the training set, obtain the training weight, and generate the anti-vibration hammer defect detection model; and verify the anti-vibration hammer defect detection model using the verification set;

[0103] The defect detection unit is used to perform anti-shock hammer defect detection based on the test set by using the trained anti-shock hammer defect detection model to obtain a detection result.

[0104] As a further implementation, the structural feature extraction module includes:

[0105] The first branch structure includes a convolution layer with a convolution kernel size of 1×1. The first branch structure adopts a residual structure, which can effectively retain global information;

[0106] The second branch structure includes a convolution layer with a convolution kernel size of 1×1, a first strip convolution layer, a second strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the first strip convolution layer is 1×3, and the convolution kernel size of the second strip convolution layer is 3×1;

[0107] The third branch structure includes a convolution layer with a convolution kernel size of 1×1, a third strip convolution layer, a fourth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the third strip convolution layer is 1×5, and the convolution kernel size of the fourth strip convolution layer is 5×1;

[0108] The fourth branch structure includes a convolution layer with a convolution kernel size of 1×1, a fifth strip convolution layer, a sixth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the fifth strip convolution layer is 1×7, and the convolution kernel size of the sixth strip convolution layer is 7×1;

[0109] The output ends of the first branch structure, the second branch structure, the third branch structure and the fourth branch structure are spliced ​​with the first branch structure after the first splicing, which can effectively extract the structural characteristics of the shock-absorbing hammer and reduce the amount of calculation.

[0110] The execution process of each unit can be performed according to the process steps of the anti-vibration hammer defect detection method guided by the structural feature extraction module in Example 1, and will not be described one by one in this embodiment.

[0111] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0112] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0113] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0114] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0115] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A shock-absorbing hammer defect detection method guided by a structural feature extraction module, characterized in that: The method includes: Acquire historical anti-vibration hammer images and form a data set, divide the data set into a training set, a validation set, and a test set; and annotate normal samples and defective samples of the anti-vibration hammer in the training set; Based on the strip structure characteristics of the shock-absorbing hammer and the task requirements, a structural feature extraction module is constructed; and the structural feature extraction module is added to the YOLOv7 model to obtain an improved YOLOv7 model; Using the training set, the improved YOLOv7 model is trained to obtain training weights, and a shock-absorbing hammer defect detection model is generated; and using the verification set, the shock-absorbing hammer defect detection model is verified; Based on the test set, the trained anti-vibration hammer defect detection model is used to perform anti-vibration hammer defect detection and obtain the detection results; The improved YOLOv7 model is based on the original YOLOv7 model, adding the structural feature extraction module to the backbone network, and correspondingly connecting the neck network through three structural feature extraction modules, and outputting three feature maps of different scales to the detection output layer through the neck network; The three structural feature extraction modules are respectively recorded as a first structural feature extraction module, a second structural feature extraction module and a third structural feature extraction module; The structural feature extraction module is used to capture and extract the structural features of the anti-vibration hammer and its defects. The anti-vibration hammer has a strip-shaped structural feature and a symmetrical structure. The structural feature extraction module comprises: The first branch structure includes a convolution layer with a convolution kernel size of 1×1, and the first branch structure adopts a residual structure; The second branch structure includes a convolution layer with a convolution kernel size of 1×1, a first strip convolution layer, a second strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the first strip convolution layer is 1×3, and the convolution kernel size of the second strip convolution layer is 3×1; The third branch structure includes a convolution layer with a convolution kernel size of 1×1, a third strip convolution layer, a fourth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the third strip convolution layer is 1×5, and the convolution kernel size of the fourth strip convolution layer is 5×1; The fourth branch structure includes a convolution layer with a convolution kernel size of 1×1, a fifth strip convolution layer, a sixth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the fifth strip convolution layer is 1×7, and the convolution kernel size of the sixth strip convolution layer is 7×1; The output ends of the first branch structure, the second branch structure, the third branch structure and the fourth branch structure are spliced ​​with the first branch structure after being spliced ​​for the first time.

2. The anti-vibration hammer defect detection method based on structural feature extraction module guidance according to claim 1 is characterized in that: The defective samples include samples of anti-vibration hammer falling off, samples of anti-vibration hammer deformation, samples of anti-vibration hammer intersection, and samples of anti-vibration hammer rust.

3. The anti-vibration hammer defect detection method based on structural feature extraction module guidance according to claim 2 is characterized in that: The length-to-width ratios of the normal samples, rusted samples and deformed samples are all 5:1, the length-to-width ratio of the fallen samples is 3:1, and the length-to-width ratio of the intersecting samples is 7:

1.

4. The anti-vibration hammer defect detection method based on structural feature extraction module guidance according to claim 1 is characterized in that: The improved YOLOv7 model includes a backbone network, a neck network and a detection output layer; The backbone network includes two groups of first structural feature extraction modules, first ELAN modules and first maximum pooling modules, second structural feature extraction modules, second ELAN modules and second maximum pooling modules, and a group of SPPCSPC modules and third structural feature extraction modules. The neck network includes an upsampling branch and a downsampling branch, the upsampling branch is sequentially provided with a first UP module, a first ELAN-H module, a second UP module and a second ELAN-H module; the first UP module is connected to the third structural feature extraction module; the first structural feature extraction module is connected to the second ELAN-H module through the first CBS module and the first Concat module; the second UP module is also connected to the first Concat module; the second structural feature extraction module is connected to the first ELAN-H module through the second CBS module and the second Concat module; the first UP module is also connected to the second Concat module; The down-sampling branch dimensions are sequentially arranged as a third maximum pooling module, a third Concat module, a third ELAN-H module, a fourth maximum pooling module, and a fourth Concat module, the second ELAN-H module is connected to the third maximum pooling module, the first ELAN-H module is connected to the third Concat module, and the fourth Concat module is connected to the fourth ELAN-H module; The second ELAN-H module, the third Concat module, and the fourth ELAN-H module are respectively connected to the rearranged convolution module and output three feature maps of different sizes to the detection output layer; Among them, Concat refers to concatenation, UP refers to upsampling, ELAN refers to a multi-branch stacking module, and ELAN-H is a multi-branch stacking module with half the number of channels.

5. The anti-vibration hammer defect detection system guided by the structural feature extraction module is characterized by: The system includes: An acquisition unit, used to acquire historical anti-vibration hammer images and form a data set; A data set division and labeling unit, used to divide the data set into a training set, a validation set and a test set; and label normal samples and defective samples of the shock-absorbing hammer in the training set; The improved YOLOv7 model generation unit is used to construct a structural feature extraction module based on the strip-shaped structural feature of the anti-vibration hammer; and the structural feature extraction module is added to the YOLOv7 model to obtain the improved YOLOv7 model; wherein the structural feature extraction module is used to capture and extract the structural features of the anti-vibration hammer and its defects, and the anti-vibration hammer has a strip-shaped structural feature and a symmetrical structure; A model training and verification unit, used to perform model training on the improved YOLOv7 model using a training set, obtain training weights, and generate an anti-shock hammer defect detection model; and verify the anti-shock hammer defect detection model using a verification set; A defect detection unit, used for performing anti-shock hammer defect detection based on a test set and using a trained anti-shock hammer defect detection model to obtain a detection result; The improved YOLOv7 model is based on the original YOLOv7 model, adding the structural feature extraction module to the backbone network, and correspondingly connecting the neck network through three structural feature extraction modules, and outputting three feature maps of different scales to the detection output layer through the neck network; The three structural feature extraction modules are respectively recorded as a first structural feature extraction module, a second structural feature extraction module and a third structural feature extraction module; The structural feature extraction module is used to capture and extract the structural features of the anti-vibration hammer and its defects. The anti-vibration hammer has a strip-shaped structural feature and a symmetrical structure. The structural feature extraction module comprises: The first branch structure includes a convolution layer with a convolution kernel size of 1×1, and the first branch structure adopts a residual structure; The second branch structure includes a convolution layer with a convolution kernel size of 1×1, a first strip convolution layer, a second strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the first strip convolution layer is 1×3, and the convolution kernel size of the second strip convolution layer is 3×1; The third branch structure includes a convolution layer with a convolution kernel size of 1×1, a third strip convolution layer, a fourth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the third strip convolution layer is 1×5, and the convolution kernel size of the fourth strip convolution layer is 5×1; The fourth branch structure includes a convolution layer with a convolution kernel size of 1×1, a fifth strip convolution layer, a sixth strip convolution layer, and a convolution layer with a convolution kernel size of 3×3, which are connected in sequence; the convolution kernel size of the fifth strip convolution layer is 1×7, and the convolution kernel size of the sixth strip convolution layer is 7×1; The output ends of the first branch structure, the second branch structure, the third branch structure and the fourth branch structure are spliced ​​with the first branch structure after being spliced ​​for the first time.

6. The anti-vibration hammer defect detection system based on structural feature extraction module guidance according to claim 5 is characterized in that: The improved YOLOv7 model includes a backbone network, a neck network and a detection output layer; The backbone network includes two groups of first structural feature extraction modules, first ELAN modules and first maximum pooling modules, second structural feature extraction modules, second ELAN modules and second maximum pooling modules, and a group of SPPCSPC modules and third structural feature extraction modules. The neck network includes an upsampling branch and a downsampling branch, the upsampling branch is sequentially provided with a first UP module, a first ELAN-H module, a second UP module and a second ELAN-H module; the first UP module is connected to the third structural feature extraction module; the first structural feature extraction module is connected to the second ELAN-H module through the first CBS module and the first Concat module; the second UP module is also connected to the first Concat module; the second structural feature extraction module is connected to the first ELAN-H module through the second CBS module and the second Concat module; the first UP module is also connected to the second Concat module; The down-sampling branch dimensions are sequentially arranged as a third maximum pooling module, a third Concat module, a third ELAN-H module, a fourth maximum pooling module, and a fourth Concat module, the second ELAN-H module is connected to the third maximum pooling module, the first ELAN-H module is connected to the third Concat module, and the fourth Concat module is connected to the fourth ELAN-H module; The second ELAN-H module, the third Concat module, and the fourth ELAN-H module are respectively connected to the rearranged convolution module and output three feature maps of different sizes to the detection output layer; Among them, Concat refers to concatenation, UP refers to upsampling, ELAN refers to a multi-branch stacking module, and ELAN-H is a multi-branch stacking module with half the number of channels.