Overhead ground wire detection method and system suitable for inspection and repair robot
By improving the lightweight model MobileNetV2 and introducing the Enhanced Feature Attention (ECA) mechanism, the detection accuracy of the inspection and repair robot for overhead ground wire defects has been improved, solving the problem of poor detection performance on resource-constrained equipment and achieving efficient and intelligent detection.
Patent Information
- Application Number
- CN202410872118.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-01
- Publication Date
- 2024-11-15
AI Technical Summary
Existing inspection and repair robots, especially those with lightweight models, do not perform well in detecting defects in overhead ground wires, making it difficult to achieve efficient and accurate target detection on resource-constrained equipment.
The lightweight detection model MobileNetV2 is adopted, and an enhanced feature attention mechanism (ECA) is introduced to improve it, forming the ECA-MobileNetV2 model. By combining data augmentation and appropriate training parameters, the model's ability to identify defects in overhead ground wires is improved.
It improves the accuracy, recall, and precision of overhead ground wire defect detection, enables efficient detection on resource-constrained equipment, and enhances the intelligence and safety of inspection and repair robots.
Smart Images

Figure CN118968439A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of overhead ground wire detection, and in particular relates to an overhead ground wire detection method and system suitable for an inspection and repair robot. Background Art
[0002] High-voltage transmission lines are an important part of the power system, and their working conditions have an important impact on the safe and stable operation of the power system. Because they are located in high ground and have poor insulation levels, they are easily struck by lightning. It is very important to take necessary measures to protect them from lightning. Installing overhead ground wires is the most direct and reliable protection measure. Due to the large span of the transmission line, the overhead ground wire is exposed to the natural environment for a long time, especially the transmission lines located in remote mountainous areas, which are more susceptible to bad weather, which will eventually lead to overhead ground wire breakage, loose strands and other faults, causing huge economic losses and serious electrical accidents.
[0003] At present, the measures taken for the maintenance of overhead ground wire faults are that the high-voltage operators of the power system locate the appropriate position of the broken strands, handle them manually, and use special auxiliary tools to carry out the broken strand repair task. The broken strands of the ground wire are generally located at the maximum stress point in the center of the line. Specially trained operators need to use special mobile equipment to reach the broken strands. In this case, the personnel are at high altitude and in a strong electromagnetic field environment, which is dangerous, time-consuming, and labor-intensive. In addition, during the repair process, they must wear shielding clothing, which reduces the efficiency and flexibility of the entire repair operation, and significantly increases the labor intensity and danger.
[0004] With the development of smart grids, the development of power grids has been closely integrated with the progress of social science and technology. The use of inspection and repair robots to replace manual repairs is not restricted by environmental factors and can avoid a large amount of repetitive labor. The image recognition algorithm carried by the inspection and repair robots can identify defects in overhead ground wires.
[0005] Transmission line image algorithms mainly include traditional image algorithms and image algorithms based on deep learning. Traditional transmission line target detection methods often rely on manually designed feature extractors and manually adjusted thresholds. Their performance is limited by the expressiveness and generalization capabilities of features, and it is difficult to meet the actual requirements in complex scenarios. Image algorithms based on deep learning can directly learn high-level and abstract feature expressions from raw data, and have stronger adaptability and generalization capabilities. Therefore, many important breakthroughs have been made in the field of image recognition.
[0006] Image detection models based on deep learning are divided into large models and lightweight models. When large models are deployed in actual applications, considering the detection speed, a graphics processing unit (GPU) is often required, which is expensive and complex to deploy in practice. At the same time, with the popularity of mobile terminals and edge computing, the demand for running efficient models on resource-constrained devices is increasing. Lightweight models can reduce the model's computing and memory requirements while maintaining model performance, thereby improving the system's response speed and efficiency. Therefore, considering the cost and equipment constraints, it is a better choice to use lightweight models for transmission line detection. However, different models have different detection effects on different recognition targets. For model selection, it is necessary to form a corresponding data set for model training and make a selection based on the results.
[0007] In addition, since the obstacles and line damage in the overhead ground wire have small feature areas in the picture and the background is complex, the feature differences are not obvious, and it is not possible to effectively identify them based on the feature information of obstacles or line damage. In addition, the lightweight model has a small number of parameters and limited feature learning capabilities, so the detection effect is not good. Summary of the invention
[0008] In view of the above-mentioned deficiencies in the prior art, the present invention provides an overhead ground wire detection method and system suitable for a patrol and repair robot. The patrol and repair robot involved in the present invention is not equipped with a graphics processing unit (GPU), and aims to solve the problem of improving target detection accuracy when the equipment is limited and the cost is taken into consideration.
[0009] The technical solution adopted by the present invention is:
[0010] An overhead ground wire detection method applicable to a patrol and repair robot comprises the following steps:
[0011] Step 1: Obtain the overhead ground wire defect dataset and preprocess the dataset to obtain the training set and verification set of overhead ground wire defect images;
[0012] Step 2: Select the training equipment and training parameters of the detection model, and set the evaluation indicators;
[0013] Step 3: Select a suitable lightweight detection model;
[0014] Step 4: Improve the lightweight detection model to enhance its ability to identify overhead ground wires.
[0015] The step 1 comprises the following steps:
[0016] S1.1. The present invention builds an experimental platform based on the inspection and repair robot's perspective, and installs obstacles such as shock-absorbing hammers on the simulated transmission line, while simulating the damage of the overhead ground wire in the natural environment;
[0017] S1.2. Deploy the video recording device on the inspection and repair robot, and operate the robot to walk normally on the line and record the video. At the same time, choose to walk and record the video multiple times under different weather and light conditions;
[0018] S1.3, sampling the obtained video data frame by frame, and screening the images;
[0019] S1.4, crop the image, keep the ground line part in the middle, and remove irrelevant information on both sides of the image;
[0020] S1.5, data augmentation, vertical flipping, horizontal flipping and adding Gaussian noise to the image;
[0021] S1.6. The final data set of the present invention has 4068 images, and the data set is divided into a training set and a validation set according to an 8:2 ratio.
[0022] In S1.5, the formulas for vertical flipping and horizontal flipping are:
[0023]
[0024] Where: (x, y) represents the pixel coordinates in the source image, (x′, y′) represents the pixel coordinates after flipping, and f H is the height of the original image, f W is the original image width.
[0025] The formula for Gaussian noise is:
[0026]
[0027] Where: f(x) represents the probability density of Gaussian noise, x represents the pixel of the overhead ground wire image, σ represents the standard deviation, and u represents the mean.
[0028] The step 2 comprises the following steps:
[0029] S2.1. Select a suitable device for model training. In order to save training time, the model training device selected by the present invention is: the operating system uses Window11 64-bit, the CPU is Intel Core i9-12900H, the GPU is NVIDIAGeForce RTX 3060Laptop, the running memory is 16G, the CUDA is CUDA11.6, and the development language is Python3.8;
[0030] S2.2. Select appropriate experimental parameters before training. The parameters selected in the present invention are: batch size is 32, initial learning rate is set to 0.001, momentum is set to 0.9, optimizer is Adam, and the number of training rounds is set to 200 rounds;
[0031] S2.3, Evaluation indicators:
[0032] The evaluation indicators of the present invention are precision (P), recall (R), F1 score, accuracy (Accuracy), parameter quantity (Params), computational amount GFLOPs and model size, wherein precision, recall F1 score, and accuracy evaluate the detection effect of the model, and parameter quantity and computational amount evaluate the complexity of the model. Their formulas are as follows:
[0033]
[0034] Params=IK 2 C+C
[0035] Where: T P It indicates the number of positive samples correctly predicted by the model as positive, T N Indicates the number of negative samples correctly predicted by the model as negative, F P Indicates the number of negative samples that the model incorrectly predicts as positive, F N It indicates the number of positive samples that the model incorrectly predicts as negative samples; I is the input size, K is the convolution kernel size, and C is the output size.
[0036] The step 3 comprises the following steps:
[0037] S3.1. According to the detection effect comparison model, the present invention uses the same data set to train the network structures of five different frameworks, namely MobileNetV2, GoogLeNet, ShuffleNet, AlexNet and ResNet, and compares the precision, recall rate, F1 score and accuracy of overhead ground wire detection by different models;
[0038] S32. Comprehensively evaluate the complexity of five different models: MobileNetV2, GoogLeNet, ShuffleNet, AlexNet and ResNe, and compare the final number of parameters, amount of computation and model size of the models;
[0039] Based on S31 and S32, the present invention selects the MobileNetV2 network for subsequent improvement.
[0040] In step 4, an attention mechanism is added to the MobileNetV2 network to enable the model to better understand the image data, thereby improving the performance and effect of the model. The specific steps are as follows:
[0041] S4.1. Add three attention mechanisms, ECA, CBAM and SE, to the MobileNetV2 network structure respectively, compare the detection effects of the network models after adding the three attention mechanisms, and compare the precision, recall, F1 score and accuracy of the three improved models for overhead ground wire detection;
[0042] It is preferred to add the ECA attention mechanism to the MobileNetV2 network structure;
[0043] S42. Compare the model complexity of the three improved MobileNetV2 network models described in S4.1, and compare the final parameter quantity, computational complexity, and model size of the MobileNetV2 network model;
[0044] Combining S41 and S42, we invent a model for an overhead ground wire detection method suitable for a patrol and repair robot: the ECA-MobileNetV2 network model.
[0045] The ECA-MobileNetV2 network model includes a convolutional layer, an enhanced feature linear bottleneck layer, an average pooling layer in a feature extraction network, and a Dropout layer and a Linear layer in a classification network;
[0046] The convolution layer contains 1×1 convolution and ReLU activation function. The main function of this module is to extract features. The input feature map first undergoes a convolution operation with a convolution kernel of 1×1, and then passes through a ReLU activation function. The formula of the ReLU activation function is as follows:
[0047] ReLU(x)=max(x,0);
[0048] The ReLU activation function provides a very simple nonlinear transformation. Given an element x, the ReLU activation function is defined as the maximum value of the element and 0.
[0049] The enhanced feature linear bottleneck layer is a linear bottleneck layer structure that adds an ECA attention mechanism. The linear bottleneck layer structure is a bottleneck structure that uses linear activation at the end of the convolution in the inverted residual, wherein the activation function is a ReLU activation function; this structure can reduce the loss of nonlinear features and retain low-dimensional information;
[0050] The image processing process of the inverted residual structure is:
[0051] The input feature map first undergoes a 1×1 convolution to increase the feature dimension, then uses a 3×3 depth-separable convolution to filter, and finally uses a 1×1 convolution to reduce the feature dimension to obtain the output feature of this layer, and then the input feature and the output feature are added; the process of the inverted residual structure can be called "compression-convolution-expansion"; this structure can improve the ability of gradients to propagate between network layers.
[0052] Depthwise separable convolution splits convolution into two independent steps: depthwise convolution and pointwise convolution;
[0053] The characteristic of the deep convolution is that a convolution kernel only performs convolution on one channel and only outputs the feature map of one channel. The calculation amount of the deep convolution is H×W×K 2 ×M; where: H represents the height of each channel, W represents the width of each channel, K represents the convolution kernel size, and M represents the input channel;
[0054] The processing process of the input feature map by the point-by-point convolution is similar to the standard convolution, but the convolution kernel size is 1×1. The working process is to align the convolution kernel with each position of the input image, perform element-by-element multiplication and sum, and then put the output result into the corresponding position of the output feature map. The calculation amount of the point-by-point convolution is H×W×M×N, where N represents the output channel;
[0055] Therefore, the calculation of the depth-separable convolution is: H×W×K 2 ×M+H×W×M×N; while the standard convolution calculation amount is H×W×K 2 ×M×N, and the standard convolution kernel is usually 3×3; generally, N in the depthwise separable convolution and the standard convolution is a large value. At this time, it can be seen that the computational complexity of the depthwise separable convolution is only one eighth of that of the standard convolution; the depthwise separable convolution greatly reduces the amount of parameter kernel computation while maintaining the representation capability of the model.
[0056] The ECA attention mechanism workflow is as follows:
[0057] First, the input feature map is subjected to a global average pooling operation, and then a one-dimensional convolution operation with a convolution kernel size of K is performed. Then, a sigmoid activation function is used to limit the calculated attention weights to the range of [0, 1]. Finally, these weights are multiplied with the original feature response to obtain the weighted feature representation.
[0058] The calculation formula of the above sigmoid activation function is:
[0059]
[0060] e represents a natural constant, whose value is approximately 2.718; x represents an input feature map;
[0061] The core mechanism of the ECA attention mechanism is to calculate the attention weight in each channel through a one-dimensional convolution operation to dynamically adjust the importance of the channel, strengthen the model's attention to important features, and reduce the attention to minor features; the convolution kernel size of the one-dimensional convolution can be freely selected, and it is self-adaptive through a function. The calculation formula for the adaptive convolution kernel size is:
[0062]
[0063] In the formula, K represents the adaptive convolution kernel size; Represents the adaptive convolution kernel size;
[0064] C is the characteristic dimension of the input channel, b = 1, γ = 2, |x odd Represented as the nearest odd integer to x.
[0065] The average pooling layer is used to reduce the size of feature maps, reduce the amount of calculation and required video memory, and speed up model reasoning capabilities;
[0066] The Dropout layer is used to connect the feature extraction network of ECA-MobileNetV2. During the forward propagation process of the model training stage, the activation values of certain neurons are stopped with a certain probability, which can effectively avoid overfitting and improve detection accuracy.
[0067] The Linear layer is used to further map the feature map abstracted by the convolution to a label space of a specific dimension to obtain the loss or output the prediction result.
[0068] An overhead ground wire detection system suitable for a patrol and repair robot, comprising an image display module, a detection and recognition module, a robot monitoring module and a human-computer interaction interface;
[0069] The image display module is used to transmit the images captured by the camera deployed on the robot to the public screen for display. The public screen is also provided with a defect detection signal light, which flashes to remind the operator when a defect is detected.
[0070] The detection and recognition module stores the ECA-MobileNetV2 model and can load the ECA-MobileNetV2 model for target recognition;
[0071] The robot monitoring module can monitor the status of the robot and cooperate with the image display module and the detection and recognition module to control the robot to move or stop;
[0072] The human-computer interaction interface is designed using WinForm in Visual Studio 2022 using C# language.
[0073] The present invention provides an overhead ground wire detection method and system suitable for a patrol and repair robot, and the technical effects are as follows:
[0074] 1) The present invention produces a ground wire defect dataset that conforms to the perspective of an overhead ground wire inspection and repair robot, and selects a lightweight detection model (MobileNetV2 model) for improvement in the absence of a GPU, and proposes a detection model that integrates the ECA attention mechanism in the MobileNetV2 model: the ECA-MobileNetV2 model. The complexity of this detection model is not much different from that of the MobileNetV2 model, but it has a better detection effect for overhead ground wire defect detection. The various detection indicators in the embodiment show that the improved model can better complete actual detection in engineering applications.
[0075] 2) The present invention designs an automatic defect detection system based on the overhead ground wire inspection robot combined with the ECA-MobileNetV2 model. The robot automatically stops when a defect is detected, making the robot more intelligent.
[0076] 3) The present invention designs a host computer interface according to the automatic defect detection system and the robot workflow, which is beneficial for operators to monitor the robot and realize human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0077] Figure 1 The present invention is a flow chart of an overhead ground wire detection method applicable to a patrol and repair robot.
[0078] Figure 2 The model structure diagram of the overhead ground wire detection method applicable to the inspection and repair robot of the present invention.
[0079] Figure 3 This is a deep convolution process diagram of the overhead ground wire detection method suitable for an inspection and repair robot of the present invention.
[0080] Figure 4 This is a diagram of the point-by-point convolution process in the overhead ground wire detection method applicable to the inspection and repair robot of the present invention.
[0081] Figure 5 This is a diagram of the standard convolution process in the overhead ground wire detection method applicable to the inspection and repair robot of the present invention.
[0082] Figure 6 This is a structural diagram of the ECA model in the overhead ground wire detection method applicable to the patrol and repair robot of the present invention.
[0083] Figure 7 This is a parameter interface configuration diagram of the overhead ground wire detection system applicable to the inspection and repair robot of the present invention.
[0084] Figure 8This is a diagram of the homepage interface of the overhead ground wire detection system applicable to the inspection and repair robot of the present invention.
[0085] Fig. 9 The present invention is a flowchart of the overhead ground wire detection system applicable to the inspection and repair robot.
[0086] Fig.10 The present invention is a flow chart of detecting defects of an overhead ground wire detection system applicable to a patrol and repair robot. DETAILED DESCRIPTION
[0087] Example:
[0088] like Figure 1 As shown, the present invention provides a flow chart of an overhead ground wire detection method applicable to a patrol and repair robot, comprising the following steps:
[0089] S1, obtain an overhead ground wire defect data set, and preprocess the data set to obtain a training set and a verification set of overhead ground wire defect images;
[0090] S2. Selecting training equipment, training parameters and evaluation indicators for the detection model;
[0091] S3, select a suitable lightweight detection algorithm;
[0092] S4. Improve the algorithm to enhance its ability to identify overhead ground wires.
[0093] The step of S1 comprises the following steps:
[0094] S11. The present invention builds an experimental platform based on the inspection and repair robot's perspective, and installs obstacles such as shock-absorbing hammers on the simulated transmission line, while simulating the damage of the overhead ground wire in the natural environment;
[0095] S12, deploying a video recording device on the inspection and repair robot, and operating the robot to walk normally on the line and record video, and selecting different weather and light conditions to walk and record video multiple times;
[0096] The benefits of selecting different weather conditions for image acquisition in S12 are: enabling the defect recognition model to achieve better learning effects on the defects of overhead ground wires and enhancing the generalization ability of the model;
[0097] S13, sampling the obtained video data frame by frame, and screening the images;
[0098] S14, cropping the image, retaining the ground line portion in the middle, and removing irrelevant information on both sides of the image;
[0099] The benefits of S14 are: reducing the input image size, reducing the computational cost, and improving the performance and stability of the target detection algorithm; by focusing on the core area of the overhead ground wire, the algorithm can more attentively analyze and identify targets related to the inspection task, reducing the processing of irrelevant information, thereby improving the accuracy and efficiency of recognition;
[0100] S15, data augmentation, vertical flipping, horizontal flipping and adding Gaussian noise to the image;
[0101] The benefits of S15 are: improving the size and quality of the data set, making the model better adaptable to various scenarios and changes, and enhancing the generalization ability of the model.
[0102] Furthermore, the formulas for vertical flipping and horizontal flipping are:
[0103]
[0104]
[0105] Where (x, y) represents the pixel coordinates in the source image, (x′, y′) represents the pixel coordinates after flipping, and f H is the height of the original image, f W is the original image width;
[0106] Furthermore, the formula of the Gaussian noise is:
[0107]
[0108] f(x) represents the probability density of Gaussian noise, x represents the pixels of the overhead ground wire image, σ represents the standard deviation, and u represents the mean;
[0109] S16. The final data set of the present invention has 4068 pictures, and the data set is divided into a training set and a validation set according to an 8:2 ratio.
[0110] The step of S2 comprises the following steps:
[0111] S21. Select a suitable device for model training. In order to save training time, the model training device selected by the present invention is: the operating system uses Window11 64-bit, the CPU is Intel Core i9-12900H, the GPU is NVIDIAGeForce RTX 3060Laptop, the running memory is 16G, the CUDA is CUDA11.6, and the development language is Python3.8;
[0112] S22. Select appropriate experimental parameters before training. The parameters selected in the present invention are: batch size is 32, initial learning rate is set to 0.001, momentum is set to 0.9, optimizer is Adam, and the number of training rounds is set to 200 rounds;
[0113] S23, evaluation indicators:
[0114] The evaluation indicators of the present invention are precision (P), recall (R), F1 score, accuracy (Accuracy), parameter quantity (Params), computational amount GFLOPs and model size, wherein precision, recall F1 score, and accuracy evaluate the detection effect of the model, and parameter quantity and computational amount evaluate the complexity of the model. Their formulas are as follows:
[0115]
[0116] Params=IK 2 C+C
[0117] Where: T P It indicates the number of positive samples correctly predicted by the model as positive, T N Indicates the number of negative samples correctly predicted by the model as negative, F P Indicates the number of negative samples that the model incorrectly predicts as positive, F N It indicates the number of positive samples that the model incorrectly predicts as negative samples; I is the input size, K is the convolution kernel size, and C is the output size.
[0118] The step of S3 includes the following steps:
[0119] S31. According to the detection effect comparison model, the present invention uses the same data set to train the network structures of five different frameworks, namely MobileNetV2, GoogLeNet, ShuffleNet, AlexNet and ResNet, and compares the precision, recall rate, F1 score and accuracy of overhead ground wire detection by different models; the experimental results are shown in Table 1 below:
[0120] Table 1 Precision, recall, F1 score and accuracy of different models for overhead ground wire detection
[0121]
[0122] From Table 1 above, we can see that MobileNetV2 outperforms other models in terms of precision, recall, F1 score and accuracy, and the detection effects are all above 90%, making it more suitable for overhead ground wire defect detection.
[0123] S32. Comprehensively evaluate the complexity of five different models: MobileNetV2, GoogLeNet, ShuffleNet, AlexNet and ResNe, and compare the final number of parameters, amount of computation and model size of the models; the comparison results are shown in Table 2 below:
[0124] Table 2 Comparison of different model parameters, computational complexity and model size
[0125]
[0126] From the data in Table 2 above, we know that the GoogLeNet network model, AlexNet network model and ResNet network model are large networks in terms of parameter quantity, computational complexity and model size, which are not conducive to deployment under limited equipment conditions; the MobileNetV2 network model and ShuffleNet network model are lightweight networks in terms of parameter quantity, computational complexity and model size, which are conducive to deployment under limited equipment conditions; although the complexity of the ShuffleNet network is slightly lower than that of the MobileNetV2, according to Table 1, the ShuffleNet network has a poor effect on overhead ground wire defect detection;
[0127] Therefore, through comprehensive consideration of detection effect and model complexity, the present invention finally chooses to use the MobileNetV2 network model for subsequent improvements to achieve better detection effect.
[0128] The algorithm is improved in S4. Specifically, an attention mechanism is added to the MobileNetV2 network to enable the model to better understand the image data, thereby improving the performance and effect of the model. The specific steps are as follows:
[0129] S41. Add three attention mechanisms, ECA, CBAM and SE, to the network structure respectively, compare the detection effects of the network models after adding the three attention mechanisms, and compare the precision, recall rate, F1 score and accuracy of the three improved models for overhead ground wire detection; the comparison results are shown in Table 3:
[0130] Table 3 Comparison of precision, recall, F1 score and accuracy of three improved models for overhead ground wire detection
[0131]
[0132] From the data in Table 3 above, we can see that adding the ECA attention mechanism to the MobileNetV2 network has the best effect on overhead ground wire detection, with the precision rate increasing from 92.3% to 95.75%, an increase of 3.45%; the recall rate increasing from 91.61% to 95.39%, an increase of 3.78%; the F1 score value increasing from 91.95% to 95.73%, an increase of 3.78%; the accuracy rate increasing from 92.31% to 95.71%, an increase of 3.4%; and all indicators have increased to more than 95%, which is of great help to practical engineering applications;
[0133] S42. Comparing the model complexity of the three improved models described in S41, and comparing the final number of parameters, amount of calculation and model size of the models, the experimental results are shown in Table 4 below:
[0134] Table 4 Comparison of parameters, computational complexity and model size of the three improved models
[0135]
[0136] It can be seen from Table 4 above that after the ECA attention mechanism is added to the MobileNetV2 model, the model size is the smallest, and the number of parameters and the amount of computation are basically unchanged. When the SE attention mechanism or CBAM attention mechanism is added to the MobileNetV2 model, the number of parameters and the amount of computation are increased to a certain extent; the ECA attention mechanism is lighter than the global pooling operation of SE and CBAM, and has smaller computational complexity and parameters. Therefore, the introduction of the ECA module in the MobileNetV2 network model requires less computing resources;
[0137] Further, Figure 2 The structure diagram of the ECA-MobileNetV2 detection model includes a convolutional layer, an enhanced feature linear bottleneck layer, an average pooling layer in a feature extraction network, and a Dropout layer and a Linear layer in a classification network;
[0138] Furthermore, the convolution layer includes a 1×1 convolution and a ReLU activation function. The main function of this module is to extract features. The input feature map first undergoes a convolution operation with a convolution kernel of 1×1, and then undergoes a ReLU activation function. The formula of the ReLU activation function is as follows:
[0139] ReLU(x)=max(x,0)
[0140] ReLU provides a very simple nonlinear transformation. Given an element x, the ReLU activation function is defined as the maximum value of the element and 0.
[0141] Furthermore, the enhanced feature linear bottleneck layer is a linear bottleneck layer structure with an ECA attention mechanism added;
[0142] Furthermore, the linear bottleneck layer structure is a bottleneck structure using linear activation at the end of the convolution in the inverted residual, wherein the activation function is a ReLU activation function; this structure can reduce the loss of nonlinear features and retain low-dimensional information;
[0143] Furthermore, the image processing process of the inverted residual structure is as follows: the input feature map is first subjected to a 1×1 convolution to increase the feature dimension, then a 3×3 depth-separable convolution is used for filtering, and finally a 1×1 convolution is used to reduce the feature dimension to obtain the output feature of this layer, and then the input feature and the output feature are added; the process of the inverted residual structure can be called "compression-convolution-expansion"; this structure can improve the ability of gradient propagation between network layers;
[0144] Furthermore, the depthwise separable convolution divides the convolution into two independent steps: depthwise convolution and pointwise convolution, as Figure 3 The characteristic of the deep convolution is that a convolution kernel only performs convolution on one channel and only outputs the feature map of one channel. The calculation amount of the deep convolution is H×W×K 2 ×M; where H represents the height of each channel, W represents the width of each channel, K represents the convolution kernel size, and M represents the input channel; Figure 4 The processing process of the input feature map by point-by-point convolution is similar to the standard convolution, such as Figure 5 As shown, the convolution kernel size is 1×1. The working process is to align the convolution kernel with each position of the input image, perform element-by-element multiplication and sum, and then put the output result into the position corresponding to the output feature map. The point-by-point convolution calculation amount is H×W×M×N, where N represents the output channel; therefore, the calculation of the depth-separable convolution is: H×W×K 2 ×M+H×W×M×N; while the standard convolution calculation amount is H×W×K 2 ×M×N, and the standard convolution kernel is usually 3×3; generally, N in the depthwise separable convolution and the standard convolution is a large value. At this time, it can be seen that the computational complexity of the depthwise separable convolution is only one eighth of that of the standard convolution; the depthwise separable convolution greatly reduces the number of parameters and kernel computations while maintaining the representation capability of the model;
[0145] Furthermore, if Figure 6 The figure shows the structure of the ECA attention mechanism model. The workflow of the attention mechanism is as follows: first, the input feature map is globally averaged pooled, then a one-dimensional convolution operation with a convolution kernel size of K is performed, and then a sigmoid activation function is used to limit the calculated attention weights to the range of [0,1], and finally these weights are multiplied with the original feature response to obtain the weighted feature representation;
[0146] The calculation formula of the above sigmoid activation function is:
[0147]
[0148] The core mechanism of the ECA attention mechanism is to calculate the attention weight in each channel through a one-dimensional convolution operation to dynamically adjust the importance of the channel, strengthen the model's attention to important features, and reduce the attention to minor features; the convolution kernel size of the one-dimensional convolution can be freely selected, and it is self-adaptive through a function. The calculation formula for the adaptive convolution kernel size is:
[0149]
[0150] Where C is the characteristic dimension of the input channel, b = 1, γ = 2, |x odd Expressed as the nearest odd number to x;
[0151] Furthermore, the average pooling layer has the following functions: reducing the size of feature maps, reducing the amount of computation and required video memory, and accelerating the model reasoning capability;
[0152] Furthermore, the Dropout layer is connected to the feature extraction network of ECA-MobileNetV2. During the forward propagation process in the model training phase, the Dropout layer stops the activation values of certain neurons with a certain probability, which can effectively avoid overfitting and improve detection accuracy. The Linear layer is used to further map the feature map abstracted by the convolution to a label space of a specific dimension in order to obtain the loss or output the prediction result.
[0153] The present invention also provides an overhead ground wire detection system suitable for a patrol and repair robot, comprising an image display module, a detection and recognition module, a robot monitoring module and a human-computer interaction interface;
[0154] Furthermore, the image display module is responsible for transmitting the images captured by the camera deployed on the robot to the public screen for display. The public screen is equipped with a defect detection signal light, which flashes to remind the operator when a defect is detected.
[0155] Furthermore, the detection and recognition module stores the ECA-MobileNetV2 model and can load the ECA-MobileNetV2 model for target recognition;
[0156] Furthermore, the robot monitoring module can monitor the status of the robot and cooperate with the first two modules to control the robot to move or stop;
[0157] Furthermore, the human-computer interaction interface is designed using WinForm in Visual Studio 2022 using C# language;
[0158] Further, the human-computer interaction interface, please refer to Figure 7 , Figure 8 The present invention produces a human-machine interaction interface according to the inspection robot requirements. First, the system prompts the user to configure parameters, such as Figure 7 As shown in the figure; in this interface, you need to configure the IP address of the input camera to realize remote login; and open the pre-set serial port to complete the communication with the robot; after clicking Save configuration, you can enter the main interface of human-computer interaction; as shown in the figure Figure 8 As shown, the main interface can monitor the robot's movements and status, display the real-time image of the camera, provide operations such as capturing pictures and recording videos, and click the defect detection switch button to select whether to detect defects;
[0159] Furthermore, the defect detection switch button is located on the right side of the human-computer interaction main interface and is controlled by the detection and recognition module. When the automatic detection function is turned on, the detection and recognition module loads the ECA-MobileNetV2 model for target recognition;
[0160] Furthermore, the image display module displays the images captured by the camera in two large picture frames in the middle of the human-computer interaction main interface. The camera connected to the upper picture frame is used to detect overhead ground wire defects, and the camera connected to the lower picture frame is used to locate overhead ground wire defects. The defect detection signal light is located in the upper picture frame of the main interface. When a defect is detected, the defect detection signal light will flash, thereby reminding the staff to operate the robot to repair the defect.
[0161] Furthermore, if Fig. 9 The figure shows the workflow of the whole system. First, the operator enters the parameter configuration interface to configure the parameters and connect the robot. If the connection is unsuccessful, the operator continues to configure the parameters until the connection is successful. After the connection is successful, the operator enters the main interface of the human-computer interaction interface and clicks the defect detection switch button to select whether to perform defect detection. If the operator selects no, the inspection operation is performed manually. If the operator selects yes, the robot enters the automatic inspection state. After entering the automatic inspection state, the defect detection and recognition module loads the ECA-MobileNetV2 detection model. When the detection model detects a defect, the robot stops moving and the defect detection signal light flashes. When no defect is detected, the robot travels normally.
[0162] Furthermore, the detection model detects defects in the following process: Fig.10As shown, when a defect is detected, the detection and identification module gives a signal, which is transmitted to the robot monitoring module. The robot monitoring module terminates the robot's moving motion and gives a robot stop command to feedback to the image display module. Finally, the defect detection signal light in the upper image frame flashes, prompting the operator to find a defect in the overhead ground wire.
Claims
1. A method for detecting overhead ground wires suitable for a patrol and repair robot, characterized in that The following steps are involved: Step 1: Obtain the overhead ground wire defect dataset and preprocess the dataset to obtain the training set and verification set of overhead ground wire defect images; Step 2: Select the training equipment and training parameters of the detection model, and set the evaluation indicators; Step 3: Select a lightweight detection model; Step 4: Improve the lightweight detection model to enhance its ability to identify overhead ground wires.
2. The method for detecting an overhead ground wire suitable for an inspection and repair robot according to claim 1, characterized in that: The step 1 comprises the following steps: S1.
1. Build an experimental platform based on the inspection and repair robot’s perspective, install the anti-vibration hammer obstacle on the simulated transmission line, and simulate the damage of the overhead ground wire in the natural environment; S1.
2. Deploy the video recording device on the inspection and repair robot, and operate the robot to walk normally on the line and record the video. At the same time, choose to walk and record the video multiple times under different weather and light conditions; S1.3, sampling the obtained video data frame by frame, and screening the images; S1.4, crop the image, keep the ground line part in the middle, and remove irrelevant information on both sides of the image; S1.5, data augmentation, vertical flipping, horizontal flipping and adding Gaussian noise to the image; S1.
6. Divide the dataset into training set and validation set according to the ratio of 8:
2.
3. The method for detecting an overhead ground wire suitable for an inspection and repair robot according to claim 2, characterized in that: In S1.5, the formulas for vertical flipping and horizontal flipping are: Where: (x, y) represents the pixel coordinates in the source image, (x′, y′) represents the pixel coordinates after flipping, and f H is the height of the original image, f W is the original image width; The formula for Gaussian noise is: Where: f(x) represents the probability density of Gaussian noise, x represents the pixel of the overhead ground wire image, σ represents the standard deviation, and u represents the mean.
4. The method for detecting an overhead ground wire suitable for an inspection and repair robot according to claim 1, characterized in that: The step 2 comprises the following steps: S2.
1. Select appropriate equipment for model training; S2.
2. Select appropriate experimental parameters before training; S2.3, Evaluation indicators: Evaluation indicators include precision (P), recall (R), F1 score, accuracy, parameter quantity (Params), computational load GFLOPs and model size. Precision, recall F1 score and accuracy evaluate the detection effect of the model, while parameter quantity and computational load evaluate the complexity of the model. Their formulas are as follows: Params=I 2 C+C Where: T P It indicates the number of positive samples correctly predicted by the model as positive, T N Indicates the number of negative samples correctly predicted by the model as negative, F P Indicates the number of negative samples that the model incorrectly predicts as positive, F N It indicates the number of positive samples that the model incorrectly predicts as negative samples; I is the input size, K is the convolution kernel size, and C is the output size.
5. The method for detecting an overhead ground wire suitable for a patrol and repair robot according to claim 1, characterized in that: In step 3, the MobileNetV2 network is selected for subsequent improvement.
6. The method for detecting an overhead ground wire suitable for a patrol and repair robot according to claim 5, characterized in that: In step 4, an attention mechanism is added to the MobileNetV2 network to enable the model to better understand the image data, thereby improving the performance and effect of the model, as follows: an ECA attention mechanism is added to the MobileNetV2 network structure to obtain an ECA-MobileNetV2 network model; The ECA-MobileNetV2 network model includes a convolutional layer, an enhanced feature linear bottleneck layer, an average pooling layer in a feature extraction network, and a Dropout layer and a Linear layer in a classification network; The convolution layer contains 1×1 convolution and ReLU activation function. This module is used to extract features. The input feature map first undergoes a convolution operation with a convolution kernel of 1×1, and then undergoes a ReLU activation function. The formula of the ReLU activation function is as follows: ReLU(x)=max(x,0) The ReLU activation function provides a very simple nonlinear transformation. Given an element x, the ReLU activation function is defined as the maximum value of the element and 0.
7. The method for detecting overhead ground wires applicable to a patrol and repair robot according to claim 6, characterized in that: The enhanced feature linear bottleneck layer is a linear bottleneck layer structure that adds an ECA attention mechanism. The linear bottleneck layer structure is a bottleneck structure that uses a linear activation at the end of the convolution in the inverted residual, wherein the activation function is a ReLU activation function.
8. The method for detecting overhead ground wires applicable to a patrol and repair robot according to claim 6, characterized in that: The ECA attention mechanism workflow is as follows: First, the input feature map is subjected to a global average pooling operation, and then a one-dimensional convolution operation with a convolution kernel size of K is performed. Then, a sigmoid activation function is used to limit the calculated attention weights to the range of [0, 1]. Finally, these weights are multiplied with the original feature response to obtain the weighted feature representation. The calculation formula of the above sigmoid activation function is: e represents a natural constant, whose value is approximately 2.718; x represents an input feature map; The convolution kernel size of the one-dimensional convolution can be freely selected, and it is self-adaptive through a function. The calculation formula of the adaptive convolution kernel size is: In the formula, K represents the adaptive convolution kernel size; Represents the adaptive convolution kernel size; C is the characteristic dimension of the input channel, b = 1, γ = 2, |x odd Represented as the nearest odd integer to x.
9. The method for detecting overhead ground wires applicable to a patrol and repair robot according to claim 6, characterized in that: The average pooling layer is used to reduce the size of feature maps, reduce the amount of calculation and required video memory, and speed up the model reasoning capability; The Dropout layer is used to connect the feature extraction network of ECA-MobileNetV2; The Linear layer is used to further map the feature map abstracted by the convolution to a label space of a specific dimension to obtain the loss or output the prediction result.
10. An overhead ground wire detection system suitable for a patrol and repair robot, characterized in that: It includes image display module, detection and recognition module, robot monitoring module and human-computer interaction interface; The image display module is used to transmit the images captured by the camera deployed on the robot to the public screen for display. The public screen is also provided with a defect detection signal light, which flashes to remind the operator when a defect is detected. The detection and recognition module stores the ECA-MobileNetV2 model and can load the ECA-MobileNetV2 model for target recognition; The robot monitoring module can monitor the status of the robot and cooperate with the image display module and the detection and recognition module to control the robot to move or stop; The human-computer interaction interface is designed using WinForm in Visual Studio 2022 using C# language.
