A method for recognizing line environment scenes for tension insulators based on deep vision

Abstract

The present invention discloses a system for recognizing a scene in an environment of a tension insulator line based on deep vision, comprising: a robot vision module based on a pan / tilt platform, for collecting images of an overhead line environment in which the robot is located; an offline training module, for annotating and labeling the collected visual images using a deep neural network to construct an original data set, and for performing step-by-step parameter training and evaluation based on an improved YOLOv5 detection algorithm; a real-time scene analysis module, for detecting scene information in an overhead line environment in real time based on a trained model, and the tension insulator detection robot can adjust its motion planning algorithm at the parameter and strategy levels through the real-time scene information to improve the efficiency and effect of the robot's movement and operation on the overhead line.

Description

Technical Field

[0001] The invention relates to a method for recognizing a tension insulator line environment scene based on depth vision, and belongs to the technical field of robot vision recognition equipment. Background Art

[0002] In the power system, the tension insulator is an important insulating element, which can effectively isolate the live wire from the ground and keep the overhead lines in place, playing an important role in electrical insulation and mechanical support. However, due to various reasons, the tension insulator may be damaged or aged during use, which will bring huge risks to the safe operation of the power system, so the tension insulator needs to be regularly inspected and maintained.

[0003] Traditional maintenance of tension insulators mainly relies on manual inspections, which is time-consuming and labor-intensive and cannot effectively obtain environmental information in real time. In addition, traditional monitoring methods cannot accurately identify overhead lines in complex environments. To address this problem, various transmission line tension insulator detection robots have emerged. Such robots can replace manual work on overhead lines and perform health checks on insulators. However, at present, most tension insulator detection robots rely on manual remote control and can only move along the conductors on overhead lines without obstacles. However, when equipment such as shock-absorbing hammers and spacers appear on overhead lines, it is difficult for robots to cross these obstacles of different sizes and continue to work. In order to improve work efficiency and reduce risks, it is necessary to develop robots that can autonomously identify scenes on overhead lines and take corresponding actions.

[0004] Intelligence is an important development direction in the field of electric power robots. The perception technology based on visual sensors is becoming more and more mature, and its application in the field of mobile robot motion planning is becoming more and more extensive. For the visual system of the tension insulator inspection robot, its motion planning algorithm will be adjusted accordingly from the parameter to the strategy level for different overhead line environments. Therefore, accurate recognition of scene information in the robot's visual input has high application value and significance. Summary of the invention

[0005] The purpose of the present invention is to provide a method for recognizing scene of a tension insulator line environment based on depth vision, accurately recognize scene information in robot vision input, and provide technical support for the intelligent operation of the tension insulator detection robot on the line.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A method for recognizing a tension insulator line environment scene based on deep vision comprises the following steps:

[0008] Step S1: using a robot vision module based on a pan / tilt system to collect an image of the overhead line environment where the robot is located;

[0009] Step S2: Perform offline training based on the offline training module, use the deep neural network to annotate and generate labels for the collected visual images to construct the original data set, and perform step-by-step training and evaluation of the neural network parameters; the specific working steps are: (1) Collect image data of different scenes under the overhead line environment to form a label set T = (t1, t2, ..., tn) of the scene category, where tn represents the nth scene corresponding to the pipeline; (2) Select multiple scene image data under different labels to form an original image data set P = (p1, p2, ..., pn), where pn represents the nth original image data, and divide the data set into a training set, a validation set, and a test set in a ratio of 8:1:1; (3) Use the improved target detection algorithm YOLOv5 to extract, transfer and fuse the sample data, and iterate repeatedly until the termination condition is met to obtain the trained model;

[0010] Step S3: Perform real-time scene analysis, detect the scene information in the overhead line environment in real time based on the trained model, output the scene category of the overhead line environment where the tension insulator detection robot is located, and adjust the parameters and strategies of the robot's motion planning algorithm according to the detection results to improve the efficiency and effectiveness of the robot's movement and operation on the overhead line.

[0011] Furthermore, the offline training module described in step S2 includes a data acquisition unit, a sample annotation unit and a deep network parameter training and evaluation unit; the data acquisition unit and the sample annotation unit are mainly used to extract discrete time series training samples, balance the distribution of samples of each category, and assign labels according to several scene types of 0-conductor (obstacle-free), 1-shock hammer, 2-vertical insulator, 3-horizontal insulator, 4-pole tower, 5-spacer, 6-tension clamp, and 7-abnormal scene.

[0012] Furthermore, the deep network parameter training and evaluation unit of the offline training module in step S2 is implemented based on the improved YOLOv5 algorithm; the specific improvements are: integrating the attention mechanism unit into the original network, improving the decoupling head mechanism, and optimizing the loss function.

[0013] The improved YOLOv5 detection network includes a backbone feature extraction network (Backbone), a path aggregation network (Neck), and a prediction network (Head). The overall network detection process is as follows:

[0014] (1) Backbone feature extraction network Backbone is responsible for extracting information features from the input sample data. The network architecture includes CBS module, CONV module, C3 module and SPP module. Among them, the SiLU activation function is used in the CBS module, and the formula is as follows:

[0015]

[0016] (2) The path aggregation network Neck uses a combination of upsampling and downsampling to achieve the fusion of feature information of different scales. The fused feature layer is passed to the prediction network Head, and the redundant information is eliminated by non-maximum suppression (NMS) to obtain the scene prediction category with a confidence level higher than the set threshold.

[0017] The integration of the attention mechanism unit is as follows: add the SE attention mechanism in the path aggregation network Neck to filter irrelevant information and enhance the attention between feature channels, thereby improving recognition accuracy; the operation of the SE attention mechanism module is divided into squeeze and excitation:

[0018] (1) Squeezing is to perform global average pooling on the input feature map with C channels to generate a 1*1*C vector so that each value has the global receptive field of the channel. The calculation method is as follows:

[0019]

[0020] Where Z represents the average eigenvalue of the feature map, F sq represents the squeeze operation, H represents the feature map height, W represents the feature map width, and u(i,j) represents a single eigenvalue of the feature map;

[0021] (2) Incentive, two fully connected layers are used for adaptive learning and weights are generated. The calculation method is as follows:

[0022] S=F ex =σ(w2×δ(w1×Z))

[0023] Where S represents the weight, F ex represents the excitation operation, w1 and w2 are the weights of the two fully connected layers, × represents the product in the fully connected layer, δ represents the Relu activation function of the connection layer, and σ represents the Sigmoid function;

[0024] The decoupling head mechanism is improved as follows: a task-specific context decoupling head mechanism (Task-Specific Context Decoupling, TSCODE) is introduced to fuse features from deep layers in classification tasks and embed rich semantic information into the current feature map, thereby improving the overall performance of the model;

[0025] The optimization of the loss function is as follows: based on the cross entropy loss function used by the original YOLOv5 network, a random optimization cross entropy improvement method is introduced, combined with adaptive smoothing parameters and mutation operations.

[0026] The cross entropy loss function calculation formula is:

[0027]

[0028] Among them, H(p,q) represents the cross entropy loss, m is the amount of sample data, n is the number of labels, and P(x i ) is the true distribution of the target, Q(x i ) is the model prediction probability distribution;

[0029] The parameter update strategy of the random optimization cross entropy improvement method is as follows:

[0030] V k+1 =α1V ck +α2V gk +α3V k-1

[0031] Where V ck 、V gk are the distribution parameters of the current elite sample and the global elite sample in the kth iteration, V k-1 is the sample distribution parameter in the k-1th iteration, V k+1 is the sample distribution parameter in the k+1th iteration, α1, α2, α3 represent smoothing parameters;

[0032] Furthermore, the adaptive smoothing parameter adjustment formula is:

[0033]

[0034] α2=1-α1-α3 0.5≤α1≤0.7

[0035] Among them, K N is the total number of iterations set in advance, K is the number of current iterations, α3 max , α3 min Represents the maximum and minimum values ​​of α3.

[0036] Furthermore, a mutation operation is used to perturb the updated parameters in each iteration process, thereby increasing the randomness of the samples and reducing the probability of the algorithm falling into a local optimum. The perturbation function used is as follows:

[0037]

[0038] in, represents the sample variance in the K+1th iteration when no disturbance is applied, Represents the variance of the sample after the perturbation, Z k+1 represents the perturbation added in the K+1th iteration.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] (1) The present invention uses a deep convolutional neural network to recognize scene information in the insulator line environment, which can effectively identify the overhead line environment information where the current tension insulator detection robot is located, facilitate rapid adjustment of its motion planning strategy, and has good robustness and real-time performance, and can meet the requirements of intelligent applications.

[0041] (2) The present invention improves the YOLOv5 algorithm, integrates the attention mechanism unit, and optimizes the decoupling head mechanism and loss function to achieve irrelevant information filtering, deep feature fusion, and sample parameter adjustment strategy optimization to improve the recognition accuracy of the model in the overhead line environment, which has important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the environmental scene recognition of the tension insulator line of the present invention;

[0043] Figure 2 It is the overall structure diagram of the detection network;

[0044] Figure 3 It is the module structure diagram of SE attention mechanism;

[0045] Figure 4 It is a schematic diagram of the deep feature fusion process of the TSCODE decoupling head. DETAILED DESCRIPTION

[0046] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0047] like Figure 1 As shown, the present invention provides a method for identifying a tension insulator line environment scene based on depth vision, comprising the following steps:

[0048] S1: Use the robot vision module based on the pan-tilt structure to collect images of the overhead line environment where the robot is located; in this embodiment, this is accomplished by a binocular camera equipped on the pan-tilt structure mounted on the robot, which can observe scene images in front and behind the line when the robot moves along the overhead line, and transmit the image information to the real-time scene analysis module to achieve real-time detection of the state of the scene around the robot.

[0049] S2: Offline training module, using deep neural network to annotate and generate labels for the collected visual images to construct the original data set, and perform step-by-step training and evaluation of neural network parameters. The specific working steps are: (1) Collect image data of different scenes in the overhead line environment to form a label set T = (t1, t2, ..., tn) of the scene category, where tn represents the nth scene corresponding to the pipeline; (2) Select multiple scene image data under different labels to form an original image data set P = (p1, p2, ..., pn), where pn represents the nth original image data, and divide the data set into training set, validation set and test set according to the ratio of 8:1:1; (3) Use the improved target detection algorithm YOLOv5 to extract, transfer and fuse the sample data, and iterate repeatedly until the termination condition is met to obtain the model with the best relative detection effect.

[0050] S3: Real-time scene analysis module, which detects scene information in the overhead line environment in real time based on the trained model, outputs the scene category of the overhead line environment where the tension insulator inspection robot is located, and adjusts the robot's motion planning algorithm at the parameter and strategy levels based on the detection results to improve the efficiency and effectiveness of the robot's movement and operation on the overhead line.

[0051] The data set involved in the S2 module in this embodiment comes from the self-collected insulator line environment scene data set. The data set has a total of 1403 images, including eight scene types, namely 0-conductor (obstacle-free), 1-shock hammer, 2-vertical insulator, 3-horizontal insulator, 4-tower, 5-spacer, 6-tension clamp, and 7-abnormal scene. Considering that the sample size of the data set is not large, in order to prevent overfitting during model training, data enhancement technology is used to randomly rotate, translate, scale and flip the original images, and finally the sample size is expanded to 5000.

[0052] The deep network parameter training and evaluation unit of the S2 module is mainly based on the improved YOLOv5 algorithm for functional implementation. The specific improvement directions are: integrating the attention mechanism unit in the original network, improving the decoupling head mechanism, and optimizing the loss function.

[0053] like Figure 2 As shown in the figure, the detection network includes a backbone feature extraction network (Backbone), a path aggregation network (Neck) and a prediction network (Head). The overall network detection process is as follows:

[0054] (1) Backbone is responsible for extracting information features from the input sample data. The network architecture mainly includes CBS module, CONV module, C3 module and SPP module. Among them, the SiLU activation function is used in the CBS module, and the formula is as follows:

[0055]

[0056] (2) Neck uses a combination of upsampling and downsampling to achieve the fusion of feature information of different scales. The fused feature layer is passed to the Head network, and redundant information is eliminated through non-maximum suppression (NMS) to obtain the scene prediction category with a confidence level higher than the set threshold.

[0057] In this embodiment, the integration of the attention mechanism unit is mainly reflected in: adding the SE attention mechanism in the Neck network to filter irrelevant information and enhance the attention between feature channels, thereby improving recognition accuracy. Figure 3 As shown, the operation of the SE module can be mainly divided into Squeeze and Excitation:

[0058] (1) Squeezing refers to performing global average pooling on the input feature map with C channels to generate a 1*1*C vector. This is equivalent to each value having the global receptive field of the channel it belongs to. The calculation method is as follows:

[0059]

[0060] Where Z represents the average eigenvalue of the feature map, F sq represents a squeeze operation, H represents the feature map height, W represents the feature map width, and u(i, j) represents a single eigenvalue of the feature map.

[0061] (2) Incentive, which refers to using two fully connected layers for adaptive learning and generating weights. The calculation method is as follows:

[0062] S=F ex =σ(w2×δ(w1×Z))

[0063] Where S represents the weight, F ex represents the excitation operation, w1 and w2 are the weights of the two fully connected layers, × represents the product in the fully connected layer, δ represents the Relu activation function of the connection layer, and σ represents the Sigmoid function;

[0064] The Relu activation function formula is as follows:

[0065] δ(x)=max(0,x)

[0066] The Sigmiod activation function formula is as follows:

[0067]

[0068] max(0, x) represents the maximum value operation between x and 0.

[0069] In this embodiment, the improvement of the decoupling header mechanism is mainly reflected in: introducing a task-specific context decoupling header mechanism (Task-Specific Context Decoupling, TSCODE). Figure 4 As shown in the figure, the TSCODE mechanism can fuse features from deep layers in classification tasks and embed rich semantic information into the current feature map, thereby improving the overall performance of the model.

[0070] In terms of loss function optimization in this embodiment, based on the cross entropy loss function used by the original YOLOv5 network, a random optimization cross entropy improvement method is introduced, combined with adaptive smoothing parameters and mutation operations.

[0071] The cross entropy loss function is a commonly used loss function in classification problems. Its calculation formula is:

[0072]

[0073] Among them, H(p, q) represents the cross entropy loss, m is the amount of sample data, n is the number of labels, P(x i ) is the true distribution of the target, Q(x i ) is the model prediction probability distribution.

[0074] The improved random optimization cross entropy method further refines the concept of elite samples in the cross entropy loss function, and its parameter update strategy is shown in the following formula:

[0075] V k+1 =α1V ck +α2V gk +α3V k-1

[0076] Where V ck 、V gk are the distribution parameters of the current elite sample and the global elite sample in the kth iteration, V k-1 is the sample distribution parameter in the k-1th iteration, V k+1 is the sample distribution parameter in the k+1th iteration, and α1, α2, and α3 represent smoothing parameters.

[0077] The adaptive smoothing parameter adjustment formula is:

[0078]

[0079] α2=1-α1-α3 0.5≤α1≤0.7

[0080] Where KN is the total number of iterations set in advance, and K is the number of current iterations. max , α3 minRepresents the maximum and minimum values ​​of α3.

[0081] Furthermore, a mutation operation is used to perturb the updated parameters in each iteration, thereby increasing the randomness of the samples and reducing the probability of the algorithm falling into a local optimum. The perturbation function used is as follows:

[0082]

[0083] in, represents the sample variance in the K+1th iteration when no disturbance is applied, Represents the variance of the sample after the perturbation, Z k+1 represents the perturbation added in the K+1th iteration.

[0084] Based on the improved YOLOv5 detection algorithm, the model parameters are trained and evaluated step by step, the number of iterations is set to 300, and the initial learning rate is set to 0.01. The trained detection model is used to detect the scene status in the overhead line environment in real time, and the corresponding environmental category information is output. The tension insulator detection robot can use this information to adjust its motion planning algorithm at the parameter and strategy level to improve the efficiency and effect of the robot's movement and operation on the overhead line.

[0085] In addition to the above embodiments, the present invention may also have other implementation modes. Any technical solutions formed by equivalent replacement or equivalent transformation shall fall within the protection scope required by the present invention.

Claims

1. A method for recognizing a tension insulator line environment scene based on depth vision, characterized in that: The following steps are involved: Step S1: using a robot vision module based on a pan / tilt system to collect an image of the overhead line environment where the robot is located; Step S2: Perform offline training based on the offline training module, use the deep neural network to annotate and generate labels for the collected visual images to construct the original data set, and perform step-by-step training and evaluation of the neural network parameters; The specific working steps are as follows: (1) Collect image data of different scenes in the overhead line environment to form a label set T = (t1, t2, ..., tn) of the scene category, where tn represents the nth scene corresponding to the pipeline; (2) Select multiple scene image data under different labels to form an original image data set P = (p1, p2, ..., pn), where pn represents the nth original image data, and divide the data set into a training set, a validation set, and a test set in a ratio of 8:1:1; (3) Use the improved target detection algorithm YOLOv5 to extract, transfer and fuse features of the sample data, and iterate repeatedly until the termination condition is met to obtain the trained model; The improved target detection algorithm YOLOv5 has been improved by integrating the attention mechanism unit into the original network, improving the decoupling head mechanism, and optimizing the loss function. The improved target detection algorithm YOLOv5 detection network includes the backbone feature extraction network Backbone, the path aggregation network Neck and the prediction network Head. The overall network detection process is: ① The backbone feature extraction network Backbone is responsible for extracting information features from the input sample data. The network architecture includes CBS module, CONV module, C3 module and SPP module. Among them, the SiLU activation function is used in the CBS module, and the formula is as follows: ② The path aggregation network Neck uses a combination of upsampling and downsampling to achieve the fusion of feature information of different scales. The fused feature layer is passed to the prediction network Head, and redundant information is eliminated by non-maximum suppression to obtain the scene prediction category with a confidence level higher than the set threshold; The integration of the attention mechanism unit is as follows: before the multi-scale fusion features processed in the path aggregation network Neck are passed to the prediction network Head, the SE attention mechanism module is introduced after the CBS module of CSP2_1 to filter irrelevant information and enhance the attention between feature channels, thereby improving recognition accuracy; the operation of the SE attention mechanism module is divided into squeezing and excitation: (1) Squeezing is to perform global average pooling on the input feature map with C channels to generate a 1*1*C vector so that each value has the global receptive field of the channel. The calculation method is as follows: Where Z represents the average eigenvalue of the feature map, F sq represents the squeeze operation, H represents the feature map height, W represents the feature map width, and u(i,j) represents a single eigenvalue of the feature map; (2) Incentive, two fully connected layers are used for adaptive learning and weights are generated. The calculation method is as follows: S=F ex =σ(w2×δ(w1×Z)) Where S represents weight, Fe x represents the excitation operation, w1 and w2 are the weights of the two fully connected layers, × represents the product in the fully connected layer, δ represents the Relu activation function of the connection layer, and σ represents the Sigmoid function; The decoupling head mechanism is improved as follows: a context decoupling head mechanism for specific tasks is introduced to fuse features from deep layers in classification tasks and embed rich semantic information into the current feature map, thereby improving the overall performance of the model; The optimization of the loss function is as follows: based on the cross entropy loss function used by the original YOLOv5 network, a random optimization cross entropy improvement method is introduced, combined with adaptive smoothing parameters and mutation operations; The cross entropy loss function calculation formula is: Among them, H(p,q) represents the cross entropy loss, m is the amount of sample data, n is the number of labels, and P(x i ) is the true distribution of the target, Q(x i ) is the model prediction probability distribution; The parameter update strategy of the random optimization cross entropy improvement method is as follows: V k+1 =α1V ck +α2V gk +α3V k-1 Where V ck 、V gk are the distribution parameters of the current elite sample and the global elite sample in the kth iteration, V k-1 is the sample distribution parameter in the k-1th iteration, V k+1 is the sample distribution parameter in the k+1th iteration, α1, α2, α3 represent smoothing parameters; Step S3: Perform real-time scene analysis, detect the scene information in the overhead line environment in real time based on the trained model, output the scene category of the overhead line environment where the tension insulator detection robot is located, and adjust the parameters and strategies of the robot's motion planning algorithm according to the detection results to improve the efficiency and effectiveness of the robot's movement and operation on the overhead line.

2. The method for recognizing a tension insulator line environment scene based on depth vision according to claim 1, characterized in that: The offline training module described in step S2 includes a data acquisition unit, a sample annotation unit and a deep network parameter training and evaluation unit; the data acquisition unit and the sample annotation unit are mainly used to extract discrete time series training samples, balance the distribution of samples of each category, and assign labels according to several scene types of 0-conductor, 1-shock hammer, 2-vertical insulator, 3-horizontal insulator, 4-pole tower, 5-spacer, 6-tension clamp, and 7-abnormal scene.

3. The method for recognizing a tension insulator line environment scene based on depth vision according to claim 1, characterized in that: The adaptive smoothing parameter adjustment formula is: α2＝1-α1-α3 0.5≤α1≤0.7 Among them, K N is the total number of iterations set in advance, K is the number of current iterations, α3 max , α3 min Represents the maximum and minimum values ​​of α3.

4. The method for recognizing a tension insulator line environment scene based on depth vision according to claim 1, characterized in that: The mutation operation is used to perturb the updated parameters in each iteration process, thereby increasing the randomness of the samples and reducing the probability of the algorithm falling into the local optimum; the perturbation function used is as follows: in, represents the sample variance in the K+1th iteration when no disturbance is applied, Represents the variance of the sample after the perturbation, Z k+1 represents the perturbation added in the K+1th iteration.

Images