Power transmission line construction equipment perception method and system based on improved YOLOv7 model

Abstract

A power transmission line construction equipment perception method and system based on an improved YOLOv7 model, the method first trains an improved YOLOv7 model with a feature pyramid network with a cross-connection structure and bidirectional weighted fusion based on a historical power transmission line construction equipment image set, and then inputs a real-time photographed power transmission line construction equipment image into the trained improved YOLOv7 model to obtain a perception result of the power transmission line construction equipment. The present application effectively reduces the interference of noise on the precision of the model, and improves the convergence speed and robustness of the model.

Description

Technical Field

[0001] This invention belongs to the field of power transmission line detection technology, specifically relating to a method and system for sensing power transmission line construction equipment based on the YOLOv7 model. Background Technology

[0002] With the rapid development of smart grids, transmission lines, as an important component of the power system, are seeing continuous improvement in the level of intelligence in their construction and maintenance. There is also an increasing number of publications on machine learning-based detection of transmission line equipment. For example, the paper "Aerial Insulator Detection of Transmission Lines Based on Improved Faster RCNN" (Yi Jiyu, Chen Cifa, Gong Guoqiang [J]. Computer Engineering, 2021, 47(06):292-298+304.) improves the detection accuracy of the Faster RCNN model for insulators under occlusion conditions by introducing occlusion masks to generate difficult samples for training. However, the two-stage detection algorithm Faster RCNN requires generating candidate regions first, followed by classification and localization, making its detection speed much slower than the one-stage algorithm. The paper "A Surface Defect Detection Algorithm for Wind Turbine Blades Based on HSCA-YOLOv7" (Li Bing, Bai Yunshan, Zhao Kuan, et al. A Surface Defect Detection Algorithm for Wind Turbine Blades Based on HSCA-YOLOv7 [J]. China Electric Power, 2023, 56(10):43-52.) introduces an improved spatial pyramid pooling module and proposes a hybrid spatial channel attention mechanism based on the YOLOv7 model, which improves the accuracy of the model in detecting surface defects on wind turbine blades. However, in practical situations, the YOLOv7 model still suffers from the problem of losing feature information, and the accuracy for detecting small targets needs to be improved. Summary of the Invention

[0003] The purpose of this invention is to address the aforementioned problems in the existing technology by providing a sensing method and system for transmission line construction equipment based on the YOLOv7 model.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows:

[0005] In a first aspect, this invention proposes a method for sensing transmission line construction equipment based on an improved YOLOv7 model, comprising:

[0006] S1. Training the improved YOLOv7 model based on historical images of transmission line construction equipment, including:

[0007] By improving the neck network of the YOLOv7 model, feature fusion is performed on the extracted images of transmission line construction equipment. The network weights are updated using the gradient of the loss function, and the optimization is iterated until the optimal hyperparameters are obtained. Based on the optimal hyperparameters, the trained improved YOLOv7 model is obtained.

[0008] The neck network is a pyramidal network with a cross-connection structure and bidirectional weighted fusion.

[0009] S2. Input the real-time captured images of the power transmission line construction equipment into the trained improved YOLOv7 model to obtain the perception results of the power transmission line construction equipment.

[0010] The cross-connection structure includes: a second CBS module of the convolutional layer in the neck network is connected to a first Concat module of the higher layers;

[0011] The bidirectional weighted fusion includes top-down weighted fusion and bottom-up weighted fusion. Each feature fusion is followed by a separable convolutional layer, and a specific fusion weight allocation strategy is adopted. The weighted fusion process is shown in the following equation:

[0012]

[0013] In the above formula, P i out P represents the input and output characteristics of the i-th layer, respectively. i td Let ω be an intermediate feature of the i-th layer, i = 3, 4, 5, and D(·) be a depthwise separable convolution operation. j Let ε be the fusion weight of the j-th fusion node, and let ε be a local minimum value.

[0014] The specific fusion weight allocation strategy includes: each feature fusion node is assigned an initial fusion weight value. When the network performs backpropagation, the gradient of each fusion weight can be calculated based on the loss function. Then, the fusion weight values ​​are updated according to the gradient descent method. This process is iterated until the maximum number of iterations is reached. The update formula for the fusion weight is:

[0015]

[0016] In the above formula, ω l ω l ′ represents the fusion weights before and after the update of the l-th fusion node. For learning rate, This is the gradient of the loss function with respect to the fusion weights.

[0017] The loss function is an EIOU loss function improved based on Huber loss, and its expression is as follows:

[0018]

[0019] In the above formula, L Huber-EIoU Let L be the loss function. δ(IoU,1) is the Huber loss based on the intersection-union ratio (IoU), ρ(b,b) gt ρ(w,w) represents the Euclidean distance between the center point of the predicted bounding box and the center point of the actual bounding box. gt ρ(h,h) represents the Euclidean distance between the predicted bounding box width and the actual bounding box width. gt ) represent the Euclidean distance between the predicted bounding box height and the actual bounding box height, respectively. w c h , where are the width and height of the minimum bounding rectangle, respectively, and δ is the threshold for controlling the loss function to change from mean squared error to mean absolute error.

[0020] The training described in S1 also includes:

[0021] The MP module in the neck network is used to extract features from images of power transmission line construction equipment. The MP module adopts an improved channel prior attention mechanism module with high efficiency channel attention mechanism module. The final feature map F output by this module is... icpca for:

[0022]

[0023] In the above formula, SA(F) ca F represents the output feature map of the high-efficiency channel attention mechanism module. ca The input channel prior attention mechanism module outputs a feature map, where DwConv is a depthwise convolution, and Branch is a branch. i For the i-th branch, and Branch0 is a residual connection, Conv 1×1 This is a convolution operation with a kernel size of (1,1).

[0024] The output feature map F ca It was obtained based on the following method:

[0025] The input feature map F of dimension H×W×C is subjected to global average pooling to obtain a feature map of size 1×1×C. This feature map is then subjected to a 1D convolution with kernel size k and a sigmoid function, followed by weight normalization to obtain another feature map of size 1×1×C. Multiplying this feature map with the input feature map yields the output feature map F. ca The weights ω and the kernel size k are calculated according to the following formulas:

[0026] ω=σ(CID k (y))

[0027]

[0028] In the above formula, σ is the sigmoid activation function, CID is the one-dimensional convolution, y is the channel feature, C is the number of channels in the feature map, a and b are hyperparameters, and odd is the odd property used to filter out variables with odd absolute values.

[0029] Secondly, this invention proposes a perception system for transmission line construction equipment based on an improved YOLOv7 model, including an improved YOLOv7 model construction module, an improved YOLOv7 model training module, and a transmission line construction equipment perception module.

[0030] The improved YOLOv7 model building module is used to build an improved YOLOv7 model, wherein the neck network of the improved YOLOv7 model is a feature pyramid network with a cross-connection structure and bidirectional weighted fusion.

[0031] The improved YOLOv7 model training module is used to train the constructed improved YOLOv7 model based on historical images of transmission line construction equipment, including:

[0032] The extracted features of the transmission line construction equipment images are fused using a neck network, and the network weights are updated using the gradient of the loss function. The optimization is iterated until the optimal hyperparameters are obtained. Based on the optimal hyperparameters, a trained improved YOLOv7 model is obtained.

[0033] The transmission line construction equipment perception module is used to input real-time captured images of the transmission line construction equipment into a trained improved YOLOv7 model to obtain the perception results of the transmission line construction equipment.

[0034] The cross-connection structure includes: a second CBS module of the convolutional layer in the neck network is connected to a first Concat module of the higher layers;

[0035] The bidirectional weighted fusion includes top-down weighted fusion and bottom-up weighted fusion. Each feature fusion is followed by a separable convolutional layer, and a specific fusion weight allocation strategy is adopted. The weighted fusion process is shown in the following equation:

[0036]

[0037]

[0038] In the above formula, P i out P represents the input and output characteristics of the i-th layer, respectively. i td Let ω be an intermediate feature of the i-th layer, i = 3, 4, 5, and D(·) be a depthwise separable convolution operation. jLet ε be the fusion weight of the j-th fusion node, and let ε be a local minimum value.

[0039] The specific fusion weight allocation strategy includes: each feature fusion node is assigned an initial fusion weight value. When the network performs backpropagation, the gradient of each fusion weight can be calculated based on the loss function. Then, the fusion weight values ​​are updated according to the gradient descent method. This process is iterated until the maximum number of iterations is reached. The update formula for the fusion weight is:

[0040]

[0041] In the above formula, ω l ω l ′ represents the fusion weights before and after the update of the l-th fusion node. For learning rate, This is the gradient of the loss function with respect to the fusion weights.

[0042] The loss function is an EIOU loss function improved based on Huber loss, and its expression is as follows:

[0043]

[0044] In the above formula, L Huber-EIoU Let L be the loss function. δ (IoU,1) is the Huber loss based on the intersection-union ratio (IoU), ρ(b,b) gt ρ(w,w) represents the Euclidean distance between the center point of the predicted bounding box and the center point of the actual bounding box. gt ρ(h,h) represents the Euclidean distance between the predicted bounding box width and the actual bounding box width. gt ) represent the Euclidean distance between the predicted bounding box height and the actual bounding box height, respectively. w c h , where are the width and height of the minimum bounding rectangle, respectively, and δ is the threshold for controlling the loss function to change from mean squared error to mean absolute error.

[0045] The training also includes:

[0046] Feature extraction is performed on images of power transmission line construction equipment based on the MP module in the neck network. The MP module in the neck network adopts an improved channel prior attention mechanism module with high efficiency channel attention mechanism module. The final feature map F output by this module is... icpca for:

[0047]

[0048] In the above formula, SA(F) ca F represents the output feature map of the high-efficiency channel attention mechanism module. caThe input channel prior attention mechanism module outputs a feature map, where DwConv is a depthwise convolution, and Branch is a branch. i For the i-th branch, and Branch0 is a residual connection, Conv 1×1 This is a convolution operation with a kernel size of (1,1).

[0049] The output feature map F ca It was obtained based on the following method:

[0050] The input feature map F of dimension H×W×C is subjected to global average pooling to obtain a feature map of size 1×1×C. This feature map is then subjected to a 1D convolution with kernel size k and a sigmoid function, followed by weight normalization to obtain another feature map of size 1×1×C. Multiplying this feature map with the input feature map yields the output feature map F. ca The weights ω and the kernel size k are calculated according to the following formulas:

[0051] ω=σ(CID k (y))

[0052]

[0053] In the above formula, σ is the sigmoid activation function, CID is the one-dimensional convolution, y is the channel feature, C is the number of channels in the feature map, a and b are hyperparameters, and odd is the odd property used to filter out variables with odd absolute values.

[0054] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0055] 1. The proposed method for sensing equipment in power transmission line construction based on an improved YOLOv7 model uses a feature pyramid network with a cross-connection structure and bidirectional weighted fusion. On the one hand, by connecting the CBS module of the convolutional layer to the Concat module of the higher layer, this cross-connection structure allows some features output by the backbone network to skip the intermediate layers of the neck network, thereby reducing information redundancy and loss. On the other hand, based on this cross-connection structure, a bidirectional weighted fusion method is introduced, and a specific fusion weight allocation strategy is adopted to ensure that features from different levels can remain balanced during the fusion process. These measures effectively reduce the interference of noise on the model accuracy.

[0056] 2. The present invention proposes a method for sensing transmission line construction equipment based on an improved YOLOv7 model. This method replaces the CIOU loss function used in the traditional YOLOv7 model with an EIOU loss function improved based on Huber loss. This solves the problem that the CIOU loss function cannot effectively express the situation when the aspect ratio of the predicted box and the actual box is the same but the width and height values ​​are different, thereby improving the convergence speed and robustness of the model.

[0057] 3. The present invention proposes a perception method for transmission line construction equipment based on an improved YOLOv7 model. The MP module in the neck network adopts an improved channel prior attention mechanism module with a high-efficiency channel attention mechanism module to improve the expressive ability of the target equipment, reduce the interference of complex transmission line scenes on the perception model, and improve the model's target perception accuracy for transmission line construction equipment. Attached Figure Description

[0058] Figure 1 This is a network structure diagram of the improved YOLOv7 model described in Example 1.

[0059] Figure 2 The results show the comparison of loss values ​​for each scheme in Example 1.

[0060] Figure 3 The results show a comparison of the detection effects of each scheme in Example 1.

[0061] Figure 4 This is a structural diagram of the system described in Example 2. Detailed Implementation

[0062] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings.

[0063] This invention proposes a sensing method for transmission line construction equipment based on an improved YOLOv7 model. The network structure of the improved YOLOv7 model is as follows: Figure 1As shown. The neck network of this model is a Feature Pyramid Network (ScFPN) with a cross-connection structure. The convolutional layers consist of three CBS modules, the intermediate layers consist of two UPSample modules, two Concat modules, one CBS module, and one ELAN module, and the higher layers consist of two Concat modules, two MP modules, and three ELAN modules. The first CBS module in the convolutional layers is connected to the C3 layer in the backbone network and the second Concat module in the intermediate layers; the second CBS module in the convolutional layers is connected to the C4 layer in the backbone network, the first Concat module in the intermediate layers, and also to the first Concat module in the higher layers; the third CBS module in the convolutional layers is connected to the SPPCS module in the backbone network. The PC module and the first UPSample module of the intermediate layer are connected. The first UPSample module of the intermediate layer is sequentially connected to the first Concat module of the intermediate layer, the ELAN module of the intermediate layer, the CBS module of the intermediate layer, the second UPSample module of the intermediate layer, the second Concat module of the intermediate layer, the first ELAN module of the upper layer, the first MP module of the upper layer, the first Concat module of the upper layer, the second ELAN module of the upper layer, the second MP module of the upper layer, the second Concat module of the upper layer, and the third ELAN module of the upper layer. All three ELAN modules of the upper layer are connected to the head network, and the SPCSPC module in the backbone network is connected to the second Concat module of the upper layer. Through ScFPN, each node has the characteristics of bidirectional feature fusion and skip connection. At the same time, path weights (i.e., fusion weights of fusion nodes) are added. The network can learn to autonomously choose the degree of acceptance of each path, thereby having the ability to distinguish the importance of each path feature and retain the small target features of important transmission line construction equipment.

[0064] This invention also designs an improved channel prior attention mechanism (ICPCA) module with high efficiency channel attention mechanism to improve the convolutional pooling module (MP) of the YOLOv7 neck network, so as to improve the expressive ability of the target device and thus improve the model's target perception ability of power transmission line construction equipment.

[0065] The method proposed in this invention can accurately sense the construction equipment of each transmission line, maintain a high confidence level even when there is obstruction, and better balance detection accuracy and speed, significantly improving the real-time performance and accuracy of equipment sensing.

[0066] Example 1:

[0067] A method for sensing transmission line construction equipment based on an improved YOLOv7 model is proposed. This method targets four types of transmission line construction equipment: angle steel towers, steel pipe towers, cranes, and excavators, and is implemented sequentially according to the following steps:

[0068] 1. Constructing such Figure 1 The improved YOLOv7 model shown.

[0069] 2. Denoising and color correction are performed on the sample set of historical images of various types of transmission line construction equipment to obtain a high-quality image dataset of transmission line construction equipment, and the type of transmission line construction equipment is labeled in the images.

[0070] 3. Rotate and scale the processed images of power transmission line construction equipment to expand the dataset, and divide the dataset into training and testing sets.

[0071] 4. Train the improved YOLOv7 model by inputting the training set, including:

[0072] 4.1 Feature extraction is performed on images of transmission line construction equipment based on the MP module in the neck network. The MP module in the neck network adopts a channel prior attention mechanism module CPCA (abbreviated as ICPCA), which is an improvement on the high-efficiency channel attention mechanism module ECA. The final feature map F output by ICPCA is... icpca for:

[0073]

[0074] In the above formula, SA(F) ca F represents the output feature map of the high-efficiency channel attention mechanism module. ca The input channel prior annotation mechanism module outputs a feature map, where DwConv is a depthwise convolution, and Branch is used. i For the i-th branch, and Branch0 is a residual connection, Conv 1×1 For convolution operations with a kernel size of (1,1);

[0075] The output feature map F ca It was obtained based on the following method:

[0076] The ECA attention mechanism mainly consists of three modules: Global Average Pooling (GAP), a fully connected layer, and a Sigmoid activation function. First, a feature map F of size H×W×C is input. After global average pooling, a feature map of size 1×1×C is obtained. This feature map is then subjected to a 1D convolution with kernel size k and a Sigmoid function, followed by weight normalization, resulting in another feature map of size 1×1×C. Multiplying this feature map with the input feature map yields the output feature map F. ca The weights ω and the kernel size k are calculated according to the following formulas:

[0077] ω=σ(CID k (y))

[0078]

[0079] In the above formula, σ is the sigmoid activation function, CID is the one-dimensional convolution, y is the channel feature, C is the number of channels in the feature map, a and b are hyperparameters, and odd is the odd property used to filter out variables with odd absolute values.

[0080] 4.2. Feature fusion is performed on the extracted transmission line construction equipment image features using the neck network ScFPN. The feature fusion is a bidirectional weighted fusion, including top-down weighted fusion and bottom-up weighted fusion. Each feature fusion is followed by a separable convolutional layer. The input of each feature fusion node in the ScFPN network is assigned a fusion weight, that is, a weighted sum is performed after each fusion. By upsampling, we can ensure that... Having the same resolution, weighted feature fusion is performed followed by a depthwise separable convolutional layer to obtain P3. out .and This is from P3 out After downsampling, and Weighted feature fusion is performed, followed by a depthwise separable convolutional layer to obtain the final feature. The specific feature fusion process of ScFPN is shown in the following equation:

[0081]

[0082]

[0083] In the above formula, P i out P represents the input and output characteristics of the i-th layer, respectively. i td Let ω be an intermediate feature of the i-th layer, i = 3, 4, 5, and D(·) be a depthwise separable convolution operation. j Let be the fusion weight of the j-th fusion node;

[0084] For the fusion weight ω of each fusion node j A specific fusion weight allocation strategy is employed, including: each feature fusion node is assigned an initial fusion weight value; the network weights are adjusted using a fast normalization formula, which divides the weight of each feature by the sum of all weight values ​​to improve the training stability of the network; during backpropagation, the gradient of each fusion weight is calculated based on the loss function, and the fusion weight values ​​are updated using gradient descent, iterating continuously until the maximum number of iterations is reached. The update formula for the fusion weights is:

[0085]

[0086] In the above formula, ω l ω l ′ represents the fusion weights before and after the update of the l-th fusion node. For learning rate, Let be the gradient of the loss function with respect to the fusion weights;

[0087] 4.3 Update network weights using the gradient of the improved EIOU loss function based on Huber loss:

[0088]

[0089] In the above formula, L Huber-EIoU Let L be the loss function. δ (IoU,1) represents the Huber loss based on the Intersection over Union (IoU). When the IoU is close to 1, meaning the overlap between the predicted and ground truth bounding boxes is high, the loss function represents the mean squared error. When the IoU is far from 1, the loss function represents the mean absolute error, ρ(b,b). gt ρ(w,w) represents the Euclidean distance between the center point of the predicted bounding box and the center point of the actual bounding box. gt ρ(h,h) represents the Euclidean distance between the predicted bounding box width and the actual bounding box width. gt ) represent the Euclidean distance between the predicted bounding box height and the actual bounding box height, respectively. w c h , respectively, are the width and height of the minimum bounding rectangle, and δ is the threshold for controlling the loss function to change from mean squared error to mean absolute error;

[0090] 4.4. Continuously iterate and optimize until the optimal hyperparameters are obtained;

[0091] 4.5 Input the optimal hyperparameters into the model to obtain the trained improved YOLOv7 model.

[0092] 5. Use the test set to verify the effectiveness of the trained improved YOLOv7 model.

[0093] 6. First, transmit the images of power transmission line construction equipment captured in real time by the drone to the cloud server for data processing. Then, input the processed images into the trained improved YOLOv7 model to obtain the perception results of various power transmission line construction equipment.

[0094] The evaluation index values ​​of the sensing results of various transmission line construction equipment are shown in Table 1:

[0095] Table 1 Evaluation Indicators of Perception Results for Various Transmission Line Construction Equipment

[0096] Equipment type Accuracy (%) Recall rate (%) Average accuracy (%) Angle steel tower 90.07 85.03 93.31 steel pipe tower 90.35 85.93 92.78 crane 94.38 92.11 92.97 excavator 96.73 89.47 94.12 All types 92.88 88.14 92.30 .

[0097] To examine the effectiveness of the method described in this invention, a transmission line construction equipment perception method based on the Faster-RCNN model was used as Scheme 1, a transmission line construction equipment perception method based on the YOLOv7 model was used as Scheme 2, and a transmission line construction equipment perception method based on the improved YOLOv7 model described in Example 1 was used as Scheme 3. The following comparative tests were conducted:

[0098] (1) Comparison of loss values

[0099] The comparison results of the loss values ​​of the above schemes are as follows: Figure 2 As shown in the figure, the loss value of Scheme 1 gradually converges after 353 iterations, eventually converging to around 0.191; Scheme 2 converges faster, starting to converge after 364 iterations and eventually reaching around 0.238; while Scheme 3 converges rapidly to 0.059 after only 187 iterations, with the final loss value also decreasing to 0.051. Compared to Schemes 1 and 2, the method proposed in this invention can converge to a lower loss value at a faster speed, demonstrating a significant advantage in perception accuracy.

[0100] (2) Comparison of detection results

[0101] The comparison results of the detection effects of the above schemes are as follows: Figure 3 As shown. (Through) Figure 3 As can be seen, the Faster-RCNN algorithm used in Scheme 1, a two-stage object detection algorithm, can capture the features and shape of objects more precisely, and has certain advantages in detecting multi-scale objects. Its accuracy, recall, and mean precision are all above 85%, but its detection speed is relatively slow, reaching only 36.86 FPS. In contrast, Scheme 2, using YOLOv7 as a single-stage object detection algorithm, has higher accuracy, mean precision, and faster detection speed, reaching 91.56%, 87.32%, and 48.13 FPS respectively, representing improvements of 3.24%, 1.89%, and 30.58% compared to Scheme 1. However, its recall rate is 85.34%, a decrease of 1.92% compared to Scheme 1. This is because YOLOv7 is weaker in detecting small objects and is prone to missed detections. The improved YOLOv7 model proposed in Scheme 3 balances detection accuracy and speed. Its accuracy is improved by 8.41% and 5.17% compared to Schemes 1 and 2, respectively; its recall is improved by 2.21% and 4.13%, respectively; and its mean precision is improved by 8.69% and 6.80%, respectively. Simultaneously, Scheme 3 is 22.73% faster than Scheme 1, but 6.00% slower than Scheme 2. This is because Scheme 3 improves upon the traditional YOLOv7 model based on the ICPCA attention mechanism, enhancing the model's ability to capture spatial dependencies.

[0102] Example 2:

[0103] like Figure 4 As shown, a transmission line construction equipment perception system based on an improved YOLOv7 model includes an improved YOLOv7 model construction module, an improved YOLOv7 model training module, and a transmission line construction equipment perception module.

[0104] The improved YOLOv7 model building module is used to build, for example... Figure 1 The improved YOLOv7 model shown here has a neck network that is a feature pyramid network with a cross-connection structure and bidirectional weighted fusion. The cross-connection structure includes a second CBS module of the convolutional layer in the neck network connected to a first Concat module of the higher layers.

[0105] The improved YOLOv7 model training module is used to train the constructed improved YOLOv7 model based on historical images of transmission line construction equipment, and includes the following steps:

[0106] (1) Feature extraction is performed on images of power transmission line construction equipment based on the MP module in the neck network. The MP module in the neck network adopts a channel prior attention mechanism module improved by a high-efficiency channel attention mechanism module. The feature map F output by this module is the final feature map F. icpca for:

[0107]

[0108] In the above formula, SA(F ca F represents the output feature map of the high-efficiency channel attention mechanism module. ca The input channel prior attention mechanism module outputs a feature map, where DwConv is a depthwise convolution, and Branch is a branch. i For the i-th branch, and Branch0 is a residual connection, Conv 1×1 This is a convolution operation with a kernel size of (1,1).

[0109] The output feature map F ca It was obtained based on the following method:

[0110] The input feature map F of dimension H×W×C is subjected to global average pooling to obtain a feature map of size 1×1×C. This feature map is then subjected to a 1D convolution with kernel size k and a sigmoid function, followed by weight normalization to obtain another feature map of size 1×1×C. Multiplying this feature map with the input feature map yields the output feature map F. ca The weights ω and the kernel size k are calculated according to the following formulas:

[0111] ω=σ(CID k (y))

[0112]

[0113] In the above formula, σ is the sigmoid activation function, CID is the one-dimensional convolution, y is the channel feature, C is the number of channels in the feature map, a and b are hyperparameters, and odd is the odd property used to filter out variables with odd absolute values.

[0114] (2) Feature fusion is performed on the extracted transmission line construction equipment image features through the neck network. The feature fusion is a bidirectional weighted fusion, including top-down weighted fusion and bottom-up weighted fusion. Each feature fusion is followed by a separable convolutional layer. The input of each feature fusion node in the ScFPN network is assigned a fusion weight, that is, a weighted sum is performed once for each fusion. By upsampling, we can ensure that... Having the same resolution, after weighted feature fusion, it is passed through a depthwise separable convolutional layer to obtain and Then it is by After downsampling, and Weighted feature fusion is performed, followed by a depthwise separable convolutional layer to obtain the final feature. The specific feature fusion process of ScFPN is shown in the following equation:

[0115]

[0116] In the above formula, P i out P represents the input and output characteristics of the i-th layer, respectively. i td Let ω be an intermediate feature of the i-th layer, i = 3, 4, 5, and D(·) be a depthwise separable convolution operation. j Let be the fusion weight of the j-th fusion node;

[0117] For the fusion weight ω of each fusion node j A specific fusion weight allocation strategy is employed, including: each feature fusion node is assigned an initial fusion weight value; the network weights are adjusted using a fast normalization formula, which divides the weight of each feature by the sum of all weight values ​​to improve the training stability of the network; during backpropagation, the gradient of each fusion weight is calculated based on the loss function, and the fusion weight values ​​are updated using gradient descent, iterating continuously until the maximum number of iterations is reached. The update formula for the fusion weights is:

[0118]

[0119] In the above formula, ω l ω l ′ represents the fusion weights before and after the update of the l-th fusion node. For learning rate, Let be the gradient of the loss function with respect to the fusion weights;

[0120] (3) Update the network weights using the gradient of the loss function, iteratively optimizing until the optimal hyperparameters are obtained. The loss function is an improved EIOU loss function based on Huber loss, and its expression is:

[0121]

[0122] In the above formula, L Huber-EIoU Let L be the loss function. δ (IoU,1) is the Huber loss based on the intersection-union ratio (IoU), ρ(b,b) gt ρ(w,w) represents the Euclidean distance between the center point of the predicted bounding box and the center point of the actual bounding box. gt ρ(h,h) represents the Euclidean distance between the predicted bounding box width and the actual bounding box width. gt ) represent the Euclidean distance between the predicted bounding box height and the actual bounding box height, respectively. w c h , respectively, are the width and height of the minimum bounding rectangle, and δ is the threshold for controlling the loss function to change from mean squared error to mean absolute error;

[0123] (4) The improved YOLOv7 model is trained based on the optimal hyperparameters.

[0124] The transmission line construction equipment perception module is used to input images of the transmission line construction equipment captured in real time by the UAV into a trained improved YOLOv7 model to obtain the perception results of the transmission line construction equipment.

Claims

1. A method for sensing transmission line construction equipment based on an improved YOLOv7 model, characterized in that: include: S1. Training the improved YOLOv7 model based on historical images of transmission line construction equipment, including: Feature extraction of images of transmission line construction equipment is performed by improving the MP module in the neck network of the YOLOv7 model. The extracted features of the transmission line construction equipment images are then fused through the neck network. The network weights are updated using the gradient of the loss function, and the optimization is iterated until the optimal hyperparameters are obtained. The improved YOLOv7 model is then trained based on the optimal hyperparameters. The neck network is a characteristic pyramid network with a cross-connection structure and bidirectional weighted fusion. The MP module employs an improved channel prior attention mechanism module based on a high-efficiency channel attention mechanism module. The feature map ultimately output by this module is... for: ； ； In the above formula, The output feature map of the high-efficiency channel attention mechanism module The input channel prior attention mechanism module outputs a feature map. For depthwise convolution, For the i-th branch, and For residual connection, The kernel size is Convolution operations; S2. Input the real-time captured images of the power transmission line construction equipment into the trained improved YOLOv7 model to obtain the perception results of the power transmission line construction equipment.

2. The method for sensing transmission line construction equipment based on an improved YOLOv7 model according to claim 1, characterized in that: The cross-connection structure includes: the second CBS module of the convolutional layer in the neck network is connected to the first Concat module of the higher layer; The bidirectional weighted fusion includes top-down weighted fusion and bottom-up weighted fusion. Each feature fusion is followed by a separable convolutional layer, and a specific fusion weight allocation strategy is adopted. The weighted fusion process is shown in the following equation: ； ； ； ； In the above formula, , These are the input and output characteristics of the i-th layer, respectively. For the intermediate characteristics of the i-th layer, , For depthwise separable convolution operations, Let be the fusion weight of the j-th fusion node. It is a local minimum; The specific fusion weight allocation strategy includes: each feature fusion node is assigned an initial fusion weight value. When the network performs backpropagation, the gradient of each fusion weight can be calculated based on the loss function. Then, the fusion weight values ​​are updated according to the gradient descent method. This process is iterated until the maximum number of iterations is reached. The update formula for the fusion weight is: ； In the above formula, , The first The fusion weights of each fusion node before and after the update. For learning rate, This is the gradient of the loss function with respect to the fusion weights.

3. A method for sensing transmission line construction equipment based on an improved YOLOv7 model according to claim 1 or 2, characterized in that: The loss function is an EIOU loss function improved based on Huber loss, and its expression is as follows: ； ； In the above formula, For loss function, For intersection-union ratio Huber's losses To predict the Euclidean distance between the center point of the bounding box and the center point of the actual bounding box, To predict the Euclidean distance between the frame width and the actual frame width, These are the Euclidean distances between the predicted bounding box height and the actual bounding box height, respectively. , These are the width and height of the minimum bounding rectangle, respectively. The threshold used to control the loss function from mean squared error to mean absolute error.

4. The method for sensing transmission line construction equipment based on an improved YOLOv7 model according to claim 1, characterized in that: The output feature map It was obtained based on the following method: enter dimensional feature map After global average pooling, a size of [size missing] is obtained. Feature map; the resulting feature map is processed by a convolution kernel of size [missing information]. The 1D convolution and sigmoid function are applied, and after weight normalization, a result of size is obtained. The feature map is obtained by multiplying the obtained feature map with the input feature map to obtain the output feature map. , where the weight kernel size The following formula is used for calculation: ； ； In the above formula, It is the sigmoid activation function. For one-dimensional convolution, As a channel feature, The number of channels in the feature map. , For hyperparameters, This property, which is odd-numbered, is used to filter out variables whose absolute value is odd.

5. A sensing system for transmission line construction equipment based on an improved YOLOv7 model, characterized in that: This includes improvements to the YOLOv7 model building module, the YOLOv7 model training module, and the power transmission line construction equipment perception module; The improved YOLOv7 model building module is used to build an improved YOLOv7 model, wherein the neck network of the improved YOLOv7 model is a feature pyramid network with a cross-connection structure and bidirectional weighted fusion. The improved YOLOv7 model training module is used to train the constructed improved YOLOv7 model based on historical images of transmission line construction equipment, including: The MP module in the neck network of the improved YOLOv7 model is used to extract features from images of transmission line construction equipment. The extracted features are then fused using the neck network. The network weights are updated using the gradient of the loss function, and the optimization is iterated until the optimal hyperparameters are obtained. The improved YOLOv7 model is then trained based on the optimal hyperparameters. The MP module employs an improved channel prior attention mechanism module based on a high-efficiency channel attention mechanism module. The feature map ultimately output by this module is... for: ； ； In the above formula, The output feature map of the high-efficiency channel attention mechanism module The input channel prior attention mechanism module outputs a feature map. For depthwise convolution, For the i-th branch, and For residual connection, The kernel size is Convolution operations; The transmission line construction equipment perception module is used to input real-time captured images of the transmission line construction equipment into a trained improved YOLOv7 model to obtain the perception results of the transmission line construction equipment.

6. The sensing system for transmission line construction equipment based on an improved YOLOv7 model according to claim 5, characterized in that: The cross-connection structure includes: the second CBS module of the convolutional layer in the neck network is connected to the first Concat module of the higher layer; The bidirectional weighted fusion includes top-down weighted fusion and bottom-up weighted fusion. Each feature fusion is followed by a separable convolutional layer, and a specific fusion weight allocation strategy is adopted. The weighted fusion process is shown in the following equation: ； ； ； ； In the above formula, , These are the input and output characteristics of the i-th layer, respectively. For the intermediate characteristics of the i-th layer, , For depthwise separable convolution operations, Let be the fusion weight of the j-th fusion node. It is a local minimum; The specific fusion weight allocation strategy includes: each feature fusion node is assigned an initial fusion weight value. When the network performs backpropagation, the gradient of each fusion weight can be calculated based on the loss function. Then, the fusion weight values ​​are updated according to the gradient descent method. This process is iterated until the maximum number of iterations is reached. The update formula for the fusion weight is: ； In the above formula, , The first The fusion weights of each fusion node before and after the update. For learning rate, This is the gradient of the loss function with respect to the fusion weights.

7. A sensing system for transmission line construction equipment based on an improved YOLOv7 model according to claim 5 or 6, characterized in that: The loss function is an EIOU loss function improved based on Huber loss, and its expression is as follows: ； ； In the above formula, For loss function, For intersection-union ratio Huber's losses To predict the Euclidean distance between the center point of the bounding box and the center point of the actual bounding box, To predict the Euclidean distance between the frame width and the actual frame width, These are the Euclidean distances between the predicted bounding box height and the actual bounding box height, respectively. , These are the width and height of the minimum bounding rectangle, respectively. The threshold used to control the loss function from mean squared error to mean absolute error.

8. The sensing system for transmission line construction equipment based on an improved YOLOv7 model according to claim 5, characterized in that: The output feature map It was obtained based on the following method: enter dimensional feature map After global average pooling, a size of [size missing] is obtained. Feature map; the resulting feature map is processed by a convolution kernel of size [missing information]. The 1D convolution and sigmoid function are applied, and after weight normalization, a result of size is obtained. The feature map is obtained by multiplying the obtained feature map with the input feature map to obtain the output feature map. , where the weight kernel size The following formula is used for calculation: ； ； In the above formula, It is the sigmoid activation function. For one-dimensional convolution, As a channel feature, The number of channels in the feature map. , For hyperparameters, This property, which is odd-numbered, is used to filter out variables whose absolute value is odd.

Images