Power transmission line safety hidden danger monitoring method and device, fusion terminal and medium
By constructing a lightweight target YOLOv5 network model, the problems of reliance on human resources and high model complexity in the monitoring of safety hazards in power transmission lines are solved, and rapid and accurate safety hazard detection is achieved on the fusion terminal.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING SMARTCHIP MICROELECTRONICS TECHNOLOGY CO LTD
- Filing Date
- 2022-10-20
- Publication Date
- 2026-06-26
AI Technical Summary
In existing technologies, monitoring of safety hazards in power transmission lines relies on human resources, resulting in high operation and maintenance costs and safety risks. Furthermore, existing deep learning models are complex, slow in detection speed, and consume a lot of power when running on fusion terminals, making it difficult to achieve automated monitoring.
The target YOLOv5 network model is adopted, with the backbone network using Focus and Mobilenetv2 structures, the neck network using depthwise separable convolutional modules, and the detection head network using four detection branches. By combining the anchorless bounding box prediction mechanism and data augmentation training, a lightweight network model is constructed for security risk monitoring of fusion terminals.
It enables rapid, accurate, and low-latency monitoring of power line safety hazards on integrated terminals, reduces network model complexity, is suitable for edge intelligence mode, and improves detection speed and accuracy.
Smart Images
Figure CN115546726B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of power transmission line monitoring technology, and in particular to a method, device, fusion terminal and medium for monitoring safety hazards in power transmission lines. Background Technology
[0002] With the rapid development of urbanization, foreign object intrusion has gradually become a major safety hazard for power transmission lines, requiring a large investment in the inspection and maintenance of these lines.
[0003] In current technologies, monitoring safety hazards in power transmission lines primarily relies on human resources. This not only significantly increases the cost of power transmission line operation and maintenance but also risks unforeseen safety issues, seriously threatening the personal safety of maintenance personnel. Therefore, the automated and efficient monitoring of safety hazards in power transmission lines has become particularly important. Summary of the Invention
[0004] This invention aims to at least partially solve one of the technical problems in related technologies. Therefore, the first objective of this invention is to propose a method for monitoring safety hazards in transmission lines. This method can automatically monitor safety hazards in transmission lines at a fusion terminal and has advantages such as fast detection speed, high detection accuracy, and low latency.
[0005] A second objective of this invention is to provide a computer-readable storage medium.
[0006] The third objective of this invention is to propose a converged terminal.
[0007] The fourth objective of this invention is to provide a power transmission line safety hazard monitoring device.
[0008] To achieve the above objectives, a first aspect of the present invention proposes a method for monitoring safety hazards of power transmission lines, applied to a converged terminal. The method includes: acquiring a current environmental image of the power transmission line; inputting the current environmental image into a pre-trained target YOLOv5 network model, wherein the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed using four detection branches; and determining the safety hazard information of the power transmission line based on the output of the target YOLOv5 network model.
[0009] According to the transmission line safety hazard monitoring method of the present invention, the backbone network of the target YOLOv5 network model adopts the Focus structure and Mobilenetv2 structure, the neck network adopts the depth-separable convolutional module, and the detection head network adopts four detection branches to construct the network model. This realizes the lightweighting of the network model and reduces the complexity of the network model, thereby enabling automatic monitoring of transmission line safety hazards at the fusion terminal. It also has the advantages of fast detection speed, high detection accuracy, and low latency.
[0010] In some embodiments of the present invention, the target YOLOv5 network model is trained according to the following steps: acquiring environmental images of the transmission line and labeling safety hazards in the environmental images to obtain a sample set; using the sample set to train the constructed YOLOv5 network model based on the anchorless bounding box prediction mechanism and a pre-set loss function, so as to determine the model parameters of the target YOLOv5 network model.
[0011] In some embodiments of the present invention, labeling safety hazards in the environmental image to obtain a sample set includes: labeling safety hazards in the environmental image to obtain a standard file; and performing data augmentation processing on the environmental image and the standard file to obtain the sample set.
[0012] In some embodiments of the present invention, the sample set includes a training set, and training the constructed YOLOv5 network model using the sample set includes: inputting the training set into the constructed YOLOv5 network model for training to determine the training model parameters of the YOLOv5 network model; calculating the total loss of the training set according to the loss function at a preset testing frequency; stopping training when the total loss of the training set shows a trend of first decreasing and then increasing, and determining the training model parameters corresponding to the minimum value of the total loss of the training set as the model parameters of the target YOLOv5 network model.
[0013] In some embodiments of the present invention, the loss function is: L = L box +L conf +L cls Where L is the total loss, L box To predict the localization loss of the bounding box, L conf For confidence loss, L cls Classification of losses due to safety hazards.
[0014] In some embodiments of the present invention, the localization loss of the predicted bounding box is calculated using the following formula: Where N is the number of detection branches, S 2For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, For area-weighted summation, h'w' is the area of the predicted bounding box, hw is the area of the entire image, and L... CIoU The CIoU loss function is used. In some embodiments of the present invention, the confidence loss is calculated using the following formula: Where N is the number of detection branches, S 2 For each of the detection branches, B is the resolution, and B is the number of predicted bounding boxes generated by each grid at that resolution, log(|p gt -p pred |) is the cross-entropy loss function, (p) gt -p pred ) 2 For Focal weights, p gt p represents the true probability of a potential safety hazard. pred This represents the predicted probability of a potential safety hazard.
[0015] In some embodiments of the present invention, the classified loss of the safety hazard is calculated using the following formula: Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. Let C be the indicator function, and C be the category of safety hazard. This represents the true probability that the current safety hazard belongs to category c. This represents the predicted probability that the current safety hazard belongs to category c.
[0016] In some embodiments of the present invention, determining the safety hazard information of the transmission line based on the output of the target YOLOv5 network model includes: using the DIoU_NMS algorithm to remove redundant detection boxes in the output of the target YOLOv5 network model to obtain the remaining detection boxes in the output of the target YOLOv5 network model; and determining the category and confidence level of the safety hazard corresponding to the remaining detection boxes.
[0017] In some embodiments of the present invention, the categories of safety hazards include one or more of the following: construction machinery, fireworks, flames, kites, bird nests, and plastic films.
[0018] To achieve the above objectives, a second aspect of the present invention provides a computer-readable storage medium storing a power transmission line safety hazard monitoring program thereon. When the power transmission line safety hazard monitoring program is executed by a processor, it implements the power transmission line safety hazard monitoring method described in any of the above embodiments.
[0019] According to the computer-readable storage medium of the present invention, the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed with four detection branches. This achieves lightweighting of the network model and reduces its complexity, thereby enabling automatic monitoring of power transmission line safety hazards at the fusion terminal. It also has advantages such as fast detection speed, high detection accuracy, and low latency.
[0020] To achieve the above objectives, a third aspect of the present invention provides a fusion terminal, which includes a memory, a processor, and a transmission line safety hazard monitoring program stored in the memory and executable on the processor. When the processor executes the transmission line safety hazard monitoring program, it implements the transmission line safety hazard monitoring method described in any of the above embodiments.
[0021] According to the fusion terminal of the present invention, the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed with four detection branches. This achieves lightweighting of the network model and reduces its complexity, thereby enabling automatic monitoring of power transmission line safety hazards in the fusion terminal. It also has advantages such as fast detection speed, high detection accuracy, and low latency.
[0022] To achieve the above objectives, a fourth aspect of the present invention proposes a power transmission line safety hazard monitoring device, applied to a fusion terminal. The device includes: an acquisition module for acquiring a current environmental image of the power transmission line; and a determination module for inputting the current environmental image into a pre-trained target YOLOv5 network model and determining the safety hazard information of the power transmission line based on the output of the target YOLOv5 network model. The target YOLOv5 network model's backbone network uses a Focus structure and a Mobilenetv2 structure, its neck network uses a depthwise separable convolutional module, and its detection head network is constructed using four detection branches.
[0023] According to the embodiment of the present invention, the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed with four detection branches. This achieves lightweighting of the network model and reduces its complexity, thereby enabling automatic monitoring of transmission line safety hazards at the fusion terminal. It also has advantages such as fast detection speed, high detection accuracy, and low latency.
[0024] In some embodiments of the present invention, the target YOLOv5 network model is obtained through an annotation module and a training module. The annotation module is used to acquire environmental images of the transmission line and annotate safety hazards in the environmental images to obtain a sample set. The training module is used to train the constructed YOLOv5 network model using the sample set based on the anchorless bounding box prediction mechanism and a pre-set loss function to determine the model parameters of the target YOLOv5 network model.
[0025] In some embodiments of the present invention, the sample set includes a training set, and the training module is further configured to input the training set into the constructed YOLOv5 network model for training, to determine the training model parameters of the YOLOv5 network model, and to calculate the total loss of the training set according to the loss function at a preset testing frequency, and to stop training when the total loss of the training set shows a trend of first decreasing and then increasing, and to determine the training model parameters corresponding to the minimum value of the total loss of the training set as the model parameters of the target YOLOv5 network model.
[0026] In some embodiments of the present invention, the loss function is: L = L box +L conf +L cls Where L is the total loss, L box To predict the localization loss of the bounding box, L conf For confidence loss, L cls Classification of losses due to safety hazards.
[0027] In some embodiments of the present invention, the localization loss of the predicted bounding box is calculated using the following formula: Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, For area-weighted summation, h'w' is the area of the predicted bounding box, hw is the area of the entire image, and L... CIoU The CIoU loss function is used. In some embodiments of the present invention, the confidence loss is calculated using the following formula: Where N is the number of detection branches, S 2 For each of the detection branches, B is the resolution, and B is the number of predicted bounding boxes generated by each grid at that resolution, log(|p gt -p pred |) is the cross-entropy loss function, (p) gt -p pred ) 2 For Focal weights, pgt p represents the true probability of a potential safety hazard. pred This represents the predicted probability of a potential safety hazard.
[0028] In some embodiments of the present invention, the classified loss of the safety hazard is calculated using the following formula: Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. Let C be the indicator function, and C be the category of safety hazard. This represents the true probability that the current safety hazard belongs to category c. This represents the predicted probability that the current safety hazard belongs to category c.
[0029] In some embodiments of the present invention, the determining module is further configured to use the DIoU_NMS algorithm to remove redundant detection boxes in the output of the target YOLOv5 network model, obtain the remaining detection boxes in the output of the target YOLOv5 network model, and determine the category and confidence level of the security risks corresponding to the remaining detection boxes.
[0030] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description
[0031] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:
[0032] Figure 1 This is a flowchart illustrating a method for monitoring safety hazards in transmission lines according to an embodiment of the present invention;
[0033] Figure 2 This is a structural block diagram of an inverted residual module according to an embodiment of the present invention;
[0034] Figure 3 This is a flowchart illustrating a method for monitoring safety hazards in power transmission lines according to another embodiment of the present invention;
[0035] Figure 4 This is a schematic diagram of a scenario for an anchorless bounding box prediction mechanism according to an embodiment of the present invention;
[0036] Figure 5 This is a flowchart illustrating a method for monitoring safety hazards in power transmission lines according to another embodiment of the present invention;
[0037] Figure 6This is a flowchart illustrating a method for monitoring safety hazards in power transmission lines according to another embodiment of the present invention;
[0038] Figure 7 This is a flowchart illustrating a method for monitoring safety hazards in power transmission lines according to another embodiment of the present invention;
[0039] Figure 8 This is a structural block diagram of a fusion terminal according to an embodiment of the present invention;
[0040] Figure 9 This is a structural block diagram of a power transmission line safety hazard monitoring device according to an embodiment of the present invention. Detailed Implementation
[0041] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0042] The following describes in detail, with reference to the accompanying drawings, the method, apparatus, fusion terminal, and medium for monitoring safety hazards in transmission lines according to embodiments of the present invention.
[0043] Figure 1 A flowchart illustrating a method for monitoring safety hazards in transmission lines according to an embodiment of the present invention is shown. This method is applied to a converged terminal, such as... Figure 1 As shown, the method includes the following steps:
[0044] S11: Obtain the current environmental image of the transmission line;
[0045] S13: Input the current environment image into the pre-trained target YOLOv5 network model, wherein the backbone network of the target YOLOv5 network model adopts the Focus structure and Mobilenetv2 structure, the neck network adopts the depthwise separable convolutional module, and the detection head network is constructed with four detection branches;
[0046] S15: Determine the safety hazard information of the transmission line based on the output of the target YOLOv5 network model.
[0047] According to the transmission line safety hazard monitoring method of the present invention, the backbone network of the target YOLOv5 network model adopts the Focus structure and Mobilenetv2 structure, the neck network adopts the depth-separable convolutional module, and the detection head network adopts four detection branches to construct the network model. This realizes the lightweighting of the network model and reduces the complexity of the network model, thereby enabling automatic monitoring of transmission line safety hazards at the fusion terminal. It also has the advantages of fast detection speed, high detection accuracy, and low latency.
[0048] Understandably, if a deep learning + cloud computing model is used for monitoring safety hazards in power transmission lines, it requires uploading images of the current environment of the transmission lines obtained on-site to a cloud computing center via network communication. The powerful computing capabilities of the cloud computing center are then used to store and analyze these images, and relevant target detection algorithms are used to determine the safety hazards of the transmission lines. However, this method suffers from severe transmission latency and cannot handle sensitive data. Furthermore, while an "edge intelligence" model offers lower transmission latency, the complex network structure of the deep learning models in this technology leads to slow detection speeds, high power consumption, and difficulty in running on converged terminals with limited hardware resources. In other words, the existing methods for monitoring safety hazards in power transmission lines suffer from problems such as complex deep learning model network structures, slow detection speeds, high power consumption, and unsuitability for deployment on converged terminals.
[0049] In the transmission line safety hazard monitoring method of this invention, firstly, the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure. Since Mobilenetv2 can exponentially reduce the complexity of the network structure, thereby reducing storage and computational overhead and power consumption, it is suitable for converged terminals. Compared with a backbone network using a Focus structure + CSP structure, it can significantly reduce the complexity of the network structure. Secondly, the neck network of the target YOLOv5 network model adopts a depthwise separable convolutional module, which can significantly reduce the complexity of the network structure without changing the network computational accuracy. Thirdly, the detection head network of the target YOLOv5 network model adopts four detection branches, which can perform safety hazard detection on feature maps of different scales, thereby improving detection accuracy. Therefore, the transmission line safety hazard monitoring method of this invention simplifies the network model structure without reducing detection accuracy and can be applied to converged terminals to achieve automatic transmission line safety hazard monitoring based on edge intelligence.
[0050] Specifically, the Mobilenetv2 architecture consists of inverted residual modules. Please refer to... Figure 2The inverted residual module includes a first pointwise (PW) convolution, a second pointwise convolution, and a depthwise (DW) convolution. The output of the first PW convolution is used as the input of the DW convolution, and the output of the DW convolution is used as the input of the second PW convolution. The activation functions of the first PW convolution and the DW convolution are both ReLU6 activation functions, and the activation function of the second PW convolution is a linear activation function.
[0051] The neck network of the target YOLOv5 network model adopts an FPN+PAN structure. Compared with the scheme of using 3*3 convolutional modules in the neck network in related technologies, the embodiment of the present invention replaces the 3*3 convolutional modules with depth-separable convolutional modules, thereby reducing the number of parameters and computational cost.
[0052] In some embodiments, based on the three detection branches of the detection head network in related technologies, an additional detection branch is added for detecting smaller safety hazards, thus obtaining the detection head network of the target YOLOv5 network model. The resolutions of the four detection branches can be different. It is understood that a smaller resolution (e.g., 19*19) detection branch can detect larger safety hazards, while a larger resolution (e.g., 76*76) detection branch can detect smaller safety hazards. In power transmission line safety hazard monitoring, the distance between safety hazards such as cranes, tower cranes, construction machinery, fireworks, flames, kites, bird nests, and plastic films and the fusion terminal varies greatly, resulting in significant variations in the size of safety hazards in the current environmental image. The probability of smaller safety hazards appearing also increases accordingly. By adding a detection branch for detecting smaller safety hazards, the accuracy of safety hazard monitoring can be effectively improved.
[0053] In one example, the fusion terminal is mounted on a high-voltage tower and contains a pre-trained target YOLOv5 network model. After acquiring an image of the current environment of the transmission line, the fusion terminal performs data preprocessing to ensure the processed image meets the input requirements of the target YOLOv5 network model. The processed image is then input into the target YOLOv5 network model, enabling the fusion terminal to automatically determine safety hazards related to the transmission line based on the model's output. Data preprocessing includes proportional image scaling and / or padding.
[0054] Please combine Figure 3 In some embodiments of the present invention, the target YOLOv5 network model is trained according to the following steps:
[0055] S17: Obtain environmental images of the transmission line and label the safety hazards in the environmental images to obtain a sample set;
[0056] S19: Based on the anchorless bounding box prediction mechanism and the pre-set loss function, the constructed YOLOv5 network model is trained using a sample set to determine the model parameters of the target YOLOv5 network model.
[0057] Specifically, the environmental image of the transmission line can be a pre-captured image of the surrounding environment of the transmission line. The anchorless bounding box prediction mechanism uses the anchorless encoding and decoding scheme of FCOS to predict the bounding box. Compared with the anchored bounding box prediction mechanism, it avoids complex calculations, reduces the complexity of the algorithm, and improves the detection speed.
[0058] The resolution of each detection branch can be different. Taking a single detection branch as an example, when its resolution is S... 2 In this case, an image is divided into S*S grids, with each grid responsible for predicting B bounding boxes. When training the model based on the anchorless bounding box prediction mechanism, each location (x, y) on the feature map Fi is treated as a training sample, rather than an anchor box from an anchor-based detector. The location (x, y) is then mapped back to the input image, as shown below. Figure 4 As shown, the center point of the grid (x) c ,y c The ground truth (GT) bounding box of the input image is defined as {B i}, in, These are the coordinates of the top-left corner of the actual bounding box. These are the coordinates of the bottom right corner of the actual bounding box, c (i) This represents the category of safety hazards within the ground truth bounding box. If the center point of a mesh falls within the ground truth bounding box, the mesh is identified as a positive sample; otherwise, it is identified as a negative sample. The 4D real vector t* = (l*, t*, r*, b*) serves as the regression target for this location, where l*, t*, r*, and b* are the target offsets from this location to the four sides of the ground truth bounding box. The relationship between the regression target and the ground truth bounding box coordinates is shown below:
[0059]
[0060]
[0061]
[0062]
[0063] Furthermore, corresponding to the training objective, the network model of this embodiment predicts the bounding box coordinates of the 4D vector p and the 4D vector t = (l,t,r,b) of the classification label, and compares them with the true values.
[0064] Please combine Figure 5In some embodiments of the present invention, step S17 includes:
[0065] S171: Annotate safety hazards in environmental images to obtain standard documents;
[0066] S173: Perform data augmentation on environmental images and standard files to obtain a sample set.
[0067] This increases the number of samples in the sample set.
[0068] Specifically, the Lableme open-source annotation tool can be used to annotate safety hazards in environmental images to obtain standard XML files. The environment surrounding power transmission lines is complex; data augmentation of environmental images and standard files can supplement the dataset and improve the network's robustness. Data augmentation includes one or more of random horizontal flipping, random translation, random cropping, and Mosaic techniques. It is worth noting that the images and annotations generated by data augmentation are intermediate results generated during the training of the YOLOv5 network model and do not occupy storage space on the fusion terminal where the target YOLOv5 network model is deployed.
[0069] Please combine Figure 6 In some embodiments of the present invention, the sample set includes a training set, and step S19 includes:
[0070] S191: Input the training set into the constructed YOLOv5 network model for training to determine the training model parameters of the YOLOv5 network model;
[0071] S193: Calculate the total loss of the training set according to the loss function based on the pre-set test frequency;
[0072] S195: When the total loss of the training set shows a trend of first decreasing and then increasing, stop training and determine the training model parameters corresponding to the minimum total loss of the training set as the model parameters of the target YOLOv5 network model.
[0073] In one example, the training set is input into the constructed YOLOv5 network model on the server for training. Every 10 iterations, the total loss of the training set is calculated according to the loss function. When the total loss of the training set shows a trend of first decreasing and then increasing, it indicates that overfitting has occurred, and training is stopped at this point. The model that is saved when the total loss of the training set reaches the minimum value is the best prediction model. The model parameters of the best prediction model are determined as the model parameters of the target YOLOv5 network model, and the network structure of the best prediction model is determined as the network structure of the target YOLOv5 network model.
[0074] In some embodiments, the sample set also includes a test set. After step S195, the method further includes: inputting the test set into the target YOLOv5 network model, and determining the model's precision, recall, map, and other performance metrics based on the output of the target YOLOv5 network model, thereby facilitating the understanding of the actual performance metrics of the target YOLOv5 network model.
[0075] In some embodiments of the present invention, the loss function is: L = L box +L conf +L cls Where L is the total loss, L box To predict the localization loss of the bounding box, L conf For confidence loss, L cls Classification of losses due to safety hazards.
[0076] Thus, the loss function of the target YOLOv5 network model consists of three parts, which helps to improve detection accuracy.
[0077] In some embodiments of the present invention, the localization loss of the predicted bounding box is calculated using the following formula: Where N is the number of detection branches, S2 is the resolution of each detection branch, and B is the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, For area-weighted summation, h'w' is the area of the predicted bounding box, hw is the area of the entire image, and L... CIoU Let CIoU be the loss function.
[0078] Thus, by employing the CIoU loss function in the localization loss of the predicted bounding box and introducing area weighting of the predicted bounding box, it becomes easier to find the optimization objective and increase the weight of small-sized predicted bounding boxes, making the network model pay more attention to the problems in training small-sized predicted bounding boxes.
[0079] Specifically, when there is a safety hazard within the predicted bounding box at positions i, j, and k, the indicator function... The value of is 1; when there are no safety hazards within the predicted bounding boxes at i, j, and k, the indicator function is 1. The value of is 0.
[0080] In some embodiments of the present invention, the confidence loss is calculated using the following formula: Where N is the number of detection branches, S 2 For each detection branch, the resolution is given, B is the number of predicted bounding boxes generated per grid at that resolution, and log(|p gt -p pred |) is the cross-entropy loss function, (p) gt -p pred )2 For Focal weights, p gt p represents the true probability of a potential safety hazard. pred This represents the predicted probability of a potential safety hazard.
[0081] Thus, by introducing Focal weights and reducing the weight of easily classified samples, the model pays more attention to difficult samples, preventing a large number of simple samples from dominating the optimization direction of the model and solving the problem of imbalance between easy and difficult samples.
[0082] Specifically, when a real security risk exists within the predicted bounding box at points i, j, and k, p gt The value of p is 1; when there is no actual safety hazard within the predicted bounding box at points i, j, and k, p gt The value of p is 0. pred The value of is between 0 and 1.
[0083] In some embodiments of the present invention, the classification loss of safety hazards is calculated using the following formula: Where N is the number of detection branches, S 2 For each detection branch, B represents the resolution, and B represents the number of predicted bounding boxes generated per grid cell at that resolution. Let C be the indicator function, and C be the category of safety hazard. This represents the true probability that the current safety hazard belongs to category c. This represents the predicted probability that the current safety hazard belongs to category c.
[0084] Thus, the classification loss of safety hazards is calculated based on the cross-entropy loss function.
[0085] Specifically, when the security hazard within the predicted bounding box at points i, j, and k actually belongs to category c, The value is 1; when the safety hazard within the predicted bounding box at points i, j, and k does not actually belong to category c. The value of is 0. The value of is between 0 and 1.
[0086] It should be noted that p in each loss function gt p pred , and All are related to i, j, and k. To simplify the formula, i, j, and k are not shown in the formula as superscript or subscript.
[0087] Please combine Figure 7 In some embodiments of the present invention, step S15 includes:
[0088] S151: Use the DIoU_NMS algorithm to remove redundant detection boxes from the output of the target YOLOv5 network model to obtain the remaining detection boxes in the output of the target YOLOv5 network model;
[0089] S153: Determine the category and confidence level of the safety hazard corresponding to the remaining detection frames.
[0090] Therefore, by considering not only the overlapping area between the detection box and the security hazard, but also the distance to the center point, the detection accuracy of overlapping targets can be greatly improved. It is understandable that if the IoU_NMS algorithm is used to remove redundant detection boxes from the output of the target YOLOv5 network model, it calculates the corresponding IoU value between the highest-scoring detection box and other detection boxes. When the IoU value exceeds a set threshold (e.g., 0.5 or 0.7), the boxes exceeding the threshold are suppressed or removed. This approach only considers the overlapping area, which can lead to incorrect suppression and removal, especially when the detection boxes contain each other.
[0091] Specifically, in some embodiments of the present invention, the categories of safety hazards include one or more of the following: construction machinery, fireworks, flames, kites, bird nests, and plastic films.
[0092] It should be noted that the specific values mentioned above are only for illustrating the implementation of the present invention in detail, and should not be construed as limiting the present invention. In other examples, implementation methods, or embodiments, other values may be selected according to the present invention, and no specific limitations are made here.
[0093] To implement the above embodiments, this invention also proposes a computer-readable storage medium storing a power transmission line safety hazard monitoring program thereon. When the power transmission line safety hazard monitoring program is executed by a processor, it implements the power transmission line safety hazard monitoring method of any of the above embodiments.
[0094] According to the computer-readable storage medium of the present invention, the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed with four detection branches. This achieves lightweighting of the network model and reduces its complexity, thereby enabling automatic monitoring of power transmission line safety hazards at the fusion terminal. It also has advantages such as fast detection speed, high detection accuracy, and low latency.
[0095] For example, when the transmission line safety hazard monitoring program is executed by the processor, the following steps of the transmission line safety hazard monitoring method are implemented:
[0096] S11: Obtain the current environmental image of the transmission line;
[0097] S13: Input the current environment image into the pre-trained target YOLOv5 network model, wherein the backbone network of the target YOLOv5 network model adopts the Focus structure and Mobilenetv2 structure, the neck network adopts the depthwise separable convolutional module, and the detection head network is constructed with four detection branches;
[0098] S15: Determine the safety hazard information of the transmission line based on the output of the target YOLOv5 network model.
[0099] It should be noted that the above explanation of the embodiments and beneficial effects of the method for monitoring safety hazards of transmission lines also applies to the computer-readable storage medium of the embodiments of the present invention. To avoid redundancy, it will not be elaborated in detail here.
[0100] To implement the above embodiments, this invention also proposes a converged terminal. Figure 8 This is a structural block diagram of a fusion terminal according to an embodiment of the present invention. Figure 8 As shown, the fusion terminal 100 includes a memory 102, a processor 104, and a transmission line safety hazard monitoring program 106 stored in the memory 102 and run on the processor 104. When the processor 104 executes the transmission line safety hazard monitoring program 106, it implements the transmission line safety hazard monitoring method of any of the above embodiments.
[0101] According to the fusion terminal 100 of the present invention, the backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed with four detection branches. This achieves lightweighting of the network model and reduces the complexity of the network model, thereby enabling automatic monitoring of power transmission line safety hazards in the fusion terminal. It also has the advantages of fast detection speed, high detection accuracy, and low latency.
[0102] For example, when the transmission line safety hazard monitoring program 106 is executed by the processor 104, the following steps of the transmission line safety hazard monitoring method are implemented:
[0103] S11: Obtain the current environmental image of the transmission line;
[0104] S13: Input the current environment image into the pre-trained target YOLOv5 network model, wherein the backbone network of the target YOLOv5 network model adopts the Focus structure and Mobilenetv2 structure, the neck network adopts the depthwise separable convolutional module, and the detection head network is constructed with four detection branches;
[0105] S15: Determine the safety hazard information of the transmission line based on the output of the target YOLOv5 network model.
[0106] It should be noted that the above explanation of the embodiments and beneficial effects of the method for monitoring safety hazards of transmission lines also applies to the fusion terminal 100 of the present invention. To avoid redundancy, it will not be elaborated in detail here.
[0107] To achieve the above embodiments, this invention also proposes a power transmission line safety hazard monitoring device. Figure 9 This is a structural block diagram of a power transmission line safety hazard monitoring device according to an embodiment of the present invention. The power transmission line safety hazard monitoring device of this embodiment is applied to a converged terminal, such as... Figure 9 As shown, the power transmission line safety hazard monitoring device 300 includes an acquisition module 302 and a determination module 304. The acquisition module 302 is used to acquire the current environmental image of the power transmission line. The determination module 304 is used to input the current environmental image into a pre-trained target YOLOv5 network model, and determine the safety hazard information of the power transmission line based on the output of the target YOLOv5 network model. The backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network is constructed with four detection branches.
[0108] According to the embodiment of the present invention, the transmission line safety hazard monitoring device 300 uses a Focus structure and a Mobilenetv2 structure for the backbone network of the target YOLOv5 network model, a depthwise separable convolutional module for the neck network, and four detection branches for the detection head network. This achieves lightweighting of the network model and reduces its complexity, thereby enabling automatic monitoring of transmission line safety hazards at the fusion terminal. It also has advantages such as fast detection speed, high detection accuracy, and low latency.
[0109] In some embodiments of the present invention, the target YOLOv5 network model is obtained through an annotation module and a training module. The annotation module is used to acquire environmental images of the transmission line and annotate safety hazards in the environmental images to obtain a sample set. The training module is used to train the constructed YOLOv5 network model using the sample set based on an anchorless bounding box prediction mechanism and a pre-set loss function to determine the model parameters of the target YOLOv5 network model.
[0110] In some embodiments of the present invention, the annotation module is also used to annotate safety hazards in environmental images to obtain standard documents, and to perform data augmentation processing on environmental images and standard documents to obtain a sample set.
[0111] In some embodiments of the present invention, the sample set includes a training set. The training module is further configured to input the training set into the constructed YOLOv5 network model for training, in order to determine the training model parameters of the YOLOv5 network model, calculate the total loss of the training set according to the loss function at a pre-set testing frequency, stop training when the total loss of the training set shows a trend of first decreasing and then increasing, and determine the training model parameters corresponding to the minimum value of the total loss of the training set as the model parameters of the target YOLOv5 network model.
[0112] In some embodiments of the present invention, the loss function is: L = L box +L conf +L cls Where L is the total loss, L box To predict the localization loss of the bounding box, L conf For confidence loss, L cls Classification of losses for safety hazards.
[0113] In some embodiments of the present invention, the localization loss of the predicted bounding box is calculated using the following formula: Where N is the number of detection branches, S2 is the resolution of each detection branch, and B is the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, For area-weighted summation, h'w' is the area of the predicted bounding box, hw is the area of the entire image, and L... CIoU Let CIoU be the loss function.
[0114] In some embodiments of the present invention, the confidence loss is calculated using the following formula: Where N is the number of detection branches, S 2 For each detection branch, the resolution is given, B is the number of predicted bounding boxes generated per grid at that resolution, and log(|p gt -p pred |) is the cross-entropy loss function, (p) gt -p pred ) 2 For Focal weights, p gt p represents the true probability of a potential safety hazard. pred This represents the predicted probability of a potential safety hazard.
[0115] In some embodiments of the present invention, the classification loss of safety hazards is calculated using the following formula: Where N is the number of detection branches, S 2 For each detection branch, B represents the resolution, and B represents the number of predicted bounding boxes generated per grid cell at that resolution. Let C be the indicator function, and C be the category of safety hazard. This represents the true probability that the current safety hazard belongs to category c. This represents the predicted probability that the current safety hazard belongs to category c.
[0116] In some embodiments of the present invention, the determining module 304 is further configured to use the DIoU_NMS algorithm to remove redundant detection boxes in the output of the target YOLOv5 network model, obtain the remaining detection boxes in the output of the target YOLOv5 network model, and determine the category and confidence level of the security risks corresponding to the remaining detection boxes.
[0117] In some embodiments of the present invention, the categories of safety hazards include one or more of the following: construction machinery, fireworks, flames, kites, bird nests, and plastic films.
[0118] It should be noted that the above explanation of the embodiments and beneficial effects of the method for monitoring safety hazards of transmission lines also applies to the transmission line safety hazard monitoring device 300 of the present invention. To avoid redundancy, it will not be elaborated in detail here.
[0119] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.
[0120] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0121] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0122] Furthermore, the terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying relative importance, or implicitly specifying the number of technical features indicated in this embodiment. Therefore, features defined with terms such as "first" and "second" in the embodiments of this invention can explicitly or implicitly indicate that the embodiment includes at least one of those features. In the description of this invention, the word "multiple" means at least two or more, such as two, three, four, etc., unless otherwise explicitly specified in the embodiments.
[0123] In this invention, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific implementation.
[0124] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.
Claims
1. A method for monitoring safety hazards in power transmission lines, characterized in that, Applied to converged terminals, the method includes: Acquire current environmental images of the power transmission line; The current environment image is input into a pre-trained target YOLOv5 network model. The backbone network of the target YOLOv5 network model adopts the Focus structure and Mobilenetv2 structure, the neck network adopts the depth-separable convolutional module, and the detection head network is constructed with four detection branches. The YOLOv5 network model is trained based on the anchorless bounding box prediction mechanism. The safety hazard information of the transmission line is determined based on the output of the target YOLOv5 network model. The categories of safety hazards include one or more of the following: construction machinery, fireworks, flames, kites, bird nests, and plastic film. The target YOLOv5 network model is trained according to the following steps: Obtain environmental images of the transmission line and label the safety hazards in the environmental images to obtain a sample set; Based on a pre-set loss function, the constructed YOLOv5 network model is trained using the sample set to determine the model parameters of the target YOLOv5 network model. The loss function is: ， in, For the total loss, To predict the localization loss of the bounding box, For confidence loss, Classification of losses due to safety hazards; The losses from the aforementioned safety hazards are calculated using the following formula: ， Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, As a category of safety hazards, This represents the true probability that the current safety hazard belongs to category c. This represents the predicted probability that the current safety hazard belongs to category c.
2. The method for monitoring safety hazards in transmission lines according to claim 1, characterized in that, The environmental images are labeled to obtain a sample set, including: Safety hazards in the environmental images are labeled to obtain standard documents; The environmental images and the standard files are subjected to data augmentation processing to obtain the sample set.
3. The method for monitoring safety hazards in transmission lines according to claim 1, characterized in that, The sample set includes a training set, and the step of training the constructed YOLOv5 network model using the sample set includes: The training set is input into the constructed YOLOv5 network model for training, so as to determine the training model parameters of the YOLOv5 network model; Calculate the total loss of the training set according to the loss function based on the pre-set testing frequency; When the total loss of the training set shows a trend of first decreasing and then increasing, training is stopped, and the training model parameters corresponding to the minimum value of the total loss of the training set are determined as the model parameters of the target YOLOv5 network model.
4. The method for monitoring safety hazards in transmission lines according to claim 1, characterized in that, The localization loss of the predicted bounding box is calculated using the following formula: ， Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, For area weighting, To predict the area of the bounding box, The area of the entire map. Let CIoU be the loss function.
5. The method for monitoring safety hazards in transmission lines according to claim 1, characterized in that, The confidence loss is calculated using the following formula: ， Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. Let cross-entropy be the loss function. Focal weights This represents the true probability of a potential safety hazard. This represents the predicted probability of a potential safety hazard.
6. The method for monitoring safety hazards in transmission lines according to any one of claims 1-5, characterized in that, The step of determining the safety hazard information of the transmission line based on the output of the target YOLOv5 network model includes: The DIoU_NMS algorithm is used to remove redundant detection boxes from the output of the target YOLOv5 network model, thereby obtaining the remaining detection boxes in the output of the target YOLOv5 network model. Determine the category and confidence level of the safety hazard corresponding to the remaining detection frames.
7. A computer-readable storage medium, characterized in that, It stores a power transmission line safety hazard monitoring program, which, when executed by the processor, implements the power transmission line safety hazard monitoring method according to any one of claims 1-6.
8. A converged terminal, characterized in that, The method includes a memory, a processor, and a transmission line safety hazard monitoring program stored in the memory and executable on the processor. When the processor executes the transmission line safety hazard monitoring program, it implements the transmission line safety hazard monitoring method according to any one of claims 1-6.
9. A power transmission line safety hazard monitoring device, characterized in that, The device, applied to a converged terminal, includes: The acquisition module is used to acquire current environmental images of the transmission line; The determination module is used to input the current environment image into a pre-trained target YOLOv5 network model and determine the safety hazard information of the transmission line based on the output of the target YOLOv5 network model. The backbone network of the target YOLOv5 network model adopts a Focus structure and a Mobilenetv2 structure, the neck network adopts a depthwise separable convolutional module, and the detection head network adopts four detection branches. The YOLOv5 network model is trained based on the anchorless bounding box prediction mechanism. The categories of safety hazards include one or more of the following: construction machinery, fireworks, flames, kites, bird nests, and plastic film. The target YOLOv5 network model is obtained through an annotation module and a training module, wherein... The annotation module is used to acquire environmental images of the transmission line and annotate safety hazards in the environmental images to obtain a sample set; The training module is used to train the constructed YOLOv5 network model using the sample set based on the anchorless bounding box prediction mechanism and a pre-set loss function, so as to determine the model parameters of the target YOLOv5 network model. The loss function is: ， in, For the total loss, To predict the localization loss of the bounding box, For confidence loss, Classification of losses due to safety hazards; The losses from the aforementioned safety hazards are calculated using the following formula: ， Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, As a category of safety hazards, This represents the true probability that the current safety hazard belongs to category c. This represents the predicted probability that the current safety hazard belongs to category c.
10. The power transmission line safety hazard monitoring device according to claim 9, characterized in that, The sample set includes a training set. The training module is further configured to input the training set into the constructed YOLOv5 network model for training, to determine the training model parameters of the YOLOv5 network model, and to calculate the total loss of the training set according to the loss function at a pre-set testing frequency. When the total loss of the training set shows a trend of first decreasing and then increasing, training is stopped, and the training model parameters corresponding to the minimum value of the total loss of the training set are determined as the model parameters of the target YOLOv5 network model.
11. The power transmission line safety hazard monitoring device according to claim 9, characterized in that, The localization loss of the predicted bounding box is calculated using the following formula: ， Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. For indicator functions, For area weighting, To predict the area of the bounding box, The area of the entire map. Let CIoU be the loss function.
12. The power transmission line safety hazard monitoring device according to claim 9, characterized in that, The confidence loss is calculated using the following formula: ， Where N is the number of detection branches, S 2 For each of the detection branches, B represents the resolution, and B represents the number of predicted bounding boxes generated by each grid at that resolution. Let cross-entropy be the loss function. Focal weights This represents the true probability of a potential safety hazard. This represents the predicted probability of a potential safety hazard.
13. The transmission line safety hazard monitoring device according to any one of claims 9-12, characterized in that, The determining module is further configured to use the DIoU_NMS algorithm to remove redundant detection boxes in the output of the target YOLOv5 network model, obtain the remaining detection boxes in the output of the target YOLOv5 network model, and determine the category and confidence level of the security risks corresponding to the remaining detection boxes.