Method and device for detecting suspension wire clamp offset fault based on directed bounding box

Abstract

The present application provides a method and device for detecting the offset fault of a suspension wire clamp based on a directed bounding box. The method includes: preprocessing the images collected during the inspection of a drone; inputting the preprocessed images into a segmentation model, using the segmentation model to extract the position information of the suspension wire clamp and the hull, and generating a segmentation mask; according to the segmentation mask, calculating the minimum circumscribed matrix of the suspension wire clamp and the hull, and generating corresponding rotation frame annotation data; inputting the rotation frame annotation data and the preprocessed image into a pre-trained target detection model for target feature extraction and fusion, and generating a detection result; according to the rotation angles of the directed bounding boxes corresponding to the suspension wire clamp and the hull respectively, calculating the angle between the directed bounding box of the suspension wire clamp and the directed bounding box of the hull, and judging whether the suspension wire clamp has an offset fault according to the angle. The present application can improve the accuracy and efficiency of the detection of suspension wire clamp faults, thereby ensuring the safe operation of the power system.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for detecting a suspension wire clamp offset fault based on a directed bounding box. Background Art

[0002] In the power system, the suspension clamp is an indispensable key component of the transmission line. Its main function is to support and fix the conductor to ensure the stable operation of the transmission line. The performance of the suspension clamp directly affects the service life of the transmission line, line loss and power supply reliability. However, due to the influence of various factors such as the natural environment, mechanical stress, and material aging, the suspension clamp may have an offset failure. This offset failure will cause uneven distribution of conductor tension and may even cause a line break accident, which seriously threatens the safe operation of the power system.

[0003] During long-term outdoor operation, suspension clamps may deviate to a certain extent due to factors such as wind force, line direction and terrain. In severe cases, the suspension clamps may break, resulting in power outages. Therefore, accurately identifying the slight deviation and wear of the suspension clamps and promptly handling the deviated suspension clamps are crucial to maintaining the stability of the power system.

[0004] At present, the detection of suspension clamp faults mainly relies on regular manual inspections. However, manual inspections have the following problems: traditional detection methods often cannot accurately extract the characteristic information of suspension clamps, resulting in low accuracy of detection results; and traditional manual inspection methods are inefficient and cannot meet the needs of large-scale and efficient power inspections. Inspection work consumes a lot of manpower and time, and it is impossible to achieve real-time monitoring and processing of suspension clamp offset faults. Summary of the invention

[0005] In view of this, an embodiment of the present application provides a method and device for detecting a suspension wire clamp offset fault based on a directed bounding box to solve the problems of low accuracy of detection results, easy missed detection and misjudgment, and low fault detection efficiency in the prior art.

[0006] According to a first aspect of an embodiment of the present application, a method for detecting an offset fault of a suspension wire clamp based on a directed bounding box is provided, comprising: obtaining images collected during an unmanned aerial vehicle inspection and preprocessing the images; inputting the preprocessed images into a segmentation model, using the segmentation model to extract position information of the suspension wire clamp and the hull, and generating a segmentation mask; according to the segmentation mask, using a connected region detection algorithm to calculate a minimum circumscribed matrix of the suspension wire clamp and the hull, and generating corresponding rotation box annotation data according to the minimum circumscribed matrix; inputting the rotation box annotation data and the preprocessed images into a pre-trained target detection model, using the target detection model to extract and fuse target features, and generating a detection result; according to the rotation angles of the directed bounding boxes corresponding to the suspension wire clamp and the hull in the detection results, calculating the angle between the directed bounding box of the suspension wire clamp and the directed bounding box of the hull, and judging whether an offset fault occurs to the suspension wire clamp according to the angle.

[0007] According to a second aspect of an embodiment of the present application, a device for detecting an offset fault of a suspension wire clamp based on a directed bounding box is provided, comprising: an acquisition module, configured to acquire images collected during an unmanned aerial vehicle inspection and to preprocess the images; a segmentation module, configured to input the preprocessed images into a segmentation model, extract position information of the suspension wire clamp and the hull using the segmentation model, and generate a segmentation mask; a calculation module, configured to calculate the minimum circumscribed matrix of the suspension wire clamp and the hull using a connected region detection algorithm according to the segmentation mask, and generate corresponding rotation box annotation data according to the minimum circumscribed matrix; a detection module, configured to input the rotation box annotation data and the preprocessed images into a pre-trained target detection model, extract and fuse target features using the target detection model, and generate a detection result; a judgment module, configured to calculate the angle between the directed bounding box of the suspension wire clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes corresponding to the suspension wire clamp and the hull in the detection result, and judge whether an offset fault occurs to the suspension wire clamp according to the angle.

[0008] According to a third aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the above method are implemented when the processor executes the computer program.

[0009] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0010] At least one of the above technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects:

[0011] By acquiring images collected during the drone inspection process and preprocessing the images; inputting the preprocessed images into the segmentation model, using the segmentation model to extract the position information of the suspension wire clamp and the hull, and generating a segmentation mask; according to the segmentation mask, using the connected region detection algorithm to calculate the minimum circumscribed matrix of the suspension wire clamp and the hull, and generating corresponding rotation box annotation data according to the minimum circumscribed matrix; inputting the rotation box annotation data and the preprocessed images into the pre-trained target detection model, using the target detection model to extract and fuse target features, and generating detection results; according to the rotation angles of the directed bounding boxes corresponding to the suspension wire clamp and the hull in the detection results, the angle between the directed bounding box of the suspension wire clamp and the directed bounding box of the hull is calculated, and judging whether the suspension wire clamp has an offset fault according to the angle. The present application can improve the accuracy and efficiency of the detection of suspension wire clamp faults, thereby ensuring the safe operation of the power system. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0013] Figure 1 It is a flow chart of a suspension wire clamp offset fault detection method based on a directed bounding box provided in an embodiment of the present application;

[0014] Figure 2 is a schematic diagram of parameters corresponding to a directional bounding box included in the detection result provided in an embodiment of the present application;

[0015] Figure 3 is a schematic diagram of the structure of the weighted residual attention module provided in an embodiment of the present application;

[0016] Figure 4 is a schematic diagram of three different angles provided in the embodiments of the present application;

[0017] Figure 5 It is a structural schematic diagram of a suspension wire clamp offset fault detection device based on a directed bounding box provided in an embodiment of the present application;

[0018] Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0019] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0020] In the power system, the suspension clamp is a vital component of the transmission line, which is mainly used to support and fix the conductors to ensure the stable operation of the transmission line. The working state of the suspension clamp directly affects the service life, line loss and power supply reliability of the transmission line. However, since the suspension clamp is exposed to the natural environment all year round, it will be affected by various factors such as wind, mechanical stress and material aging, which may cause the suspension clamp to offset. Once the suspension clamp is offset, it may cause uneven distribution of conductor tension, and even cause line breakage accidents in severe cases, posing a serious threat to the safe operation of the power system. Therefore, detecting and repairing the offset failure of the suspension clamp is of great significance to ensure the stability and reliability of the power system.

[0021] Since the suspension clamps are operated outdoors for a long time, they may deviate to a certain extent due to factors such as wind force, line direction and terrain. If this deviation is not detected and handled promptly and effectively, it may cause the suspension clamps to break in severe cases, leading to power outages. Accurately identifying the slight deviation and wear of the suspension clamps and performing maintenance and treatment in a timely manner is an important means to ensure the stability of the power system. However, existing detection methods have obvious shortcomings and are difficult to meet this requirement.

[0022] At present, the detection of suspension clamp faults mainly relies on manual inspections. Inspectors regularly go to the transmission line site to detect the status of the suspension clamps through visual inspections, measurements and other means. Although this method can detect and handle suspension clamp faults to a certain extent, it also has obvious limitations and defects. For example, the traditional manual inspection method has the following problems:

[0023] 1. Low detection accuracy: Traditional manual inspection methods often fail to accurately extract the characteristic information of the suspension wire clamp, resulting in low detection accuracy. Manual inspection relies on the experience and subjective judgment of the inspectors, which is prone to missed inspections and misjudgments.

[0024] 2. Inefficiency: Manual inspection is inefficient, especially when faced with large-scale power line inspection tasks, it is difficult to achieve efficient and comprehensive inspection. Inspection work requires a lot of manpower and time, and it is impossible to monitor and handle the offset fault of the suspension clamp in real time.

[0025] These shortcomings indicate that the existing suspension line clamp fault detection method is difficult to effectively meet actual needs. A new, more efficient and accurate detection method is urgently needed to improve the accuracy and efficiency of suspension line clamp fault detection and ensure the safe operation of the power system.

[0026] In view of the problems existing in the prior art, in order to effectively solve the problem of possible offset faults of suspension clamps in transmission lines, the present application proposes a suspension clamp offset fault detection method based on a directed bounding box. During the drone inspection process, the collected images are first preprocessed, such as denoising and contrast enhancement, to improve the image quality. The segmentation model (such as the SAM model) is used to obtain the position information of the suspension clamp and the hull, and then the minimum circumscribed matrix of the two is calculated by connecting the area to automatically generate the annotation data, which solves the problem of manual annotation and saves a lot of manpower and time. The image and the annotation data are passed to the improved target detection model (such as the YOLOv8-OBB network model) to extract features from the preprocessed image, extract the key feature information of the suspension clamp and the hull, and classify and identify according to the extracted feature information. The angle between the directed bounding box of the suspension clamp and the hull is used to determine whether the suspension clamp is offset.

[0027] The contents of the technical solution of the present application are described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Figure 1 It is a flow chart of a suspension wire clamp offset fault detection method based on a directed bounding box provided in an embodiment of the present application. Figure 1 The directed bounding box-based suspension clip offset fault detection method can be executed by the background server. Figure 1 As shown, the method for detecting the suspension wire clamp offset fault based on a directed bounding box may specifically include:

[0029] S101, acquiring images collected during the drone inspection process and preprocessing the images;

[0030] S102, inputting the preprocessed image into a segmentation model, using the segmentation model to extract position information of the suspension line clamp and the hull, and generating a segmentation mask;

[0031] S103, calculating the minimum circumscribed matrix of the suspension line clamp and the hull using a connected region detection algorithm according to the segmentation mask, and generating corresponding rotation frame annotation data according to the minimum circumscribed matrix;

[0032] S104, inputting the rotation box annotation data and the preprocessed image into a pre-trained target detection model, using the target detection model to extract and fuse target features, and generating a detection result;

[0033] S105, calculating the angle between the directed bounding box of the suspension clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes corresponding to the suspension clamp and the hull in the detection results, and judging whether the suspension clamp has an offset fault according to the angle.

[0034] First, use drones to inspect transmission lines and collect image data of suspension clamps. Drones carry high-resolution cameras to ensure that the collected images have sufficient clarity and details. In order to improve image quality and enhance image features, the collected images need to be preprocessed, which may include but is not limited to the following operations:

[0035] 1. Denoising:

[0036] Image denoising algorithms, such as median filtering and Gaussian filtering, are used to remove noise and interference in the image and improve the clarity of the image. This step helps to make subsequent image segmentation and feature extraction more accurate.

[0037] 2. Enhance contrast:

[0038] By using image enhancement techniques such as histogram equalization and contrast stretching, the contrast of the image is improved to make the difference between the hanging wire clamp and the background in the image more obvious. Enhancing the contrast can help better identify the outline and boundary of the hanging wire clamp.

[0039] 3. Image scaling:

[0040] Scale the image as needed to ensure that the image resolution is suitable for subsequent processing steps. Image scaling can be adjusted according to specific application scenarios and algorithm requirements to achieve the best results.

[0041] Furthermore, in order to ensure that the quality of the preprocessed image meets the requirements, the preprocessing effect needs to be verified, for example, manually checking the preprocessed image to ensure that the noise is effectively removed, the contrast is enhanced, and the image clarity meets the requirements of subsequent processing. In addition, objective evaluation indicators such as peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) can be used to compare the images before and after preprocessing to quantify the improvement effect of image quality.

[0042] Through the preprocessing steps provided in the above embodiment, the quality of the image can be effectively improved, laying a good foundation for subsequent image segmentation and feature extraction. The above process ensures that the feature information of the suspension clamp can be accurately extracted, improving the accuracy and reliability of the entire offset fault detection method.

[0043] In some embodiments, the segmentation model adopts a zero-sample image segmentation model, and uses the segmentation model to extract the position information of the suspension line clamp and the hull, and generate a segmentation mask, including:

[0044] Input the preprocessed image into a zero-shot image segmentation model, which includes an image encoder, a hint encoder, and a mask decoder;

[0045] Encoding the preprocessed image using an image encoder so as to convert the preprocessed image into a high-level feature representation and generate a feature map;

[0046] A cue encoder is used to generate a cue embedding according to the cue information, and a mask decoder is used to generate a segmentation mask according to the feature map and the cue embedding, wherein the segmentation mask is used to locate the position information of the suspension line clamp and the hull.

[0047] Specifically, the preprocessed image is first input into the zero-sample image segmentation model. The segmentation model may be a SAM (Segment Anything Model) model, which adopts an encoder-decoder structure similar to U-Net, including an image encoder, a hint encoder, and a mask decoder. The processing of these three parts is described in detail below in conjunction with specific embodiments, which may include the following:

[0048] 1. Encode using image encoder:

[0049] The preprocessed image is input into the image encoder. The image encoder encodes the input image and converts it into a high-level feature representation. During the encoding process, the image encoder captures the key information in the image and generates a feature map.

[0050] 2. Generate prompt embeddings using prompt encoder

[0051] Based on the input prompt information, the prompt encoder generates the corresponding prompt embedding. The prompt information can be a specific prompt provided by the user to guide the model to focus on specific areas or features in the image.

[0052] 3. Generate segmentation mask using mask decoder

[0053] The feature map generated by the image encoder and the hint embedding generated by the hint encoder are input into the mask decoder. The mask decoder combines the feature map and hint embedding to generate a segmentation mask. The segmentation mask is used to locate the specific location information of the suspension wire clamp and the hull.

[0054] The processing step of the mask decoder includes decoding the feature map, combining the information from the cue embedding, and outputting an accurate segmentation mask containing the location information of the suspension clip and the hull.

[0055] Finally, the generated segmentation mask can be verified to ensure its accuracy. Verification methods include visual inspection and evaluation metrics such as IoU (Intersection over Union) to quantify the accuracy of the segmentation effect.

[0056] Through the method of the present embodiment described above, the zero-sample image segmentation model can be used to effectively extract the location information of the suspension line clamp and the hull, and generate an accurate segmentation mask. This process ensures that the subsequent processing steps can be performed based on accurate location information, improving the reliability and accuracy of the overall detection method. As a zero-sample image segmentation model, the SAM model can perform image segmentation based on prompt information. The encoder-decoder structure it adopts generates a feature map through an image encoder, generates a prompt embedding through a prompt encoder, and generates an accurate segmentation mask in combination with a mask decoder, thereby effectively locating the position of the suspension line clamp and the hull.

[0057] In some embodiments, according to the segmentation mask, a connected region detection algorithm is used to calculate the minimum circumscribed matrix of the suspension line clamp and the hull, and corresponding rotation box annotation data is generated according to the minimum circumscribed matrix, including:

[0058] Based on the segmentation mask, the segmentation and positioning coordinate information of the suspension wire clamp and the hull in the preprocessed image is extracted;

[0059] The connected area detection algorithm is used to process the segmented positioning coordinate information to determine the connected area between the suspension line clamp and the hull;

[0060] The minimum circumscribed matrix of the connected area is calculated to generate the rotation box of the suspension clamp and the hull, and the minimum circumscribed matrix is ​​automatically saved as the rotation box annotation data, where the rotation box annotation data is used to train the target detection model and perform offset fault detection of the suspension clamp.

[0061] Specifically, this embodiment uses a connected region detection algorithm to calculate the coordinate information of the preprocessed image and the segmented and positioned image, obtains the minimum circumscribed matrix of the suspension clamp and the hull, and automatically saves it as the annotation data used for training the target detection model. These annotation data contain the information of the rotation box and are used to train the model for offset fault detection of the suspension clamp.

[0062] In one example, after generating segmentation masks of the suspension clamp and the hull using a zero-shot image segmentation model (such as a SAM model), segmentation location coordinate information of the suspension clamp and the hull is extracted from the preprocessed image based on the generated segmentation mask. The segmentation location coordinate information refers to the specific position and boundary of each connected area in the segmentation mask.

[0063] Furthermore, the extracted segmentation location coordinate information is processed using a connected region detection algorithm to determine the connected region between the suspension line clamp and the hull. The connected region detection algorithm can identify independent connected regions in the image and distinguish them.

[0064] Furthermore, the connected regions are analyzed and the minimum circumscribed matrix (rotation box) of each connected region is calculated. The minimum circumscribed matrix is ​​the smallest rectangular box that can completely surround the connected region. Based on the calculated minimum circumscribed matrix, the rotation box of the suspension clamp and the hull is generated. The rotation box includes not only the boundary of the connected region, but also its rotation angle information for subsequent offset detection.

[0065] Furthermore, the calculated rotation box is automatically saved as the annotation data used for training the target detection model (such as the YOLOv8-OBB model). The annotation data includes the position information of the suspension line clamp and the hull and their rotation angle.

[0066] In practical applications, the generated rotation box annotation data can also be verified to ensure its accuracy and completeness. Verification methods include visual inspection and evaluation indicators such as IoU (Intersection over Union) to quantify the quality of the annotation data.

[0067] Through the method of the present embodiment described above, accurate rotation frame annotation data can be generated according to the segmentation mask. This process greatly saves the time and cost of manual annotation, and ensures that the position information of the suspension line clamp and the hull can be accurately annotated, providing reliable data support for subsequent target detection and offset fault detection.

[0068] In some embodiments, the target detection model includes a backbone network, a neck network, and a detection head, and the target detection model is used to extract and fuse target features and generate detection results, including:

[0069] The backbone network is used to extract features from input images with annotation information and generate feature maps of multiple scales;

[0070] The feature map generated by the backbone network is input into the neck network, and the feature map is fused using the neck network to generate a fused feature map;

[0071] The fused feature map is input into the detection head, and the detection head is used to forward propagate the fused feature map to generate a detection result, wherein the detection result includes the coordinates, category information and rotation angle of the directed bounding boxes corresponding to the suspension line clamp and the hull respectively.

[0072] Specifically, the backbone network extracts features through a dual attention mechanism, the neck network performs feature fusion, and the detection head uses the eSE module (i.e., weighted residual attention module) to improve the accuracy of rotation angle prediction. Finally, the angle value of the directed bounding box of the suspension clamp and the hull is output to determine whether there is an abnormal offset problem with the suspension clamp.

[0073] The following is a detailed description of the process of generating detection results using a target detection model (such as a YOLOv8-OBB model) in the present application in conjunction with the accompanying drawings and specific embodiments. Figure 2 is a schematic diagram of the parameters corresponding to the directional bounding box included in the detection result provided in the embodiment of the present application, Figure 3 is a schematic diagram of the structure of the weighted residual attention module provided in the embodiment of the present application. Figures 2 to 3 As shown, using the target detection model to generate detection results can specifically include the following:

[0074] The YOLOv8-OBB network model mainly consists of three parts: the backbone network (Backbone) is responsible for feature extraction of the input image; the neck network (Neck) is responsible for feature fusion of the feature map of the backbone network; the detection head (Head) is responsible for generating the final detection result based on the fused feature map. In the detection head part, in addition to predicting the traditional bounding box coordinates (x, y, w, h) and category information, it is also necessary to predict the rotation angle α (the five parameters of the oriented bounding box are as follows Figure 2 As shown in the figure), the detection head supports the detection of rotated bounding boxes, and the detection head contains three decoupled branches: angle prediction branch, bounding box regression branch and classification branch.

[0075] First, feature extraction is performed to input the image with annotation information into the object detection model. The annotation information includes data such as the position and rotation angle of the suspension clamp and the hull.

[0076] The backbone network extracts features from the input image through structures such as convolutional layers and pooling layers. A dual attention mechanism is added to the backbone network to enhance the ability to extract target features and reduce background interference information. After being processed by the backbone network, feature maps of multiple scales are generated, which contain semantic information at different levels from low-level details to high-level abstraction.

[0077] Then, feature fusion is performed, and the feature map generated by the backbone network is input into the neck network. The neck network is responsible for fusing the feature map, combining feature information from different scales, and generating a fused feature map. Feature fusion helps improve the model's detection accuracy and robustness for targets.

[0078] Finally, the detection results are generated and the fused feature maps are input into the detection head. The detection head consists of three decoupled branches: angle prediction branch, bounding box regression branch, and classification branch.

[0079] The eSE module is introduced into the angle prediction branch network. The eSE module contains an average pooling layer, a fully connected layer, and a SiLU activation function. This module helps improve the model's accuracy in predicting the rotation angle of a oriented bounding box.

[0080] The detection head is used to forward propagate the fused feature map to generate the detection results, which include the coordinates (x, y, w, h), category information and rotation angle α of the directed bounding box of the suspension clamp and the hull.

[0081] Furthermore, the angle between the suspension clamp and the hull is calculated based on the angles α1 and α2 between the long line of the directional bounding box of the suspension clamp and the hull and the horizontal direction output by the YOLOv8-OBB model. , determine whether there is an offset abnormality problem.

[0082] when When it is within the preset range (60º to 120º), the suspension clamp is considered normal; otherwise, the suspension clamp is determined to have an offset fault and a fault detection report is generated.

[0083] Through the method of the present embodiment described above, the improved YOLOv8-OBB model can be used to efficiently and accurately extract and fuse target features and generate detection results. This process ensures that the position information of the suspension clamp and the hull can be accurately detected and analyzed, and improves the accuracy and reliability of the suspension clamp offset fault detection.

[0084] In some embodiments, the target detection model further includes:

[0085] A dual attention mechanism is added to the backbone network, which is used to weaken the complex background information in the feature map; the detection head includes an angle prediction branch, a bounding box regression branch and a classification branch, wherein a weighted residual attention module is added to the angle prediction branch, and the weighted residual attention module includes an average pooling layer, a fully connected layer and an activation function layer.

[0086] Specifically, in order to improve the generalization ability of the model in complex scenes, a dual attention mechanism is first added to the backbone network area to obtain more feature information, weaken the complex background information in the feature map, and enhance the network model's ability to extract target features. In addition, the eSE module is inserted at the appropriate position of the angle prediction branch to reduce the problem of small angle deviation on the accuracy of oriented bounding box regression. The angle prediction branch network structure includes an eSE module and a convolution layer. The structure of the eSE module (such as Figure 3 As shown in Figure 3, the proposed method includes an average pooling layer, a fully connected layer and a SiLU activation function.

[0087] For example, in one example, an image with labeled information is input into the target detection model, and the backbone network of the model is responsible for feature extraction of the image. The backbone network extracts feature information from the image through structures such as convolutional layers and pooling layers. To enhance the feature extraction effect, a dual attention mechanism is added to the backbone network.

[0088] The dual attention mechanism is used to weaken the complex background information in the feature map and enhance the network's ability to extract target features. Specifically, the dual attention mechanism weights the spatial and channel dimensions of the feature map, allowing the network to pay more attention to the target area and reduce the interference of background noise.

[0089] Furthermore, the feature map generated by the backbone network is input into the neck network. The neck network fuses the feature maps of different scales to generate a fused feature map. The fused feature map contains richer semantic information, which helps to improve the detection accuracy.

[0090] Furthermore, the fused feature map is input into the detection head. The detection head consists of an angle prediction branch, a bounding box regression branch, and a classification branch, which are responsible for predicting the rotation angle, bounding box coordinates, and target category, respectively.

[0091] A weighted residual attention module (eSE module) is added to the angle prediction branch to improve the accuracy of rotation angle prediction. The specific structure of the eSE module includes:

[0092] Average pooling layer: Perform global average pooling on the feature map to generate a channel-level feature vector.

[0093] Fully connected layer: Input the channel-level feature vector into the fully connected layer to learn the weight relationship between features.

[0094] SiLU activation function: Use the SiLU activation function to perform nonlinear transformation on the output of the fully connected layer to enhance the feature representation capability.

[0095] Weighted residual connection: Add the weighted residual information back to the original feature map to improve the network's expressiveness and the accuracy of rotation angle prediction.

[0096] Finally, the detection head is used to forward propagate the fused feature map to generate the detection results, which include the directed bounding box coordinates (x, y, w, h), category information, and rotation angle α of the suspension clamp and the hull.

[0097] Through the above method of this embodiment, the improved target detection model is used to extract and fuse target features efficiently and accurately, and generate detection results. The dual attention mechanism in the backbone network enhances the ability to extract target features, and the weighted residual attention module in the angle prediction branch improves the accuracy of rotation angle prediction. Finally, the directed bounding box angle value of the suspension clamp and the hull is output to determine whether the suspension clamp has an abnormal offset problem.

[0098] In some embodiments, calculating the angle between the directed bounding box of the suspension clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes corresponding to the suspension clamp and the hull in the detection results includes:

[0099] According to the first angle between the long line of the directed bounding box of the suspension line clamp and the horizontal direction, and the second angle between the long line of the directed bounding box of the hull and the horizontal direction in the rotation angle, the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull is calculated using the first angle and the second angle, wherein the angle is the absolute value of the difference between the first angle and the second angle.

[0100] Specifically, this embodiment will calculate the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes corresponding to the suspension line clamp and the hull in the detection results. The process of calculating the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull in this application is described in detail below in conjunction with the accompanying drawings and specific embodiments. Figure 4 is a schematic diagram of three different angles provided in the embodiments of the present application, wherein Figure 4 The left side shows the angle between the long line of the directional bounding box and the horizontal direction (including α1 and α2), and the right side shows the angle between the directional bounding boxes of the two targets.

[0101] In one example, the improved YOLOv8-OBB model is used to detect the image and output the rotation angle information of the directed bounding boxes of the suspension clamp and the hull. The angle between the long line of the directed bounding box of the suspension clamp and the horizontal direction (the first angle α1) and the angle between the long line of the directed bounding box of the hull and the horizontal direction (the second angle α2) are obtained from the model output.

[0102] According to the first angle α1 and the second angle α2, the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull is calculated. The calculation formula is as follows:

[0103]

[0104] In one example, setting the angle The preset normal range is 60º to 120º. When it is within this range, the suspension wire clamp is considered to be normal; otherwise, it is considered that the suspension wire clamp has an offset failure.

[0105] For example, if the calculated angle Satisfy the conditions ∈(60º,120º), the suspension clamp is considered to be in normal condition and no processing is required; if the angle If it is not within the preset range, it is determined that the suspension clamp has an offset failure and needs to be handled by relevant personnel.

[0106] Through the method of the present embodiment, the angle between the suspension clamp and the hull can be accurately calculated using the rotation angle information output by the model, and whether the suspension clamp has an offset fault can be determined based on the calculation result. This process improves the accuracy and reliability of the suspension clamp offset detection and ensures the safe operation of the power system.

[0107] In some embodiments, judging whether the suspension clamp has an offset fault according to the included angle includes:

[0108] It is determined whether the angle is within a preset angle range. If so, it is determined that the state of the suspension clamp is normal. Otherwise, it is determined that the suspension clamp has an offset fault and a fault detection report is generated.

[0109] Specifically, in one example, under normal circumstances, the suspension clamp and the hull are in a 90° vertical state, and the offset between the two is considered normal if it is within 30°, and it is considered abnormal if it is greater than 30°. Therefore, the angle between the long line of the directional bounding box of the suspension clamp and the horizontal direction is α1∈[0,180º], and the angle between the long line of the directional bounding box of the hull and the horizontal direction is α2∈[0,180º].

[0110] Furthermore, the angle between the suspension clamp and the hull is , when β∈(60º,120º), it is normal; others are considered offset faults and need to be processed. For example, a fault detection report can be generated based on the judgment results. If there is an offset fault, the report should include specific offset conditions and processing suggestions so that maintenance personnel can handle it in a timely manner.

[0111] According to the technical solution provided by the embodiment of the present application, in order to effectively improve the detection efficiency of the suspension clamp offset fault in the power system, the present embodiment inputs the preprocessed image into the SAM model to extract the position information of the suspension clamp and the hull, performs connected area detection on the preprocessed image according to the position information of the target, calculates the minimum external matrix of the relevant target to complete the data labeling work; transmits the data image to the improved YOLOv8 model to achieve high-precision directed bounding box detection, and finally uses the angle information of the directed bounding box of the suspension clamp and the hull to realize the offset fault detection of the suspension clamp. The technical solution of the present application has at least the following advantages:

[0112] 1. Save manpower and time:

[0113] The SAM model (zero-sample image segmentation model) and connected region detection algorithm can automatically calculate the minimum circumscribed matrix of the suspension wire clamp. By automatically generating rotating frame annotation data, the problem of manual annotation data in traditional methods is effectively solved, greatly saving a lot of manpower and time costs.

[0114] 2. Improve detection accuracy:

[0115] The improved YOLOv8-OBB model is used to detect the offset fault of the suspension wire clamp. The dual attention mechanism is introduced in the backbone network to obtain more feature information of the suspension wire clamp. This mechanism effectively reduces the interference of background information on the suspension wire clamp in complex scenes and enhances the model's ability to extract target features. In addition, the weighted residual attention module (eSE module) is introduced in the angle prediction branch, which significantly improves the regression accuracy of the directed bounding box, making the detection of the suspension wire clamp in complex scenes more accurate and reliable.

[0116] 3. Improve inspection efficiency:

[0117] This algorithm can realize large-scale and efficient power inspection work. Through efficient and accurate detection of suspension clamp offset faults, the efficiency of power inspection is significantly improved, and the frequency and workload of manual inspection are reduced. While ensuring the safe operation of the power system, resource allocation is optimized and the efficiency of the overall inspection work is improved.

[0118] Therefore, in summary, the technical solution of the present application demonstrates significant technical advantages in reducing labor costs, improving detection accuracy and improving inspection efficiency, providing strong technical guarantees for the safe and stable operation of the power system.

[0119] The following is an embodiment of the device of the present application at the bottom of the window, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application. Figure 5 Schematic diagram of the structure of the suspension clamp offset fault detection device based on the directed bounding box provided in the embodiment of the present application. Figure 5 As shown, the suspension wire clamp offset fault detection device based on a directed bounding box includes:

[0120] The acquisition module 501 is configured to acquire images collected during the drone inspection process and pre-process the images;

[0121] The segmentation module 502 is configured to input the pre-processed image into the segmentation model, extract the position information of the suspension line clamp and the hull using the segmentation model, and generate a segmentation mask;

[0122] The calculation module 503 is configured to calculate the minimum circumscribed matrix of the suspension line clamp and the hull according to the segmentation mask using a connected region detection algorithm, and generate corresponding rotation box annotation data according to the minimum circumscribed matrix;

[0123] The detection module 504 is configured to input the rotation box annotation data and the preprocessed image into a pre-trained target detection model, use the target detection model to extract and fuse target features, and generate a detection result;

[0124] The judgment module 505 is configured to calculate the angle between the directed bounding box of the suspension clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes corresponding to the suspension clamp and the hull in the detection results, and judge whether the suspension clamp has an offset failure according to the angle.

[0125] In some embodiments, the segmentation model adopts a zero-shot image segmentation model. Figure 5 The segmentation module 502 inputs the preprocessed image into a zero-shot image segmentation model, which includes an image encoder, a prompt encoder and a mask decoder; the image encoder is used to encode the preprocessed image so as to convert the preprocessed image into a high-level feature representation and generate a feature map; the prompt encoder is used to generate a prompt embedding according to the prompt information, and the mask decoder is used to generate a segmentation mask according to the feature map and the prompt embedding, wherein the segmentation mask is used to locate the position information of the suspension line clamp and the hull.

[0126] In some embodiments, Figure 5 The calculation module 503 extracts the segmentation positioning coordinate information of the suspension wire clamp and the hull in the preprocessed image based on the segmentation mask; uses the connected area detection algorithm to process the segmentation positioning coordinate information and determine the connected area of ​​the suspension wire clamp and the hull; calculates the minimum circumscribed matrix of the connected area, generates the rotation box of the suspension wire clamp and the hull, and automatically saves the minimum circumscribed matrix as rotation box annotation data, wherein the rotation box annotation data is used to train the target detection model and perform offset fault detection of the suspension wire clamp.

[0127] In some embodiments, the target detection model includes a backbone network, a neck network, and a detection head. Figure 5 The detection module 504 uses the backbone network to extract features from the input image with annotation information to generate feature maps of multiple scales; inputs the feature map generated by the backbone network into the neck network, and uses the neck network to fuse the feature map to generate a fused feature map; inputs the fused feature map into the detection head, and uses the detection head to forward propagate the fused feature map to generate a detection result, wherein the detection result includes the coordinates, category information and rotation angle of the directed bounding boxes corresponding to the suspension line clamp and the hull respectively.

[0128] In some embodiments, a dual attention mechanism is added to the backbone network, and the dual attention mechanism is used to weaken the complex background information in the feature map; the detection head includes an angle prediction branch, a bounding box regression branch, and a classification branch, wherein a weighted residual attention module is added to the angle prediction branch, and the weighted residual attention module includes an average pooling layer, a fully connected layer, and an activation function layer.

[0129] In some embodiments, Figure 5 The judgment module 505 calculates the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull according to the first angle between the long line of the directed bounding box of the suspension line clamp and the horizontal direction in the rotation angle, and the second angle between the long line of the directed bounding box of the hull and the horizontal direction, using the first angle and the second angle, wherein the angle is the absolute value of the difference between the first angle and the second angle.

[0130] In some embodiments, Figure 5 The judgment module 505 judges whether the angle is within a preset angle range. If the angle is within the angle range, it is judged that the state of the suspension clamp is normal. Otherwise, it is judged that the suspension clamp has an offset fault and a fault detection report is generated.

[0131] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0132] Figure 6 Schematic diagram of the structure of the electronic device 6 provided in the embodiment of the present application. Figure 6 As shown, the electronic device 6 of this embodiment includes: a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and executable on the processor 601. When the processor 601 executes the computer program 603, the steps in the above-mentioned various method embodiments are implemented. Alternatively, when the processor 601 executes the computer program 603, the functions of each module / unit in the above-mentioned various device embodiments are implemented.

[0133] Exemplarily, the computer program 603 may be divided into one or more modules / units, which are stored in the memory 602 and executed by the processor 601 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program 603 in the electronic device 6.

[0134] The electronic device 6 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 6 may include, but is not limited to, a processor 601 and a memory 602. Those skilled in the art will appreciate that Figure 6 It is only an example of the electronic device 6 and does not constitute a limitation of the electronic device 6. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0135] The processor 601 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0136] The memory 602 may be an internal storage unit of the electronic device 6, for example, a hard disk or memory of the electronic device 6. The memory 602 may also be an external storage device of the electronic device 6, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. Further, the memory 602 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device. The memory 602 may also be used to temporarily store data that has been output or is to be output.

[0137] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0138] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0140] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. Multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0141] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0142] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0143] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0144] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for detecting suspension clamp offset fault based on directed bounding box, characterized in that: include: Acquire images collected during the drone inspection process and pre-process the images; The preprocessed image is input into a segmentation model, the location information of the suspension wire clamp and the hull is extracted using the segmentation model, and a segmentation mask is generated; According to the segmentation mask, a minimum circumscribed matrix of the suspension clamp and the hull is calculated using a connected region detection algorithm, and corresponding rotation frame annotation data is generated according to the minimum circumscribed matrix; Inputting the rotation box annotation data and the preprocessed image into a pre-trained target detection model, using the target detection model to extract and fuse target features, and generating a detection result; Calculate the angle between the directed bounding box of the suspension clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes respectively corresponding to the suspension clamp and the hull in the detection result, and determine whether the suspension clamp has an offset fault according to the angle; The target detection model includes a backbone network, a neck network and a detection head, and the target detection model is used to extract and fuse target features and generate detection results, including: Using the backbone network to extract features from an input image with labeled information, generating feature maps of multiple scales; Inputting the feature map generated by the backbone network into the neck network, and using the neck network to perform feature fusion on the feature map to generate a fused feature map; Inputting the fused feature map into the detection head, and using the detection head to forward propagate the fused feature map to generate the detection result, wherein the detection result includes the coordinates, category information and rotation angle of the directed bounding boxes corresponding to the suspension line clamp and the hull respectively; The target detection model also includes: A dual attention mechanism is added to the backbone network, and the dual attention mechanism is used to weaken the complex background information in the feature map; the detection head includes an angle prediction branch, a bounding box regression branch and a classification branch, wherein a weighted residual attention module is added to the angle prediction branch, and the weighted residual attention module includes an average pooling layer, a fully connected layer and an activation function layer.

2. The method according to claim 1, characterized in that The segmentation model adopts a zero-sample image segmentation model, and the segmentation model is used to extract the position information of the suspension line clamp and the hull, and generate a segmentation mask, including: Inputting the preprocessed image into the zero-shot image segmentation model, wherein the zero-shot image segmentation model comprises an image encoder, a hint encoder and a mask decoder; encoding the preprocessed image using the image encoder so as to convert the preprocessed image into a high-level feature representation and generate a feature map; The prompt encoder is used to generate a prompt embedding according to the prompt information, and the mask decoder is used to generate the segmentation mask according to the feature map and the prompt embedding, wherein the segmentation mask is used to locate the position information of the suspension line clamp and the hull.

3. The method according to claim 1, characterized in that The method of calculating the minimum circumscribed matrix of the suspension clamp and the hull using a connected region detection algorithm according to the segmentation mask and generating corresponding rotation frame annotation data according to the minimum circumscribed matrix includes: Based on the segmentation mask, extracting segmentation positioning coordinate information of the suspension line clamp and the hull in the preprocessed image; Using the connected area detection algorithm, the segmented positioning coordinate information is processed to determine the connected area between the suspension line clamp and the hull; The minimum circumscribed matrix of the connected area is calculated, a rotation frame of the suspension clamp and the hull is generated, and the minimum circumscribed matrix is ​​automatically saved as the rotation frame annotation data, wherein the rotation frame annotation data is used to train the target detection model and perform offset fault detection of the suspension clamp.

4. The method according to claim 1, characterized in that: The calculating the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes respectively corresponding to the suspension line clamp and the hull in the detection result comprises: According to a first angle between the long line of the directed bounding box of the suspension line clamp and the horizontal direction in the rotation angle, and a second angle between the long line of the directed bounding box of the hull and the horizontal direction, the angle between the directed bounding box of the suspension line clamp and the directed bounding box of the hull is calculated using the first angle and the second angle, wherein the angle is the absolute value of the difference between the first angle and the second angle.

5. The method according to claim 1, characterized in that The step of judging whether the suspension clamp has an offset fault according to the included angle includes: It is determined whether the angle is within a preset angle range. If the angle is within the angle range, it is determined that the state of the suspension clamp is normal. Otherwise, it is determined that the suspension clamp has an offset fault and a fault detection report is generated.

6. A suspension clamp offset fault detection device based on a directed bounding box, characterized in that: include: An acquisition module is configured to acquire images collected during the inspection process of the drone and pre-process the images; a segmentation module configured to input the preprocessed image into a segmentation model, extract position information of the suspension wire clamp and the hull using the segmentation model, and generate a segmentation mask; A calculation module is configured to calculate the minimum circumscribed matrix of the suspension line clamp and the hull according to the segmentation mask using a connected region detection algorithm, and generate corresponding rotation box annotation data according to the minimum circumscribed matrix; A detection module is configured to input the rotation box annotation data and the preprocessed image into a pre-trained target detection model, use the target detection model to extract and fuse target features, and generate a detection result; a judgment module configured to calculate an angle between the directed bounding box of the suspension clamp and the directed bounding box of the hull according to the rotation angles of the directed bounding boxes respectively corresponding to the suspension clamp and the hull in the detection result, and judge whether the suspension clamp has an offset fault according to the angle; The target detection model includes a backbone network, a neck network and a detection head. The detection module is used to use the backbone network to extract features from an input image with annotation information to generate feature maps of multiple scales; the feature map generated by the backbone network is input into the neck network, and the feature map is fused using the neck network to generate a fused feature map; the fused feature map is input into the detection head, and the fused feature map is forward propagated using the detection head to generate the detection result, wherein the detection result includes the coordinates, category information and rotation angle of the directed bounding boxes corresponding to the suspension line clamp and the hull respectively; A dual attention mechanism is added to the backbone network, and the dual attention mechanism is used to weaken the complex background information in the feature map; the detection head includes an angle prediction branch, a bounding box regression branch and a classification branch, wherein a weighted residual attention module is added to the angle prediction branch, and the weighted residual attention module includes an average pooling layer, a fully connected layer and an activation function layer.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images