Edge end composite insulator heating defect judgment system, device and method

Abstract

The invention discloses an edge end composite insulator heating defect judgment system, device and method. The method comprises the following steps: receiving an infrared picture heating detection request from a handle of the unmanned aerial vehicle through an infrared image acquisition module; the model training conversion module is used for training and converting a composite insulator image detection segmentation model; the composite insulator detection module is used for automatically identifying and positioning a target of a composite insulator; the composite insulator segmentation module extracts a complete shape contour of the composite insulator; the composite insulator center line extraction module is used for extracting the center line of a composite insulator core rod; a composite insulator center line temperature fitting extraction module extracts a corresponding temperature value; and the composite insulator heating defect detection module judges the heating condition of the corresponding composite insulator. The method has the beneficial effects that the heating defect identification of the unmanned aerial vehicle infrared picture of the power transmission line at the edge end of the inspection site is realized, and the intelligent and automatic level of the heating defect detection of the composite insulator of the power transmission line is improved.

Description

Technical Field

[0001] This invention relates to the field of power infrared image detection technology, and in particular to a system, device and method for judging heating defects in edge-end composite insulators. Background Technology

[0002] With the continuous advancement of smart grid construction, the safe and stable operation of transmission lines has received increasing attention. Composite insulators and other components are crucial parts of transmission line hardware, and their operational status directly affects the reliability of the line. During long-term operation, composite insulators on transmission lines are prone to localized overheating due to poor contact, corrosion, or mechanical damage. If not detected in time, this can lead to equipment damage or even line failure.

[0003] Currently, drones equipped with infrared thermal imagers have become the primary means of inspecting composite insulators on power transmission lines, offering advantages such as high efficiency, flexibility, and wide coverage, and are widely used for detecting overheating in composite insulators. However, existing technologies still have significant shortcomings: infrared images collected by drones typically need to be transmitted back to a backend for interpretation by professionals, relying on human experience to determine whether composite insulators are overheating abnormally. This is not only inefficient but also susceptible to subjective factors, failing to meet the dual requirements of real-time performance and accuracy. Especially in complex field environments, inspection personnel lack professional analytical tools and cannot make quick decisions on-site, hindering the effectiveness of inspections. Furthermore, existing edge computing devices largely rely on foreign architecture chips (Nvidia), posing supply chain security risks and failing to meet the urgent need for independent control in the power system. Therefore, there is an urgent need for an edge-end overheating detection solution and equipment based on a domestically produced hardware platform, possessing real-time autonomous analysis capabilities in the field. Summary of the Invention

[0004] The present invention aims to overcome the above-mentioned deficiencies in the prior art and provides a system, equipment and method for judging the thermal defects of composite insulators at the edge of the inspection site by using infrared images from drones.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A system for judging heating defects in composite insulators at their edges includes an infrared image acquisition module, a model training and conversion module, a composite insulator detection module, a composite insulator segmentation module, a composite insulator centerline extraction module, a composite insulator centerline temperature fitting and extraction module, and a composite insulator heating defect detection module, wherein: The infrared image acquisition module receives an infrared image heat detection request from the drone handle; The model training and conversion module realizes the training and conversion of the composite insulator image detection and segmentation model; The composite insulator detection module is based on a composite insulator image detection and segmentation model to automatically identify and locate composite insulator targets in infrared images from the drone handle; The composite insulator segmentation module performs fine segmentation on the minimum circumscribed rectangle region containing the composite insulator output by the composite insulator detection module, so as to accurately extract the complete shape contour of the composite insulator. The composite insulator centerline extraction module extracts the centerline of the composite insulator core rod based on extracting the complete morphological outline of the composite insulator. The composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value from the pixel point set of the composite insulator core rod centerline output by the composite insulator centerline extraction module. The composite insulator heating defect detection module determines the heating status of the corresponding composite insulator based on the temperature value extracted by the composite insulator centerline temperature fitting and extraction module.

[0006] This invention aims to identify thermal defects in composite insulators at the edge of power transmission lines during inspections using UAV infrared images. It achieves this by segmenting and judging the temperature of the composite insulators, combined with modifications to domestically produced chips. This solves the problem of identifying thermal defects in composite insulators at the edge of power transmission lines during inspections. The invention implements end-to-end edge processing, from image acquisition, target detection, fine segmentation, centerline extraction, temperature acquisition to defect judgment. Relying on a domestically produced ARM+NPU hardware platform, it ensures the system's autonomy and real-time field analysis capabilities, significantly improving the intelligence and automation level of thermal defect detection in composite insulators of power transmission lines.

[0007] Preferably, the infrared image acquisition module accepts an infrared image heat detection request from the drone handle via a POST interface. The request information includes... <Unique image identifier, image name, image data, and location of the image> The image data is transmitted using Base64 encoding.

[0008] Preferably, the specific training and transformation process of the composite insulator image detection and segmentation model is as follows: (11) The composite insulator image detection and segmentation model adopts the YOLOv8 network architecture based on deep learning for target detection and semantic segmentation, and performs instance segmentation modeling on infrared images of composite insulators of transmission lines. (12) By collecting composite insulator image data under different working conditions, climate conditions and shooting angles, and by using manual annotation tools to perform pixel-level accurate annotation of composite insulator regions, a training dataset containing category labels and segmentation masks is generated. (13) The YOLOv8-Seg network was trained end-to-end using the training dataset. During the training process, the backbone network CSPDarknet53 extracted multi-scale features, the neck network PAN-FPN achieved feature fusion, the detection head output bounding boxes and confidence scores, and the segmentation head upsampled the feature maps through deconvolution operations, outputting a binary segmentation mask M∈R(H×W) with the same resolution as the input image, where H and W represent the height and width of the image matrix, respectively. i,j =1 indicates that pixel (i,j) belongs to the composite insulator region, M i,j =0 indicates background; (14) After training is completed, save the model parameters as a .pt model in PyTorch format as the basis for subsequent model conversion; (15) The trained .pt model is converted into a general intermediate representation format .onnx model through the ONNX export interface, preserving the complete network structure and weight information, and achieving cross-platform compatibility; (16) For edge computing devices equipped with domestic NPU, call their dedicated model compilation toolchain to further compile and optimize the .onnx model into a dedicated inference format .axmodel model suitable for the target NPU. The final generated .axmodel model can be directly loaded into the edge computing device with domestic ARM architecture chip for real-time execution of composite insulator detection and segmentation tasks.

[0009] As a preferred embodiment, the composite insulator detection module operates as follows: (21) The Base64 encoding of the image in the infrared image acquisition module is converted into an infrared image and saved for later processing; (22) The input infrared image is loaded by the edge device to perform forward inference using the .axmodel model. Multi-scale features are extracted through the backbone network, and high and low level semantic information is fused through the neck network. The target bounding box coordinates, category label and confidence score of the composite insulator are output by the detection head. (23) The overlapping detection boxes are filtered by the non-maximum suppression algorithm, and the optimal detection result is retained, that is, the smallest bounding rectangle containing the composite insulator. The pixel coordinates of the smallest bounding rectangle are as follows: [(X L ,Y L ), (X R ,Y R )] Among them, (X) L ,Y L () represents the top left pixel of the detected target, (X) R ,Y R () indicates the lower right pixel of the detected target.

[0010] Preferably, the segmentation process in the composite insulator segmentation module adopts the YOLOv8-Seg network, a lightweight semantic segmentation model based on the YOLOv8 architecture, to extract the boundary contour of the segmentation result, and outputs a series of continuous pixel coordinate matrices C={(x1,y1),(x2,y2),...,(xn,yn)} distributed along the shape of the composite insulator.

[0011] Preferably, the composite insulator centerline extraction module implements the following extraction process for the composite insulator core rod centerline: (31) Using OpenCV, obtain the minimum bounding rectangle Q based on the continuous pixel coordinate matrix C of the composite insulator shape. Q=[(x1,y1),(x2,y2),(x3,y3),(x4,y4)] Here, (x1,y1), (x2,y2), (x3,y3), and (x4,y4) represent the four vertices of the minimum bounding rectangle. (32) Obtain the centroid b-center of the smallest bounding rectangle and the lengths (a, b) of each side of this bounding rectangle, where a represents the length of the smallest bounding rectangle and b represents the width of the smallest bounding rectangle; compare the lengths of each side of the rectangle, and take the point where the longest side is located as the straight line to obtain its slope k. (33) Form a center line L1 with the centroid b-center and the slope k, and move up and down by one pixel in the direction perpendicular to this center line to obtain two other center lines L2 and L3 parallel to this center line. (34) For centerline L1, generate its discrete point sequence in the image using the Bresenham line algorithm. Determine its row and column intervals by obtaining the boundary range of the minimum bounding rectangle of the composite insulator. Traverse each pixel on the centerline and determine whether its horizontal axis coordinates simultaneously satisfy the row and column constraints of the minimum bounding rectangle. If satisfied, the point is located inside the minimum bounding rectangle and is placed in the centerline pixel set D1. Similarly, obtain the corresponding centerline pixel sets D2 and D3 for the other two parallel centerlines L2 and L3.

[0012] Where D={D1,D2,D3} represents the set of pixels of the three center lines within the smallest rectangle of the composite insulator, and each row represents the set of pixels of one center line.

[0013] Preferably, the composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value based on the output centerline pixel set as follows: (41) Using the obtained centerline pixel coordinate sequence as a spatial index, call the temperature data interface provided by the UAV infrared camera SDK to access the radiation temperature matrix of the original infrared image, thereby extracting the precise temperature value corresponding to each pixel and forming a temperature sequence distributed along the centerline. (42) By constructing the intermediate conversion layer software QEMU, the key decoding and temperature analysis logic is rewritten and compiled into ARM native instructions using cross-compilation technology. At the same time, by encapsulating RESTful API services or local Socket communication methods, seamless integration with the system is achieved. (43) The module obtains the centerline temperature matrix T as follows:

[0014] Among them, (t) 11 )...(t) 1i )...(t) 1n Temperature sequence distributed along centerline L1; (t) 21 )...(t) 2i )...(t) 2n Temperature sequence distributed along centerline L2; (t) 31 )...(t) 3i )...(t) 3n Temperature sequence distributed along centerline L3.

[0015] Preferably, the composite insulator heating defect detection module determines the heating status of the corresponding composite insulator based on the temperature matrix as follows: (51) For the centerline temperature matrix T, obtain the median t of the temperature. i The column containing the temperature it takes, and all temperatures in that column are set to t. i ; (52) Obtain t i The temperature value in column i+1, and the sum of each temperature and t. i The tangent vectors formed by the temperatures are compared, and the maximum value of the temperatures of the two closest tangent vectors is taken as the temperature of this column. This process is repeated to fit the temperature matrix T into a temperature vector T1. T1={t1,t i ,...,t n}; (53) Determine whether the insulator is heating up based on the temperature vector.

[0016] The present invention also provides a device for judging the heating defects of edge-end composite insulators, including a communication interface, an ARM architecture processor, an NPU image processor, a memory, and an edge-end composite insulator heating defect judgment system running in the device.

[0017] This invention also provides a method for judging heating defects in edge-end composite insulators, based on an edge-end composite insulator heating defect judgment device, and the specific implementation process is as follows: (1) A connection is established with the drone handle through the communication interface. The Wi-Fi wireless communication method is adopted. After the connection is established, the edge composite insulator heating defect judgment device receives the infrared image heating detection request from the drone handle through the post interface. After receiving the request, the infrared image acquisition module decodes the Base64 encoded image data and restores it into an infrared image, and stores it locally for subsequent processing. (2) The composite insulator detection module is started, loading the .axmodel format model already deployed on the domestic ARM architecture chip NPU image processor. After forward inference of the input infrared image, the model outputs the target bounding box coordinates, category label, and confidence score of the composite insulator. The minimum bounding box containing the composite insulator is obtained by filtering through the non-maximum suppression algorithm. The pixel coordinates of its upper left and lower right corners are denoted as (X... L ,Y L ) and (X R ,Y R This provides a precise region for subsequent segmentation; (3) The composite insulator segmentation module uses the minimum bounding rectangle as the input region, calls the segmentation head in the same .axmodel model, and performs refined semantic segmentation on the region; through backbone feature extraction and multi-scale fusion, it outputs a pixel-level segmentation mask, accurately extracts the complete morphological contour of the composite insulator, and generates a continuous pixel coordinate sequence C={(x1,y1),(x2,y2),...,(x n ,y n )}; (4) Based on the complete morphological outline, the composite insulator centerline extraction module is started. The minimum bounding rectangle Q of the outline is calculated using OpenCV tools to obtain its centroid b-center and the slope k of the long side direction. The centerline L1 is constructed using the slope k and the centroid b-center, and its parallel centerlines L2 and L3 are generated. The discrete pixel points of the three lines are generated by the Bresenham line algorithm, and the points located within the minimum bounding rectangle are selected to form the centerline pixel point set D={D1,D2,D3}, which represents the potential center path of the composite insulator core rod. (5) The composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value based on the set of centerline pixels; by constructing a QEMU dynamic binary translation environment, the x86 instruction is compatible with the ARM architecture processor, and combined with local Socket communication, the SDK is called to access the radiation temperature matrix of the original infrared image; using the centerline pixels as the index, the temperature of each point is extracted to form the original temperature matrix T distributed along the centerline; (6) The composite insulator heating defect detection module processes the temperature matrix T and takes the median temperature t. i Using its column as a benchmark, the temperature of adjacent columns is fitted and corrected using the tangent vector comparison method, effectively filtering out outlier interference and generating a smooth temperature vector T1={t1,t... i ,...,t n The system determines the heating status based on a preset threshold, and the result is transmitted back to the drone handle via a communication interface, enabling real-time, autonomous defect identification at the inspection site.

[0018] The beneficial effects of this invention are: it realizes the edge processing of the entire process from image acquisition, target detection, fine segmentation, centerline extraction, temperature acquisition to defect judgment. Relying on the domestic ARM+NPU hardware platform, it ensures the system's autonomy and controllability and real-time field analysis capabilities, realizes the identification of heating defects in UAV infrared images of power transmission lines at the edge of the inspection site, and significantly improves the intelligence and automation level of heating defect detection of composite insulators of power transmission lines. Attached Figure Description

[0019] Figure 1 This is a system framework diagram of the present invention; Figure 2 This is a block diagram of the device of the present invention; Figure 3 This is a flowchart of the method of the present invention. Detailed Implementation

[0020] The present invention will now be further described with reference to the accompanying drawings and specific embodiments.

[0021] To better achieve the identification of thermal defects in composite insulators of transmission lines at the edge of the inspection site using infrared images from drones, this invention provides a system, device, and method for judging thermal defects in composite insulators of transmission lines at the edge of the inspection site based on a domestically produced ARM architecture chip, thereby realizing the intelligent identification of thermal defects in composite insulators of transmission lines at the inspection site.

[0022] like Figure 1In the described embodiment, a system for judging heating defects in edge-end composite insulators includes an infrared image acquisition module, a model training and conversion module, a composite insulator detection module, a composite insulator segmentation module, a composite insulator centerline extraction module, a composite insulator centerline temperature fitting and extraction module, and a composite insulator heating defect detection module, wherein: The infrared image acquisition module primarily accepts infrared image heat detection requests from the drone controller via a POST interface. The requested information includes... <Unique image identifier, image name, image data, and location of the image> The image data is transmitted using Base64 encoding.

[0023] The model training and conversion module implements the training and conversion of composite insulator image detection and segmentation models. The specific process is as follows: (11) The model adopts the YOLOv8 network architecture based on deep learning for object detection and semantic segmentation, and performs instance segmentation modeling on infrared images of composite insulators of transmission lines. (12) By collecting a large amount of composite insulator image data under different working conditions, climate conditions and shooting angles, and by using manual annotation tools to perform pixel-level accurate annotation of composite insulator regions, a training dataset containing category labels and segmentation masks is generated. (13) The YOLOv8-Seg network was trained end-to-end using the training dataset. During the training process, the backbone network CSPDarknet53 extracted multi-scale features, the neck network PAN-FPN achieved feature fusion, the detection head output bounding boxes and confidence scores, and the segmentation head upsampled the feature maps through deconvolution operations, outputting a binary segmentation mask M∈R(H×W) with the same resolution as the input image, where H and W represent the height and width of the image matrix, respectively. i,j =1 indicates that pixel (i,j) belongs to the composite insulator region, M i,j =0 indicates background; the loss function uses a combination of object detection loss (such as CIoU Loss) and segmentation loss (a weighted sum of Binary Cross-Entropy Loss and Dice Loss) to ensure that the model achieves optimal performance in both localization accuracy and contour segmentation accuracy; (14) After training is completed, save the model parameters as a .pt model in PyTorch format as the basis for subsequent model conversion; (15) To achieve efficient conversion and deployment of the model on domestic ARM architecture chips, the trained .pt models are sequentially converted in format and optimized for hardware adaptation. Specifically, the trained .pt models are converted into .onnx models in a general intermediate representation format through the ONNX (Open Neural Network Exchange) export interface, preserving the complete network structure and weight information to achieve cross-platform compatibility; (16) For edge computing devices equipped with domestically produced NPUs, their dedicated model compilation toolchain (RKNN Toolkit, etc.) is called to further compile and optimize the .onnx model into a dedicated inference format .axmodel model suitable for the target NPU. This process includes key technologies such as operator fusion, weight quantization (e.g., FP32 to INT8 conversion), and memory layout optimization, which significantly improves the inference speed of the model at the edge and reduces power consumption. The final generated .axmodel model can be directly loaded into edge computing devices with domestically produced ARM architecture chips for real-time execution of composite insulator detection and segmentation tasks.

[0024] The composite insulator detection module is based on a composite insulator image detection and segmentation model to automatically identify and locate composite insulator targets in infrared images from a drone handle. Its specific operation is as follows: (21) The Base64 encoding of the image in the infrared image acquisition module is converted into an infrared image and saved for later processing; (22) The input infrared image is loaded by the edge device to perform forward inference using the .axmodel model. Multi-scale features are extracted through the backbone network, and high and low level semantic information is fused through the neck network. The target bounding box coordinates, category label and confidence score of the composite insulator are output by the detection head. (23) The overlapping detection boxes are filtered by the non-maximum suppression (NMS) algorithm, and the best detection result is retained, that is, the smallest bounding rectangle containing the composite insulator. The pixel coordinates of the smallest bounding rectangle are as follows: [(X L ,Y L ), (X R ,Y R )] Among them, (X) L ,Y L () represents the top left pixel of the detected target, (X) R ,Y R () indicates the lower right pixel of the detected target.

[0025] The composite insulator segmentation module performs fine segmentation on the minimum bounding rectangle region containing the composite insulator output by the composite insulator detection module to accurately extract the complete shape contour of the composite insulator. The segmentation process adopts a lightweight semantic segmentation model (YOLOv8-Seg network) based on the YOLOv8 architecture to extract the boundary contour of the segmentation result. The output is a series of continuous pixel coordinate matrices C={(x1,y1),(x2,y2),...,(xn,yn)} distributed along the shape of the composite insulator.

[0026] The composite insulator centerline extraction module extracts the centerline of the composite insulator core rod based on extracting the complete morphological outline of the composite insulator. The extraction process is as follows: (31) Using OpenCV, obtain the minimum bounding rectangle Q based on the continuous pixel coordinate matrix C of the composite insulator shape. Q=[(x1,y1),(x2,y2),(x3,y3),(x4,y4)] Here, (x1,y1), (x2,y2), (x3,y3), and (x4,y4) represent the four vertices of the minimum bounding rectangle. (32) Obtain the centroid b-center of the smallest bounding rectangle and the lengths (a, b) of each side of this bounding rectangle, where a represents the length of the smallest bounding rectangle and b represents the width of the smallest bounding rectangle; compare the lengths of each side of the rectangle, considering the shooting direction of the UAV and the fact that composite insulators are generally slender, take the point where the long side is located as the straight line, and obtain its slope k. (33) Form a center line L1 with the centroid b-center and the slope k, and move up and down by one pixel in the direction perpendicular to this center line to obtain two other center lines L2 and L3 parallel to this center line. (34) For centerline L1, generate its discrete point sequence in the image using the Bresenham line algorithm. Determine its row and column intervals by obtaining the boundary range of the minimum bounding rectangle of the composite insulator. Traverse each pixel on the centerline and determine whether its horizontal axis coordinates simultaneously satisfy the row and column constraints of the minimum bounding rectangle. If satisfied, the point is located inside the minimum bounding rectangle and is placed in the centerline pixel set D1. Similarly, obtain the corresponding centerline pixel sets D2 and D3 for the other two parallel centerlines L2 and L3.

[0027] Where D={D1,D2,D3} represents the set of pixels of the three center lines within the smallest rectangle of the composite insulator, and each row represents the set of pixels of one center line.

[0028] The composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value from the pixel set of the composite insulator core rod centerline output by the composite insulator centerline extraction module. The process is as follows: (41) Using the obtained centerline pixel coordinate sequence as a spatial index, call the temperature data interface provided by the drone (DJI) infrared camera SDK to access the radiation temperature matrix of the original infrared image, thereby extracting the precise temperature value corresponding to each pixel point and forming a temperature sequence distributed along the centerline. (42) Since DJI's official SDK only provides dynamic link libraries based on the x86 architecture, it cannot run directly on domestic ARM architecture edge computing devices, thus cross-platform adaptation is required. This invention constructs an intermediate conversion layer software QEMU, uses cross-compilation technology to rewrite and compile the key decoding and temperature parsing logic into native ARM instructions, and achieves seamless integration with the system by encapsulating RESTful API services or local Socket communication methods; this intermediate conversion layer software is compatible with mainstream domestic ARM chips, ensuring efficient and stable acquisition of temperature data at the edge; The module obtains the centerline temperature matrix T as follows:

[0029] Among them, (t) 11 )...(t) 1i )...(t) 1n Temperature sequence distributed along centerline L1; (t) 21 )...(t) 2i )...(t) 2n Temperature sequence distributed along centerline L2; (t) 31 )...(t) 3i )...(t) 3n Temperature sequence distributed along centerline L3.

[0030] The composite insulator heating defect detection module determines the heating status of the corresponding composite insulator based on the temperature value extracted by the composite insulator centerline temperature fitting and extraction module. The process is as follows: (51) For the centerline temperature matrix T, obtain the median t of the temperature. i The column containing the temperature it takes, and all temperatures in that column are set to t. i ; (52) Obtain t i The temperature value in column i+1, and the sum of each temperature and t. i By comparing the tangent vectors formed by the temperatures, the maximum value of the temperatures of the two closest tangent vectors is taken as the temperature of this column, and so on. This effectively removes the temperature error caused by outlier temperatures. The temperature matrix T is then fitted into a temperature vector T1. T1={t1,t i ,...,t n}; (53) Determine whether the insulator is heating up based on the temperature vector. The heating status is determined by the specific temperature range, as shown in the table below.

[0031]

[0032] like Figure 2 As shown, the present invention also provides a device for judging the heating defects of edge-end composite insulators, including a communication interface, an ARM architecture processor, an NPU image processor, a memory, and an edge-end composite insulator heating defect judgment system running in the device.

[0033] like Figure 3 As shown, the present invention also provides a method for judging heating defects in edge-end composite insulators, based on an edge-end composite insulator heating defect judgment device, and the specific implementation process is as follows: (1) A connection is established with the drone handle via a communication interface, primarily using Wi-Fi wireless communication to ensure real-time and stable data transmission. After the connection is established, the edge-end composite insulator heating defect judgment device receives an infrared image heating detection request from the drone handle via a post interface. This request includes a unique image identifier, image name, shooting location, and Base64 encoded image data. Upon receiving the request, the infrared image acquisition module decodes the Base64 encoded image data to restore it to an infrared image and stores it locally for subsequent processing. (2) The composite insulator detection module is started, loading the .axmodel format model already deployed on the domestic ARM architecture chip NPU image processor. This model is generated by training the YOLOv8-Seg network, converting it through the ONNX intermediate format, and compiling and optimizing it using the RKNN Toolkit. It is specifically designed for NPU hardware acceleration. After forward inference of the input infrared image, the model outputs the target bounding box coordinates, category label, and confidence score of the composite insulator. The minimum bounding box containing the composite insulator is obtained by filtering through the non-maximum suppression (NMS) algorithm. The pixel coordinates of its upper left and lower right corners are denoted as (X... L ,Y L ) and (X R ,Y R This provides a precise region for subsequent segmentation; (3) The composite insulator segmentation module uses the minimum bounding rectangle as the input region, calls the segmentation head in the same .axmodel model, and performs refined semantic segmentation on the region; through backbone feature extraction and multi-scale fusion, it outputs a pixel-level segmentation mask, accurately extracts the complete morphological contour of the composite insulator, and generates a continuous pixel coordinate sequence C={(x1,y1),(x2,y2),...,(x n ,y n )}; (4) Based on the complete morphological outline, the composite insulator centerline extraction module is started. The minimum bounding rectangle Q of the outline is calculated using OpenCV tools to obtain its centroid b-center and the slope k of the long side direction. The centerline L1 is constructed using the slope k and the centroid b-center, and its parallel centerlines L2 and L3 are generated. The discrete pixel points of the three lines are generated by the Bresenham line algorithm, and the points located within the minimum bounding rectangle are selected to form the centerline pixel point set D={D1,D2,D3}, which represents the potential center path of the composite insulator core rod. (5) The composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value based on the set of centerline pixels; since DJI infrared SDK only supports x86 architecture, this invention constructs a QEMU dynamic binary translation environment to achieve compatible operation of x86 instructions on ARM architecture processors, and combines local Socket communication to call the SDK to access the radiation temperature matrix of the original infrared image; using the centerline pixels as indexes, the temperature of each point is extracted to form the original temperature matrix T distributed along the centerline; (6) The composite insulator heating defect detection module processes the temperature matrix T and takes the median temperature t. i Using its column as a benchmark, the temperature of adjacent columns is fitted and corrected using the tangent vector comparison method, effectively filtering out outlier interference and generating a smooth temperature vector T1={t1,t... i ,...,t n The system determines the overheating status based on preset thresholds: temperatures ≤70℃ are normal, 70℃ < temperature < 120℃ are moderate overheating, and ≥120℃ are severe overheating. The determination results are transmitted back to the drone's controller via a communication interface, enabling real-time, autonomous defect identification at the inspection site.

[0034] This invention realizes edge processing of the entire process from image acquisition, target detection, fine segmentation, centerline extraction, temperature acquisition to defect judgment. Relying on the domestic ARM+NPU hardware platform, it ensures the system's independent controllability and real-time field analysis capabilities, and significantly improves the intelligence and automation level of thermal defect detection of composite insulators in power transmission lines.

Claims

1. A system for judging heating defects in composite insulators at their edge ends, characterized in that, It includes an infrared image acquisition module, a model training and conversion module, a composite insulator detection module, a composite insulator segmentation module, a composite insulator centerline extraction module, a composite insulator centerline temperature fitting and extraction module, and a composite insulator heating defect detection module, among which: The infrared image acquisition module receives an infrared image heat detection request from the drone handle; The model training and conversion module realizes the training and conversion of the composite insulator image detection and segmentation model; The composite insulator detection module is based on a composite insulator image detection and segmentation model to automatically identify and locate composite insulator targets in infrared images from the drone handle; The composite insulator segmentation module performs fine segmentation on the minimum circumscribed rectangle region containing the composite insulator output by the composite insulator detection module, so as to accurately extract the complete shape contour of the composite insulator. The composite insulator centerline extraction module extracts the centerline of the composite insulator core rod based on extracting the complete morphological outline of the composite insulator. The composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value from the pixel point set of the composite insulator core rod centerline output by the composite insulator centerline extraction module. The composite insulator heating defect detection module determines the heating status of the corresponding composite insulator based on the temperature value extracted by the composite insulator centerline temperature fitting and extraction module.

2. The edge-end composite insulator heating defect judgment system according to claim 1, characterized in that, The infrared image acquisition module accepts an infrared image heat detection request from the drone controller via a POST interface. The request information includes... <Unique image identifier, image name, image data, and location of the image> The image data is transmitted using Base64 encoding.

3. The edge-end composite insulator heating defect judgment system according to claim 2, characterized in that, The specific training and transformation process of the composite insulator image detection and segmentation model is as follows: (11) The composite insulator image detection and segmentation model adopts the YOLOv8 network architecture based on deep learning for target detection and semantic segmentation, and performs instance segmentation modeling on infrared images of composite insulators of transmission lines. (12) By collecting composite insulator image data under different working conditions, climate conditions and shooting angles, and by using manual annotation tools to perform pixel-level accurate annotation of composite insulator regions, a training dataset containing category labels and segmentation masks is generated. (13) The YOLOv8-Seg network was trained end-to-end using the training dataset. During the training process, the backbone network CSPDarknet53 extracted multi-scale features, the neck network PAN-FPN achieved feature fusion, the detection head output bounding boxes and confidence scores, and the segmentation head upsampled the feature maps through deconvolution operations, outputting a binary segmentation mask M∈R(H×W) with the same resolution as the input image, where H and W represent the height and width of the image matrix, respectively. i,j =1 indicates that pixel (i,j) belongs to the composite insulator region, M i,j =0 indicates background; (14) After training is completed, save the model parameters as a .pt model in PyTorch format as the basis for subsequent model conversion; (15) The trained .pt model is converted into a general intermediate representation format .onnx model through the ONNX export interface, preserving the complete network structure and weight information, and achieving cross-platform compatibility; (16) For edge computing devices equipped with domestic NPU, call their dedicated model compilation toolchain to further compile and optimize the .onnx model into a dedicated inference format .axmodel model suitable for the target NPU. The final generated .axmodel model can be directly loaded into the edge computing device with domestic ARM architecture chip for real-time execution of composite insulator detection and segmentation tasks.

4. The edge-end composite insulator heating defect judgment system according to claim 3, characterized in that, The specific operation of the composite insulator detection module is as follows: (21) The Base64 encoding of the image in the infrared image acquisition module is converted into an infrared image and saved for later processing; (22) The input infrared image is loaded by the edge device to perform forward inference using the .axmodel model. Multi-scale features are extracted through the backbone network, and high and low level semantic information is fused through the neck network. The target bounding box coordinates, category label and confidence score of the composite insulator are output by the detection head. (23) The overlapping detection boxes are filtered by the non-maximum suppression algorithm, and the optimal detection result is retained, that is, the smallest bounding rectangle containing the composite insulator. The pixel coordinates of the smallest bounding rectangle are as follows: [(X L ,AND L ), (X R ,AND R )] Among them, (X) L ,Y L () represents the top left pixel of the detected target, (X) R ,Y R () indicates the lower right pixel of the detected target.

5. The edge-end composite insulator heating defect judgment system according to claim 4, characterized in that, The segmentation process in the composite insulator segmentation module adopts the YOLOv8-Seg network, a lightweight semantic segmentation model based on the YOLOv8 architecture, to extract the boundary contour of the segmentation result. The output is a series of continuous pixel coordinate matrices C={(x1,y1),(x2,y2),...,(xn,yn)} distributed along the shape of the composite insulator.

6. The edge-end composite insulator heating defect judgment system according to claim 5, characterized in that, The composite insulator centerline extraction module implements the following extraction process for the composite insulator core rod centerline: (31) Using OpenCV, obtain the minimum bounding rectangle Q based on the continuous pixel coordinate matrix C of the composite insulator shape. Q=[(x1,y1),(x2,y2),(x3,y3),(x4,y4)] Here, (x1,y1), (x2,y2), (x3,y3), and (x4,y4) represent the four vertices of the minimum bounding rectangle. (32) Obtain the centroid b-center of the smallest bounding rectangle and the lengths (a, b) of each side of this bounding rectangle, where a represents the length of the smallest bounding rectangle and b represents the width of the smallest bounding rectangle; compare the lengths of each side of the rectangle, and take the point where the longest side is located as the straight line to obtain its slope k. (33) Form a center line L1 with the centroid b-center and the slope k, and move up and down by one pixel in the direction perpendicular to this center line to obtain two other center lines L2 and L3 parallel to this center line. (34) For centerline L1, generate its discrete point sequence in the image using the Bresenham line algorithm. Determine its row and column intervals by obtaining the boundary range of the minimum bounding rectangle of the composite insulator. Traverse each pixel on the centerline and determine whether its horizontal axis coordinates simultaneously satisfy the row and column constraints of the minimum bounding rectangle. If satisfied, the point is located inside the minimum bounding rectangle and is placed in the centerline pixel set D1. Similarly, obtain the corresponding centerline pixel sets D2 and D3 for the other two parallel centerlines L2 and L3. Where D={D1,D2,D3} represents the set of pixels of the three center lines within the smallest rectangle of the composite insulator, and each row represents the set of pixels of one center line.

7. The edge-end composite insulator heating defect judgment system according to claim 6, characterized in that, The process by which the composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value from the output centerline pixel set is as follows: (41) Using the obtained centerline pixel coordinate sequence as a spatial index, call the temperature data interface provided by the UAV infrared camera SDK to access the radiation temperature matrix of the original infrared image, thereby extracting the precise temperature value corresponding to each pixel and forming a temperature sequence distributed along the centerline. (42) By constructing the intermediate conversion layer software QEMU, the key decoding and temperature analysis logic is rewritten and compiled into ARM native instructions using cross-compilation technology. At the same time, by encapsulating RESTful API services or local Socket communication methods, seamless integration with the system is achieved. (43) The module obtains the centerline temperature matrix T as follows: Among them, (t) 11 )...(t) 1i )...(t) 1n Temperature sequence distributed along centerline L1; (t) 21 )...(t) 2i )...(t) 2n Temperature sequence distributed along centerline L2; (t) 31 )...(t) 3i )...(t) 3n Temperature sequence distributed along centerline L3.

8. The edge-end composite insulator heating defect judgment system according to claim 7, characterized in that, The process by which the composite insulator heating defect detection module determines the heating status of the corresponding composite insulator based on the temperature matrix is ​​as follows: (51) For the centerline temperature matrix T, obtain the median t of the temperature. i The column containing the temperature it takes, and all temperatures in that column are set to t. i ; (52) Obtain t i The temperature value in column i+1, and the sum of each temperature and t. i The tangent vectors formed by the temperatures are compared, and the maximum value of the temperatures of the two closest tangent vectors is taken as the temperature of this column. This process is repeated to fit the temperature matrix T into a temperature vector T1. T1={t1,t i ,...,t n }; (53) Determine whether the insulator is heating up based on the temperature vector.

9. The device for judging heating defects in edge-end composite insulators according to claim 8, characterized in that, It includes a communication interface, an ARM architecture processor, an NPU image processor, a memory, and an edge-end composite insulator heating defect detection system running in the edge-end composite insulator heating defect detection device.

10. The method for judging heating defects in edge-end composite insulators according to claim 9, characterized in that, The specific implementation process of the equipment for judging heating defects in edge-end composite insulators is as follows: (1) A connection is established with the drone handle through the communication interface. The Wi-Fi wireless communication method is adopted. After the connection is established, the edge composite insulator heating defect judgment device receives the infrared image heating detection request from the drone handle through the post interface. After receiving the request, the infrared image acquisition module decodes the Base64 encoded image data and restores it into an infrared image, and stores it locally for subsequent processing. (2) The composite insulator detection module is started, loading the .axmodel format model already deployed on the domestic ARM architecture chip NPU image processor. After forward inference of the input infrared image, the model outputs the target bounding box coordinates, category label, and confidence score of the composite insulator. The minimum bounding box containing the composite insulator is obtained by filtering through the non-maximum suppression algorithm. The pixel coordinates of its upper left and lower right corners are denoted as (X... L ,Y L ) and (X R ,Y R This provides a precise region for subsequent segmentation; (3) The composite insulator segmentation module uses the minimum bounding rectangle as the input region, calls the segmentation head in the same .axmodel model, and performs refined semantic segmentation on the region; through backbone feature extraction and multi-scale fusion, it outputs a pixel-level segmentation mask, accurately extracts the complete morphological contour of the composite insulator, and generates a continuous pixel coordinate sequence C={(x1,y1),(x2,y2),...,(x n ,y n )}; (4) Based on the complete morphological outline, the composite insulator centerline extraction module is started. The minimum bounding rectangle Q of the outline is calculated using OpenCV tools to obtain its centroid b-center and the slope k of the long side direction. The centerline L1 is constructed using the slope k and the centroid b-center, and its parallel centerlines L2 and L3 are generated. The discrete pixel points of the three lines are generated by the Bresenham line algorithm, and the points located within the minimum bounding rectangle are selected to form the centerline pixel point set D={D1,D2,D3}, which represents the potential center path of the composite insulator core rod. (5) The composite insulator centerline temperature fitting and extraction module extracts the corresponding temperature value based on the set of centerline pixels; By constructing a QEMU dynamic binary translation environment, x86 instructions can be run compatiblely on ARM architecture processors. Combined with local Socket communication, the SDK is called to access the radiation temperature matrix of the original infrared image. Using the centerline pixels as indices, the temperature of each point is extracted to form the original temperature matrix T distributed along the centerline. (6) The composite insulator heating defect detection module processes the temperature matrix T and takes the median temperature t. i Using its column as a benchmark, the temperature of adjacent columns is fitted and corrected using the tangent vector comparison method, effectively filtering out outlier interference and generating a smooth temperature vector T1={t1,t... i ,...,t n The system determines the heating status based on a preset threshold, and the result is transmitted back to the drone handle via a communication interface, enabling real-time, autonomous defect identification at the inspection site.

Images