Power distribution unmanned aerial vehicle inspection defect intelligent identification method, system, equipment and medium

Abstract

The invention discloses a power distribution unmanned aerial vehicle inspection defect intelligent identification method, system and device and a medium, and belongs to the technical field of power inspection, and the method comprises the steps: inputting multi-modal data collected by a power distribution unmanned aerial vehicle into a pre-constructed defect semantic hierarchical label tree, and positioning a defect region based on multi-modal weak supervision contrast learning, generating a defect region structured sample; performing multi-modal thinking chain reasoning on the structural sample of the defect area to generate an interpretable reasoning chain, and judging the defect type and attribute based on the interpretable reasoning chain; and performing cross-modal lightweight distillation on a defect type and attribute judgment result, migrating knowledge to a lightweight model, and deploying the lightweight model to an unmanned aerial vehicle end-side reasoning engine to realize real-time defect identification reasoning. The problems that in power distribution unmanned aerial vehicle inspection defect identification, marking cost is high, reasoning cannot be explained, and end-side real-time performance is poor are solved, sample marking cost can be reduced, reasoning interpretability and reliability are improved, and end-side real-time deployment is achieved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power inspection, and particularly relates to a power distribution unmanned aerial vehicle (UAV) inspection defect intelligent identification method, system, device and medium. BACKGROUND

[0002] As the last mile of the power system, the equipment of the power distribution network is exposed to the outdoor environment for a long time, and is prone to typical defects such as bird nest foreign matter (causing line short circuit), insulator damage (causing flashover failure), wire breakage (causing power interruption), and equipment corrosion (reducing insulation performance). According to the DL / T1575-2016 Power Distribution Line UAV Inspection Technical Guidelines, power distribution defects account for more than 62% of the total power distribution accidents. The UAV inspection technology has become the mainstream means of power distribution inspection due to its high efficiency (the daily inspection mileage per person is increased to 20 km) and strong safety (avoiding high-altitude operation risks). However, the existing defect identification technology still has three core pain points: 1. High sample labeling cost and low efficiency. The power distribution defect scene has strong diversity (such as differences in bird nest shape in different seasons and insulator damage level classification), and traditional manual labeling requires professional personnel to label the defect area pixel by pixel, with a single defect labeling time of more than 30 minutes, and the labeling cost accounting for more than 65% of the total algorithm development cost. The existing weak supervision labeling algorithm is mainly for general scenes (such as ImageNet), and is not adapted to the semantic features of power distribution equipment (such as exclusive structures such as "insulator umbrella skirt" and "wire clamp"), resulting in a defect region segmentation IoU (intersection over union) of only 68% to 72%, which cannot meet the sample quality requirements. 2. Defect reasoning is not interpretable and has poor reliability. The existing model mainly relies on a single visual feature (such as visible light image texture), ignores the cooperation of multi-modal data such as infrared thermal imaging (device temperature anomaly) and laser radar (spatial geometric information), resulting in a bird nest and tree branch misjudgment rate of ≥20% and an insulator minor damage (area <5%) missing rate of ≥18%. Moreover, the model reasoning process is a "black box", and cannot output the logical basis for defect judgment (such as "why is this area judged as a bird nest instead of a tree branch"), which reduces the trust of the inspection personnel in the results and requires manual review, thereby increasing the operation and maintenance cost. 3. The performance of the end-side deployment is insufficient and the real-time performance is poor. The parameter quantity of the multi-modal large model (such as ViT-B / 16+GPT-4V) is hundreds of millions to several billion, and needs to rely on cloud computing power. However, the 4G / 5G network coverage rate in the power distribution scene is low (only 65%) and fluctuates greatly, resulting in a reasoning delay of more than 500 ms, which cannot meet the real-time inspection requirements of the UAV (the end-side delay requirement is ≤200 ms). The existing lightweight model mainly optimizes the visual feature, and does not consider the preservation of the text reasoning ability, resulting in a defect identification accuracy of 10% to 15% after distillation, which cannot balance "lightweight" and "high accuracy". SUMMARY

[0003] The purpose of the present application is to provide a power distribution unmanned aerial vehicle inspection defect intelligent identification method, system, device and medium to reduce sample labeling cost, improve inference interpretability and reliability, and realize end-side real-time deployment, aiming at the problems in the prior art.

[0004] To achieve the above purpose, the present application has the following technical solutions: In a first aspect, a power distribution unmanned aerial vehicle inspection defect intelligent identification method is provided, comprising: Inputting multi-modal data collected by a power distribution unmanned aerial vehicle into a pre-constructed defect semantic hierarchical label tree, and locating a defect area based on multi-modal weak supervision contrast learning to generate a defect area structured sample; Performing multi-modal thought chain reasoning on the defect area structured sample to generate an interpretable reasoning chain, and determining a defect type and attribute based on the interpretable reasoning chain; Performing cross-modal lightweight distillation on the defect type and attribute determination result, migrating knowledge to a lightweight model, and deploying it to an unmanned aerial vehicle end-side reasoning engine to realize real-time defect identification reasoning.

[0005] As a preferred scheme, the defect semantic hierarchical label tree is a three-level semantic label tree of "device category-defect type-attribute parameter", wherein the first-level label is a device category, corresponding to the main object of power distribution unmanned aerial vehicle inspection; the second-level label is a defect type, corresponding to the typical defects associated with each device category; and the third-level label is an attribute parameter, corresponding to the quantitative attributes of each defect type. A tree structure model is adopted to represent the three-level semantic label tree, wherein, V represents a node set, , is a first-level node representing a device category, is a second-level node representing a defect type, is a third-level node representing an attribute parameter; E is an edge set, represents the parent-child association of nodes and ; L is a label weight set, represents the semantic association weight of edge , which is set based on power distribution expert experience and defect occurrence frequency; The semantic similarity of the three-level semantic label tree is calculated using cosine similarity, and for any two label nodes , , the semantic similarity S( ) is calculated according to the following formula:

[0006] In the formula, a semantic embedding vector of a node v , a shortest path , a path length d , a weight of a kth edge of a path.

[0007] As a preferred scheme, the step of locating a defect region based on the multi-modal weakly supervised contrast learning and generating a structured sample of the defect region comprises: extracting multi-modal features of visual, text, and point cloud from the multi-modal data collected by the power distribution unmanned aerial vehicle; realizing semantic alignment of the "visual-text-point cloud" features through cross-modal contrast learning, and forcing the defect region features to be similar to the text description features and the non-defect region features to be significantly different from the text description features; based on the "visual-text-point cloud" features after semantic alignment, performing defect region segmentation and defect region labeling in combination with a defect semantic hierarchical label tree to generate a structured sample of the defect region.

[0008] As a preferred scheme, when the multi-modal features of visual, text, and point cloud are extracted from the multi-modal data collected by the power distribution unmanned aerial vehicle, the multi-modal data collected by the power distribution unmanned aerial vehicle comprises visible light images, infrared thermal images, laser radar point clouds, and text descriptions of corresponding defects; when the multi-modal features are extracted, a lightweight convolutional neural network (CNN) is used to extract multi-scale features of the visible light images and output visual feature maps; the infrared images are processed by temperature threshold segmentation, and infrared feature maps are extracted and fused into visual joint features; when the point cloud features are extracted, the point cloud is first down-sampled by a voxel grid (VoxelGrid), and a point net (PointNet) is used to extract spatial geometric features, and then the point cloud features are mapped to the image coordinate system through region of interest pooling (RoI Pooling); text feature extraction is based on a defect semantic hierarchical label tree, and the text description T is parsed into a label sequence, and a text feature is generated through DistilBERT encoding.

[0009] As a preferred scheme, in the step of realizing semantic alignment of the "visual-text-point cloud" features through cross-modal contrast learning, forcing the defect region features to be similar to the text description features and the non-defect region features to be significantly different from the text description features, the expression of the loss function is defined as:

[0010] In the formula, the visual-text contrast loss is calculated according to the following formula: the contrast loss between the visual joint features and the text features

[0011] wherein, B is the batch size; C is the number of feature map channels; is the visual feature vector of the b-th sample and c-th channel; is the text feature of the b-th sample, is the temperature coefficient; is the point cloud-textual contrastive loss, the contrastive loss between the point cloud mapping feature and the textual feature is calculated as follows:

[0012] is the visual-point cloud contrastive loss, the contrastive loss between the visual joint feature and the point cloud mapping feature is calculated as follows:

[0013] wherein, is the L2 norm.

[0014] As a preferred scheme, in the step of performing defect region segmentation based on the "visual-textual-point cloud" feature after semantic alignment, and combining the defect semantic hierarchical label tree to perform defect region labeling to generate a defect region structured sample, the similarity graph between the visual joint feature and the textual feature is calculated; and the similarity graph between the point cloud mapping feature and the textual feature is calculated; the similarity graphs are fused to obtain a defect region mask by Otsu threshold segmentation; based on the obtained defect region mask, the corresponding label is labeled for the defect region in combination with the defect semantic hierarchical label tree to generate a structured sample library.

[0015] As a preferred scheme, in the step of performing multi-modal thought chain reasoning on the defect region structured sample to generate an interpretable reasoning chain, and determining the defect type and attribute based on the interpretable reasoning chain, the visible light image and the infrared image are fused at the pixel level by using an attention fusion mechanism, and the expression is as follows: wherein, the attention weight is calculated by ; the visual-infrared fusion feature , the point cloud feature , and the textual label feature ​, through the cross-attention module and the embedding matrix of the defect semantic label tree , to obtain the fusion feature , the calculation expression is ; the reasoning chain probability model is expressed as , wherein, is the initial state "to be reasoned", to correspond to the equipment identification, defect type determination and attribute parameter quantization states respectively, represents the reasoning probability from state k-1 to k; in the equipment identification stage, based on the fusion feature and the first-level node V1 of the label tree, the equipment category probability is calculated through the expression , wherein, is the equipment identification weight matrix, and the equipment category with the maximum probability is selected as the result; in the defect type determination, the associated second-level node in the label tree is filtered according to the equipment identification result, and the defect type probability is calculated through the expression , wherein, is the defect type determination weight matrix, is the semantic weight; in the attribute parameter quantization stage, the associated third-level node is filtered based on the defect type result, the attribute parameters are quantized through regression, the severity is determined based on the proportion of the defect area of the visual feature through , and the size or temperature is directly quantized based on the point cloud volume or infrared temperature; the reasoning chain output integrates the three-step reasoning results into a natural language form.

[0016] As a preferred scheme, the cross-modal lightweight distillation of the defect type and attribute determination results is used to transfer the knowledge to the lightweight model and deploy it to the unmanned aerial vehicle end-side reasoning engine to realize real-time defect identification reasoning; in the steps of the reasoning, the student model obtains key information from the teacher model through knowledge distillation; the distillation loss function is the weighted sum of multi-task losses, including feature alignment loss, semantic consistency loss and task loss; the feature alignment loss uses MSE loss to ensure that the multi-modal feature distribution of the student model is consistent with that of the teacher model, so that the student model has the feature extraction ability of the teacher model; the semantic consistency loss is based on cosine similarity loss, which ensures that the reasoning chain semantics of the student model are consistent with those of the teacher model, and ensures that the reasoning logics of the two are the same; the task loss is composed of weighted classification loss and regression loss, the classification loss calculates the classification error of the equipment and defect type using cross-entropy loss, the regression loss calculates the regression error of the position, size and temperature using L1 loss, and the accuracy of the student model in the defect identification task is ensured; The distillation weight is determined through the power distribution defect data set; The student model after distillation is optimized, the optimization including model quantization, using INT8 quantization technology to convert the model weight from 32-bit floating point number to 8-bit integer; and operator optimization, optimizing the core operator with the help of the TensorRT framework to reduce the memory access delay; the optimized model is converted into ONNX format and deployed to the unmanned aerial vehicle end side inference engine.

[0017] In a second aspect, an intelligent defect identification system for power distribution unmanned aerial vehicle inspection is provided, comprising: A defect area positioning module is configured to input multi-modal data collected by the power distribution unmanned aerial vehicle into a pre-constructed defect semantic hierarchical label tree, and locate a defect area based on multi-modal weak supervision contrast learning to generate a defect area structured sample; A thought chain reasoning module is configured to perform multi-modal thought chain reasoning on the defect area structured sample to generate an interpretable reasoning chain, and determine a defect type and attribute based on the interpretable reasoning chain; A lightweight distillation module is configured to perform cross-modal lightweight distillation on the defect type and attribute determination result, and deploy the knowledge to a lightweight model to the unmanned aerial vehicle end side inference engine to realize real-time defect identification reasoning.

[0018] As a preferred scheme, when the defect area positioning module inputs the multi-modal data collected by the power distribution unmanned aerial vehicle into the pre-constructed defect semantic hierarchical label tree, the defect semantic hierarchical label tree is a three-level semantic label tree of "device category-defect type-attribute parameter", wherein the first-level label is a device category, corresponding to the main object of the power distribution unmanned aerial vehicle inspection; the second-level label is a defect type, corresponding to a typical defect associated with each device category; and the third-level label is an attribute parameter, corresponding to a quantitative attribute of each defect type. A tree structure model is adopted represents a three-level semantic label tree, wherein, V represents a node set, , is a first-level node representing a device category, is a second-level node representing a defect type, is a third-level node representing an attribute parameter; E is an edge set, represents the parent-child association of nodes and . L is a label weight set, represents the semantic association weight of edge , which is set based on power distribution expert experience and defect occurrence frequency; The semantic similarity of the three-level semantic label tree is calculated using cosine similarity, and for any two label nodes , , the semantic similarity S( Calculate according to the following formula:

[0019] In the formula, For nodes v semantic embedding vector, for and The shortest path, d For path length, Let be the weight of the k-th edge of the path.

[0020] As a preferred embodiment, the defect region localization module locates the defect region based on multimodal weakly supervised contrastive learning. When generating structured samples of the defect region, it extracts multimodal features of vision, text, and point cloud from the multimodal data collected by the power distribution drone. Through cross-modal contrastive learning, semantic alignment of "visual-text-point cloud" features is achieved, forcing the features of defective regions to be similar to the text description features, and the features of non-defective regions to be significantly different from the text description features; Based on the semantically aligned "visual-text-point cloud" features, defect regions are segmented, and defect regions are labeled using a hierarchical semantic label tree to generate structured samples of defect regions.

[0021] As a preferred embodiment, when the defect area localization module extracts multimodal features (visual, textual, and point cloud) from the multimodal data collected by the power distribution drone, the multimodal data includes visible light images, infrared thermal images, lidar point clouds, and corresponding textual descriptions of defects. During multimodal feature extraction, a lightweight convolutional neural network (CNN) is used to extract multi-scale features from the visible light images, outputting a visual feature map. Infrared images are segmented using a temperature threshold, and infrared feature maps are extracted and fused into joint visual features. When extracting point cloud features, the point cloud is first downsampled using a voxel grid, and spatial geometric features are extracted using a point net (PointNet). Then, region-of-interest pooling (RoIPooling) maps the point cloud features to the image coordinate system. Text feature extraction is based on a hierarchical semantic label tree for defects, parsing the text description T into a label sequence and generating text features using DistilBERT encoding.

[0022] As a preferred embodiment, the defect region localization module achieves semantic alignment of "visual-text-point cloud" features through cross-modal contrastive learning, forcing the defect region features to be similar to the text description features. When the non-defect region features differ significantly from the text description features, the expression for the loss function is defined as follows:

[0023] In the formula, The visual-text contrast loss is calculated as follows and the text feature

[0024] In the formula, B is the batch size; C is the number of feature map channels; is the visual feature vector of the b-th sample and the c-th channel; is the text feature of the b-th sample, is the temperature coefficient; The point cloud-text contrast loss is calculated as follows and the text feature

[0025] The visual-point cloud contrast loss is calculated as follows and the point cloud mapping feature

[0026] In the formula, is the L2 norm.

[0027] As a preferred solution, the defect region positioning module performs defect region segmentation based on the "visual-text-point cloud" features after semantic alignment, and performs defect region labeling in combination with a defect semantic hierarchical label tree to generate a defect region structured sample. When calculating the visual joint feature and the text feature , a similarity map of the point cloud mapping feature and the text feature is calculated; the similarity maps are fused to obtain a defect region mask through Otsu segmentation; based on the obtained defect region mask, the corresponding label is labeled for the defect region in combination with the defect semantic hierarchical label tree to generate a structured sample library.

[0028] As a preferred solution, the thought chain reasoning module uses an attention fusion mechanism to perform pixel-level fusion on the visible light image and the infrared image, and the expression is wherein the attention weight is calculated through The visual-infrared fusion feature and the point cloud feature​​​​ , text label features , through the cross-attention module and the embedding matrix of the defect semantic label tree , to obtain fusion features , the calculation expression is ; the reasoning chain probability model is expressed as , wherein, is the initial state "to be reasoned", to correspond to the device identification, defect type determination and attribute parameter quantization states respectively, indicates the reasoning probability from state k-1 to k; in the device identification stage, based on the fusion features and the first-level node V1 of the label tree, the device category probability is calculated through the expression , wherein, is the device identification weight matrix, and the device category with the maximum probability is selected as the result; in the defect type determination, the associated second-level node in the label tree is filtered according to the device identification result, and the defect type probability is calculated through the expression , wherein, is the defect type determination weight matrix, is the semantic weight; in the attribute parameter quantization stage, the associated third-level node is filtered based on the defect type result, the attribute parameters are quantized through regression, the severity is determined based on the proportion of the defect area of the visual feature through , and the size or temperature is directly quantized based on the point cloud volume or infrared temperature; the reasoning chain output integrates the results of the three reasoning steps into a natural language form.

[0029] As a preferred scheme, the lightweight distillation module makes the student model obtain key information from the teacher model through knowledge distillation; the distillation loss function is the weighted sum of multi-task losses, including feature alignment loss, semantic consistency loss and task loss; the feature alignment loss uses MSE loss to ensure that the multi-modal feature distribution of the student model is consistent with that of the teacher model, so that the student model has the feature extraction ability of the teacher model; the semantic consistency loss is based on cosine similarity loss, which ensures that the reasoning chain semantics of the student model are consistent with those of the teacher model, and ensures that the reasoning logics of the two are the same; the task loss is composed of weighted classification loss and regression loss, the classification loss calculates the classification error of the device and the defect type using cross-entropy loss, the regression loss calculates the regression error of the position, size and temperature using L1 loss, and the accuracy of the student model in the defect identification task is ensured; The distillation weight is determined by verifying the power distribution defect data set; The student model after distillation is optimized, the optimization including model quantization, using INT8 quantization technology to convert model weights from 32-bit floating point numbers to 8-bit integers, and operator optimization, using the TensorRT framework to optimize core operators to reduce memory access delay; the optimized model is converted into an ONNX format and deployed to a UAV side inference engine.

[0030] In a third aspect, an electronic device is provided, comprising a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the power distribution UAV inspection defect intelligent identification method.

[0031] In a fourth aspect, a computer readable storage medium is provided, the computer readable storage medium storing at least one instruction, the at least one instruction being executed by a processor to implement the power distribution UAV inspection defect intelligent identification method.

[0032] Compared with the prior art, the first aspect of the present application has at least the following beneficial effects: To solve the problems of high labeling cost, uninterpretable reasoning and poor real-time performance in defect identification of power distribution UAV inspection, the application combines multi-modal weakly supervised contrastive learning with defect semantic hierarchical label tree to realize automatic segmentation of defect area under the guidance of a small amount of text, accurately locates the defect area by using multi-modal feature alignment, and generates structured samples of the defect area. The structured samples of the defect area are subjected to multi-modal thought chain reasoning to generate an interpretable reasoning chain, accurately determine the defect type and attribute, and improve the reasoning interpretability and reliability. The reasoning process is traceable and verifiable. Through cross-modal lightweight distillation, the defect type and attribute determined by multi-modal thought chain reasoning are subjected to knowledge transfer, and the lightweight model is deployed to the UAV side reasoning engine to realize real-time defect identification reasoning and meet the real-time inspection requirements of the power distribution UAV. The application has been verified. In sample labeling, the multi-modal weakly supervised contrastive learning combined with the defect semantic hierarchical label tree improves the labeling efficiency by 4.3 times, reduces the single defect labeling cost from 30 yuan to 6.9 yuan, shortens the 100,000-frame data labeling period from 100 days to 23 days, and the defect area segmentation IoU reaches 87.2%, which is 17.2 percentage points higher than that of the traditional weakly supervised algorithm. In defect reasoning, the multi-modal thought chain reasoning realizes the visual output of "step-by-step reasoning + semantic basis", improves the result trust of the inspection personnel from 65% to 92%, reduces the artificial review rate from 100% to 10%, and makes the defect type accuracy reach 95.2%, the bird nest and tree branch misjudgment rate is 3.1%, which is 16.9 percentage points lower than that of the traditional single-modal model. In terms of side deployment performance, the model parameter amount after cross-modal lightweight distillation reaches 7.8M, which is compressed by 99.94% compared with the teacher model, the side storage occupancy is reduced from 50GB to 30MB, the side reasoning delay is 112ms, the total delay is ≤200ms, which is improved by 57.6% compared with cloud reasoning. In the inspection application, the UAV inspection efficiency reaches 22km / day, which is 4.4 times that of manual inspection, the 100-pole line inspection time is shortened from 20 days to 5 days, the defect omission rate is reduced from 15% to 4.8%, and the core problems of defect identification of power distribution UAV inspection are effectively solved.

[0033] It can be understood that the beneficial effects of the above-mentioned second aspect to fourth aspect can be referred to the related description in the above-mentioned first aspect, which will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creating labor intensity.

[0035] Figure 1 The power distribution unmanned aerial vehicle inspection defect intelligent identification method flow chart of the embodiment of the application; Figure 2 The power distribution unmanned aerial vehicle inspection defect intelligent identification method principle architecture diagram of the embodiment of the application; Figure 3 The power distribution unmanned aerial vehicle inspection defect intelligent identification system structure block diagram of the embodiment of the application. DETAILED DESCRIPTION

[0036] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, techniques, etc., in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to one skilled in the art that the application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the application with unnecessary detail. The embodiments of the application are described herein using flowcharts to illustrate the operations performed by devices according to embodiments of the application. It will be understood that the operations as set forth in the flowcharts are not necessarily performed in the order shown. Rather, according to the needs of the various embodiments, the steps can be performed in different order, or concurrently, or can be omitted entirely. Other operations can also be added to, or removed from, the processes described herein.

[0037] Please refer to Figure 1The embodiment of the application provides a power distribution unmanned aerial vehicle inspection defect intelligent identification method, which fuses multi-modal weak supervision contrast labeling (MW-SCL), multi-modal thought chain reasoning (M-CoT), cross-modal light distillation (Light-MMD) and defect semantic hierarchical label tree construction algorithm, constructs a whole-link technical framework of "sample efficient construction-defect explainable identification-end side real-time deployment", solves the problems of high labeling cost, unexplainable reasoning and poor end side real-time performance of typical defects such as bird nest foreign matter, insulator damage, wire breakage and equipment rust in the power distribution scene, is suitable for power distribution unmanned aerial vehicle autonomous inspection, defect intelligent classification, operation and maintenance decision assistance and other tasks, and takes into account the collaborative inspection of power grid live working robots, AR defect visualization and other extended applications. The power distribution unmanned aerial vehicle inspection defect intelligent identification method of the embodiment of the application is used for overcoming the problems of high labeling cost, unexplainable reasoning and poor end side real-time performance in the existing power distribution unmanned aerial vehicle inspection defect identification technology, and specific purposes include: 1. Reducing sample labeling cost. Through the combination of the MW-SCL labeling algorithm and the defect semantic hierarchical label tree, weak supervision labeling of "a small amount of text guidance + multi-modal feature alignment" is realized, the labeling efficiency is improved by more than 4 times, the single defect labeling cost is reduced by 75%, and the defect region segmentation IoU is ensured to be greater than or equal to 85%; 2. Improving the explainability and reliability of reasoning. The M-CoT reasoning algorithm integrates visible light, infrared and laser radar multi-modal features to generate a reasoning chain conforming to the power distribution professional knowledge, the defect misjudgment rate is reduced to less than 5%, and the reasoning process is traceable and verifiable; 3. Realizing end side real-time deployment. The Light-MMD distillation algorithm is used to migrate the large model knowledge to a light model, the parameter quantity is compressed to less than 8M, the end side reasoning delay is less than or equal to 150ms, the accuracy rate is reduced by less than or equal to 3%, and the real-time inspection requirement of the power distribution unmanned aerial vehicle is met. The power distribution unmanned aerial vehicle inspection defect intelligent identification method mainly includes the following steps: S1, inputting multi-modal data collected by a power distribution unmanned aerial vehicle into a pre-constructed defect semantic hierarchical label tree, and positioning a defect region based on multi-modal weak supervision contrast learning to generate a defect region structured sample; S2, performing multi-modal thought chain reasoning on the defect region structured sample to generate an explainable reasoning chain, and determining a defect type and attribute based on the explainable reasoning chain; S3, performing cross-modal light distillation on the defect type and attribute determination result, migrating the knowledge to a light model, and deploying the light model to an unmanned aerial vehicle end side reasoning engine to realize real-time defect identification reasoning.

[0038] In one possible implementation, step S1 constructs a three-level semantic tag tree of "equipment category - defect type - attribute parameter" to address the characteristics of diverse power distribution defect types and complex semantic relationships, providing structured semantic constraints for subsequent annotation and reasoning. In terms of the tag tree structure design, the first-level tags (equipment categories) cover the core equipment for power distribution inspection, including five categories: "insulators," "conductors," "towers," "transformers," and "ring main units," corresponding to the main objects inspected by power distribution drones. The second-level tags (defect types) are based on the "Power Distribution Line Operation Regulations," with each equipment category associated with typical defects. For example, "insulators" are associated with "damage," "spontaneous explosion," and "contamination," while "conductors" are associated with "broken strands," "wear," and "bird's nest foreign objects," totaling 12 core defect categories. In the third-level tags (attribute parameters), each defect type includes quantitative attributes such as "location," "severity," and "size / temperature." For example, for "bird's nest foreign objects," the location (conductor #XX pole segment, tower crossarm), severity (mild: coverage section <10%, moderate: 10%~30%, severe: >30%), and size (volume / m³) are included, and for "equipment corrosion," the temperature (temperature difference from the environment / ℃) is included.

[0039] In the mathematical representation of the tag tree, a tree structure model is adopted. Represents a label tree, where, V Represents a set of nodes. , This is a primary node (equipment category, 5 in total). It is a secondary node (12 defect types). It is a three-level node (with 36 attribute parameters); E Let be the set of edges. Represents a node and Parent-child relationships (e.g., "insulator" - "damaged" - "location"); L For the set of label weights, Representing an edge The semantic association weights are set based on the experience of power distribution experts and the frequency of defect occurrence. For example, the weight of "conductor-bird's nest foreign object" is l=1.2 (high frequency of occurrence), and the weight of "power distribution-oil leakage" is l=0.8 (low frequency of occurrence).

[0040] The semantic similarity of the tag tree is calculated using cosine similarity, for any two tag nodes. , Its semantic similarity S ( Calculate according to the following formula:

[0041] in, For nodes vsemantic embedding vectors of the labels (generated by a BERT model pre-trained on a power distribution defect corpus, dimension 128), is the shortest path between the two labels, is d the path length, is k the weight of the edge of the path. This formula ensures that labels with close semantic associations (such as "insulator breakage" and "insulator self-explosion") have higher similarity, providing constraints for subsequent multi-modal alignment.

[0042] In one possible implementation, step S1 is based on multi-modal weakly supervised contrast learning (MW-SCL) combined with a defect semantic hierarchical label tree, achieving weakly supervised labeling of "a small amount of text description → automatic segmentation of defect regions". The core is to accurately locate the defect region through visual-text-laser radar multi-modal feature alignment. The step of locating the defect region based on multi-modal weakly supervised contrast learning in step S1 to generate a structured sample of the defect region includes: (1) Multi-modal feature extraction. First, input the multi-modal data collected by the power distribution unmanned aerial vehicle. This includes a visible light image with resolution H = 1080, W = 1920 , an infrared thermal image with a temperature range of -20°C to 150°C , a laser radar point cloud with a point cloud number N ≥ , and a text description T corresponding to the defect (such as "10kv east line #087 pole conductor above bird nest, covering 30% of the cross section"). Then, perform feature extraction. In terms of visual feature extraction, use a lightweight CNN (MobileNetV3) to extract multi-scale features of the visible light image, outputting a visual feature map . Process the infrared image through temperature threshold segmentation (such as rust area temperature > ambient temperature 5°C) to extract an infrared feature map , and fuse it into a visual joint feature through (Conv is a 1x1 convolutional dimension reduction). In point cloud feature extraction, first downsample the point cloud through VoxelGrid (voxel size is 0.05 cubic meters), and use PointNet to extract spatial geometric features (M is the number of voxels). Then, map the point cloud features to the image coordinate system through RoIPooling (region of interest pooling), obtaining . In text feature extraction, based on the defect semantic hierarchical label tree, parse the text description T into a label sequence (such as "conductor - bird nest foreign matter - location": #087 pole - severity - severe), and generate a text feature through DistilBERT encoding.

[0043] (2) Through cross-modal contrastive learning, semantic alignment of "visual-text-point cloud" features is achieved, forcing the features of defective regions to be similar to the text description features, and the features of non-defective regions to be significantly different from the text description features. The loss function is defined as:

[0044] in, For visual-text contrast loss, compute joint visual features. Text features The contrast loss is calculated using the following formula:

[0045] in, B For batch size, C The number of feature map channels. For the first b The sample, the first c Visual feature vectors of each channel, For the first Text features of each sample (negative sample). For the temperature coefficient (optimized through cross-validation in power distribution scenarios), this loss ensures that the visual features of the defective area are highly similar to the corresponding text features. For point cloud-text contrast loss, similar to Calculate point cloud mapping features Text features The contrast loss is calculated using the following formula:

[0046] This loss function utilizes the spatial geometric information of point clouds (such as the volume and location of bird nests) to enhance text semantic alignment and avoid mislabeling of visually similar objects (such as bird nests and tree branches). For visual-point cloud contrast loss, compute visual joint features Point cloud mapping features The contrast loss is used to ensure consistency within multimodal features, and the formula is as follows:

[0047] in, Using the L2 norm, this loss minimizes the difference between visual and point cloud features, improving annotation robustness.

[0048] (3) Defect region segmentation and annotation. Based on the feature extraction network trained with MW-SCL loss, defect region segmentation is performed on the input multimodal data. Specifically, the visual joint features are calculated first. Text features Similarity graph , Secondly, calculate the point cloud mapping features. Text features Similarity graph , Then, the similarity maps are merged. (Visual features have higher weights), and the defect region mask is obtained through Otsu thresholding. (1 represents the defect area); finally, combined with the hierarchical label tree of defect semantics, the defect area is automatically labeled with three-level labels (such as "wire-bird's nest foreign object-location: #087 pole-severity: severe"), generating a structured sample library.

[0049] In one possible implementation, step S2 integrates multimodal features and a defect semantic label tree through a three-step reasoning logic of "feature fusion - step-by-step reasoning - result output" to generate an interpretable reasoning chain, achieving accurate determination of defect type and attributes. In the multimodal feature fusion stage, the structured samples output from multimodal weakly supervised contrastive learning are used as input for further processing. First, the visible light image... Pixel-level fusion with infrared images, utilizing an attention fusion mechanism. Attention weights pass Calculations show that this mechanism can highlight defect areas, such as the high-temperature infrared characteristics of corroded areas. Next, the visual-infrared fusion features are then... Point cloud features Text label features The embedding matrix of the cross-attention module and the defect semantic label tree is used. (by all tag nodes) The components are fused together to obtain fusion characteristics. The formula is The CrossAttn cross-attention mechanism ensures that the fused features are closely linked to the power distribution semantics.

[0050] The step-by-step inference logic, as the core of M-CoT, breaks down the inference process into three steps: "equipment identification → defect type determination → attribute parameter quantification." Each step of the inference is based on fused features and a semantic label tree. The probabilistic model of the inference chain is represented as follows: ,in The initial state is "to be reasoned". to These correspond to equipment identification, defect type determination, and attribute parameter quantification status, respectively. This represents the inference probability from state k-1 to k. During the device identification stage, based on fused features... With the first-level node V1 of the tag tree, through the formula Calculate the probability of device category, where For device recognition, the weight matrix is identified and the device class with the highest probability is selected as the result. In defect type determination, the associated secondary nodes in the label tree are filtered according to the device recognition result, and the defect type probability is calculated by the formula wherein is the weight matrix for defect type determination, is the semantic weight. The attribute parameter quantification stage is based on the defect type result to filter the associated tertiary nodes, and the attribute parameters are quantified by regression, such as the position using the spatial coordinates of the laser radar point cloud by calculation, the severity is determined based on the proportion of the defect area covered by the visual features by calculation. The size or temperature is directly quantified based on the point cloud volume or infrared temperature. Finally, the inference chain output integrates the three-step inference results into natural language form, such as "Step 1: The similarity between the fused features and the 'wire' semantic embedding is the highest (0.92) → Device recognition is 'wire'; Step 2: The similarity between the fused features and the 'bird nest foreign matter' semantic embedding × weight = 0.88 × 1.2 = 1.056, which is higher than 'broken strand' (0.75 × 1.0 = 0.75) → Defect type is 'bird nest foreign matter'; Step 3: Laser radar point cloud volume 0.12 m³ → Size 0.12 m³; Defect area covers 32% of the wire cross section → Severity'severe'; Point cloud spatial coordinates correspond to line #087 pole → Position '#087 pole'; Conclusion: There is a severe bird nest foreign matter on the wire #087 pole segment with a volume of 0.12 m³", thereby realizing accurate and interpretable determination of defect type and attributes.

[0051] In a possible implementation, step S3 is to realize real-time deployment of the UAV side by migrating the knowledge of the above-mentioned multi-modal thinking chain reasoning large model (teacher model, parameter quantity 13B) to a light-weight model (student model, parameter quantity 8M) through cross-modal light-weight distillation (Light-MMD), the core of which is a triple constraint of “feature alignment-semantic consistency-task loss” to ensure that the performance loss after light-weighting is minimized. (1) Teacher and student model architecture. The teacher model adopts a combination of “ViT-B / 16 (vision) + PointNet++ (point cloud) + BERT (text) + Transformer (reasoning)”, with a parameter quantity of 13B. Although the accuracy is high, the model is bulky. The student model selects “MobileViT-XXS (vision) + simplified PointNet (point cloud) + DistilBERT (text) + light-weight Transformer (reasoning)”, with a parameter quantity of only 8M, which has the advantages of small size and fast reasoning speed, and aims to obtain key information from the teacher model through knowledge distillation; (2) Distillation loss function. The distillation loss function is the weighted sum of multi-task loss, and the core role is to maintain the performance close to the teacher model while ensuring the light-weight of the student model. It mainly includes three parts: first, the feature alignment loss. The MSE loss is used to ensure that the multi-modal feature distribution of the student model is consistent with that of the teacher model, so that the student model learns the feature extraction ability of the teacher model; second, the semantic consistency loss. Based on the cosine similarity loss, the semantics of the reasoning chain of the student model is ensured to be consistent with that of the teacher model, so that the reasoning logic of the two is the same; finally, the task loss. It is composed of weighted classification loss and regression loss. The former uses cross-entropy loss to calculate the classification error of equipment and defect types, and the latter uses L1 loss to calculate the regression error of position, size and temperature, aiming to ensure the accuracy of the student model in the defect identification task. Finally, through the power distribution defect dataset verification, the distillation weight is determined to control the accuracy of the student model to decrease within 3%; (3) End-side deployment optimization. To adapt to the end-side hardware of the UAV (such as NVIDIA Jetson Orin NX), the student model after distillation is optimized in multiple aspects. First, model quantization. Using INT8 quantization technology, the model weight is converted from 32-bit floating point number to 8-bit integer, which greatly compresses the model volume by 75% and improves the reasoning speed by 2 times; second, operator optimization. With the help of the TensorRT framework, the core operators such as convolution and attention are optimized to reduce the memory access delay; finally, inference engine deployment. The optimized model is converted into ONNX format and deployed to the UAV end-side inference engine to realize real-time defect identification reasoning.

[0052] See Figure 2 The power distribution UAV inspection defect intelligent identification method of the embodiment of the present application proposes: 1. Defect semantic hierarchical label tree construction method Aiming at the characteristics of power distribution equipment and defect semantic association, a three-level label tree of "equipment category-defect type-attribute parameter" is constructed, and the semantic association degree is calculated by hierarchical cosine similarity, which provides structured constraints for multi-modal labeling and reasoning. Unlike existing flat structure labels, the label tree of the invention can achieve precise semantic constraints and improve multi-modal alignment accuracy.

[0053] 2. MW-SCL weakly supervised labeling algorithm By fusing visible light-infrared-laser radar multi-modal features, a "vision-text-point cloud" triple contrast loss function is designed, realizing "a small amount of text guidance → defect area automatic segmentation + label labeling", solving the problems of low precision and high cost of traditional weakly supervised labeling. Compared with the existing technology using only visual-text features, the invention introduces point cloud contrast loss, which improves the labeling accuracy in power distribution scenarios.

[0054] 3. M-CoT multi-modal reasoning algorithm The algorithm decomposes defect recognition into three-step reasoning of "equipment recognition-defect type determination-attribute parameter quantization", based on multi-modal fusion features and label tree semantic constraints, to generate an interpretable natural language reasoning chain, breaking the "black box" limitation of traditional models. Unlike existing reasoning that directly outputs defect results, the reasoning chain of the invention is traceable, verifiable, and integrates multi-modal features, making it more reliable.

[0055] 4. Light-MMD cross-modal distillation algorithm The algorithm designs a triple distillation loss function of "feature alignment-semantic consistency-task loss", which migrates multi-modal large model knowledge to a lightweight model, ensuring the balance between "lightweight" and "high accuracy" in end-side deployment. Unlike existing distillation techniques that only optimize visual features, the invention introduces semantic consistency loss to retain the reasoning chain generation capability, with an accuracy drop of ≤3%.

[0056] 5. Multi-module collaborative defect recognition framework The framework integrates four modules of "label tree-MW-SCL-M-CoT-Light-MMD" to form a full-link technology framework of "sample labeling-model reasoning-end-side deployment", which is suitable for power distribution unmanned aerial vehicle inspection scenarios. Unlike existing methods that optimize single modules, the four modules of the invention support each other, with label tree constraints on labeling and reasoning, and distillation retaining reasoning capability, achieving optimal overall performance.

[0057] Please refer to Figure 3 Another embodiment of the invention further provides a power distribution unmanned aerial vehicle inspection defect intelligent recognition system, comprising: A defect area positioning module 301 is configured to input multi-modal data collected by a power distribution unmanned aerial vehicle into a pre-constructed defect semantic hierarchical label tree, and position a defect area based on multi-modal weak supervision contrast learning to generate a structured sample of the defect area. The thought chain reasoning module 302 is configured to perform multimodal thought chain reasoning on the defect area structured sample, generate an interpretable reasoning chain, and determine the defect type and attribute based on the interpretable reasoning chain. The lightweight distillation module 303 is configured to perform cross-modal lightweight distillation on the defect type and attribute determination result, transfer the knowledge to a lightweight model, and deploy the lightweight model to an unmanned aerial vehicle end-side reasoning engine to realize real-time defect recognition reasoning.

[0058] In a possible implementation, when the defect area positioning module 301 inputs the multimodal data collected by the power distribution unmanned aerial vehicle into a pre-constructed defect semantic hierarchical label tree, the defect semantic hierarchical label tree is a three-level semantic label tree of “device category-defect type-attribute parameter”, wherein the first-level label is a device category, which corresponds to the main object of the power distribution unmanned aerial vehicle inspection; the second-level label is a defect type, which corresponds to a typical defect associated with each device category; and the third-level label is an attribute parameter, which corresponds to a quantitative attribute of each defect type. A tree structure model is adopted represents the three-level semantic label tree, wherein, V represents a node set, , is a first-level node representing a device category, is a second-level node representing a defect type, is a third-level node representing an attribute parameter; E is an edge set, represents a node and a parent-child association of L is a label weight set, represents a semantic association weight of an edge , which is set based on power distribution expert experience and defect occurrence frequency; Cosine similarity is used to calculate the semantic similarity of the three-level semantic label tree. For any two label nodes , , the semantic similarity S( ) is calculated according to the following formula:

[0059] In the formula, is a semantic embedding vector of a node v , is the shortest path between and , d is the path length, is the weight of the kth edge of the path.

[0060] In a possible implementation, the defect region positioning module 301 positions the defect region based on multi-modal weakly supervised contrast learning, and when generating the defect region structured sample, performs multi-modal feature extraction on the multi-modal data collected by the power distribution unmanned aerial vehicle in a visual, text, and point cloud manner. Through cross-modal contrast learning, semantic alignment of "visual-text-point cloud" features is achieved, and defect region features are forced to be similar to text description features, and non-defect region features are forced to be significantly different from text description features. Based on the "visual-text-point cloud" features after semantic alignment, defect region segmentation is performed, and defect region labeling is performed in combination with a defect semantic hierarchical label tree to generate a defect region structured sample.

[0061] In a possible implementation, when the defect region positioning module 301 performs multi-modal feature extraction on the multi-modal data collected by the power distribution unmanned aerial vehicle in a visual, text, and point cloud manner, the multi-modal data collected by the power distribution unmanned aerial vehicle includes visible light images, infrared thermal images, laser radar point clouds, and text descriptions of corresponding defects. When performing multi-modal feature extraction, a lightweight convolutional neural network (CNN) is used to extract multi-scale features of the visible light images, and a visual feature map is output. The infrared images are processed by temperature threshold segmentation, and an infrared feature map is extracted and fused into a visual joint feature. When extracting the point cloud feature, the point cloud is first down-sampled by a voxel grid (VoxelGrid), and a point net (PointNet) is used to extract spatial geometric features. Then, the point cloud feature is mapped to an image coordinate system through a region of interest pooling (RoIPooling). Text feature extraction is based on a defect semantic hierarchical label tree, the text description T is parsed into a label sequence, and a text feature is generated through DistilBERT encoding.

[0062] In a possible implementation, when the defect region positioning module 301 achieves semantic alignment of "visual-text-point cloud" features through cross-modal contrast learning, forces defect region features to be similar to text description features, and forces non-defect region features to be significantly different from text description features, the expression of the loss function is defined as follows:

[0063] In the formula, The visual-text contrast loss is calculated according to the following formula: The contrast loss between the visual joint feature and the text feature

[0064] In the formula, B is the batch size; C is the number of feature maps; is the visual feature vector of the bth sample and the cth channel; is the text feature vector of the bth sample; ​Text features of each sample Temperature coefficient; For point cloud-text contrast loss, the point cloud mapping features are calculated using the following formula. Text features Comparative loss:

[0065] For the visual-point cloud contrast loss, the visual joint features are calculated using the following formula. Point cloud mapping features Comparative loss:

[0066] In the formula, It is an L2 norm.

[0067] In one possible implementation, the defect region localization module 301 performs defect region segmentation based on semantically aligned "visual-textual-point cloud" features, and combines a hierarchical defect semantic label tree to annotate the defect region. When generating structured samples of the defect region, it calculates the visual joint features. Text features Similarity map; and, calculating point cloud mapping features. Text features The similarity graphs are fused together, and the defect region mask is obtained by Otsu threshold segmentation. Based on the obtained defect region mask, the corresponding labels are labeled for the defect region in combination with the defect semantic hierarchical label tree, and a structured sample library is generated.

[0068] In one possible implementation, the thought chain reasoning module 302 utilizes an attention fusion mechanism to analyze visible light images. Pixel-level fusion with infrared images, the expression is: Among them, attention weight pass Calculations show that the visual-infrared fusion features are... Point cloud features Text label features The embedding matrix of the cross-attention module and the defect semantic label tree is used. , obtain fusion features The calculation expression is: The inference chain probabilistic model is represented as ,in, The initial state is "to be reasoned". to These correspond to equipment identification, defect type determination, and attribute parameter quantification status, respectively. This represents the inference probability from state k-1 to k; in the device identification stage, it is based on fused features. With the first-level node V1 of the tag tree, through the expression Calculate the probability of device category, where, For equipment identification, a weight matrix is ​​used to select the equipment category with the highest probability as the result. In defect type determination, secondary nodes associated with the label tree are filtered based on the equipment identification results, using an expression... Calculate the probability of defect type, where, Weight matrix for determining defect type, For semantic weights; in the attribute parameter quantification stage, based on the defect type results, related third-level nodes are filtered, and attribute parameters are quantified through regression. The severity is based on the proportion of defect areas in visual features. The determination is based on direct quantization of size or temperature using point cloud volume or infrared temperature; the inference chain output integrates the three-step inference results into a natural language form.

[0069] In one possible implementation, the lightweight distillation module 303 enables the student model to obtain key information from the teacher model through knowledge distillation; the distillation loss function is a weighted sum of multi-task losses, including feature alignment loss, semantic consistency loss, and task loss; the feature alignment loss uses MSE loss to ensure that the multimodal feature distribution of the student model is consistent with that of the teacher model, allowing the student model to have the feature extraction capability of the teacher model; the semantic consistency loss is based on cosine similarity loss to ensure that the semantics of the inference chain of the student model are consistent with those of the teacher model, ensuring that the inference logic of the two is the same; the task loss is composed of a weighted sum of classification loss and regression loss, the classification loss uses cross-entropy loss to calculate the classification error of the device and defect type, and the regression loss uses L1 loss to calculate the regression error of position, size, and temperature, ensuring the accuracy of the student model in the defect identification task; The distillation weights were determined by validating the power distribution defect dataset. The distilled student model is optimized, including model quantization, which uses INT8 quantization technology to convert the model weights from 32-bit floating-point numbers to 8-bit integers; and operator optimization, which uses the TensorRT framework to optimize the core operators and reduce memory access latency; the optimized model is then converted to ONNX format and deployed to the UAV edge inference engine.

[0070] Another embodiment of the present invention also proposes an electronic device, including a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the intelligent identification method for defects in power distribution drone inspections.

[0071] Another embodiment of the present invention also proposes a computer-readable storage medium storing at least one instruction, which, when executed by a processor, implements the intelligent identification method for defects in power distribution drone inspections.

[0072] The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include any entity or device capable of carrying the computer program code, a medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals. For ease of explanation, the above content only shows the parts related to the embodiments of the present invention; for specific technical details not disclosed, please refer to the method section of the embodiments of the present invention. This computer-readable storage medium is non-transitory and can be stored in storage devices formed by various electronic devices, enabling the execution process described in the method of the embodiments of the present invention.

[0073] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A method for intelligent identification of defects in power distribution drone inspections, characterized in that, include: The multimodal data collected by the power distribution drone is input into a pre-constructed hierarchical label tree for defect semantics, and the defect region is located based on multimodal weak supervised contrastive learning to generate structured samples of the defect region. Multimodal reasoning is performed on structured samples of defective regions to generate interpretable reasoning chains, and the defect type and attributes are determined based on the interpretable reasoning chains. Cross-modal lightweight distillation is performed on the defect type and attribute determination results. The knowledge is then transferred to the lightweight model and deployed to the UAV edge inference engine to achieve real-time defect identification and inference.

2. The intelligent defect identification method for power distribution drone inspection according to claim 1, characterized in that, The defect semantic hierarchical tag tree is a three-level semantic tag tree of "equipment category - defect type - attribute parameter". The first-level tag is the equipment category, which corresponds to the main objects of the power distribution drone inspection; the second-level tag is the defect type, which corresponds to the typical defects associated with each equipment category; and the third-level tag is the attribute parameter, which corresponds to the quantitative attributes of each defect type. Tree structure model This represents a three-level semantic tag tree, where, V Represents a set of nodes. , To represent the first-level node of the equipment category, For secondary nodes representing defect types, A third-level node representing attribute parameters; E Let be the set of edges. Represents a node and The parent-child relationship; L For the set of label weights, Representing an edge The semantic association weights are set based on the experience of power distribution experts and the frequency of defect occurrence. The semantic similarity of the three-level semantic tag tree is calculated using cosine similarity. For any two tag nodes... , Semantic similarity S ( Calculate according to the following formula: In the formula, For nodes v semantic embedding vector, for and The shortest path, d For path length, Let be the weight of the k-th edge of the path.

3. The intelligent defect identification method for power distribution drone inspection according to claim 1, characterized in that, The steps for locating defect regions and generating structured samples of defect regions based on multimodal weakly supervised contrastive learning include: Multimodal feature extraction from visual, textual, and point cloud data collected by power distribution drones; Through cross-modal contrastive learning, semantic alignment of "visual-text-point cloud" features is achieved, forcing the features of defective regions to be similar to the text description features, and the features of non-defective regions to be significantly different from the text description features. Based on the semantically aligned "visual-text-point cloud" features, defect regions are segmented, and defect regions are labeled using a hierarchical semantic label tree to generate structured samples of defect regions.

4. The intelligent defect identification method for power distribution drone inspection according to claim 3, characterized in that, When extracting multimodal features (visual, textual, and point cloud) from the multimodal data collected by the power distribution drone, the multimodal data collected by the power distribution drone includes visible light images, infrared thermal images, lidar point clouds, and corresponding textual descriptions of defects. For multimodal feature extraction, a lightweight convolutional neural network (CNN) is used to extract multi-scale features from the visible light images, outputting visual feature maps. Infrared images are segmented using a temperature threshold, and infrared feature maps are extracted and fused into joint visual features. For point cloud feature extraction, point clouds are first downsampled using a voxel grid, and spatial geometric features are extracted using a point net (PointNet). Then, region-of-interest (RoI) pooling is used to map the point cloud features to the image coordinate system. For textual feature extraction, a hierarchical label tree based on defect semantics is used to parse the text description T into a label sequence, and DistilBERT encoding is used to generate textual features.

5. The intelligent defect identification method for power distribution drone inspection according to claim 3, characterized in that, In the step of achieving semantic alignment of "visual-text-point cloud" features through cross-modal contrastive learning, and forcing defect region features to be similar to text description features while ensuring that non-defect region features are significantly different from text description features, the expression for the loss function is defined as follows: In the formula, Visual-text contrast loss is calculated using the following formula to compute joint visual features. Text features Comparative loss: In the formula, B Batch size; C This refers to the number of channels in the feature map. Let be the visual feature vector of the b-th sample and the c-th channel; For the first Text features of each sample Temperature coefficient; For point cloud-text contrast loss, the point cloud mapping features are calculated using the following formula. Text features Comparative loss: For the visual-point cloud contrast loss, the visual joint features are calculated using the following formula. Point cloud mapping features Comparative loss: In the formula, It is an L2 norm.

6. The intelligent defect identification method for power distribution drone inspection according to claim 5, characterized in that, In the step of segmenting defect regions based on semantically aligned "visual-text-point cloud" features and labeling defect regions using a hierarchical semantic label tree to generate structured samples of defect regions, the visual joint features are calculated. Text features Similarity map; and, calculating point cloud mapping features. Text features The similarity graphs are fused together, and the defect region mask is obtained by Otsu threshold segmentation. Based on the obtained defect region mask, the corresponding labels are labeled for the defect region in combination with the defect semantic hierarchical label tree, and a structured sample library is generated.

7. The intelligent defect identification method for power distribution drone inspection according to claim 1, characterized in that, In the step of performing multimodal reasoning on structured samples of defect regions to generate interpretable reasoning chains, and determining the defect type and attributes based on these interpretable reasoning chains, an attention fusion mechanism is used to analyze visible light images. Pixel-level fusion with infrared images, the expression is: Among them, attention weight pass Calculations show that the visual-infrared fusion features are... Point cloud features Text label features The embedding matrix of the cross-attention module and the defect semantic label tree is used. , obtain fusion features The calculation expression is: The inference chain probabilistic model is represented as ,in, The initial state is "to be reasoned". to These correspond to equipment identification, defect type determination, and attribute parameter quantification status, respectively. This represents the inference probability from state k-1 to k; in the device identification stage, it is based on fused features. With the first-level node V1 of the tag tree, through the expression Calculate the probability of device category, where, For equipment identification, a weight matrix is ​​used to select the equipment category with the highest probability as the result. In defect type determination, secondary nodes associated with the label tree are filtered based on the equipment identification results, using an expression... Calculate the probability of defect type, where, Weight matrix for determining defect type, For semantic weights; in the attribute parameter quantification stage, based on the defect type results, related third-level nodes are filtered, and attribute parameters are quantified through regression. The severity is based on the proportion of defect areas in visual features. The determination is based on direct quantization of size or temperature using point cloud volume or infrared temperature; the inference chain output integrates the three-step inference results into a natural language form.

8. The intelligent defect identification method for power distribution drone inspection according to claim 1, characterized in that, In the step of performing cross-modal lightweight distillation on the defect type and attribute determination results, transferring knowledge to a lightweight model, and then deploying it to the UAV edge-side inference engine to achieve real-time defect identification and inference, knowledge distillation enables the student model to obtain key information from the teacher model. The distillation loss function is a weighted sum of multi-task losses, including feature alignment loss, semantic consistency loss, and task loss. The feature alignment loss uses MSE loss to ensure that the multimodal feature distribution of the student model is consistent with that of the teacher model, allowing the student model to have the feature extraction capability of the teacher model. The semantic consistency loss is based on cosine similarity loss to ensure that the semantics of the inference chain of the student model are consistent with those of the teacher model, ensuring that the inference logic of both is the same. The task loss is composed of a weighted sum of classification loss and regression loss. The classification loss uses cross-entropy loss to calculate the classification error of the device and defect type, and the regression loss uses L1 loss to calculate the regression error of position, size, and temperature, ensuring the accuracy of the student model in the defect identification task. The distillation weights were determined by validating the power distribution defect dataset. The distilled student model is optimized, including model quantization, which uses INT8 quantization technology to convert the model weights from 32-bit floating-point numbers to 8-bit integers; and operator optimization, which uses the TensorRT framework to optimize the core operators and reduce memory access latency. The optimized model was converted to ONNX format and deployed to the drone-side inference engine.

9. A power distribution drone inspection defect intelligent identification system, characterized in that, include: The defect region localization module is used to input the multimodal data collected by the power distribution drone into a pre-constructed hierarchical label tree for defect semantics, and to locate the defect region based on multimodal weak supervised contrastive learning, generating a structured sample of the defect region. The thought chain reasoning module is used to perform multimodal thought chain reasoning on the structured samples of the defect area, generate an interpretable reasoning chain, and determine the defect type and attributes based on the interpretable reasoning chain. The lightweight distillation module is used to perform cross-modal lightweight distillation on the defect type and attribute determination results. After the knowledge is transferred to the lightweight model, it is deployed to the UAV edge inference engine to realize real-time defect identification and inference.

10. The intelligent defect identification system for power distribution drone inspection according to claim 9, characterized in that, When the defect area localization module inputs the multimodal data collected by the power distribution drone into the pre-constructed hierarchical semantic labeling tree, the hierarchical semantic labeling tree is a three-level semantic labeling tree of "equipment category - defect type - attribute parameter". The first-level label is the equipment category, which corresponds to the main objects inspected by the power distribution drone; the second-level label is the defect type, which corresponds to the typical defects associated with each equipment category; and the third-level label is the attribute parameter, which corresponds to the quantitative attributes of each defect type. Tree structure model This represents a three-level semantic tag tree, where, V Represents a set of nodes. , To represent the first-level node of the equipment category, For secondary nodes representing defect types, A third-level node representing attribute parameters; E Let be the set of edges. Represents a node and The parent-child relationship; L For the set of label weights, Representing an edge The semantic association weights are set based on the experience of power distribution experts and the frequency of defect occurrence. The semantic similarity of the three-level semantic tag tree is calculated using cosine similarity. For any two tag nodes... , Semantic similarity S ( Calculate according to the following formula: In the formula, For nodes v semantic embedding vector, for and The shortest path, d For path length, Let be the weight of the k-th edge of the path.

11. The intelligent defect identification system for power distribution drone inspection according to claim 9, characterized in that, The defect region localization module locates defect regions based on multimodal weak supervised contrastive learning. When generating structured samples of defect regions, it extracts multimodal features of vision, text, and point cloud from the multimodal data collected by the power distribution drone. Through cross-modal contrastive learning, semantic alignment of "visual-text-point cloud" features is achieved, forcing the features of defective regions to be similar to the text description features, and the features of non-defective regions to be significantly different from the text description features. Based on the semantically aligned "visual-text-point cloud" features, defect regions are segmented, and defect regions are labeled using a hierarchical semantic label tree to generate structured samples of defect regions.

12. The intelligent defect identification system for power distribution drone inspection according to claim 11, characterized in that, When the defect area localization module extracts multimodal features (visual, textual, and point cloud) from the multimodal data collected by the power distribution drone, the multimodal data includes visible light images, infrared thermal images, lidar point clouds, and corresponding textual descriptions of defects. For multimodal feature extraction, a lightweight convolutional neural network (CNN) is used to extract multi-scale features from the visible light images, outputting a visual feature map. Infrared images are segmented using a temperature threshold, and infrared feature maps are extracted and fused into joint visual features. For point cloud feature extraction, point clouds are first downsampled using a voxel grid, and spatial geometric features are extracted using a point net (PointNet). Then, region of interest (RoI) pooling is used to map the point cloud features to the image coordinate system. For textual feature extraction, a hierarchical label tree based on defect semantics is used to parse the text description T into a label sequence, and DistilBERT encoding is used to generate textual features.

13. The intelligent defect identification system for power distribution drone inspection according to claim 11, characterized in that, The defect region localization module achieves semantic alignment of "visual-text-point cloud" features through cross-modal contrastive learning, forcing defect region features to be similar to text description features. When non-defect region features differ significantly from text description features, the expression for the loss function is defined as follows: In the formula, Visual-text contrast loss is calculated using the following formula to compute joint visual features. Text features Comparative loss: In the formula, B Batch size; C This refers to the number of channels in the feature map. Let be the visual feature vector of the b-th sample and the c-th channel; For the first Text features of each sample Temperature coefficient; For point cloud-text contrast loss, the point cloud mapping features are calculated using the following formula. Text features Comparative loss: For the visual-point cloud contrast loss, the visual joint features are calculated using the following formula. Point cloud mapping features Comparative loss: In the formula, It is an L2 norm.

14. The intelligent defect identification system for power distribution drone inspection according to claim 13, characterized in that, The defect region localization module performs defect region segmentation based on semantically aligned "visual-text-point cloud" features, and annotates the defect regions using a hierarchical semantic label tree. When generating structured samples of defect regions, it calculates visual joint features. Text features Similarity map; and, calculating point cloud mapping features. Text features The similarity graphs are fused together, and the defect region mask is obtained by Otsu threshold segmentation. Based on the obtained defect region mask, the corresponding labels are labeled for the defect region in combination with the defect semantic hierarchical label tree, and a structured sample library is generated.

15. The intelligent defect identification system for power distribution drone inspection according to claim 9, characterized in that, The thought chain reasoning module utilizes an attention fusion mechanism to analyze visible light images. Pixel-level fusion with infrared images, the expression is: Among them, attention weight pass Calculations show that the visual-infrared fusion features are... Point cloud features Text label features The embedding matrix of the cross-attention module and the defect semantic label tree is used. , obtain fusion features The calculation expression is: The inference chain probabilistic model is represented as ,in, The initial state is "to be reasoned". to These correspond to equipment identification, defect type determination, and attribute parameter quantification status, respectively. This represents the inference probability from state k-1 to k; in the device identification stage, it is based on fused features. With the first-level node V1 of the tag tree, through the expression Calculate the probability of device category, where, For equipment identification, a weight matrix is ​​used to select the equipment category with the highest probability as the result. In defect type determination, secondary nodes associated with the label tree are filtered based on the equipment identification results, using an expression... Calculate the probability of defect type, where, Weight matrix for determining defect type, For semantic weights; in the attribute parameter quantification stage, based on the defect type results, related third-level nodes are filtered, and attribute parameters are quantified through regression. The severity is based on the proportion of defect areas in visual features. The determination is based on direct quantization of size or temperature using point cloud volume or infrared temperature; the inference chain output integrates the three-step inference results into a natural language form.

16. The intelligent defect identification system for power distribution drone inspection according to claim 9, characterized in that, The lightweight distillation module enables the student model to acquire key information from the teacher model through knowledge distillation. The distillation loss function is a weighted sum of multi-task losses, including feature alignment loss, semantic consistency loss, and task loss. The feature alignment loss uses MSE loss to ensure that the multimodal feature distribution of the student model is consistent with that of the teacher model, allowing the student model to have the feature extraction capabilities of the teacher model. The semantic consistency loss is based on cosine similarity loss to ensure that the semantics of the inference chain of the student model are consistent with those of the teacher model, ensuring that the inference logic of both is the same. The task loss is composed of a weighted sum of classification loss and regression loss. The classification loss uses cross-entropy loss to calculate the classification error of the device and defect type, and the regression loss uses L1 loss to calculate the regression error of position, size, and temperature, ensuring the accuracy of the student model in the defect identification task. The distillation weights were determined by validating the power distribution defect dataset. The distilled student model is optimized, including model quantization, which uses INT8 quantization technology to convert the model weights from 32-bit floating-point numbers to 8-bit integers; and operator optimization, which uses the TensorRT framework to optimize the core operators and reduce memory access latency. The optimized model was converted to ONNX format and deployed to the drone-side inference engine.

17. An electronic device, characterized in that, It includes a processor and a memory, wherein the processor is used to execute a computer program stored in the memory to implement the intelligent identification method for power distribution drone inspection defects as described in any one of claims 1 to 8.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the intelligent identification method for power distribution drone inspection defects as described in any one of claims 1 to 8.

Images