Defect image enhancement method fusing reasoning and generation
By employing multimodal feature fusion and dynamic collaborative decision-making methods, the problems of multi-source data fusion and sample scarcity in UAV inspection of power equipment were solved, achieving efficient and reliable defect detection in complex environments and reducing the risk of misjudgment.
Patent Information
- Application Number
- CN202511558159.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-29
- Publication Date
- 2026-07-14
- Estimated Expiration
- 2045-10-29
AI Technical Summary
Existing technologies rely on visible light images for defect analysis in power equipment drone inspections, which cannot effectively integrate multi-source data such as infrared and lidar. This results in limited detection reliability in complex environments, and traditional data augmentation methods have failed to effectively address the problem of sample scarcity, leading to an increased risk of model misjudgment in real-world scenarios.
A multimodal feature fusion network is adopted, which combines visible light, infrared and lidar data. Visual, spatial and semantic features are fused through the Transformer architecture, and the weights are dynamically adjusted to build defect localization and feature enhancement modules. Combined with a few-shot strategy and a generative model, inference-generation collaborative decision-making is carried out, and the generated samples are optimized through feature consistency loss and manual verification.
It significantly improves the stability and reliability of defect detection in complex environments, reduces the risk of false defects, and enhances model training efficiency and the practical value of the dataset.
Smart Images

Figure CN121414610B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of power equipment testing technology, specifically a defect image enhancement method that integrates reasoning and generation. Background Technology
[0002] In the field of unmanned aerial vehicle (UAV) inspection of power equipment, invention patent CN114281093A discloses a defect detection system and method based on UAV power inspection. The system achieves defect detection through visible light image acquisition and the knowledge distillation FCOS algorithm. Its three-level architecture and image quality evaluation model demonstrate basic detection capabilities in conventional scenarios. However, this technical solution still has the following technical problems:
[0003] 1. This patent relies entirely on visible light images for defect analysis, without integrating multi-source data such as infrared and lidar. For example, in insulator contamination detection, when grayscale differences are weakened under strong light, visual features alone are insufficient to identify the degree of contamination, and the depth of defect indentations cannot be quantified through 3D modeling. Although the FCOS algorithm it employs improves the detection capability of small targets through multi-scale feature maps, it is limited by single-modal information, and still suffers from insufficient features for detecting tiny components such as hardware pins, significantly limiting the reliability of detection in complex environments.
[0004] 2. The data augmentation methods mentioned in the patent (such as resolution scaling and manual annotation) do not effectively solve the problem of sample scarcity. Taking the detection of "early microcracks in insulators" as an example, traditional geometric transformations cannot simulate the crack propagation characteristics under different lighting conditions, and the generated samples deviate significantly from the feature distribution of real defects. Although it uses knowledge distillation technology to compress the model volume, it does not perform semantic consistency verification on the generated samples, which increases the risk of misjudging false defects (such as false cracks without physical basis) in real scenarios. This limitation is particularly prominent in the detection of "conductor strand breakage" in transmission lines, where the detection accuracy drops significantly when the sample size is insufficient. Summary of the Invention
[0005] The purpose of this invention is to provide a defect image enhancement method that integrates reasoning and generation to solve the problems mentioned in the background art.
[0006] To achieve the above objectives, the present invention provides the following technical solution: a defect image enhancement method that integrates reasoning and generation, the specific steps of which are as follows:
[0007] S1. Multi-source defect data acquisition and structured processing: Collect defect images of insulators, conductors, hardware, etc., inspected by UAVs, and simultaneously record 3D point clouds (reflecting spatial structure) and scene information such as lighting and occlusion; after denoising and scaling to 512×512 pixels using BM3D, contrast is enhanced using CLAHE, and then the defect areas are coarsely marked by Canny edge detection, and a structured dataset with location and scene labels is formed by classifying by type.
[0008] S2. Multimodal Feature Fusion Network Construction: Based on the Transformer architecture, it fuses visual features (texture, shape, extracted by pre-trained Vit), spatial features (depth, relative position, from 3D point cloud), and semantic features (professionally defined, generated by industry knowledge model). Through a dynamic attention mechanism, it adjusts weights according to defect type to generate a feature map containing global and local information, providing a unified input for inference and generative models.
[0009] S3. Training of the inference enhancement sub-model: Using multimodal feature maps as input, train a model containing defect localization (improved YOLO, adaptive parameter tuning of anchor boxes) and feature enhancement (edge sharpening, Gamma adjustment) modules; adopt a small sample strategy to transfer common features from similar defects, combine with a small amount of labeled data for fine-tuning, and output defect confidence and enhanced feature maps to guide the defect region for the generation model;
[0010] S4. Construction and Adaptation of Enhanced Sub-model: Construct a model containing segmentation (SAM model, generating defect mask) and 3D generation modules: After segmentation, 3D modeling of small sample defects is performed, and 2D samples are generated through perspective and lighting simulation. The model is constrained by inference features (cosine similarity check) to ensure realism. The encapsulation module supports rapid adaptation to new defects.
[0011] S5. Inference-Generation Dynamic Collaborative Decision Making: Dynamically allocate weights based on sample size: When the sample size is sufficient (≥2000 images), inference dominates (70%), and generation assists in fuzzing; when the sample size is scarce (<500 images), generation dominates (60%), and inference verifies and eliminates false samples; the feature consistency loss function is used to adjust the weights to achieve inference-driven generation and generation-supplemented data.
[0012] S6. Enhanced Sample Authenticity Verification and Iteration: Combining machine verification (feature consistency model, marking deviations exceeding 15%) with manual review (expert verification of defects), recording typical problems, such as excessive enhancement of dirt under strong light, and providing feedback to optimize model parameters, such as adjusting the generation perspective; updating standards monthly to improve sample authenticity;
[0013] S7. Enhanced Dataset Output and Closed-Loop Optimization: Output a dataset containing enhanced images, labels, and feature maps for defect recognition training. Feedback the recognition results (accuracy, false alarm rate) to adjust inference weights and generation strategies, forming an "enhancement-training-feedback" closed loop to continuously improve the practical value of the dataset and complete the entire process optimization.
[0014] Preferably, the specific steps of step S1 are as follows:
[0015] S11. Multi-source data acquisition and basic preprocessing: Acquire defect images of insulators, conductors, and fittings inspected by UAVs, and simultaneously record 3D point clouds (reflecting spatial structure) and scene information such as lighting and occlusion; Standardize the original images: Use the BM3D algorithm to remove noise, normalize the scale to 512×512 pixels, and enhance local contrast through CLAHE to highlight the grayscale difference between defects and the background, such as the dirty areas of insulators, providing a clear data foundation for subsequent processing;
[0016] S12. Coarse labeling and structured storage of small sample defects: For small sample defects, Canny edge detection is used to coarsely locate suspected areas and generate a dataset containing location coordinates and scene labels (such as "strong light - no obstruction"). The dataset is classified and stored according to types such as "insulator damage" and "conductor strand breakage" to ensure that the data has both diversity and consistency, providing standardized input for step S2 and ensuring that defect features under different scenarios can be effectively captured.
[0017] Preferably, the specific steps of step S2 are as follows:
[0018] S21. Multi-type feature extraction and integration: Build a feature extraction network based on Transformer and integrate three types of features: visual features (texture, shape, extracted by pre-trained Vit model), spatial features (depth, relative position, from 3D point cloud), and semantic features (profession definition, generated by industry knowledge model) to fully cover the details and contextual information of defects.
[0019] Multimodal feature fusion formula: ,
[0020] In the formula: This represents the fused multimodal feature map, with dimensions of [dimension number missing]. ( Image height, For width, (The number of feature channels) is a unified input feature for both inference and generative models;
[0021] Represents dynamic attention weights ( ,and The dynamic attention mechanism adaptively adjusts based on the defect type, such as when identifying broken strands in a conductor. Increase;
[0022] Represents a visual feature vector with dimension . ( (The number of visual feature channels) is extracted by a pre-trained Vit model and includes pixel-level features such as the texture of defects (e.g., the grayscale distribution of rust spots) and shape (e.g., the geometric contour of broken strands).
[0023] Represents a spatial feature vector with dimension . ( The number of spatial feature channels is obtained by parsing three-dimensional point cloud data, including spatial information such as the depth of defects (e.g., the depth of the indentation when the insulator is broken) and the relative coordinates with surrounding components (e.g., the distance between the broken strand of the conductor and the tower).
[0024] Represents a semantic feature vector with dimension . ( (For semantic features), generated by the power industry knowledge model, containing professional definitions of defect types (such as the judgment rule encoding for "damaged porcelain insulators") and other abstract semantic information;
[0025] By dynamically weighting and fusing visual, spatial, and semantic features, the problem of single-modal features not being able to fully describe complex defects is solved, providing a unified feature input containing multi-dimensional information for subsequent reasoning and generative models;
[0026] Unlike the fixed-weight feature concatenation in existing technologies, the α, β, and γ of this invention are calculated in real time through a feedforward network. The input is the preliminary classification result of the defect area (such as 'insulator' or 'conductor'), and the output is a weight combination that meets the detection requirements of this type of defect. The network training data comes from the feature importance annotation of more than 100,000 power defect samples, ensuring that the weight allocation is specific to the power industry.
[0027] S22. Dynamic Attention Mechanism and Feature Map Generation: Design a dynamic attention mechanism to adjust feature weights according to defect type: for example, when identifying "broken strands of conductors", enhance the spatial feature weights to capture the three-dimensional shape, and when identifying "polluted insulators", increase the visual feature weights to highlight grayscale changes; fuse and generate feature maps containing global and local information to provide a unified input for steps S3 and S4, and avoid collaborative failures caused by inconsistent feature dimensions.
[0028] The dynamic attention mechanism is designed specifically for the defects in power equipment—for example, insulator contamination is characterized by visual grayscale gradients, so the visual feature weight α adaptively fluctuates between 0.6 and 0.8; conductor strand breakage depends on spatial morphology, so the spatial feature weight β is fixed at 0.5-0.7, and is dynamically corrected by the curvature value of the 3D point cloud at the defect edge (β is additionally increased by 0.1 when curvature > 0.3). This mechanism breaks through the conventional approach of equal weight fusion of multimodal features in existing technologies, directly linking the physical properties of the defect with the feature weight allocation.
[0029] Preferably, the specific steps of step S3 are as follows:
[0030] S31. Dual-module construction and functional design: Based on multimodal feature maps, an inference model with defect localization and feature enhancement is constructed. The localization module uses an improved YOLO architecture to accurately locate small sample defects, such as micro-rust, through adaptive adjustment of anchor boxes. The enhancement module performs edge sharpening (Laplacian operator) and contrast optimization (dynamic Gamma value) on the localized area to enhance the distinction between defects and background.
[0031] Inference model defect confidence calculation:
[0032] ,
[0033] In the formula: This represents the defect confidence level of the inference model output. A higher value indicates that the model is more accurate in locating defects and the features are clearer;
[0034] Indicates weighting coefficient ,and These figures respectively reflect the proportions of positioning accuracy and feature quality in the confidence score (e.g., small sample defects place more emphasis on positioning). It can be set to 0.6);
[0035] Indicates the accuracy of defect location ( The intersection-over-union (IoU) ratio is calculated from the bounding boxes output by the improved YOLO model and the manually annotated boxes. =IoU= ;
[0036] Indicates feature clarity ( The edge gradient of the defect region is calculated using the Laplacian operator and then normalized. A higher gradient value indicates a clearer edge. ;
[0037] The reliability of the quantitative inference model in identifying defects provides clear guidance on defect areas for the generative model (high-confidence areas are prioritized for sample generation), thus avoiding the waste of computing power by the generative model in non-defect areas.
[0038] S32. Small sample learning strategy and output guidance: The small sample learning strategy is adopted: common features are transferred from similar defects, such as the discontinuous edge characteristics of insulator damage, and parameters are fine-tuned by combining a small amount of labeled data; the model outputs defect confidence (reflecting the accuracy of positioning) and enhanced feature map, which clarifies the defect area for step S4 and ensures that the generated samples focus on the real defect location.
[0039] Small-sample common feature transfer employs a feature-specific design for power industry defects. For example, 'orange-yellow mottled texture + edge blur <0.2' extracted from 'fitting corrosion' samples is used as a common feature. When transferring to new samples, atypical textures (such as white oxide spots) are filtered out using semantic features of the inference model ('corrosion level' defined by the industry knowledge model), ensuring the power industry specificity of the transferred features. This strategy differs from the general transfer based on image pixels in existing technologies, directly anchoring to professional defect judgment standards.
[0040] Preferably, the specific steps of step S4 are as follows:
[0041] S41. Segmentation and 3D Generation Module Design: Construct a generative model containing segmentation and 3D generation: The segmentation module uses the SAM model to accurately segment the defect region output by inference and generates a binary mask (the defect region is 1); the 3D generation module models small sample defects based on the segmentation results and converts the 3D model into a 2D image through perspective and lighting simulation to achieve sample expansion;
[0042] S42. Inference Constraints and Parameter Adaptation Mechanism: Inference constraints are introduced during training: the features of generated samples must match the multimodal feature map of the inference model, and cosine similarity is used for verification. For example, the texture of the generated sample for "insulator damage" must be consistent with the real damage. The generation logic is encapsulated as a reusable module to support rapid adaptation to new defect types and ensure that the generated samples are rich and closely match the real characteristics.
[0043] The cosine similarity verification of the generated samples not only matches the feature vectors, but also additionally verifies the key physical parameters of power defects. For example, the generated 'damaged insulator' sample must meet the requirement of 'damaged area / total insulator area < 0.3' (compliant with the power industry standard DL / T1573-2016). This constraint is forcibly input by the semantic feature module of the inference model, which breaks through the limitation of generating samples by matching feature distribution alone in the existing technology, and realizes the dual verification of 'data features + industry standards'.
[0044] The degree of matching between generated samples and inferred features: ,
[0045] In the formula: The cosine similarity between the generated sample features and the inferred features ( The closer the value is to 1, the more consistent the characteristic distributions of the two are;
[0046] This represents the sample feature vector (with dimension D, where D is the feature dimension, such as 512-dimensional) output by the generative model, which includes features such as texture and shape of the generated sample.
[0047] This indicates that the defect feature vector (with dimension D) output by the inference model comes from the defect region features after multimodal feature fusion;
[0048] Represents the vector dot product. ;
[0049] L2 norm , Similarly;
[0050] As a constraint on the generative model, it ensures that the features of the generated virtual samples are consistent with the features of the real defects identified by the inference model, thus mathematically avoiding the generation of "pseudo-defects" (such as cracks that appear out of nowhere).
[0051] Preferably, the specific steps of step S5 are as follows:
[0052] S51. Sample size-driven dynamic weight allocation: Design a collaborative decision-making module to allocate weights according to sample size: When the sample size is sufficient (≥2000 images), the inference model takes the lead (70% weight) in optimizing defective features, and the generation model assists in correcting ambiguous areas; when the sample size is scarce (<500 images), the generation model takes the lead (60% weight) in generating virtual samples, and the inference model verifies and removes false samples.
[0053] Dynamic collaborative weight allocation formula:
[0054] ,
[0055] In the formula: Represents the weights of the generative model ( ), representing the percentage of contribution of the generative model in the enhancement process;
[0056] The weights of the inference model ( ),and Complementary;
[0057] This indicates the sample size for the current defect type (unit: sheets). ), reflecting the richness of the labeled data for such defects;
[0058] Different from the design in the prior art where the generation model is independent of the inference model, the weight allocation of the present invention is simultaneously associated with the defect confidence of the inference model ( when the inference weight is forced to increase by 0.1), ensuring that the generated samples focus on the high-confidence defect areas and avoiding the pseudo-defect problem caused by the undifferentiated expansion of samples by the generation model in the prior art.
[0059] Dynamically adjust the collaborative relationship between the two models according to the sample size - when the sample is sufficient ( ), the inference-dominated features are optimized; when the sample is scarce (0 < N < 500), the generation weight increases linearly with the decrease of the sample; when there is no sample (N = 0), the generation dominates the sample creation, realizing the adaptive enhancement of different data scenarios;
[0060] The sample size-driven weight allocation is not a simple linear adjustment, but embeds the data distribution characteristics of power defects - when the sample size < 500 (such as early microcracks), additional material mechanics constraints (such as the crack propagation direction being consistent with the stress distribution) need to be introduced into the three-dimensional modeling of the generation model, and this constraint is guided by the defect mechanics features (stress concentration area based on three-dimensional point cloud) extracted by the inference model. This mechanism has not been seen in the field of defect image enhancement in the prior art, achieving double constraints of 'physical laws + data features'.
[0061] S52. Feature consistency loss and collaborative optimization: Introduce a feature consistency loss function to calculate the feature difference between the inference and generation results. If the difference exceeds the threshold (such as 0.2), then increase the inference weight to calibrate the generation direction; achieve 'inference guiding generation accuracy, generation supplementing inference data' through dynamic collaboration, and solve the contradiction between data insufficiency and enhancement distortion in the small-sample scenario.
[0062] Preferably, the specific steps of step S6 are as follows:
[0063] S61. Design and implementation of a double-layer verification mechanism: Establish a double-layer verification of machine and human: For machine verification, use a feature consistency model to compare the key features of the enhanced image and the real defect, such as the proportion of the rust area. If the deviation exceeds 15%, it is marked as 'to be optimized'; for human verification, it is reviewed by patrol experts, focusing on checking 'pseudo-defects', such as false cracks, and recording typical problems, such as excessive enhancement of dirt under strong light;
[0064] The manual review focuses on 'pseudo-defects unique to the power industry'—such as water stains on the surface of insulators, which are easily misjudged as dirt. During the review, the surface flatness of the 3D point cloud is compared (flatness of water stain area > 0.9, flatness of dirt area < 0.6) to assist in the judgment. This judgment logic is transformed into semantic constraints of the inference model (the dirt confidence is forcibly reduced by 0.3 when flatness > 0.8), realizing the accurate conversion of human experience into model parameters and solving the problem of the disconnect between manual verification and model optimization in the existing technology.
[0065] S62. Verification Result Feedback and Model Iteration: Transform the verification results into optimization indicators: For example, if the angle deviation of "broken conductor strand" frequently exceeds the standard, adjust the three-dimensional view parameters of the generated model; if "insulator contamination" has false features, strengthen the semantic constraints of the inference model, update the verification standards monthly, such as adjusting the light weight according to the season, and improve the authenticity and reliability of the samples through iteration.
[0066] Feature consistency loss function:
[0067] ,
[0068] In the formula: Represents feature consistency loss ( The smaller the value, the smaller the feature difference between the augmented sample and the real sample;
[0069] Represents the balance coefficient ( If the focus is on the difference in feature vectors, it can be set as follows: =0.7, focusing on the difference in probability distribution, can be set as ;
[0070] This represents the feature vector (dimension D) of the augmented sample;
[0071] The feature vector (with dimension D) represents the real sample;
[0072] L2 norm represents the Euclidean distance between two eigenvectors.
[0073] ;
[0074] KL divergence represents the difference between the probability distribution of defect types in augmented samples and real samples.
[0075] (K is the total number of defect types, To increase the probability that a sample belongs to class K);
[0076] These represent the probability distributions of defect types in the enhanced sample and the real sample, respectively. );
[0077] The optimization objective for model training (minimizing via gradient descent) This approach enhances the realism of samples by constraining them from two dimensions: feature vectors and probability distributions, thus addressing the problem that generative models easily deviate from real features.
[0078] Manual verification of the weight update formula:
[0079] ,
[0080] In the formula: This represents the updated feature weights (e.g., visual feature weights). );
[0081] Represents the feature weights before the update. ;
[0082] Indicates the learning rate Control the magnitude of weight adjustment, such as This indicates a maximum adjustment of 10% each time;
[0083] Indicates manual verification score ( The inspection experts score the enhanced samples based on their matching degree with the real defects: 1 indicates complete matching, 0 indicates average matching, and -1 indicates the presence of obvious false features, such as false cracks.
[0084] Transforming human experience into a basis for adjusting model parameters—when experts determine that a certain type of feature (such as texture) enhances distortion ( When <0), by reducing the corresponding weight ( The model is optimized to compensate for the shortcomings of machine verification in judging "semantic rationality".
[0085] Preferably, the specific steps of step S7 are as follows:
[0086] S71. Enhanced Dataset Output and Recognition Application: Output a dataset containing enhanced images, defect labels (location, type), and multimodal feature maps for defect recognition algorithm training. Recognition performance metrics (accuracy, false alarm rate) are fed back to the inference and generative models; for example, if the accuracy of a certain type of defect recognition decreases, its training weight in the inference model is increased.
[0087] S72. Feedback Loop Construction and Adaptive Optimization: Establishing an "Enhancement-Training-Feedback" closed loop: After each round of enhancement, update the feature weights of the inference model and the constraints of the generation model based on the recognition effect, adapting to different defect requirements. For example, when the recognition rate of "metal rust" is low, optimize the sensitivity of the inference model to rust texture, and the generation model focuses on generating samples with different degrees of rust, gradually improving the practical value of the dataset;
[0088] The formula for feedback of recognition accuracy in closed-loop optimization:
[0089] ,
[0090] In the formula: This represents the increment of training weights for a certain type of defect in the inference model. A larger value indicates that the training weight for this type of defect needs to be increased;
[0091] : Represents the proportionality coefficient Control the magnitude of weight adjustment (e.g.) (To avoid excessive fluctuations in weighting);
[0092] Indicates the target recognition accuracy (e.g.) (i.e., 85%)
[0093] Indicates the current recognition accuracy. It is calculated from the performance of the defect identification algorithm on the test set;
[0094] This represents the number of incorrect identification samples for this type of defect. ;
[0095] This represents the total number of samples of this type of defect. , Error rate;
[0096] The performance gap in defect identification algorithms is translated into training weight adjustments for the inference model—when the accuracy of identifying a certain type of defect is low. And the error rate is high ( When (large), Increasing the training ratio of this type of defect in the model will enable a closed-loop optimization of "enhancement-identification-feedback".
[0097] The enhancement-training-feedback closed loop introduces a 'defect type priority mechanism'—for example, a broken conductor / ground wire poses the highest safety risk, and its identification accuracy target value is set accordingly. Set to the highest level (0.9 or higher), when the actual accuracy rate... When the value is below the target value, the training weight increment of the inference model The system automatically increases the sample size by 20%, prioritizing the expansion of this type of sample in the generated model (accounting for no less than 30% of the total sample). This mechanism embeds the safety priority of power line inspection into the optimization logic, which differs from the conventional design of indiscriminate optimization in existing technologies.
[0098] The beneficial effects of this invention are as follows:
[0099] 1. This invention integrates visible light, infrared, and lidar data through a multimodal feature fusion network, overcoming the limitation of prior art document CN114281093A which relies solely on visible light images. The dynamic attention mechanism can flexibly adjust the weights of visual, spatial, and semantic features according to the defect type, and can still capture key features in complex environments such as strong light and occlusion, such as subtle grayscale changes in insulator contamination. At the same time, it uses three-dimensional point clouds to analyze the spatial morphology of defects, such as the relative position of broken strands in conductors. This design effectively avoids missed detection of small parts such as hardware pins due to insufficient features, and significantly improves the stability and reliability of defect detection in complex scenarios.
[0100] 2. This invention addresses the problem of distorted samples generated by traditional data augmentation methods in the prior art. It ensures sample quality through a dual mechanism of "inference constraints + physical verification." The defect features output by the inference model directly constrain the feature distribution of the generated samples. At the same time, the laws of material mechanics are introduced to verify the physical rationality of the generated samples, such as the crack propagation direction of broken strands in conductors. This design completely avoids the generation of false defects, making the augmented samples of small defects (such as early microcracks in insulators) highly consistent with the real defect features, and significantly reducing the risk of misjudgment in practical applications.
[0101] 3. This invention overcomes the resource waste and insufficient guidance caused by static weight allocation of comparison files through a dynamic weight allocation mechanism. When samples are sufficient, the inference model dominates feature optimization, while the generation model only assists in correcting local ambiguity. When samples are scarce, the generation model dominates sample expansion, while the inference model accurately locates defective areas to guide generation. Combined with the feature consistency loss function to dynamically calibrate the output, redundant calculations are avoided, and the focus of generated samples is ensured. This significantly improves the model training efficiency and the practical value of the augmented dataset, achieving optimal resource allocation. Attached Figure Description
[0102] Figure 1 This is a flowchart of the defect image enhancement method that integrates reasoning and generation according to the present invention. Detailed Implementation
[0103] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0104] like Figure 1 As shown in the figure, this embodiment of the invention provides a defect image enhancement method that integrates inference and generation. The specific steps of the method are as follows:
[0105] S1. Multi-source defect data acquisition and structured processing: Collect defect images of insulators, conductors, hardware, etc., inspected by UAVs, and simultaneously record 3D point clouds (reflecting spatial structure) and scene information such as lighting and occlusion; after denoising and scaling to 512×512 pixels using BM3D, contrast is enhanced using CLAHE, and then the defect areas are coarsely marked by Canny edge detection, and a structured dataset with location and scene labels is formed by classifying by type.
[0106] S2. Multimodal Feature Fusion Network Construction: Based on the Transformer architecture, it fuses visual features (texture, shape, extracted by pre-trained Vit), spatial features (depth, relative position, from 3D point cloud), and semantic features (professionally defined, generated by industry knowledge model). Through a dynamic attention mechanism, it adjusts weights according to defect type to generate a feature map containing global and local information, providing a unified input for inference and generative models.
[0107] S3. Training of the inference enhancement sub-model: Using multimodal feature maps as input, train a model containing defect localization (improved YOLO, adaptive parameter tuning of anchor boxes) and feature enhancement (edge sharpening, Gamma adjustment) modules; adopt a small sample strategy to transfer common features from similar defects, combine with a small amount of labeled data for fine-tuning, and output defect confidence and enhanced feature maps to guide the defect region for the generation model;
[0108] S4. Construction and Adaptation of Enhanced Sub-model: Construct a model containing segmentation (SAM model, generating defect mask) and 3D generation modules: After segmentation, 3D modeling of small sample defects is performed, and 2D samples are generated through perspective and lighting simulation. The model is constrained by inference features (cosine similarity check) to ensure realism. The encapsulation module supports rapid adaptation to new defects.
[0109] S5. Inference-Generation Dynamic Collaborative Decision Making: Dynamically allocate weights based on sample size: When the sample size is sufficient (≥2000 images), inference dominates (70%), and generation assists in fuzzing; when the sample size is scarce (<500 images), generation dominates (60%), and inference verifies and eliminates false samples; the feature consistency loss function is used to adjust the weights to achieve inference-driven generation and generation-supplemented data.
[0110] S6. Enhanced Sample Authenticity Verification and Iteration: Combining machine verification (feature consistency model, marking deviations exceeding 15%) with manual review (expert verification of defects), recording typical problems, such as excessive enhancement of dirt under strong light, and providing feedback to optimize model parameters, such as adjusting the generation perspective; updating standards monthly to improve sample authenticity;
[0111] S7. Enhanced Dataset Output and Closed-Loop Optimization: Output a dataset containing enhanced images, labels, and feature maps for defect recognition training. Feedback the recognition results (accuracy, false alarm rate) to adjust inference weights and generation strategies, forming an "enhancement-training-feedback" closed loop to continuously improve the practical value of the dataset and complete the entire process optimization.
[0112] The specific steps of step S1 are as follows:
[0113] S11. Multi-source data acquisition and basic preprocessing: Acquire defect images of insulators, conductors, and fittings inspected by UAVs, and simultaneously record 3D point clouds (reflecting spatial structure) and scene information such as lighting and occlusion; Standardize the original images: Use the BM3D algorithm to remove noise, normalize the scale to 512×512 pixels, and enhance local contrast through CLAHE to highlight the grayscale difference between defects and the background, such as the dirty areas of insulators, providing a clear data foundation for subsequent processing;
[0114] The drone uses a quadcopter model (such as the DJI Matrice 300RTK), equipped with a 20-megapixel visible light camera, with JPG image format and PLY 3D point cloud data format, and the number of point cloud points per frame is ≥1 million.
[0115] The BM3D algorithm uses a 3×3 sliding window for denoising, with a block matching threshold of 0.6; the CLAHE-enhanced clip limit is set to 2.0, and the mesh size is 8×8.
[0116] S12. Coarse labeling and structured storage of small sample defects: For small sample defects, Canny edge detection is used to coarsely locate suspected areas and generate a dataset containing location coordinates and scene labels (such as "strong light - no obstruction"). The dataset is classified and stored according to types such as "insulator damage" and "conductor strand breakage" to ensure that the data has both diversity and consistency, providing standardized input for step S2 and ensuring that defect features under different scenarios can be effectively captured.
[0117] The structured dataset is stored in CSV format. Each row contains fields such as 'image path, defect type, location coordinates (xmin, ymin, xmax, ymax), illumination label, and occlusion label'. The storage path is divided into 'defect type / date' levels, such as 'insulator damage / 20250812 / '.
[0118] The high threshold for Canny edge detection is set to 0.3 × maximum gradient value, and the low threshold is set to 0.1 × maximum gradient value. Scene labels are classified in two dimensions according to light intensity and occlusion degree. Light intensity is divided into weak light (<300 lux), normal (300-1000 lux), and strong light (>1000 lux). Occlusion degree is divided into no occlusion, partial occlusion (<30%), and severe occlusion (≥30%).
[0119] The specific steps of step S2 are as follows:
[0120] S21. Multi-type feature extraction and integration: Build a feature extraction network based on Transformer and integrate three types of features: visual features (texture, shape, extracted by pre-trained Vit model), spatial features (depth, relative position, from 3D point cloud), and semantic features (profession definition, generated by industry knowledge model) to fully cover the details and contextual information of defects.
[0121] The number of visual feature channels is Cv=256, the number of spatial feature channels is Cs=128, and the semantic feature dimension is Cn=64. The pre-trained Vit model adopts the Vit-B / 16 architecture, and the industry knowledge model is generated based on the power equipment defect terminology database (containing 500+ professional definitions).
[0122] Multimodal feature fusion formula: ,
[0123] In the formula: This represents the fused multimodal feature map, with dimensions of [dimension number missing]. ( Image height, For width, (The number of feature channels) is a unified input feature for both inference and generative models;
[0124] Represents dynamic attention weights ( ,and The dynamic attention mechanism adaptively adjusts based on the defect type, such as when identifying broken strands in a conductor. Increase;
[0125] Represents a visual feature vector with dimension . ( (The number of visual feature channels) is extracted by a pre-trained Vit model and includes pixel-level features such as the texture of defects (e.g., the grayscale distribution of rust spots) and shape (e.g., the geometric contour of broken strands).
[0126] Represents a spatial feature vector with dimension . ( The number of spatial feature channels is obtained by parsing three-dimensional point cloud data, including spatial information such as the depth of defects (e.g., the depth of the indentation when the insulator is broken) and the relative coordinates with surrounding components (e.g., the distance between the broken strand of the conductor and the tower).
[0127] Represents a semantic feature vector with dimension . ( (For semantic features), generated by the power industry knowledge model, containing professional definitions of defect types (such as the judgment rule encoding for "damaged porcelain insulators") and other abstract semantic information;
[0128] By dynamically weighting and fusing visual, spatial, and semantic features, the problem of single-modal features not being able to fully describe complex defects is solved, providing a unified feature input containing multi-dimensional information for subsequent reasoning and generative models;
[0129] S22. Dynamic Attention Mechanism and Feature Map Generation: Design a dynamic attention mechanism to adjust feature weights according to defect type: for example, when identifying "broken strands of conductors", enhance the spatial feature weights to capture the three-dimensional shape, and when identifying "polluted insulators", increase the visual feature weights to highlight grayscale changes; fuse and generate feature maps containing global and local information to provide a unified input for steps S3 and S4, and avoid collaborative failures caused by inconsistent feature dimensions.
[0130] The feature map output is in Tensor format (H=512, W=512, C=512), and is normalized by LayerNorm with normalization parameters γ=1.0 and β=0.0 to avoid feature value distribution shift.
[0131] The dynamic attention mechanism identifies defect types by triggering conductor break weight adjustment when 'spatial feature percentage > 40%' in the feature map, and triggering insulator pollution weight adjustment when 'visual feature grayscale change rate > 20%'.
[0132] The specific steps of step S3 are as follows:
[0133] S31. Dual-module construction and functional design: Based on multimodal feature maps, an inference model with defect localization and feature enhancement is constructed. The localization module uses an improved YOLO architecture to accurately locate small sample defects, such as micro-rust, through adaptive adjustment of anchor boxes. The enhancement module performs edge sharpening (Laplacian operator) and contrast optimization (dynamic Gamma value) on the localized area to enhance the distinction between defects and background.
[0134] The localization module uses the CIoU loss function with a weight coefficient of 1.0 and a bounding box regression learning rate of 5e-4. The Gamma value of the enhancement module is calculated according to the formula Gamma=1.0+0.5×(255-average brightness of defect area) / 255 to ensure that Gamma>1 in dark areas and Gamma<1 in bright areas.
[0135] The initial dimensions of the anchor frame of the improved YOLO were set to 16×16, 32×32, and 64×64, with an adaptive adjustment step size of 5% per iteration; the Laplacian operator used a 3×3 convolution kernel, with a dynamic Gamma value range of 0.5-2.0, adaptively selected based on the average brightness of the defect area;
[0136] Inference model defect confidence calculation:
[0137] ,
[0138] In the formula: This represents the defect confidence level of the inference model output. A higher value indicates that the model is more accurate in locating defects and the features are clearer;
[0139] Indicates weighting coefficient ,and These figures respectively reflect the proportions of positioning accuracy and feature quality in the confidence score (e.g., small sample defects place more emphasis on positioning). It can be set to 0.6);
[0140] Indicates the accuracy of defect location ( The intersection-over-union (IoU) ratio is calculated from the bounding boxes output by the improved YOLO model and the manually annotated boxes. =IoU= ;
[0141] Indicates feature clarity ( The edge gradient of the defect region is calculated using the Laplacian operator and then normalized. A higher gradient value indicates a clearer edge. ;
[0142] The reliability of the quantitative inference model in identifying defects provides clear guidance on defect areas for the generative model (high-confidence areas are prioritized for sample generation), thus avoiding the waste of computing power by the generative model in non-defect areas.
[0143] S32. Small sample learning strategy and output guidance: Employ a small sample learning strategy: transfer common features from similar defects, such as the discontinuous edge characteristics of insulator damage, and fine-tune parameters by combining a small amount of labeled data; the model outputs defect confidence (reflecting the accuracy of positioning) and enhanced feature map, which clarifies the defect area for step S4 and ensures that the generated samples focus on the real defect location;
[0144] During small sample transfer, 5-10 samples are randomly selected from the same type of defect as the support set and formed training pairs with the query samples. The episode training mode is adopted, and each episode contains 10 support samples and 20 query samples.
[0145] The common feature transfer of similar defects comes from public power defect datasets (such as ELectric Power DefectDataset) and 100+ internal typical defect samples; when fine-tuning the parameters, the learning rate is set to 1e-5 and the number of iterations is 50 rounds.
[0146] The specific steps of step S4 are as follows:
[0147] S41. Segmentation and 3D Generation Module Design: Construct a generative model containing segmentation and 3D generation: The segmentation module uses the SAM model to accurately segment the defect region output by inference and generates a binary mask (the defect region is 1); the 3D generation module models small sample defects based on the segmentation results and converts the 3D model into a 2D image through perspective and lighting simulation to achieve sample expansion;
[0148] The binary mask output by the segmentation module undergoes morphological processing: first erosion (3×3 rectangular kernel, 1 iteration) and then dilation (3×3 rectangular kernel, 1 iteration) to eliminate mask edge burrs; when converting the 3D model to a 2D image, each 3D model generates 20 2D samples from different perspectives;
[0149] The input resolution of the SAM model is 512×512, and the mask generation threshold is set to 0.7. The 3D modeling adopts the Poisson reconstruction algorithm, the point cloud sampling density is 100 points / mm², and the simulated rotation angle range of the viewpoint is -30° to 30°, with an interval of 5°.
[0150] S42. Inference Constraints and Parameter Adaptation Mechanism: Inference constraints are introduced during training: the features of generated samples must match the multimodal feature map of the inference model, and cosine similarity is used for verification. For example, the texture of the generated sample for "insulator damage" must be consistent with the real damage. The generation logic is encapsulated as a reusable module to support rapid adaptation to new defect types and ensure that the generated samples are rich and closely match the real characteristics.
[0151] The reusable module is encapsulated as a Python class, which includes the methods 'load_model(), generate_sample(defect mask)' and 'validate(feature vector)'. The generate_sample method supports batch generation, with a maximum batch size of 32.
[0152] The intensity range of the simulated illumination is 200-1500 lux, with intervals of 100 lux; the pass threshold for cosine similarity verification is set to 0.8, and generated samples below this value are filtered out.
[0153] The degree of matching between generated samples and inferred features: ,
[0154] In the formula: The cosine similarity between the generated sample features and the inferred features ( The closer the value is to 1, the more consistent the characteristic distributions of the two are;
[0155] This represents the sample feature vector (with dimension D, where D is the feature dimension, such as 512-dimensional) output by the generative model, which includes features such as texture and shape of the generated sample.
[0156] This indicates that the defect feature vector (with dimension D) output by the inference model comes from the defect region features after multimodal feature fusion;
[0157] Represents the vector dot product. ;
[0158] L2 norm , Similarly;
[0159] As a constraint on the generative model, it ensures that the features of the generated virtual samples are consistent with the features of the real defects identified by the inference model, thus mathematically avoiding the generation of "pseudo-defects" (such as cracks that appear out of nowhere).
[0160] The specific steps of step S5 are as follows:
[0161] S51. Sample size-driven dynamic weight allocation: Design a collaborative decision-making module to allocate weights according to sample size: When the sample size is sufficient (≥2000 images), the inference model takes the lead (70% weight) in optimizing defective features, and the generation model assists in correcting ambiguous areas; when the sample size is scarce (<500 images), the generation model takes the lead (60% weight) in generating virtual samples, and the inference model verifies and removes false samples.
[0162] The collaborative decision-making module triggers a weight update every time it processes 100 samples. The sample volume statistics range is the same type of defect images collected within the last 30 days, excluding the generated samples. The weight allocation results are written to the log file in real time, with the format 'timestamp, defect type, inference weight, generation weight;
[0163] The threshold for a sample volume of N = 2000 is determined based on the model convergence experiments of 5 types of defects (such as insulator breakage, conductor strand breakage, etc.). At this time, the accuracy fluctuation of the inference model is <5%. N = 500 is the small sample critical value. Below this value, the generated samples improve the model performance by >10%;
[0164] Dynamic collaborative weight allocation formula:
[0165] ,
[0166] In the formula: represents the weight of the generation model ( ), indicating the proportion of the contribution of the generation model during the enhancement process;
[0167] represents the weight of the inference model ( ), which is complementary to ;
[0168] represents the sample volume of the current defect type (unit: piece, ), reflecting the richness of the labeled data of this type of defect;
[0169] According to the sample volume, dynamically adjust the collaborative relationship between the two models - when the samples are sufficient ( ), optimize the inference-dominated features. When the samples are scarce (0 < N < 500), the generation weight increases linearly as the samples decrease. When there are zero samples (N = 0), the generation dominates the sample creation, achieving adaptive enhancement in different data scenarios;
[0170] S52. Feature consistency loss and collaborative optimization: Introduce a feature consistency loss function to calculate the feature differences between the inference and generation results. If the difference exceeds the threshold (such as 0.2), then increase the inference weight to calibrate the generation direction; Through dynamic collaboration, "inference guides the generation accuracy, and generation supplements the inference data" to solve the contradiction between insufficient data and enhanced distortion in the small sample scenario;
[0171] The calculation period of the feature consistency loss function is synchronized with the model iteration (calculated once every 10 iterations). When the difference exceeds the threshold, the inference weight is increased by 0.1 each time and the loss is recalculated until the difference ≤ the threshold for 3 consecutive iterations;
[0172] The difference threshold of 0.2 for the feature consistency loss function was determined through a feature comparison experiment of 1000 sets of augmented samples and real samples. When the difference exceeds the threshold, the inference weight increases by 0.1 each time until the difference is ≤0.2.
[0173] The specific steps of step S6 are as follows:
[0174] S61. Design and implementation of a dual-layer verification mechanism: Establish a dual-layer verification system of machine and human verification: Machine verification uses a feature consistency model to compare the key features of the enhanced image with those of the real defect, such as the proportion of rust area. If the deviation exceeds 15%, it is marked as "to be optimized"; Human verification is reviewed by inspection experts, focusing on checking for "false defects", such as fake cracks, and recording typical problems, such as excessive enhancement of dirt under strong light.
[0175] The manual review was completed by three engineers with more than five years of experience in power inspection. A 'one-vote veto system' was adopted (if any expert judges it as a false defect, it will be marked). The review results were recorded in an Excel spreadsheet, including the sample ID, reviewer, judgment result, and problem description.
[0176] When calculating the percentage of rusted area using the feature consistency model verified by machine, the number of defect pixels in the binarized image / the total number of pixels in the region is used; the 15% deviation threshold corresponds to an absolute error of ≤5 pixels for the critical dimensions of the defect (such as the length of a broken strand);
[0177] S62. Verification Result Feedback and Model Iteration: Transform the verification results into optimization indicators: For example, if the angle deviation of "broken conductor strand" frequently exceeds the standard, adjust the three-dimensional view parameters of the generated model; if "insulator contamination" has false features, strengthen the semantic constraints of the inference model, update the verification standards monthly, such as adjusting the light weight according to the season, and improve the authenticity and reliability of the samples through iteration.
[0178] Typical issues should be reported to the model optimization team through the internal management system. The submission should include 'the path to the problem sample, the defect type, the suggested parameter adjustments, and an attachment (comparison chart)'. The optimization team should respond within 3 working days.
[0179] The monthly updated verification standards include a seasonal lighting parameter table: the weight of strong light in summer is increased by 20%, and the weight of weak light in winter is increased by 30%; the typical problem record format is 'defect type-problem description-frequency of occurrence', such as 'insulator contamination-false feature under strong light-15 times / month';
[0180] Feature consistency loss function:
[0181] ,
[0182] In the formula: Represents feature consistency loss ( The smaller the value, the smaller the feature difference between the augmented sample and the real sample;
[0183] Represents the balance coefficient ( If the focus is on the difference in feature vectors, it can be set as follows: =0.7, focusing on the difference in probability distribution, can be set as ;
[0184] This represents the feature vector (dimension D) of the augmented sample;
[0185] The feature vector (with dimension D) represents the real sample;
[0186] L2 norm represents the Euclidean distance between two eigenvectors.
[0187] ;
[0188] KL divergence represents the difference between the probability distribution of defect types in augmented samples and real samples.
[0189] (K is the total number of defect types, To increase the probability that a sample belongs to class K);
[0190] These represent the probability distributions of defect types in the enhanced sample and the real sample, respectively. );
[0191] The optimization objective for model training (minimizing via gradient descent) This approach enhances the realism of samples by constraining them from two dimensions: feature vectors and probability distributions, thus addressing the problem that generative models easily deviate from real features.
[0192] Manual verification of the weight update formula:
[0193] ,
[0194] In the formula: This represents the updated feature weights (e.g., visual feature weights). );
[0195] Represents the feature weights before the update. ;
[0196] Indicates the learning rate Control the magnitude of weight adjustment, such as This indicates a maximum adjustment of 10% each time;
[0197] Indicates manual verification score ( The inspection experts score the enhanced samples based on their matching degree with the real defects: 1 indicates complete matching, 0 indicates average matching, and -1 indicates the presence of obvious false features, such as false cracks.
[0198] Transforming human experience into a basis for adjusting model parameters—when experts determine that a certain type of feature (such as texture) enhances distortion ( When <0), by reducing the corresponding weight ( The model is optimized to compensate for the shortcomings of machine verification in judging "semantic rationality".
[0199] The specific steps of step S7 are as follows:
[0200] S71. Enhanced Dataset Output and Recognition Application: Output a dataset containing enhanced images, defect labels (location, type), and multimodal feature maps for defect recognition algorithm training. Recognition performance metrics (accuracy, false alarm rate) are fed back to the inference and generative models; for example, if the accuracy of a certain type of defect recognition decreases, its training weight in the inference model is increased.
[0201] The augmented dataset is divided into 'train / val / test' in a ratio of 7:2:1, and the storage structure is 'dataset / train / image folder / , dataset / train / label folder / '. The file names of the image folder and the label folder correspond one-to-one (e.g., img_001.jpg corresponds to img_001.json).
[0202] Defect labels are in JSON format, example: {"Defect Type": "Wire Strand Broken", "Location": {"x1": 100, "y1": 200, "x2": 150, "y2": 250}, "Confidence": 0.92, "Scene Label": "Strong Light - Unobstructed"}; When calculating recognition performance metrics, accuracy = number of correctly identified samples / total number of samples, false alarm rate = number of false alarms / total number of detections;
[0203] S72. Feedback Loop Construction and Adaptive Optimization: Establishing an "Enhancement-Training-Feedback" closed loop: After each round of enhancement, update the feature weights of the inference model and the constraints of the generation model based on the recognition effect, adapting to different defect requirements. For example, when the recognition rate of "metal rust" is low, optimize the sensitivity of the inference model to rust texture, and the generation model focuses on generating samples with different degrees of rust, gradually improving the practical value of the dataset;
[0204] In the closed-loop optimization, the recognition algorithm uses Faster R-CNN. During training, the batch size is set to 16, and the iteration is 100 rounds. The accuracy of the validation set is calculated every 20 rounds. If the accuracy does not improve for 3 consecutive rounds, the current enhancement cycle is terminated.
[0205] The closed-loop optimization process involves 500-1000 augmented samples per round, with a feature weight update step size of 0.05 for the inference model. The constraint adjustment for the generation model is based on the principle that for every 5% decrease in recognition accuracy, the corresponding feature constraint strength increases by 10%.
[0206] The formula for feedback of recognition accuracy in closed-loop optimization:
[0207] ,
[0208] In the formula: This represents the increment of training weights for a certain type of defect in the inference model. A larger value indicates that the training weight for this type of defect needs to be increased;
[0209] : Represents the proportionality coefficient Control the magnitude of weight adjustment (e.g.) (To avoid excessive fluctuations in weighting);
[0210] Indicates the target recognition accuracy (e.g.) (i.e., 85%)
[0211] Indicates the current recognition accuracy. It is calculated from the performance of the defect identification algorithm on the test set;
[0212] This represents the number of incorrect identification samples for this type of defect. ;
[0213] This represents the total number of samples of this type of defect. , Error rate;
[0214] The performance gap in defect identification algorithms is translated into training weight adjustments for the inference model—when the accuracy of identifying a certain type of defect is low. And the error rate is high ( When (large), Increasing the training ratio of this type of defect in the model will enable a closed-loop optimization of "enhancement-identification-feedback".
[0215] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0216] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A defect image enhancement method that integrates reasoning and generation, characterized in that: The specific steps of this method are as follows: S1. Multi-source defect data acquisition and structured processing: Acquire defect images from UAV inspections, record scene information simultaneously, standardize the original images, and then locate the defect areas using Canny edge detection to form a structured dataset. S2. Multimodal feature fusion network construction: Based on the Transformer architecture, it integrates visual, spatial and semantic features, and generates feature maps by adjusting weights according to defect type through a dynamic attention mechanism. S3. Inference Enhancement Sub-model Training: Based on multimodal feature maps, an inference model with defect localization and feature enhancement is constructed. The localization module uses an improved YOLO architecture and adaptively adjusts the anchor boxes to locate small sample defects. The enhancement module strengthens the distinction between defects and background. A few-shot learning strategy is adopted to transfer common features from similar defects. Combined with labeled data, the parameters are adjusted, and the model outputs defect confidence and enhanced feature map to clarify the defect area for step S4 and ensure that the generated sample focuses on the real defect location. S4. Construction and Adaptation of Enhanced Sub-model: Construct a generative model containing segmentation and 3D generation. The segmentation module uses the SAM model to segment the defect region of the inference output and generates a binary mask. The 3D generation module models small sample defects based on the segmentation results and converts the 3D model into a 2D image through perspective and lighting simulation. Introducing inference constraints during training: the generated sample features must match the multimodal feature map of the inference model, using cosine similarity for verification, and encapsulating the generation logic as a reusable module; S5. Inference-Generation Dynamic Collaborative Decision Making: Dynamically allocate weights according to sample size: When samples are sufficient, the inference model dominates; when samples are scarce, the generation model dominates. The inference model verifies and eliminates false samples, and the weights are adjusted using the feature consistency loss function. S6. Enhanced Sample Authenticity Verification and Iteration: Combining machine verification with manual review, recording typical problems, providing feedback to optimize model parameters, and updating standards monthly; S7. Enhanced Dataset Output and Closed-Loop Optimization: Output a dataset containing enhanced images, labels, and feature maps, and feed back the recognition results to adjust inference weights and generation strategies.
2. The defect image enhancement method according to claim 1, characterized in that: The specific steps of step S1 are as follows: S11. Multi-source data acquisition and basic preprocessing: Acquire defect images of UAV inspections, including insulators, ground wires and fittings, and simultaneously record scene information including 3D point clouds, lighting and occlusion; Standardize the original images: Use BM3D algorithm to remove noise, normalize the scale to 512×512 pixels, and enhance local contrast through CLAHE to provide a data foundation for subsequent processing. S12. Coarse labeling and structured storage of small sample defects: For small sample defects, Canny edge detection is used to locate suspected areas and generate a dataset containing location coordinates and scene labels; the dataset is stored according to type to provide standardized input for step S2.
3. The defect image enhancement method that integrates inference and generation according to claim 1, characterized in that: The specific steps of step S2 are as follows: S21. Multi-type feature extraction and integration: Construct a feature extraction network based on Transformer, integrating three types of features: visual features, spatial features, and semantic features. Coverage of defect details and contextual information; S22. Dynamic Attention Mechanism and Feature Map Generation: Design a dynamic attention mechanism to adjust feature weights according to defect type and fuse them to generate a feature map containing global and local information, providing a unified input for steps S3 and S4.
4. The defect image enhancement method according to claim 1, characterized in that: The specific steps of step S5 are as follows: S51. Sample size-driven dynamic weight allocation: Design a collaborative decision-making module to allocate weights according to sample size: When samples are sufficient, the inference model takes the lead in optimizing defective features, and the generation model assists in correcting ambiguous areas; when samples are scarce, the generation model takes the lead in generating virtual samples, and the inference model verifies and eliminates false samples. S52. Feature Consistency Loss and Collaborative Optimization: Introduce a feature consistency loss function to calculate the feature difference between the inference and the generated results. If the difference exceeds the threshold, increase the inference weight to calibrate the generation direction. Through dynamic collaboration, inference guides the generation accuracy and generates supplementary inference data.
5. The defect image enhancement method according to claim 1, characterized in that: The specific steps of step S6 are as follows: S61. Design and implementation of a dual-layer verification mechanism: Establish a dual-layer verification system of machine and human: Machine verification uses a feature consistency model to compare the key features of the enhanced image with the real defect. If the deviation exceeds 15%, it is marked as needing optimization. Human verification is reviewed by inspection experts, and typical problems are recorded. S62. Verification Result Feedback and Model Iteration: Transform the verification results into optimization indicators: If the deviation of the conductor strand breakage angle frequently exceeds the standard, adjust the three-dimensional perspective parameters of the generated model; If the insulator pollution has false features, strengthen the semantic constraints of the inference model; Update the verification standards monthly.
6. The defect image enhancement method according to claim 1, characterized in that: The specific steps of step S7 are as follows: S71. Enhanced Dataset Output and Recognition Application: Output a dataset containing enhanced images, defect labels, and multimodal feature maps for defect recognition algorithm training; feed back the recognition performance metrics to the inference and generative models. S72. Feedback Loop Construction and Adaptive Optimization: Establishing an Enhancement-Training-Feedback Loop: After each round of enhancement, the feature weights of the inference model and the constraints of the generation model are updated according to the recognition effect to adapt to different defect requirements.
Citation Information
Patent Citations
Defect detection system and method based on unmanned aerial vehicle electric power inspection
CN114281093A
Photovoltaic power station unmanned inspection method and system based on multi-mode fusion detection
CN120217107A
Power transmission line insulator multi-mode multi-type defect detection method based on MEWL-YOLO model
CN120451128A
