A method for detecting defects in transmission line fittings based on large-scale model collaboration

Abstract

This invention discloses a method for detecting defects in transmission line fittings based on a large-scale model collaboration, belonging to the field of intelligent power equipment inspection. It solves the problems of fixed thresholds, false positives and false negatives, and unreasonable cloud resource overhead in existing transmission line defect detection strategies based on large-scale model collaboration. The method involves: inputting a preprocessed transmission line fitting defect dataset into a target detection small model, which outputs the detection box coordinates, category labels, and original category prediction confidence scores of potential fitting defect images in the dataset; calculating the environmental complexity of the fitting defect images to generate an adaptive gating threshold; and processing the detection results output by the target detection small model based on the original category prediction confidence score, the adaptive gating threshold, and a preset category filtering confidence threshold, comparing the initial detection results of the target detection small model with the verification results of the large cloud model, and outputting an inspection report. This invention is applicable to transmission lines.

Description

Technical Field

[0001] This invention relates to the field of intelligent testing technology for power equipment, specifically to a method for detecting defects in transmission line fittings based on the collaboration of large and small models. Background Technology

[0002] Transmission lines are the core infrastructure for energy transmission in power systems, and their operational status directly determines the stability and reliability of power supply. Among the various components of transmission lines, line fittings play a crucial role in connecting, securing, and splicing conductors and insulators. Exposed to complex outdoor conditions for extended periods, they are highly susceptible to various defects such as corrosion, deformation, loosening, and detachment. Failure of these fittings can easily lead to safety issues such as conductor galloping, component detachment, line discharge, and even line breakage and tripping, seriously threatening the normal operation and maintenance of the entire transmission line and the safe and stable operation of the power grid.

[0003] With the deep integration of drone-based intelligent inspection and artificial intelligence technology, deep learning-based transmission line hardware defect detection technology is gradually replacing traditional manual inspection methods and becoming the mainstream means of power grid equipment condition monitoring.

[0004] Currently, the field of power transmission line defect detection mainly relies on two types of models: one is lightweight target detection small models (such as the YOLO series), but these models are not adaptable enough to complex scenarios such as uneven lighting, cluttered backgrounds, target occlusion, and low visibility, and are prone to problems such as class confidence bias, high false detection and false negative rates, and significantly limited detection accuracy; the other is multimodal large models, which have powerful scene understanding and complex feature extraction capabilities, and can achieve high-precision defect identification, but have the disadvantage of high computational resource consumption, making it difficult to adapt to the real-time requirements and bandwidth limitations of UAV inspection.

[0005] To balance detection accuracy with edge resource constraints, existing technologies have proposed various collaborative detection schemes using models of different sizes. However, existing collaborative strategies still have significant shortcomings: First, they lack adaptation mechanisms designed to suit the characteristics of transmission line inspection scenarios, making it impossible to effectively differentiate scenario complexity and resulting in insufficient flexibility in model invocation strategies. Second, the result screening and triage mechanisms are crude, often directly discarding low-class confidence results from small models or submitting them all to large models for processing, which can easily lead to resource waste or decreased accuracy. Third, the collaborative triggering logic is rigid, failing to consider the impact of environmental complexity on the reliability of model decisions, and unable to dynamically adjust the class confidence threshold according to scenario changes, making it difficult to achieve a dynamic balance between accuracy and overhead. Summary of the Invention

[0006] To address the technical problems of fixed thresholds, false positives and false negatives, and unreasonable cloud resource overhead in existing transmission line defect detection methods using large and small model collaboration strategies, this invention proposes a transmission line hardware defect detection method based on large and small model collaboration.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: a method for detecting defects in transmission line fittings based on large and small model collaboration, comprising the following steps:

[0008] Step S1: Establish and preprocess a dataset of defects in transmission line fittings;

[0009] Step S2: Input the preprocessed transmission line hardware defect dataset into the target detection small model, perform preliminary inference of the target detection small model, and output the initial detection results; the initial detection results include the detection box coordinates, category labels, and original category prediction confidence of the potential hardware defect images in the transmission line hardware defect dataset;

[0010] Step S3: Obtain the environmental complexity assessment index of hardware defect images in the transmission line hardware defect dataset;

[0011] Step S4: Generate an adaptive gating threshold based on the environmental complexity determination index;

[0012] Step S5: Based on the original category prediction confidence, adaptive gating threshold and preset category filtering confidence threshold, the initial detection results output by the target detection mini-model are processed in a tiered manner.

[0013] Step S6: Upload the questionable initial detection results from the initial detection results to the large model for verification, and align the questionable initial detection results of the target detection small model with the verification results of the large model in terms of spatial coordinates. Based on the consistency between the verification results of the large model and the questionable initial detection results of the target detection small model, perform the corresponding classification fusion processing respectively, and output the inspection report.

[0014] Furthermore, step S1 specifically includes the following steps:

[0015] Step S11: Based on the image data of transmission line hardware defects collected during transmission line channel inspections, the image data of vibration dampers, suspension clamps, weighted line hardware and related environments collected during transmission line channel inspections, and the image data of transmission line hardware defects from historical inspections, establish a transmission line hardware defect dataset.

[0016] Step S12: Preprocess the original hardware defect images in the transmission line hardware defect image data.

[0017] Furthermore, step S2 includes the following steps:

[0018] Step S21: Upload the preprocessed transmission line fitting defect dataset to the target detection mini-model deployed at the edge;

[0019] Step S22: The target detection mini-model performs forward inference on the transmission line hardware defect dataset and outputs the detection box coordinates, category labels, and original category prediction confidence of the potential defect targets;

[0020] Step S23: Based on the preset category filtering confidence threshold, remove hardware defect images in the transmission line hardware defect dataset whose original category prediction confidence is less than the preset category filtering confidence threshold.

[0021] Furthermore, step S3 specifically includes the following steps:

[0022] Step S31: Calculate the global information entropy of the hardware defect images in the data set of transmission line hardware defects after the preprocessing in step S1.

[0023] Step S32: The global information entropy obtained by normalization is used as the scene complexity index for determining the environmental complexity of the hardware defect image.

[0024] Furthermore, step S4 specifically involves constructing a square root-type nonlinear adaptive gating function based on global information entropy, and dynamically generating an adaptive gating threshold according to the scene complexity index.

[0025] Furthermore, in step S5, the mechanism for diverting and processing the detection results output by the target detection mini-model is as follows:

[0026] If the confidence level of the original category prediction is not less than the adaptive gating threshold, the current hardware defect image is determined to be a reliable sample with high category confidence, and the detection result output by the small target detection model is directly used as the final detection result.

[0027] If the confidence level of the original category prediction is between the preset category filtering confidence threshold and the adaptive gating threshold, the current hardware defect image is determined to be a suspicious sample, and the initial detection result corresponding to the suspicious sample is the suspicious initial detection result; the ROI image slice of the hardware defect image is uploaded to the large model for verification;

[0028] If the confidence level of the original category prediction is lower than the confidence level threshold of the category filter, the current hardware defect image is determined to be invalid noise or a false detection, and the detection result is discarded directly.

[0029] Furthermore, step S6 specifically includes the following steps:

[0030] Step S61: The large model deployed in the cloud receives ROI image slices uploaded by the small object detection model deployed at the edge, performs depth verification under structured cue constraints, and outputs the large model's predicted category and the large model's category prediction confidence.

[0031] Step S62: Align the detection box of the hardware defect image corresponding to the questionable initial detection result with the detection box of the large model verification result in terms of spatial coordinates, and output the final judgment category after arbitration fusion operation;

[0032] Step S63: Perform unified non-maximum suppression processing on the detection boxes of the hardware defect images corresponding to the questionable initial detection results of all target detection small models and the detection boxes of the hardware defect images after arbitration fusion, remove overlapping redundancy, and retain the detection boxes with the highest confidence.

[0033] Step S64: Encapsulate the spatial coordinates of the detection box after non-maximum suppression processing, the determined final judgment category, the final fusion category confidence level, and the scene complexity index corresponding to the hardware defect image into a data package to generate an inspection report.

[0034] Furthermore, step S61 specifically involves the large model having a pre-set dedicated instruction module for identifying defects in transmission line fittings, which is used to define the type of fitting defects and the output format.

[0035] The dedicated instruction module for identifying defects in transmission line hardware includes:

[0036] The task instruction unit is used to determine whether there are defects in the ROI image slices uploaded by the small object detection model, and output the predicted category of the large model.

[0037] The defect category constraint unit communicates with the task instruction unit and is used to forcibly limit the identification range of the large model prediction category to the preset defect category, and prohibit the output of irrelevant categories;

[0038] The precision constraint unit communicates with the task instruction unit and is used to calculate and output the large model class prediction confidence based on the output large model prediction class; that is, when the task instruction unit determines that the ROI image slice uploaded by the target detection small model does not have defects, it does not output the large model class prediction confidence.

[0039] The format constraint unit is connected to the defect category constraint unit and the accuracy constraint unit respectively, and is used to constrain the output format of the large model.

[0040] Further, step S62 specifically involves using the coordinate restoration operator to map the detection boxes of the ROI image slices generated by the large model to the coordinate system of the full-image detection boxes of the original hardware defect image; calculating the intersection-union ratio (IU / R) between the detection boxes of the restored ROI image slices and the detection boxes of the hardware defect image corresponding to the questionable initial detection results; if the IU / R is higher than the preset consistency threshold, it is determined that the two point to the same hardware defect image, and an arbitration fusion operation is performed.

[0041] The arbitration fusion operation specifically involves determining the consistency between the category of the questionable initial detection result of the small object detection model and the predicted category of the large model, based on the category label, and outputting the final judgment category.

[0042] Furthermore, the questionable detection boxes of all object detection small models and the detection boxes of the hardware defect image after arbitration fusion are collectively referred to as candidate boxes; the detection boxes of the hardware defect image corresponding to the questionable initial detection results of all object detection small models and the detection boxes of the hardware defect image after arbitration fusion are subjected to unified non-maximum suppression processing, specifically including the following steps:

[0043] Step S631: Calculate the intersection-union ratio between each pair of candidate boxes;

[0044] Step S632: Sort all candidate boxes in descending order of final fusion category confidence and store them in the pending processing set. ;

[0045] Step S633: From the set to be processed Extract the candidate boxes with the highest confidence in the final fusion category. This is retained as the final prediction result and moved into the result set. ;

[0046] Step S634: Traverse the set to be processed The remaining candidate boxes Calculate candidate boxes The candidate box with the highest confidence in the final fusion category The intersection and union ratio;

[0047] If candidate box The candidate box with the highest confidence in the final fusion category If the intersection-union ratio is greater than a preset overlap threshold, then a candidate box is determined. Redundant boxes are removed from the set to be processed. Remove from the middle;

[0048] Step S635: Repeat steps S631-S634 above until the set to be processed is reached. Empty.

[0049] The advantages of this invention over the prior art are as follows:

[0050] 1. Improved scene adaptability: The method of this invention constructs an environmental prior field by using the environmental complexity judgment index of hardware defect images in the power transmission line hardware defect dataset. It can dynamically perceive the complexity of the inspection scene, solve the technical defect of the fixed threshold in the traditional collaborative detection strategy that cannot adapt to scene changes, and improve the scene adaptability of the overall detection process.

[0051] 2. Reduced false negative and false positive rates: The method of this invention dynamically generates adaptive gating thresholds based on environmental complexity judgment indicators, which can force low-confidence suspicious samples to enter the cloud-based large model for review and arbitration, effectively reducing the false negative and false positive rates of defects and significantly improving the detection reliability in complex scenarios.

[0052] 3. Optimize cloud resource scheduling efficiency: The method of this invention, through the diversion processing and the mechanism of uploading the ROI image slices of the hardware defect images corresponding to the questionable initial detection results to the large model for verification, can start cloud large model inference only when necessary. Under the premise of maintaining high-precision detection, it significantly reduces the wireless communication bandwidth occupation and cloud computing resource overhead, making it more suitable for UAV online inspection engineering applications. Attached Figure Description

[0053] The present invention will be further described below with reference to the accompanying drawings:

[0054] Figure 1 This is a schematic diagram illustrating an example of a defect collected by the present invention;

[0055] Figure 2 This is a schematic diagram of the method flow of the present invention;

[0056] Figure 3 This is a flowchart illustrating the process of diverting and processing the detection results output by the target detection mini-model in the method of this invention. Detailed Implementation

[0057] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate relative orientations or positional relationships and are used only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0058] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0059] like Figures 1 to 3 As shown, this invention provides a method for detecting defects in transmission line fittings based on large-scale model collaboration, comprising the following steps:

[0060] Step S1: Establish and preprocess a dataset of defects in transmission line fittings;

[0061] In one embodiment, step S1 specifically includes the following steps:

[0062] Step S11: Based on the image data of transmission line hardware defects collected during transmission line corridor inspections, the image data of vibration dampers, suspension clamps, and counterweights, as well as environmental related images collected during transmission line corridor inspections, and the image data of transmission line hardware defects from historical inspections, a transmission line hardware defect dataset is established. In one embodiment, the transmission line hardware defect dataset contains over four thousand high-resolution hardware defect images. The categories of transmission line hardware defects include at least three types: vibration damper detachment, suspension clamp corrosion, and counterweight corrosion. The collected transmission line hardware defect image data is as follows: Figure 1 As shown;

[0063] Step S12: Preprocess the original hardware defect images in the transmission line hardware defect image data; specifically, firstly, normalize the size of the original hardware defect images to 640×640 pixels to adapt to the input requirements of the subsequent target detection small model; then, perform grayscale conversion to convert the RGB three-channel image into a single-channel grayscale image to reduce the complexity of subsequent information entropy calculation; finally, use Gaussian noise reduction filtering (kernel size 5×5, standard deviation σ=1.0) to eliminate the interference caused by uneven illumination and sensor noise on information entropy calculation.

[0064] Step S2: Input the preprocessed transmission line hardware defect dataset into the target detection small model, perform preliminary inference of the target detection small model, and output the initial detection results; the initial detection results include the detection box coordinates, category labels, and original category prediction confidence of potential defect targets in the transmission line hardware defect dataset.

[0065] In one embodiment, step S2 includes the following steps:

[0066] Step S21: Upload the preprocessed transmission line hardware defect dataset to the lightweight target detection mini-model deployed at the edge. The target detection mini-model adopts the YOLOv11n network structure.

[0067] Step S22: The target detection mini-model performs forward inference on the transmission line hardware defect dataset, outputting the bounding box coordinates, class labels, and original class prediction confidence of potential defect targets. ;

[0068] Step S23: Filter confidence thresholds based on preset categories. After removing defective transmission line fittings from the dataset, the confidence level of the original category prediction is... Less than the preset category filtering confidence threshold Images of hardware defects.

[0069] Step S3: Obtain the environmental complexity assessment index of the hardware defect images in the transmission line hardware defect dataset.

[0070] In one embodiment, step S3 specifically includes the following steps:

[0071] Step S31: Perform global information entropy calculation on the hardware defect images in the data set of transmission line hardware defects after the preprocessing in step S1, so as to quantify the overall complexity and environmental interference of the inspection scene corresponding to the hardware defect images.

[0072] The expression for calculating the global information entropy E is:

[0073] ;

[0074] Where L is the total number of gray levels in the image, and in this embodiment L=256;

[0075] The probability of grayscale value l appearing in the entire image of hardware defects is obtained by statistical analysis of the grayscale histogram. The specific calculation formula is as follows:

[0076] ;

[0077] In the formula, The grayscale value is The number of pixels; N is the total number of pixels in the entire image of the hardware defect.

[0078] Step S32: The global information entropy obtained through normalization is used as the global information entropy obtained through normalization. Scene complexity is used as an indicator to determine the environmental complexity of an image of hardware defects:

[0079] ;

[0080] In the formula, , Given the current sampling bit depth of the hardware defect image, this represents the theoretical maximum entropy of the current hardware defect image. The normalized global information entropy is also included. The global information entropy after normalization The closer the value is to 1, the more discrete the grayscale distribution of the scene, the more complex the texture, and the stronger the environmental interference in the image of the hardware defect; that is, the greater the environmental complexity and the more complex the background. The normalized global information entropy... The closer the value is to 0, the purer the scene and the weaker the background interference in the image of the hardware defect, that is, the less complex the environment and the simpler the background.

[0081] In a specific embodiment of the present invention, measurements were taken from a transmission line image against a clean background. Measurements were taken from images of complex scenes with dense tree canopy obstruction and interplay of light and shadow. This verifies the effectiveness of the indicator in distinguishing scene complexity.

[0082] Step S4: Generate an adaptive gating threshold based on the environmental complexity assessment index. It is used to dynamically adjust the category confidence threshold.

[0083] Specifically, by constructing a square root-type nonlinear adaptive gating function based on global information entropy, and according to the scene complexity index... Dynamically generate adaptive gating thresholds This enables real-time contraction or expansion of the decision boundary for small target detection models. It also generates adaptive gating thresholds. The model is:

[0084] ;

[0085] in, and These are the upper and lower limits of the preset class confidence threshold (i.e., the class confidence judgment threshold). Utilizing the derivative properties of the square root function, a fast response of the adaptive gating threshold is achieved in medium and low complexity environments, while saturation suppression of the adaptive gating threshold is achieved in extremely high complexity environments, preventing the adaptive gating threshold from expanding infinitely and causing excessive uploading to subsequent large models.

[0086] Step S5: Based on the original category prediction confidence score output from Step 2 Step 4 generates the adaptive gating threshold. and preset category filtering confidence thresholds All initial detection results output by the target detection mini-model are diverted and processed.

[0087] The mechanism by which the initial detection results output by the target detection mini-model are diverted and processed is as follows:

[0088] If the confidence level of the original category prediction is not less than the adaptive gating threshold, the current hardware defect image is determined to be a reliable sample with high category confidence. The initial detection result output by the small object detection model is directly used as the final detection result, without triggering the large model deployed in the cloud. Specifically, if If the current hardware defect image is determined to be a high-confidence reliable sample, the result of the target detection small model is directly adopted without triggering the collaboration of the large model deployed in the cloud. The target detection small model outputs the detection result autonomously. That is, at this time, the initial detection result output by the target small model is the final detection result.

[0089] If the original category prediction confidence level falls between the preset category filtering confidence threshold and the adaptive gating threshold, the current hardware defect image is determined to be a suspicious sample, and a local region of interest (ROI) image slice of the hardware defect image is uploaded to the cloud-based large model for verification; specifically, if The current hardware defect image is determined to be a suspicious sample. At this point, using the center of the detection bounding box of the hardware defect image as a reference, its width and height are expanded outwards by 15% respectively, creating a ROI image slice. Only this ROI image slice and its edge category labels are uploaded to the large model for verification. That is, the entire hardware defect image is not transmitted, which further reduces communication overhead and latency. This partial upload method preserves the necessary environmental information around the detection area in the hardware defect image while suppressing the transmission of invalid background data. For ease of description, the detection result output by the target small model at this time is called the initial suspicious detection result.

[0090] If the confidence score of the original category prediction is lower than the category filtering confidence score threshold, the current hardware defect image is determined to be invalid noise or a false detection and discarded, and the initial detection result is directly discarded. Specifically, if If the current hardware defect image is determined to be invalid noise or a false detection, the initial detection result is discarded directly. In this embodiment, the category filtering confidence threshold... Setting it to 0.2 can effectively block most false detections while avoiding over-detection of real but low-contrast defects.

[0091] In this embodiment, the large model deployed in the cloud adopts the GLM-4V multimodal large model.

[0092] Step S6: Upload the questionable initial detection results from the initial detection results to the large model for verification, and align the questionable initial detection results of the target detection small model with the verification results of the large model in terms of spatial coordinates. Based on the consistency between the verification results of the large model and the questionable initial detection results of the target detection small model, perform the corresponding classification fusion processing respectively, and output the inspection report.

[0093] Step S6 specifically includes the following steps:

[0094] Step S61: The large model deployed in the cloud receives ROI image slices uploaded by the small object detection model deployed at the edge, performs depth verification under structured cue constraints, and outputs the large model's predicted category and the large model's category prediction confidence.

[0095] Specifically, the large model has a built-in pre-set dedicated instruction module for identifying defects in transmission line fittings, which is used to define the types of fitting defects and the output format.

[0096] More specifically, the dedicated instruction module for identifying defects in transmission line fittings includes:

[0097] The task instruction unit is used to determine whether there are defects in the ROI image slices uploaded by the small object detection model, and output the predicted category of the large model.

[0098] The defect category constraint unit communicates with the task instruction unit and is used to forcibly limit the identification range of the large model prediction category to three preset defect categories (vibration hammer detachment, suspension clamp corrosion, and heavy hammer corrosion), and prohibit the output of irrelevant categories;

[0099] The precision constraint unit communicates with the task instruction unit and is used to calculate and output the prediction confidence of the large model class based on the output large model prediction class. That is, when the task instruction unit determines that the ROI image slice uploaded by the target detection small model does not have defects, it does not output the prediction confidence of the large model class, which effectively reduces subjective misjudgment.

[0100] The format constraint unit is communicatively connected to the defect category constraint unit and the accuracy constraint unit, respectively, and is used to constrain the output format of the large model to facilitate subsequent program parsing; in one embodiment, the output format is: category: [xxx defect / normal], confidence level: [x].

[0101] Step S62: Align the detection boxes of the hardware defect images corresponding to the questionable initial detection results of the target detection small model (hereinafter referred to as questionable detection boxes) with the detection boxes of the verification results of the large model in terms of spatial coordinates, and output the final judgment category after arbitration fusion operation. Specifically, the detection boxes of the ROI image slices generated by the large model are mapped to the coordinate system of the full-image detection boxes of the original hardware defect image using the coordinate restoration operator; calculate the intersection-over-union ratio (IoU) of the detection boxes of the restored ROI image slices and the detection boxes of the hardware defect images corresponding to the original category prediction confidence scores between the preset category filtering confidence threshold and the adaptive gating threshold. If the IoU is higher than the preset consistency threshold (0.5 in this embodiment), it is determined that the two point to the same hardware defect image, and the arbitration fusion operation is performed to output the final judgment category; otherwise, the two are regarded as independent hardware defect images and are saved separately.

[0102] The arbitration fusion operation specifically involves, based on the category label, determining the consistency between the questionable initial detection result category of the small object detection model and the predicted category of the large model, and outputting the final determined category:

[0103] The arbitration integration operation mechanism is as follows:

[0104] If the category of the questionable initial detection result output by the small target detection model matches the category predicted by the large model, then category confidence enhancement processing is performed. This processing involves selecting the category with the higher confidence score from both the large and small target detection models as the final classification category for the hardware defect image, and calculating the weighted average of the two scores as the final fused category confidence score. The output, calculated using the following formula:

[0105] ;

[0106] In the formula, The aim is to enhance the overall credibility of the recognition results through feature complementarity between the large model and the small object detection model;

[0107] The class probability distribution output by the softmax function of the object detection small model classification head, with dimension 1. ;

[0108] This represents the class probability distribution after semantic inference of the large model, normalized by Softmax, with dimension [missing information]. ;

[0109] If the category of the questionable initial detection result output by the small object detection model conflicts with the category predicted by the large model, then the scenario complexity index will be used. Perform adaptive arbitration:

[0110] when When the inspection environment in the current hardware defect image is found to be severely disturbed, the small target detection model is prone to false detection or misclassification due to background noise. In this case, the system forces the large model to predict the category as the final classification category and uses the large model's category prediction confidence as the final fusion category confidence output. This aims to use the larger model's stronger feature representation capabilities to correct the cognitive bias of the small target detection model.

[0111] when When the inspection environment in the current hardware defect image is clear, since the target detection small model is more domain-specific in extracting the subdivided features of power components, in this scenario, if there is a classification conflict between the category initially detected by the target detection small model and the category predicted by the large model, the category output by the target detection small model is taken as the final judgment category, and the original category prediction confidence of the initial detection result of the target detection small model is taken as the final fusion category confidence output, or the hardware defect image is marked as "awaiting manual review". This effectively avoids the "domain illusion" and qualitative errors that may occur in the power vertical field of the general large model.

[0112] Step S63: Perform uniform non-maximum suppression (NMS) processing on the questionable detection boxes of all object detection small models and the detection boxes of the hardware defect image after arbitration fusion to remove overlapping redundancy and retain the detection boxes with the highest confidence. The questionable detection boxes of all object detection small models and the detection boxes of the hardware defect image after arbitration fusion are collectively referred to as candidate boxes.

[0113] The questionable bounding boxes of all target detection small models and the bounding boxes of the hardware defect image after arbitration fusion are subjected to uniform non-maximum suppression (NMS) processing, which specifically includes the following steps:

[0114] Step S631: Calculate the Intersection over Union (IoU) between each pair of candidate boxes:

[0115] Iterate through all candidate boxes and calculate the values ​​of any two candidate boxes. and The crossover ratio index is calculated using the following formula:

[0116] ;

[0117] In the formula, Candidate boxes With candidate boxes The area of ​​the overlapping portion, This represents the total area of ​​the union of two candidate boxes;

[0118] Step S632: Sort all candidate boxes in descending order of final fusion category confidence and store them in the pending processing set. The final fused class confidence of the questionable detection boxes in the object detection small model is the original class prediction confidence output by the object detection small model. That is, the final fused class confidence of the questionable detection boxes, which are only output by the object detection small model and have not been verified by the large model, directly adopts the class and confidence directly output by the object detection small model.

[0119] Step S633: From the set to be processed Extract the candidate boxes with the highest confidence in the final fusion category. This is retained as the final prediction result and moved into the result set. ;

[0120] Step S634: Traverse the set to be processed The remaining candidate boxes Calculate candidate boxes The candidate box with the highest confidence in the final fusion category intersection ratio ;

[0121] like Then determine the candidate box Redundant boxes are removed from the set to be processed. Remove from the middle; among them, The preset overlap threshold is set to 0.45 in this embodiment;

[0122] Step S635: Repeat steps S631-S634 above until the set to be processed is reached. This ensures that each hardware defect image corresponds to only one optimal detection box;

[0123] Step S64: Combine the spatial coordinates of the detection box after non-maximum suppression processing, the determined defect category label, the determined final judgment category, the final fusion category confidence score, and the scene complexity index corresponding to the hardware defect image. Data is encapsulated to generate structured inspection reports that conform to power industry standards.

[0124] Regarding the specific structure of this invention, it should be noted that the connection relationships between the various component modules used in this invention are definite and achievable. Except as specifically described in the embodiments, their specific connection relationships can bring about corresponding technical effects and solve the technical problems proposed by this invention without relying on the execution of corresponding software programs. The models of the components, modules, and specific components appearing in this invention, the connection methods between them, and the conventional usage methods and expected technical effects brought about by the above technical features, unless specifically described, are all publicly disclosed content in patents, journal articles, technical manuals, technical dictionaries, and textbooks that can be obtained by those skilled in the art before the application date, or belong to conventional technology, common knowledge, and other existing technologies in this field. There is no need to elaborate, which makes the technical solution provided in this case clear, complete, and achievable, and can reproduce or obtain corresponding physical products based on this technical means.

[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting defects in transmission line fittings based on large-scale model collaboration, characterized in that, Includes the following steps: Step S1: Establish and preprocess a dataset of defects in transmission line fittings; Step S2: Input the preprocessed transmission line hardware defect dataset into the target detection small model, perform preliminary inference of the target detection small model, and output the initial detection results; the initial detection results include the detection box coordinates, category labels, and original category prediction confidence of the potential hardware defect images in the transmission line hardware defect dataset; Step S3: Obtain the environmental complexity assessment index of hardware defect images in the transmission line hardware defect dataset; Step S4: Generate an adaptive gating threshold based on the environmental complexity determination index; Step S5: Based on the original category prediction confidence, adaptive gating threshold and preset category filtering confidence threshold, the initial detection results output by the target detection mini-model are processed in a tiered manner. Step S6: Upload the questionable initial detection results from the initial detection results to the large model for verification, and align the questionable initial detection results of the target detection small model with the verification results of the large model in terms of spatial coordinates. Based on the consistency between the verification results of the large model and the questionable initial detection results of the target detection small model, perform the corresponding classification fusion processing respectively, and output the inspection report.

2. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 1, characterized in that, Step S1 specifically includes the following steps: Step S11: Based on the image data of transmission line hardware defects collected during transmission line channel inspections, the image data of vibration dampers, suspension clamps, weighted line hardware and related environments collected during transmission line channel inspections, and the image data of transmission line hardware defects from historical inspections, establish a transmission line hardware defect dataset. Step S12: Preprocess the original hardware defect images in the transmission line hardware defect image data.

3. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Upload the preprocessed transmission line fitting defect dataset to the target detection mini-model deployed at the edge; Step S22: The target detection mini-model performs forward inference on the transmission line hardware defect dataset and outputs the detection box coordinates, category labels, and original category prediction confidence of the potential defect targets; Step S23: Based on the preset category filtering confidence threshold, remove hardware defect images in the transmission line hardware defect dataset whose original category prediction confidence is less than the preset category filtering confidence threshold.

4. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 1, characterized in that, Step S3 specifically includes the following steps: Step S31: Calculate the global information entropy of the hardware defect images in the data set of transmission line hardware defects after the preprocessing in step S1. Step S32: The global information entropy obtained by normalization is used as the scene complexity index for determining the environmental complexity of the hardware defect image.

5. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 1, characterized in that, Step S4 specifically involves constructing a square root-type nonlinear adaptive gating function based on global information entropy, and dynamically generating an adaptive gating threshold according to the scene complexity index.

6. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 1, characterized in that, In step S5, the mechanism for diverting and processing the detection results output by the target detection mini-model is as follows: If the confidence level of the original category prediction is not less than the adaptive gating threshold, the current hardware defect image is determined to be a reliable sample with high category confidence, and the detection result output by the small target detection model is directly used as the final detection result. If the confidence level of the original category prediction is between the preset category filtering confidence threshold and the adaptive gating threshold, the current hardware defect image is determined to be a suspicious sample, and the initial detection result corresponding to the suspicious sample is the suspicious initial detection result. The ROI image slice of the hardware defect image was uploaded to the large model for verification; If the confidence level of the original category prediction is lower than the confidence level threshold of the category filter, the current hardware defect image is determined to be invalid noise or a false detection, and the detection result is discarded directly.

7. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 1, characterized in that, Step S6 specifically includes the following steps: Step S61: The large model deployed in the cloud receives ROI image slices uploaded by the small object detection model deployed at the edge, performs depth verification under structured cue constraints, and outputs the large model's predicted category and the large model's category prediction confidence. Step S62: Align the detection box of the hardware defect image corresponding to the questionable initial detection result with the detection box of the large model verification result in terms of spatial coordinates, and output the final judgment category after arbitration fusion operation; Step S63: Perform unified non-maximum suppression processing on the detection boxes of the hardware defect images corresponding to the questionable initial detection results of all target detection small models and the detection boxes of the hardware defect images after arbitration fusion, remove overlapping redundancy, and retain the detection boxes with the highest confidence. Step S64: Encapsulate the spatial coordinates of the detection box after non-maximum suppression processing, the determined final judgment category, the final fusion category confidence level, and the scene complexity index corresponding to the hardware defect image into a data package to generate an inspection report.

8. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 7, characterized in that, Step S61 specifically involves the following: The large model has a built-in preset dedicated instruction module for identifying defects in transmission line fittings, which is used to define the categories and output formats of fitting defects. The dedicated instruction module for identifying defects in transmission line hardware includes: The task instruction unit is used to determine whether there are defects in the ROI image slices uploaded by the small object detection model, and output the predicted category of the large model. The defect category constraint unit communicates with the task instruction unit and is used to forcibly limit the identification range of the large model prediction category to the preset defect category, and prohibit the output of irrelevant categories; The precision constraint unit communicates with the task instruction unit and is used to calculate and output the large model class prediction confidence based on the output large model prediction class; that is, when the task instruction unit determines that the ROI image slice uploaded by the target detection small model does not have defects, it does not output the large model class prediction confidence. The format constraint unit is connected to the defect category constraint unit and the accuracy constraint unit respectively, and is used to constrain the output format of the large model.

9. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 7, characterized in that, Step S62 specifically involves using the coordinate restoration operator to map the detection boxes of the ROI image slices generated by the large model to the coordinate system of the full-image detection boxes of the original hardware defect image; calculating the intersection-union ratio (IU / R) between the detection boxes of the restored ROI image slices and the detection boxes of the hardware defect images corresponding to the questionable initial detection results; if the IU / R is higher than the preset consistency threshold, it is determined that the two point to the same hardware defect image, and an arbitration fusion operation is performed. The arbitration fusion operation specifically involves determining the consistency between the category of the questionable initial detection result of the small object detection model and the predicted category of the large model, based on the category label, and outputting the final judgment category.

10. The method for detecting defects in transmission line fittings based on large-scale model collaboration according to claim 7, characterized in that, The questionable bounding boxes of all object detection small models and the bounding boxes of the hardware defect image after arbitration fusion are collectively referred to as candidate bounding boxes. The bounding boxes of the hardware defect image corresponding to the questionable initial detection results of all object detection small models and the bounding boxes of the hardware defect image after arbitration fusion are subjected to unified non-maximum suppression processing, specifically including the following steps: Step S631: Calculate the intersection-union ratio between each pair of candidate boxes; Step S632: Sort all candidate boxes in descending order of final fusion category confidence and store them in the pending processing set. ; Step S633: From the set to be processed Extract the candidate boxes with the highest confidence in the final fusion category. This is retained as the final prediction result and moved into the result set. ; Step S634: Traverse the set to be processed The remaining candidate boxes Calculate candidate boxes The candidate box with the highest confidence in the final fusion category The intersection and union ratio; If candidate box The candidate box with the highest confidence in the final fusion category If the intersection-union ratio is greater than a preset overlap threshold, then a candidate box is determined. Redundant boxes are removed from the set to be processed. Remove from the middle; Step S635: Repeat steps S631-S634 above until the set to be processed is reached. Empty.

Images