A composite insulator heat defect judgment method and system based on size model cooperation

Abstract

This invention discloses a method and system for judging thermal defects in composite insulators based on a combination of large and small models. The method includes: extracting the global temperature of an acquired infrared image containing composite insulators, detecting and segmenting the composite insulators, calculating the minimum bounding rectangle of the segmented composite insulator regions, extracting temperature sequences and sharpening temperature values ​​to obtain the sharpened minimum bounding rectangle; constructing a structured multimodal feature set; inputting the core rod temperature feature matrix and its trend graph into a fine-tuned large multimodal model to determine if thermal defects exist; inputting the core rod temperature feature matrix into a trained convolutional neural network small model to determine if thermal defects exist and their intensity; and combining the thermal defect judgment results from the large model and the small model to obtain the final thermal defect result. This invention achieves intelligent, rapid, and accurate identification of thermal defects in composite insulators.

Description

Technical Field

[0001] This invention belongs to the field of power grid transmission equipment defect detection technology, and in particular, it is a method and system for judging the heating defects of composite insulators based on the collaboration of large and small models. Background Technology

[0002] In power systems, composite insulators, as key equipment in transmission lines, are widely used in high-voltage, ultra-high-voltage, and extra-high-voltage transmission operations. Their operating status directly affects the safe operation of the power grid. However, due to long-term exposure to a complex and changing external environment, composite insulators are prone to thermal defects such as core rod decay and aging, sheath dampness, and contamination-induced overheating. In severe cases, these defects can lead to flashover, core rod breakage, and line drop accidents. Therefore, timely and accurate identification of thermal defects in composite insulators is of great significance for fault prevention and ensuring power grid safety.

[0003] Currently, the detection of heat in composite insulators mainly relies on UAV infrared imaging technology, combined with human experience and traditional machine learning methods to determine heat defects. However, these methods generally suffer from low accuracy, poor generalization ability, and susceptibility to external lighting interference, especially with a high false positive rate when there is strong background interference. In recent years, deep learning models and multimodal large models have demonstrated strong capabilities in image recognition and feature extraction due to their ability to understand images. However, single large models lack component segmentation capabilities and cannot obtain the composite insulator body temperature through segmentation, thus failing to fully utilize the composite insulator temperature information for heat determination. Small models have segmentation capabilities and can obtain the composite insulator body temperature through segmentation, but their feature extraction capabilities are weaker than those of large models. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method and system for judging the thermal defects of composite insulators based on the collaboration of large and small models. By constructing a collaborative working mechanism between large and small models, the invention fully leverages the advantages of large models in understanding images and extracting features, as well as the capabilities of small models in component segmentation, so as to achieve intelligent, rapid and accurate identification of thermal defects of composite insulators in the field of transmission line inspection.

[0005] Therefore, the present invention adopts the following technical solution.

[0006] In a first aspect, the present invention provides a method for judging the heating defects of composite insulators based on the coordination of large and small models, which includes:

[0007] Global temperature extraction is performed on the acquired infrared image containing composite insulators, and composite insulators are detected and segmented. The minimum bounding rectangle of the segmented composite insulator region is calculated. Temperature sequence extraction and temperature value sharpening are performed on the minimum bounding rectangle as the region to obtain the sharpened minimum bounding rectangle of the composite insulator.

[0008] Representative temperature distribution features are extracted from the minimum bounding rectangle of the sharpened composite insulator to generate a core rod temperature feature matrix that reflects its axial thermal characteristics. Feature alignment and normalization are performed on the core rod temperature feature matrix, the core rod temperature feature matrix trend map, and the heating state label of the composite insulator to construct a structured multimodal feature set.

[0009] The multimodal feature set is input into the multimodal large model. Through image-text-heating result feature matching, a multimodal feature training library is formed. The multimodal large model is then fine-tuned by learning from these feature training libraries.

[0010] Input the aforementioned mandrel temperature feature matrix and mandrel temperature feature matrix trend graph into the fine-tuned multimodal large model to determine whether it generates heat.

[0011] The temperature feature matrix of the mandrel is input into the trained convolutional neural network mini-model to determine whether it generates heat and the level of heat generation.

[0012] The heat assessment results of the large model and the heat defect results of the small model are combined to obtain the final heat defect result.

[0013] Furthermore, the infrared images containing composite insulators are obtained through the following steps: using the cloud object storage service interface, authentication is performed using a pre-configured AK / SK key, and a secure connection is established to the cloud image library; based on the preset infrared image catalog results, infrared image data containing composite insulators is retrieved and downloaded in segments.

[0014] While acquiring infrared image data, its metadata is extracted, including the city, pole, line, and component name corresponding to the infrared image.

[0015] Furthermore, the steps for global temperature extraction from the image are as follows: call the infrared camera SDK interface, parse the radiation data stream embedded in the infrared image, convert the original calorific value into an actual temperature matrix, traverse the matrix, calculate the highest temperature, lowest temperature, median and average value of the entire image, and perform smoothing and noise reduction processing on the temperature data.

[0016] Furthermore, the U-Net image semantic segmentation algorithm based on deep learning is used to perform pixel-level classification on the infrared image and segment the composite insulator region. Based on the segmentation results, the minimum bounding rectangle of the composite insulator region is calculated, and the centroid coordinates of the region are obtained.

[0017] Furthermore, using the minimum bounding rectangle as the region, the specific steps for temperature sequence extraction and temperature value sharpening are as follows: Using the minimum bounding rectangle as the region, read the temperature values ​​corresponding to the pixels within the region through the SDK interface, calculate the average temperature of the composite insulator's centerline and use it as a reference temperature, arrange all temperature values ​​within the region in ascending order, and determine the 95th percentile temperature T. 95% As the high temperature threshold; with T 95% The average temperature along the centerline is used as the upper and lower limits of the temperature mapping. The temperature in the region is normalized and mapped to the minimum bounding rectangle of the composite insulator, generating a sharpened minimum bounding rectangle of the composite insulator.

[0018] Furthermore, by using the tilt angle of the minimum bounding rectangle Determine the geometric direction of the minimum bounding rectangle of the composite insulator, define the width of the minimum bounding rectangle as the overall width of the composite insulator in pixels, and then set one-third of this width as the core region bandwidth for temperature analysis.

[0019] The formation process of the core rod temperature characteristic matrix is ​​as follows: passing through the centroid of the smallest circumscribed rectangle of the composite insulator. Obtain parallel to the tilt angle A straight line is used as a reference line, and a strip-shaped analysis region is formed by extending one-sixth of the width of the minimum bounding rectangle to the left and right. All line segments parallel to the reference line within this region are obtained. For each line, the sharpened temperature value of the corresponding pixel on the line is extracted in the infrared image, and a set of core rod temperature feature matrices is formed using the temperatures of all line pixel locations. ,in, Within one-third of the width of the minimum bounding rectangle, the first i The temperature values ​​of the pixels contained in the root straight line are used to remove the temperature values ​​of the last five points of the temperature feature sequence of all the aforementioned straight lines that are close to the high-pressure end metal equalizing ring.

[0020] Furthermore, the process of generating the mandrel temperature feature matrix trend chart is as follows: the mandrel temperature feature matrix is ​​converted into a visual curve image to obtain the mandrel temperature feature matrix trend chart.

[0021] Furthermore, the aforementioned mandrel temperature feature matrix trend map uses temperature value as the vertical axis and the spatial order of pixels from the low-pressure end to the high-pressure end as the horizontal axis, plotting a continuous broken line. All lines are plotted on the same coordinate axis according to this rule; the trend chart of the mandrel temperature characteristic matrix does not include the horizontal and vertical axis scales, labels and grid coordinates, but only retains the trend outline of temperature changes.

[0022] Furthermore, by combining the heat generation assessment results of the large-scale model and the heat generation defect results of the small-scale model, the following rules are used to determine whether the composite insulator has heat generation defects:

[0023] Rule 1: If the large model determines that the problem is caused by overheating, and the small model determines that the overheating is moderate or above, then the final result of the overheating defect is "the composite insulator has an overheating defect";

[0024] Rule 2: If the large model determines that there is no heat generation, but the small model determines that there is heat generation and that it is severe heat generation, then the final result of the heat generation defect is "the composite insulator has a heat generation defect";

[0025] Rule 3: In other cases, the final result for the thermal defect is "the composite insulator has no thermal defect".

[0026] Secondly, the present invention provides a composite insulator heating defect judgment system based on size model coordination, comprising:

[0027] Preprocessing module: Extracts global temperature from the acquired infrared image containing composite insulators, detects and segments the composite insulators, calculates the minimum bounding rectangle of the segmented composite insulator region; uses the minimum bounding rectangle as the region, extracts temperature sequence and sharpens temperature values ​​to obtain the sharpened minimum bounding rectangle of the composite insulator.

[0028] Feature Engineering Module: Extract representative temperature distribution features from the minimum bounding rectangle of the sharpened composite insulator to generate a core rod temperature feature matrix that reflects its axial thermal characteristics. Perform feature alignment and normalization on the core rod temperature feature matrix, the core rod temperature feature matrix trend map, and the composite insulator heating state label to construct a structured multimodal feature set.

[0029] Multimodal fine-tuning training module: Input the multimodal feature set into the multimodal large model, form a multimodal feature training library through image-text-heating result feature matching, and learn through these feature training libraries to achieve fine-tuning of the multimodal large model;

[0030] Large model judgment module: Input the mandrel temperature feature matrix and mandrel temperature feature matrix trend graph into the fine-tuned multimodal large model to determine whether there is heat generation;

[0031] Small model judgment module: Input the temperature feature matrix of the mandrel into the trained convolutional neural network small model to determine whether it generates heat and the level of heat generation;

[0032] Collaborative Judgment Module: Collaborates the thermal judgment results of the large model and the thermal defect results of the small model to obtain the final thermal defect result.

[0033] The beneficial effects of this invention are as follows: By constructing a collaborative working mechanism between a large model and a small model, this invention fully leverages the advantages of the large model in understanding images and extracting features, as well as the small model in component segmentation, to achieve intelligent, rapid, and accurate identification of thermal defects in composite insulators at the transmission line inspection site. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a flowchart of a method for judging the heating defects of composite insulators based on the synergy of large and small models according to the present invention;

[0036] Figure 2 This is a schematic diagram of the composite insulator heating defect judgment system based on the coordination of large and small models according to the present invention. Detailed Implementation

[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0038] Example 1

[0039] This embodiment presents a method for judging the heating defects of composite insulators based on the synergy of large and small models. Figure 1 As shown, the steps are as follows:

[0040] S1. Obtain infrared images containing composite insulators from the image library: Securely connect to the cloud image library via the cloud object storage service (OBS) interface using a pre-configured AK / SK key for authentication; retrieve and download infrared image data containing composite insulators in segments according to the preset infrared image catalog results; while obtaining the infrared image data, extract its metadata, including the city, tower, line, and component name corresponding to the infrared image.

[0041]

[0042] in, This indicates the acquired infrared image containing the composite insulator. This indicates the infrared image containing the composite insulator that was read, and Metadata indicates the metadata of the acquired infrared image containing the composite insulator.

[0043] S2, perform global temperature extraction on the acquired infrared image containing composite insulators, and detect and segment the composite insulators, calculate the minimum bounding rectangle of the segmented composite insulator region; use the minimum bounding rectangle as the region, perform temperature sequence extraction and temperature value sharpening to obtain the sharpened minimum bounding rectangle of the composite insulator.

[0044] The acquired infrared images are preprocessed to detect and segment composite insulator components, extract temperature from the entire image, and extract feature temperatures. The detected and segmented composite insulators are then color-enhanced based on their feature temperatures.

[0045] Specifically, the steps for global temperature extraction from the image are as follows: The infrared camera SDK interface is called to parse the radiation data stream embedded in the infrared image, and the original calorific values ​​are read step by step and converted into an actual temperature matrix. The actual temperature matrix of the entire image is as follows:

[0046] ,

[0047] Where T(x,y) represents the temperature value corresponding to pixel x and y. At the same time, it iterates through all pixels to extract the temperature and extracts the feature values ​​of the sorted temperature sequence, including the highest temperature, lowest temperature, median and average value of the whole image.

[0048] Maximum temperature: ,

[0049] Minimum temperature: ,

[0050] Median: ,

[0051] average value: ,

[0052] All temperature data, after being smoothed and denoised, is encapsulated and output along with image timestamps, city / prefecture information, pole / tower information, etc., for subsequent processing.

[0053] The U-Net image semantic segmentation algorithm based on deep learning is used to perform pixel-level classification on infrared images and segment the composite insulator region. Based on the segmentation results, the minimum bounding rectangle (MBR) of the composite insulator region is calculated and the centroid coordinates of the region are obtained.

[0054] The set of pixel coordinates of the minimum circumscribed rectangle of the composite insulator is The coordinates of the top-left and bottom-right corners of the circumscribed rectangle are as follows:

[0055] Top left corner:

[0056] Bottom right corner:

[0057] Get the pixel area covered by the smallest bounding rectangle Where w represents the width of the minimum bounding rectangle, and h represents the length of the minimum bounding rectangle. This represents the tilt angle of the smallest bounding rectangle. The rectangle defines the composite insulator region for subsequent temperature characteristic analysis and anomalous region alignment, while also determining its centroid. .

[0058] Temperature values ​​are sharpened for the detected and segmented composite insulator regions to make their temperature distribution characteristics, especially abnormal heating areas, more intuitive and obvious. The specific steps for temperature sequence extraction and temperature value sharpening, using the minimum bounding rectangle as the region, are as follows:

[0059] First, extract all pixels from the minimum bounding rectangle of the composite insulator from the detection and segmentation results, and read their corresponding temperature values. , forming a regional temperature set Where R represents the pixel region covered by the minimum bounding rectangle, and x and y represent the coordinate values ​​contained within this region R; simultaneously, the module calculates the average temperature on the centerline of the composite insulator as a reference temperature. The centerline pixel set is... The average temperature along the centerline is:

[0060]

[0061] Sort all temperature values ​​within the region in ascending order and calculate the 95th percentile temperature value T. 95% This temperature represents the significant threshold of the high-temperature anomaly region. As the lower limit of the temperature mapping, As the upper limit of the temperature mapping, these two threshold temperatures are used to normalize the temperature of the original region:

[0062]

[0063] Finally, the normalized temperature is mapped to the minimum bounding rectangle of the composite insulator to generate a sharpened minimum bounding rectangle of the composite insulator, thereby significantly highlighting the overheated area of ​​the core rod.

[0064] S3. Extract representative temperature distribution features from the minimum bounding rectangle of the sharpened composite insulator to generate a core rod temperature feature matrix that reflects its axial thermal characteristics. Perform feature alignment and normalization on the core rod temperature feature matrix, the core rod temperature feature matrix trend map, and the composite insulator heating status label (heating or not heating) to construct a structured multimodal feature set.

[0065] The tilt angle of the smallest bounding rectangle Determine the geometric direction of the minimum bounding rectangle of the composite insulator, define the width of the minimum bounding rectangle as the overall width of the composite insulator in pixels, and then set one-third of this width as the core region bandwidth for temperature analysis.

[0066] The formation process of the core rod temperature characteristic matrix is ​​as follows: passing through the centroid of the smallest circumscribed rectangle of the composite insulator. Obtain parallel to the tilt angle A straight line is used as a reference line, and a strip-shaped analysis region is formed by extending one-sixth of the width of the minimum bounding rectangle to the left and right. All line segments parallel to the reference line within this region are obtained. For each line, the sharpened temperature value of the corresponding pixel on the line is extracted in the infrared image, and a set of core rod temperature feature matrices is formed using the temperatures of all line pixel locations. ,in, Within one-third of the width of the minimum bounding rectangle, the first i The temperature values ​​of the pixels contained in the root straight line are used to remove the temperature values ​​of the last five points of the temperature feature sequence of all the aforementioned straight lines that are close to the high-voltage end metal equalization ring, so as to avoid abnormal interference caused by the segmentation of the equalization ring region.

[0067] The process of generating the trend chart of the mandrel temperature feature matrix is ​​as follows: The mandrel temperature feature matrix... The temperature characteristic matrix trend chart of the mandrel is converted into a visual curve image, resulting in a trend chart of the mandrel temperature characteristic matrix. This trend chart plots temperature values ​​on the vertical axis and the spatial order of pixels from the low-pressure end to the high-pressure end on the horizontal axis, creating a continuous broken line. All lines are plotted on the same coordinate axis according to this rule; the trend chart of the mandrel temperature characteristic matrix does not include the horizontal and vertical axis scales, labels and grid coordinates, but only retains the trend outline of temperature changes.

[0068] The trend map of the mandrel temperature feature matrix, the mandrel temperature feature matrix, and the insulator heating state are combined into a multimodal feature set through feature alignment and normalization.

[0069]

[0070] in, This is a trend chart of the mandrel temperature characteristic matrix. This is the core rod temperature characteristic matrix, where H represents the heating state of the corresponding composite insulator. If it is a composite insulator that needs to be judged, then the value of H is empty.

[0071] S4. Input the multimodal feature set into the multimodal large model, and form a multimodal feature training library by matching the features of image-text-heating results. Then, learn from these feature training libraries to achieve fine-tuning of the multimodal large model.

[0072] Fine-tuning of a general multimodal large model: This section describes the professional fine-tuning of a multimodal large model (Qwen-vl) for infrared heating diagnosis of composite insulators, enabling the identification of heating defects in composite insulators. Three types of inputs are obtained: the core rod temperature feature matrix. Trend chart of mandrel temperature characteristic matrix Based on the heating state of the corresponding composite insulator, a training sample for image-text-heating result feature matching is constructed.

[0073] First, the training samples are encapsulated as training text in JSON format;

[0074] Then, using the LLamafactory fine-tuning framework, LoRA technology is employed to fine-tune the parameters of the pre-trained model, achieving alignment between the curve trends and temperature sequences of visual features and the semantics of the text. Multimodal large-scale model training maps multidimensional features to fault descriptions.

[0075] S5. Input the mandrel temperature feature matrix and mandrel temperature feature matrix trend graph into the fine-tuned multimodal large model to determine whether there is heat generation.

[0076] Deploying the Xinference framework enables intelligent identification of thermal defects in composite insulators. In the inference phase, a structured feature text is first constructed by combining the trend map of the core rod temperature feature matrix with the corresponding core rod temperature feature matrix. This text describes key thermal features in natural language, and the corresponding image is input into Xinference's finely tuned multimodal large model. This comprehensive understanding of temperature trends and temperature sequence context results in a structured judgment.

[0077] S6. Input the temperature feature matrix of the mandrel into the trained convolutional neural network mini-model to determine whether there is heat generation, the location of heat hotspots, the temperature difference, and the heat generation level.

[0078] First, the normalized core rod temperature feature matrix is ​​input into a trained convolutional neural network model. This model extracts local temperature fluctuation features from the core rod temperature feature matrix through multiple convolutional and pooling operations, outputting the heating characteristics of the composite insulator. After determining the heating characteristics of the core rod temperature feature matrix, the most significant heating points are further located based on the model output: the positions of the maximum values ​​in the core rod temperature feature matrix are obtained. The core heat source is used as the center, and five adjacent pixels are extended to the left and right to form the heat-generating area, which is used to characterize the local thermal influence range. Then, the highest temperature within this heat-generating area is calculated. minimum value of the entire temperature series And calculate the temperature difference between the two:

[0079]

[0080] At the same time, the heating level of the composite insulator is determined according to the temperature difference range in Table 1.

[0081] Table 1 Heating Rating of Composite Insulators

[0082]

[0083] S7 combines the heat assessment results of the large model and the heat defect results of the small model to obtain the final heat defect result.

[0084] Based on the combined results of the large-scale model and the small-scale model, the following rules are used to determine whether the composite insulator is overheating:

[0085] Rule 1: If the large model determines that the problem is caused by overheating, and the small model determines that the overheating is moderate or above, then the final result of the overheating defect is "the composite insulator has an overheating defect";

[0086] Rule 2: If the large model determines that there is no heat generation, but the small model determines that there is heat generation and that it is severe heat generation, then the final result of the heat generation defect is "the composite insulator has a heat generation defect";

[0087] Rule 3: In other cases, the final result for the thermal defect is "No thermal defect in the composite insulator". The above rules are listed below:

[0088] Table 2 Rules for Judging Heating Defects

[0089]

[0090] Through the above steps, this invention achieves a closed-loop process from cloud image acquisition, local preprocessing, multimodal feature extraction, to collaborative intelligent diagnosis using both large and small models. This method fully leverages the advantages of the large model's strong feature extraction capabilities and accurate identification in complex scenarios, and the small model's segmentation capabilities to fully utilize temperature information, thereby improving the accuracy of heat detection and providing reliable technical support for the condition assessment of composite insulators in transmission lines.

[0091] The above-mentioned method for judging the heating defects of composite insulators based on the synergy of large and small models was simulated and verified. The specific contents are as follows:

[0092] The technical effectiveness of the method described in this invention was evaluated on an existing test set, including positive sample (heating) composite insulators and negative sample (normal) composite insulators. The number of positive sample (heating) composite insulators was 337, and the number of negative sample (normal) composite insulators was 1632. An objective evaluation method was used to assess the model's performance, and Table 3 shows the score results on the existing test set.

[0093] Table 3. Comparison of the technical effects of the method of the present invention and existing small model judgment methods.

[0094]

[0095] The results show that the method of this invention outperforms existing methods that rely solely on small-scale models for identifying thermal defects. In particular, this invention significantly improves the detection rate without reducing the false positive rate, demonstrating that the proposed method can effectively detect thermal defects in composite insulators.

[0096] Example 2

[0097] This embodiment provides a composite insulator heating defect judgment system based on large and small model collaboration, such as Figure 2 As shown, it consists of an infrared image acquisition module, a preprocessing module, a feature engineering module, a multimodal fine-tuning training module, a large model judgment module, a small model judgment module, and a collaborative judgment module.

[0098] Infrared image acquisition module: It securely connects to the cloud image library through the cloud object storage service (OBS) interface using a pre-configured AK / SK key for authentication; it retrieves and downloads infrared image data containing composite insulators in segments according to the preset infrared image catalog results; while acquiring infrared image data, it extracts its metadata, including the city, tower, line and component name corresponding to the infrared image;

[0099] Preprocessing module: Extracts global temperature from the acquired infrared image containing composite insulators, detects and segments the composite insulators, calculates the minimum bounding rectangle of the segmented composite insulator region; uses the minimum bounding rectangle as the region, extracts temperature sequence and sharpens temperature values ​​to obtain the sharpened minimum bounding rectangle of the composite insulator.

[0100] Feature Engineering Module: Extract representative temperature distribution features from the minimum bounding rectangle of the sharpened composite insulator to generate a core rod temperature feature matrix that reflects its axial thermal characteristics. Perform feature alignment and normalization on the core rod temperature feature matrix, the core rod temperature feature matrix trend map, and the heating state of the composite insulator to construct a structured multimodal feature set.

[0101] Multimodal fine-tuning training module: Input the multimodal feature set into the multimodal large model, form a multimodal feature training library through image-text-heating result feature matching, and learn through these feature training libraries to achieve fine-tuning of the multimodal large model;

[0102] Large model judgment module: Input the mandrel temperature feature matrix and mandrel temperature feature matrix trend graph into the fine-tuned multimodal large model to determine whether there is heat generation;

[0103] Small model judgment module: Input the temperature feature matrix of the mandrel into the trained convolutional neural network small model to determine whether it generates heat and the level of heat generation;

[0104] Collaborative Judgment Module: Collaborates the thermal judgment results of the large model and the thermal defect results of the small model to obtain the final thermal defect result.

[0105] It should be noted that each unit in the above-mentioned composite insulator thermal defect judgment system based on large and small model collaboration can be implemented entirely or partially through software, hardware, or a combination thereof. These units can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each unit. For specific limitations regarding the composite insulator thermal defect judgment system based on large and small model collaboration, please refer to the limitations of the composite insulator thermal defect judgment method (i.e., Example 1) above; both have the same function and role, and will not be repeated here.

[0106] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. It will be apparent to those skilled in the art that various modifications can be made to the above embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A method for judging heating defects in composite insulators based on size model collaboration, characterized in that, include: Global temperature extraction is performed on the acquired infrared image containing composite insulators, and composite insulators are detected and segmented. The minimum bounding rectangle of the segmented composite insulator region is calculated. Temperature sequence extraction and temperature value sharpening are performed on the minimum bounding rectangle as the region to obtain the sharpened minimum bounding rectangle of the composite insulator. Representative temperature distribution features are extracted from the minimum bounding rectangle of the sharpened composite insulator to generate a core rod temperature feature matrix that reflects its axial thermal characteristics. Feature alignment and normalization are performed on the core rod temperature feature matrix, the core rod temperature feature matrix trend map, and the heating state label of the composite insulator to construct a structured multimodal feature set. The multimodal feature set is input into the multimodal large model. Through image-text-heating result feature matching, a multimodal feature training library is formed. The multimodal large model is then fine-tuned by learning from these feature training libraries. Input the aforementioned mandrel temperature feature matrix and mandrel temperature feature matrix trend graph into the fine-tuned multimodal large model to determine whether it generates heat. The temperature feature matrix of the mandrel is input into the trained convolutional neural network mini-model to determine whether it generates heat and the level of heat generation. The heat assessment results of the large model and the heat defect results of the small model are combined to obtain the final heat defect result; The heat generation results of the large collaborative model and the heat generation defect results of the small model are used to determine whether the composite insulator has heat generation defects, based on the following rules: Rule 1: If the large model determines that the problem is caused by overheating, and the small model determines that the overheating is moderate or above, then the final result of the overheating defect is "the composite insulator has an overheating defect"; Rule 2: If the large model determines that there is no heat generation, but the small model determines that there is heat generation and that it is severe heat generation, then the final result of the heat generation defect is "the composite insulator has a heat generation defect"; Rule 3: In other cases, the final result for the thermal defect is "the composite insulator has no thermal defect".

2. The method for judging heating defects in composite insulators according to claim 1, characterized in that, Infrared images containing composite insulators are obtained through the following steps: using the cloud object storage service interface, authentication is performed using a pre-configured AK / SK key to securely connect to the cloud image library; based on the preset infrared image catalog results, infrared image data containing composite insulators is retrieved and downloaded in segments. While acquiring infrared image data, its metadata is extracted, including the city, pole, line, and component names corresponding to the infrared image.

3. The method for judging heating defects in composite insulators according to claim 1, characterized in that, The steps for global temperature extraction from an image are as follows: call the infrared camera SDK interface, parse the radiation data stream embedded in the infrared image, convert the original calorific value into an actual temperature matrix, traverse the matrix, calculate the highest temperature, lowest temperature, median and average value of the entire image, and perform smoothing and noise reduction processing on the temperature data.

4. The method for judging heating defects in composite insulators according to claim 1, characterized in that, The U-Net image semantic segmentation algorithm based on deep learning is used to perform pixel-level classification on infrared images and segment the composite insulator region. Based on the segmentation results, the minimum bounding rectangle of the composite insulator region is calculated and the centroid coordinates of the region are obtained.

5. The method for judging heating defects in composite insulators according to claim 4, characterized in that, The specific steps for temperature sequence extraction and temperature value sharpening using the minimum bounding rectangle as the region are as follows: Using the minimum bounding rectangle as the region, read the temperature values ​​corresponding to the pixels within the region through the SDK interface. Calculate the average temperature of the composite insulator's centerline and use it as a reference temperature. Arrange all temperature values ​​within the region in ascending order and determine the 95th percentile temperature T. 95% As a high temperature threshold; With T 95% The average temperature along the centerline is used as the upper and lower limits of the temperature mapping. The temperature in the region is normalized and mapped to the minimum bounding rectangle of the composite insulator, generating a sharpened minimum bounding rectangle of the composite insulator.

6. The method for judging heating defects in composite insulators according to claim 1, characterized in that, The tilt angle of the minimum bounding rectangle Determine the geometric direction of the minimum bounding rectangle of the composite insulator, define the width of the minimum bounding rectangle as the overall width of the composite insulator in pixels, and then set one-third of this width as the core region bandwidth for temperature analysis. The formation process of the core rod temperature characteristic matrix is ​​as follows: passing through the centroid of the smallest circumscribed rectangle of the composite insulator. Obtain parallel to the tilt angle A straight line is used as a reference line, and a strip-shaped analysis region is formed by extending one-sixth of the width of the minimum bounding rectangle to the left and right. All line segments parallel to the reference line within this region are obtained. For each line, the sharpened temperature value of the corresponding pixel on the line is extracted in the infrared image, and a set of core rod temperature feature matrices is formed using the temperatures of all line pixel locations. ,in, Within one-third of the width of the minimum bounding rectangle, the first i The temperature values ​​of the pixels contained in the root straight line are used to remove the temperature values ​​of the last five points of the temperature feature sequence of all the aforementioned straight lines that are close to the high-pressure end metal equalizing ring.

7. The method for judging heating defects in composite insulators according to claim 6, characterized in that, The process of generating the mandrel temperature feature matrix trend chart is as follows: the mandrel temperature feature matrix is ​​converted into a visual curve image to obtain the mandrel temperature feature matrix trend chart.

8. The method for judging heating defects in composite insulators according to claim 7, characterized in that, The aforementioned mandrel temperature feature matrix trend chart uses temperature values ​​as the vertical axis and the spatial order of pixels from the low-pressure end to the high-pressure end as the horizontal axis, plotting a continuous broken line. All lines are plotted on the same coordinate axis according to this rule; the trend chart of the mandrel temperature characteristic matrix does not include the horizontal and vertical axis scales, labels and grid coordinates, but only retains the trend outline of temperature changes.

9. A composite insulator heating defect judgment system based on size model collaboration, used to implement the composite insulator heating defect judgment method according to any one of claims 1-8, characterized in that, include: Preprocessing module: Extracts global temperature from the acquired infrared image containing composite insulators, detects and segments the composite insulators, calculates the minimum bounding rectangle of the segmented composite insulator region; uses the minimum bounding rectangle as the region, extracts temperature sequence and sharpens temperature values ​​to obtain the sharpened minimum bounding rectangle of the composite insulator. Feature Engineering Module: Extract representative temperature distribution features from the minimum bounding rectangle of the sharpened composite insulator to generate a core rod temperature feature matrix that reflects its axial thermal characteristics. Perform feature alignment and normalization on the core rod temperature feature matrix, the core rod temperature feature matrix trend map, and the composite insulator heating state label to construct a structured multimodal feature set. Multimodal fine-tuning training module: Input the multimodal feature set into the multimodal large model, form a multimodal feature training library through image-text-heating result feature matching, and learn through these feature training libraries to achieve fine-tuning of the multimodal large model; Large model judgment module: Input the mandrel temperature feature matrix and mandrel temperature feature matrix trend graph into the fine-tuned multimodal large model to determine whether there is heat generation; Small model judgment module: Input the temperature feature matrix of the mandrel into the trained convolutional neural network small model to determine whether it generates heat and the level of heat generation; Collaborative Judgment Module: Collaborates the thermal judgment results of the large model and the thermal defect results of the small model to obtain the final thermal defect result.

Images