An electric power equipment infrared image intelligent labeling system and a labeling server

Abstract

This invention relates to an intelligent annotation system and annotation server for infrared images of power equipment, belonging to the field of infrared diagnostic technology for power equipment. The system includes: an image preprocessing module for enhancing, fusing, and normalizing operating conditions of multimodal raw data, and constructing a three-dimensional temperature field, outputting multimodal feature maps, a standardized temperature matrix, and structured temperature data; and an intelligent annotation and verification module for pre-annotating equipment contours and hotspot areas based on the multimodal feature maps, standardized temperature matrix, and structured temperature data using a few-shot learning network; simultaneously, it uses structured temperature data to verify the physical laws of the pre-annotation results and calculates the annotation confidence level. This invention eliminates operating condition interference through operating condition normalization, ensures annotation interpretability through physical constraint verification, and solves the problem of sample scarcity through human-machine collaborative closed-loop and incremental learning, achieving high-precision, high-efficiency, and evolvable intelligent annotation.

Description

Technical Field

[0001] This invention belongs to the field of infrared diagnostic technology for power equipment, specifically relating to an intelligent annotation system and annotation server for infrared images of power equipment. Background Technology

[0002] Infrared thermal imaging technology is widely used in live-line testing of power equipment. By analyzing the temperature distribution on the surface of the equipment, heating defects can be detected in time, preventing failures from occurring.

[0003] However, existing infrared image annotation technologies for power equipment have the following problems: First, the annotation data is disconnected from the equipment's operating conditions. The temperature distribution of infrared images is significantly affected by operating conditions such as equipment load and ambient temperature. Existing annotation systems cannot distinguish between "normal heating caused by high load" and "abnormal heating caused by equipment defects," resulting in insufficient accuracy of the annotation results. Second, AI annotation results lack physical interpretability. Existing deep learning-based annotation methods only focus on image features and lack constraints on the physical laws of heat conduction in power equipment, which can easily lead to annotation errors that do not conform to the heating mechanism of power equipment. Third, the scarcity of fault samples leads to poor generalization ability of the annotation model. It is difficult to obtain samples of heating defects in power equipment. Existing few-sample learning methods cannot effectively utilize limited samples and lack sample augmentation mechanisms tailored to the characteristics of power equipment.

[0004] Therefore, there is an urgent need for an intelligent annotation system and annotation server that can integrate multimodal data, introduce physical constraints, and realize a closed loop of human-machine collaboration. Summary of the Invention

[0005] To address the aforementioned problems in existing technologies, this invention provides an intelligent annotation system and annotation server for infrared images of power equipment. By eliminating operating condition interference through operating condition normalization, ensuring annotation interpretability through physical constraint verification, and solving the problem of sample scarcity through human-machine collaborative closed-loop and incremental learning, it achieves high-precision, high-efficiency, and evolvable intelligent annotation.

[0006] The objective of this invention can be achieved through the following technical solutions: This disclosure provides an intelligent annotation system for infrared images of power equipment, comprising: The multimodal data acquisition module is used to simultaneously acquire infrared images, visible light images, 3D point cloud data, and equipment operating parameters of power equipment to construct multimodal raw data. The image preprocessing module is used to enhance, fuse, normalize, and construct a three-dimensional temperature field for multimodal raw data, and output multimodal feature maps, standardized temperature matrices, and structured temperature data. The intelligent annotation and verification module is used to pre-annotate the device outline and hotspot areas based on multimodal feature maps, standardized temperature matrices, and structured temperature data using a few-shot learning network; at the same time, it uses structured temperature data to verify the physical laws of the pre-annotation results and calculate the annotation confidence level, thereby achieving automatic output distribution. The human-machine collaboration and data management module is used to manually review and correct data to be manually reviewed. It is also responsible for the storage and management of all data and triggers incremental training of the intelligent annotation and verification module based on the manually corrected data. The pre-annotation of the device outline and hotspot areas includes the following steps: The three-dimensional spatial bounding boxes of each component in the structured temperature data are extracted and back-projected onto the two-dimensional image plane through perspective projection to generate the corresponding component attention mask; the component attention mask is concatenated with the multimodal feature map in the channel dimension to obtain the enhanced multimodal feature map. The enhanced multimodal feature map is input into the few-sample segmentation network, which outputs an initial device contour mask and an initial hot spot region mask. The generation of the initial hot spot region mask also takes into account the temperature statistical characteristics of each component in the structured temperature data. The initial device outline mask, initial hotspot region mask, corresponding component labels, hotspot temperature information, and original image identifiers are packaged into a pre-annotated result.

[0007] Preferably, the operating parameters of the equipment include load current and ambient temperature; the image preprocessing module includes: An image enhancement unit is used to progressively enhance and fuse infrared and visible light images to obtain an enhanced image. The cross-modal feature fusion unit is used to fuse enhanced infrared images, visible light images and 3D point cloud data at the feature level to generate a multimodal feature map. The operating condition normalization unit is used to standardize the temperature matrix of the infrared image according to the operating condition parameters of the equipment, convert the measured temperature to the equivalent temperature under standard operating conditions, and output the standardized temperature matrix. The 3D temperature field construction unit is used to map a standardized temperature matrix onto 3D point cloud data, generate the 3D temperature field of the device, register the 3D temperature field with the device CAD model, and output component-level structured temperature data.

[0008] Preferably, the output normalized temperature matrix includes the following steps: Read the measured load current from the multimodal raw data. and ambient temperature ; Read the rated load current from the equipment file database. and standard ambient temperature ; Based on the equipment's historical temperature curves, historical load current sequences, and historical ambient temperature sequences, a load-temperature coefficient is fitted. and ambient temperature coefficient ; For temperature matrix Each temperature value in Calculate the equivalent temperature under standard operating conditions: ; Obtain the standardized temperature matrix .

[0009] Preferably, the structured temperature data at the output component level includes the following steps: The pixel coordinates of the standardized temperature matrix are spatially aligned with the world coordinates of the 3D point cloud data. An interpolation algorithm is used to assign a temperature value to each point cloud coordinate to generate an initial 3D temperature field. The CAD model of the currently inspected equipment is read from the equipment archive database. The iterative nearest point algorithm is used to rigidly register the three-dimensional temperature field with the equipment CAD model to establish the mapping relationship between point cloud coordinates and CAD component identifiers. Based on the registration results, temperature statistical characteristics are calculated by dividing the regions according to the CAD model components, and component-level structured temperature data is generated, including equipment identification, component name, temperature statistical characteristics, three-dimensional bounding box, and topological connection relationship between components.

[0010] Preferably, the physical law verification and calculation of the label confidence level includes the following steps: Component spatial location verification: Calculate the geometric center pixel coordinates based on the initial hotspot region mask, and back-project them into 3D space using the known camera projection matrix to obtain the hotspot's 3D coordinates; perform spatial inclusion judgment between the hotspot's 3D coordinates and the 3D bounding boxes of each component in the structured temperature data to determine the component identifier to which the hotspot belongs; according to the device type-standard heating component mapping rules stored in the internal knowledge graph, determine whether the component identifier belongs to a component that is allowed to generate heat: if so, the spatial verification passes; otherwise, it is marked as an abnormal spatial location. Temperature threshold verification: Read the measured temperature values ​​within the initial hot spot mask coverage area from the standardized temperature matrix; read the historical temperature statistical characteristics corresponding to the component identifier from the structured temperature data; calculate the deviation between the measured temperature and the historical mean, and use the two-standard-deviation criterion to determine whether it is abnormal; when the temperature is abnormal, calculate the relative temperature difference to determine the defect level; if the measured temperature is within the normal range, it is marked as no temperature abnormality. Topology verification: Extract the topology connection relationships of component identifiers from structured temperature data; determine whether the initial hot spot region is located in a reasonable topology position in the current path or voltage-heated region: for current-heated devices, the component where the hot spot is located must be at the contact interface of the current loop; for voltage-heated devices, the component where the hot spot is located must be in the electric field concentration region; if the conditions are met, the topology verification passes; otherwise, it is marked as a topology anomaly. Comprehensive confidence calculation: Obtain the classification confidence score output by the few-sample segmentation network, calculate the physical verification score based on the results of three verifications: component spatial location verification, temperature threshold verification, and topological relationship verification, and finally obtain the label confidence score by weighted fusion of classification confidence score and physical verification score, while generating verification description.

[0011] Preferably, the automatic output splitting includes the following steps: The calculated annotation confidence level is compared with the preset confidence level threshold. If the annotation confidence level is greater than or equal to the confidence level threshold, the initial equipment contour mask, initial hot spot area mask, defect level, component identification, measured temperature value, and verification description are packaged into a structured annotation result and directly output to the human-machine collaboration and data management module for archiving. If the annotation confidence level is less than the confidence level threshold, the initial equipment contour mask, initial hot spot area mask, defect level, verification description, and reasons for low confidence level are packaged into data awaiting manual review.

[0012] Preferably, the human-machine collaboration and data management module includes a data storage unit, a manual review unit, a data synchronization unit, and an incremental learning unit; The data storage unit is used to store equipment file information and annotation results; the equipment file information includes equipment CAD model, historical annotation records, historical temperature curves, equipment model and rated parameters, equipment operating years, equipment maintenance records, and equipment family defect information; the annotation results include original image identifiers, equipment outline masks, hotspot area masks, defect levels, annotation time, and annotation source. The manual review unit is used to provide a visual interactive interface for manual reviewers to adjust the outline, hotspots and defect levels of the data to be manually reviewed, and output the corrected annotation results as the final annotation results to the data storage unit.

[0013] Preferably, the data synchronization unit is used to synchronize the annotation results corrected by the manual review unit to the equipment file information in the data storage unit, and update the historical annotation records and historical temperature curves of the corresponding equipment; The incremental learning unit is used to monitor the updates of device file information in the data storage unit. When the accumulated correction data reaches a preset threshold, it triggers the incremental training of the few-sample segmentation network in the intelligent annotation and verification module.

[0014] A second aspect of this disclosure provides an annotation server for use in an intelligent annotation system for infrared images of power equipment as described above, comprising: The data interface unit is used to receive the multimodal raw data uploaded by the multimodal data acquisition module; Storage unit, used to deploy data storage units in the human-machine collaboration and data management module; The computing unit is used to run the image preprocessing module, the intelligent annotation and verification module, and the manual review unit, data synchronization unit, and incremental learning unit in the human-computer collaboration and data management module. The network communication unit is used for data interaction with the manual review terminal.

[0015] Preferably, the annotation server is configured to perform the following operations: S1. Receive the multimodal raw data uploaded by the multimodal data acquisition module through the data interface unit; S2. The image preprocessing module is run through the computing unit to enhance, fuse, normalize the working conditions, and construct a three-dimensional temperature field for the multimodal raw data, generating multimodal feature maps, standardized temperature matrices, and structured temperature data. S3. The intelligent annotation and verification module is run through the computing unit. Based on the multimodal feature map, standardized temperature matrix and structured temperature data, the device outline and hot spot area are pre-annotated according to the few-shot learning network. The pre-annotation results are verified based on physical laws. The high-confidence annotation results and the pre-annotation results awaiting manual review and the corresponding verification instructions are output. S4. The human-machine collaboration and data management module is run through the computing unit. The high-confidence annotation results are directly stored in the storage unit. The pre-annotation results and verification instructions to be manually reviewed are pushed to the manual review terminal through the network communication unit. The manually corrected annotation results returned by the manual review terminal are received and stored in the storage unit. At the same time, the device file information is updated. S5. The incremental learning unit in the human-machine collaboration and data management module is run through the computing unit, and the incremental training of the intelligent annotation and verification module is triggered based on the manually corrected annotation results.

[0016] The beneficial effects of this invention are as follows: This invention acquires infrared images, visible light images, 3D point clouds, and operating condition parameters simultaneously through a multimodal data acquisition module, obtaining a complete data foundation. The operating condition normalization unit standardizes and corrects the infrared temperature matrix using load current and ambient temperature, eliminating the interference of operating condition fluctuations on heat generation determination and ensuring that the annotation results reflect the true state of the equipment. The cross-modal feature fusion unit deeply integrates texture, color, and geometric features, providing an accurate spatial reference for intelligent annotation. A few-sample segmentation network enables rapid pre-annotation of equipment contours and hotspots with a limited number of annotation samples, while also incorporating component spatial position verification and temperature... The system employs triple physical constraints—degree threshold verification and topological relationship verification—to integrate the heat conduction patterns in the knowledge graph with deep learning. By weighted fusion of classification confidence and physical verification scores to calculate annotation confidence, it achieves automatic routing of high-confidence results and targeted delivery of low-confidence results. Low-confidence results are corrected through manual review, and the corrected data is synchronized to the device archive database. The incremental learning unit uses an efficient parameter fine-tuning method to incrementally train the few-sample segmentation network, constructing a complete closed loop of annotation-review-correction-learning. This significantly reduces the cost of manual annotation while ensuring the physical interpretability of the annotations. Attached Figure Description

[0017] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0018] Figure 1 This is a schematic diagram of the structure of an intelligent annotation system for infrared images of power equipment provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the steps of an annotation server performing operations according to an embodiment of the present invention. Detailed Implementation

[0019] To further illustrate the technical means and effects of the present invention in achieving the intended purpose, the following detailed description of the specific implementation methods, structures, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0020] Example 1 This embodiment provides an intelligent annotation system for infrared images of power equipment, such as... Figure 1 As shown, it includes a multimodal data acquisition module, an image preprocessing module, an intelligent annotation and verification module, and a human-machine collaboration and data management module.

[0021] The multimodal data acquisition module is used to simultaneously acquire infrared images, visible light images, three-dimensional point cloud data, and equipment operating condition parameters of the power equipment to construct multimodal raw data; the equipment operating condition parameters include load current and ambient temperature.

[0022] Specifically, the multimodal data acquisition module includes an infrared thermal imager array, a visible light camera array, a lidar system, and a sensor network. The infrared thermal imager array consists of multiple infrared thermal imagers that simultaneously acquire infrared images of the power equipment from different angles, outputting radiation image data containing the mapping relationship between image pixels and temperature values. The visible light camera array is arranged at the same viewing angle as the infrared thermal imager array, simultaneously acquiring visible light images to assist in equipment contour recognition. The lidar system acquires 3D point cloud data of the equipment for spatial positioning and 3D reconstruction. The sensor network includes current transformers and temperature sensors, acquiring real-time data on the equipment's operating load current and ambient temperature.

[0023] The data is timestamped during acquisition to ensure alignment of multimodal data in the time dimension. The multimodal data acquisition module outputs the synchronized raw multimodal data to the image preprocessing module.

[0024] The image preprocessing module is connected to the multimodal data acquisition module and is used to enhance, fuse, normalize, and construct a three-dimensional temperature field for the original multimodal data, outputting multimodal feature maps, standardized temperature matrices, and structured temperature data.

[0025] Specifically, the image preprocessing module includes an image enhancement unit, a cross-modal feature fusion unit, a working condition normalization unit, and a three-dimensional temperature field construction unit, wherein: The image enhancement unit performs progressive enhancement and fusion of infrared and visible light images to improve the edge sharpness of the infrared image, including the following steps: Multi-scale decomposition: Representing the infrared image in the original data as an infrared image. Visible light images are denoted as visible light images. The Laplace pyramid decomposition method was used to analyze the infrared images. Infrared image pyramids are obtained by performing multi-scale decomposition. ,in The number of decomposition layers, For the highest resolution layer, The lowest resolution layer; synchronized with visible light images Visible light image pyramids are obtained by performing same-scale decomposition. ; Enhanced fusion at each layer: For the first layer layer( Perform the following enhanced fusion operations: For infrared image layer Contrast stretching is performed to enhance the visual contrast of its temperature distribution, resulting in an enhanced infrared layer. ; for visible light image layers Edge extraction is performed, and the gradient magnitude is calculated using the Sobel operator to obtain the edge saliency map. Then, the infrared layer will be enhanced. With edge saliency map Weighted fusion will be performed, and the fusion rules are as follows: ; in, For infrared weighting coefficients, For edge-guided weighting coefficients, For the first The result of layer fusion.

[0026] It should be noted that for the infrared image layer Contrast stretching is performed in this embodiment using a temperature threshold based on the device type, so that the stretched grayscale values ​​can highlight defect features. This includes: reading the normal operating temperature range corresponding to the current device type from the device archive database. and defect warning temperature Design a piecewise linear mapping function to map the normal temperature range to a low grayscale range and the area above the warning temperature to a high grayscale range: ; In the formula, This is the original temperature value. This represents the mapped grayscale value.

[0027] Understandably, the mapping function uses a piecewise linear function to ensure a monotonic mapping relationship between temperature values ​​and grayscale values, so that the grayscale image can intuitively reflect the relative temperature.

[0028] Visible light image layer Edge extraction is performed, including: Visible light image layer Convert to grayscale image ; The horizontal gradient is calculated using the Sobel operator. and vertical gradient And calculate the gradient magnitude. ; Gradient magnitude plot Nonmaximum suppression and double-threshold edge connectivity are performed to obtain a binary edge saliency map. .

[0029] Pyramid Reconstruction: Merging the Results of Each Layer The enhanced image is obtained by upsampling and stacking layers according to the Laplacian pyramid reconstruction method. This will enhance the image. As output, it is passed to the cross-modal feature fusion unit.

[0030] Understandably, image enhancement Compared to the infrared image It features a clearer device edge outline while retaining temperature distribution information from the infrared image.

[0031] The cross-modal feature fusion unit is used to fuse enhanced infrared images, visible light images, and 3D point cloud data at the feature level to generate a multimodal feature map, including the following steps: Multimodal feature extraction: enhancing images The data is input to a first feature extraction network, which employs a convolutional neural network structure. This network extracts the texture distribution features of the infrared image through stacked convolutional and pooling layers, and outputs a texture feature map. Its dimensions are ,in For space dimensions, This represents the number of texture feature channels.

[0032] Visible light images The input is fed into a second feature extraction network, which employs a convolutional neural network structure connected in parallel with the first feature extraction network. This network extracts the color distribution features of the visible light image and outputs a color feature map. Its dimensions are ,in This represents the number of color feature channels.

[0033] Point cloud data from multimodal raw data The input is fed into a third feature extraction network, which uses a PointNet or sparse convolutional network structure to map the 3D point cloud coordinates onto a 2D plane projection, extracting the geometric structural features of the device and outputting a geometric feature map. Its dimensions are ,in This represents the number of geometric feature channels.

[0034] Channel dimension stitching: stitching texture feature maps Color feature map and geometric feature map The feature maps are concatenated along the channel dimension to obtain the concatenated feature map. : ; Among them, splicing feature maps The dimension is .

[0035] Dimensionality reduction fusion: stitching together feature maps Input to one or more convolutional kernels of size The convolutional layers perform channel dimensionality reduction and feature interaction to generate multimodal feature maps. : ; Among them, multimodal feature maps The dimension is ,in This represents the number of output channels after dimensionality reduction. .

[0036] The operating condition normalization unit is used to standardize the temperature matrix of the infrared image based on the equipment's operating condition parameters, converting the measured temperature to the equivalent temperature under standard operating conditions, and outputting the standardized temperature matrix. This includes the following steps: Operating Parameter Acquisition: Read equipment operating parameters from multimodal raw data, including measured load current. and ambient temperature .

[0037] Retrieves the rated parameters and standard operating condition baseline values ​​of the current equipment from the equipment file database, including the rated load current. and standard ambient temperature (Take 20℃).

[0038] Correction coefficient calculation: Read the historical temperature curve, historical load current sequence, and historical ambient temperature sequence of the equipment, and use a multiple linear regression method to fit the load-temperature coefficient based on the historical data. and ambient temperature coefficient The regression model is: ; in, Historical temperature values Historical load current, Historical environmental temperature, The residual term is obtained by solving using the least squares method. and The estimated value.

[0039] ; in, The design matrix consists of historical load current and historical ambient temperature. This is the historical temperature vector.

[0040] The fitted load-temperature coefficient and ambient temperature coefficient Stored in the device profile database.

[0041] Temperature matrix normalization: acquiring infrared images from multimodal raw data The corresponding temperature matrix Temperature matrix The dimension is Each element represents the measured temperature value of the corresponding pixel. Understandably, infrared images are obtained from multimodal raw data. The corresponding temperature matrix By reading the raw digital quantization (DN) values ​​of radiation output by the infrared thermal imager, the values ​​are successively converted into radiance through radiometric calibration, equivalent blackbody temperature is obtained through Planck inversion, and then radiometric correction is performed by combining the emissivity of the device surface and the ambient temperature. Finally, the temperature values ​​of each pixel are arranged in space to form a temperature matrix.

[0042] For temperature matrix Each temperature value in Calculate the equivalent temperature under standard operating conditions. The expression is: ; In the formula, and The correction coefficients obtained from the fitting are... and For real-time operating parameters, and This is the reference value for standard operating conditions.

[0043] The equivalent temperature is calculated by subtracting the additional temperature rise caused by the load deviating from the rated value from the measured temperature. And the additional temperature rise caused by the ambient temperature deviating from the standard value. This allows us to obtain the equivalent temperature that the equipment should have under rated load and standard ambient temperature.

[0044] By performing the above calculations on all pixels, a normalized temperature matrix is ​​obtained. Its dimensions and temperature matrix same.

[0045] The 3D temperature field construction unit is used to map a standardized temperature matrix onto 3D point cloud data, generate the 3D temperature field of the device, register the 3D temperature field with the device's CAD model, and output component-level structured temperature data, including the following steps: The pixel coordinates of the standardized temperature matrix are spatially aligned with the world coordinates of the 3D point cloud data. A temperature value is assigned to each point cloud coordinate through an interpolation algorithm (since the infrared thermal imager and lidar have been jointly calibrated in advance, the mapping matrix from pixel coordinates to world coordinates is obtained through perspective projection transformation), thus generating an initial 3D temperature field. The CAD model of the currently inspected equipment (including component hierarchical structure, such as A-phase bushing, B-phase bushing, oil conservator and heat sink, etc.) is read from the equipment archive database. The three-dimensional temperature field is rigidly registered with the equipment CAD model using the ICP iterative nearest point algorithm (after iterative convergence, each point cloud coordinate obtains the component identifier), and a mapping relationship between point cloud coordinates and CAD component identifiers is established. Based on the registration results, temperature statistical characteristics (maximum temperature, minimum temperature, average temperature, and temperature variance) are calculated for regions divided according to the CAD model components. This generates a bounding box (axis-aligned bounding box, denoted as [...]) that includes equipment identification, component name, temperature statistical characteristics, and a three-dimensional spatial bounding box. x min , y min , z min , x max , y max , z max Structured temperature data at the component level, including component-level topological connections between components (such as "A-phase bushing" connected to "A-phase clamp").

[0046] It should be noted that the construction of the three-dimensional temperature field realizes the dimensionality upgrade from two-dimensional infrared images to three-dimensional component-level temperature semantics, providing an accurate spatial benchmark for subsequent intelligent annotation and physical verification, and improving the granularity and physical interpretability of the annotation.

[0047] The intelligent annotation and verification module is connected to the image preprocessing module. It is used to pre-annotate the device outline and hotspot areas based on multimodal feature maps, standardized temperature matrices and structured temperature data, using a few-shot learning network. At the same time, it uses structured temperature data to verify the physical laws of the pre-annotation results and calculates the annotation confidence level, so as to achieve automatic output distribution.

[0048] The pre-annotation of the device outline and hotspot areas includes the following steps: Component attention prior generation: Extract the 3D spatial bounding box of each component from the structured temperature data, backproject it onto the 2D image plane through perspective projection, and generate the corresponding component attention mask; cascade the component attention mask with the multimodal feature map in the channel dimension to obtain the enhanced multimodal feature map.

[0049] Specifically, extract the 3D bounding box: read the 3D bounding box of each component from the component-level structured temperature data. This bounding box is a cuboid, defined by the smallest coordinate point. and the maximum coordinate point Definition; Back projection onto a two-dimensional image plane: using a pre-calibrated camera projection matrix (size is) The coordinates of the eight vertices of the 3D bounding box are projected onto the 2D image plane to obtain the corresponding eight pixel coordinates. The convex hull of these eight pixel coordinates is then filled to obtain the component attention mask (binary image, component region is 1, others are 0) on the 2D image. Channel cascade fusion: The size of the original multimodal feature map is... (For example ). Component attention mask (size) As additional channels, they are concatenated along the channel dimension to obtain an enhanced multimodal feature map with a size of [size missing]. .

[0050] Few-shot segmentation network inference: The enhanced multimodal feature map is input into the few-shot segmentation network, which outputs an initial device contour mask and an initial hot spot region mask. The generation of the initial hot spot region mask also refers to the temperature statistical characteristics of each component in the structured temperature data, as a temperature threshold reference to distinguish between normal heating and abnormal heating.

[0051] Specifically, the few-shot segmentation network structure adopts a prototypical network architecture. The network consists of: an encoder. (e.g., ResNet-50), maps the input feature map to the embedding space; a decoder (e.g., FPN), outputting pixel-level classification results. The network has been trained with a small support set, which contains several labeled infrared images of power equipment and their contour masks.

[0052] Inference process: Enhanced multimodal feature maps Input encoder The embedded feature map is obtained. Calculate the distance between each pixel in the embedded feature map and various prototypes in the supporting samples to generate a device contour probability map. (The probability that each pixel belongs to a device region). For Threshold segmentation (threshold = 0.5) is performed to obtain the initial device contour mask. (Binary image).

[0053] Hotspot region-assisted detection: In hotspot region detection, the network relies not only on image features but also on temperature statistical features from component-level structured temperature data. Specific method: From... Read the historical average temperature of the currently detected component. and temperature standard deviation Set an abnormal temperature threshold: In the standardized temperature matrix (From the image preprocessing module) Filter out values ​​with a temperature greater than […]. The pixels are used as candidate hotspot seed points. The candidate hotspot seed points are then weighted and fused with the hotspot probability map predicted by the network to finally output the initial hotspot region mask. (Binary image).

[0054] Pre-annotation results are temporarily stored: The initial device outline mask, initial hot spot area mask, corresponding component labels (obtained from structured temperature data based on hot spot location), hot spot temperature information, and original image identifiers are packaged into pre-annotation results.

[0055] Specifically, query the component label to which the hotspot belongs: calculate the initial hotspot region mask. Geometric center pixel coordinates : ;in This represents the total number of pixels within the hotspot area. These are the pixel coordinates within the hotspot area. Using the inverse transformation of the camera projection matrix, [the following is used]... Back-projecting into three-dimensional space yields the three-dimensional coordinates of the hotspot. Traverse component-level structured temperature data All components, determine Does it fall within the 3D bounding box of this component? If it falls within a component, then use the name of that component as the component label. If the hotspot falls within the overlapping area of ​​multiple components, the component with the smallest bounding box volume is selected (for greater precision); if it does not fall within any component, the component is labeled "unknown".

[0056] Extracting hotspot temperature information: from the standardized temperature matrix In the middle, read the hotspot area mask. All temperature values ​​for the covered pixels. Calculate the highest temperature in this area. Average temperature and lowest temperature Packaging pre-annotation results: The following data is packaged into a single structure, called the pre-annotation results. Initial device contour mask Initial hotspot mask Component labels Hotspot temperature information: Original image identifiers (such as timestamps and device IDs).

[0057] The physical laws are verified and the labeled confidence level is calculated, including the following steps: Component spatial location verification: Calculate the geometric center pixel coordinates based on the initial hotspot region mask, and back-project it into 3D space using the known camera projection matrix to obtain the hotspot's 3D coordinates; perform spatial inclusion judgment between the hotspot's 3D coordinates and the 3D bounding boxes of each component in the structured temperature data to determine the component identifier to which the hotspot belongs; according to the "equipment type - standard heating component" mapping rules stored in the internal knowledge graph, determine whether the component identifier belongs to a component that is allowed to generate heat: if so, the spatial verification passes; otherwise, it is marked as an abnormal spatial location.

[0058] Specifically, calculate the pixel coordinates of the hotspot's geometric center: from the pre-annotation results. Extract the initial hotspot region mask The mask is a binary image where pixels within hotspot regions have a value of 1, and the rest have a value of 0. Let there be a total number of pixels within the hotspot region. Each pixel has coordinates as follows: Then the coordinates of the geometric center are: .

[0059] Back projection into 3D space: using a pre-calibrated camera projection matrix (size is) ) and its inverse matrix , pixel coordinates Convert to 3D spatial coordinates. Since single-pixel backprojection yields a ray, it's necessary to combine this with depth information (obtaining the depth value corresponding to that pixel from the 3D point cloud data). Let the camera intrinsic parameter matrix be... The extrinsic parameter matrix is ,but: ; Finally, the three-dimensional coordinates of the hotspot are obtained. .

[0060] Spatial inclusion determination: Traversing component-level structured temperature data All components, each component Corresponding to a three-dimensional bounding box: ; Determine if the three-dimensional coordinates of the hotspot satisfy: ; If the conditions are met, then the hotspot belongs to the component. Record component identification If it belongs to multiple components, select the component with the smallest bounding box volume; if it does not belong to any component, mark it as "unknown".

[0061] Knowledge Graph Mapping Verification: This function calls the internal knowledge graph to query the list of standard heat-generating components corresponding to the current device type (e.g., "transformer"). Examples of mapping relationships stored in the knowledge graph: Current-heated devices (e.g., clamps): {clamp body, wire connection point, terminal block}; Voltage-heated devices (e.g., bushings): {shroud skirt root, end screen, flange connection}.

[0062] judge Does it belong to the standard heating component set of this device? If it belongs to the category, the space verification passes, and the flag is recorded. ; If it does not belong to or is "unknown", then the spatial location is abnormal and should be recorded. .

[0063] Temperature threshold verification: Read the measured temperature values ​​within the initial hot spot mask coverage area from the standardized temperature matrix; read the historical temperature statistical characteristics corresponding to the component identifier from the structured temperature data; calculate the deviation between the measured temperature and the historical mean, and use the two-standard-deviation criterion to determine whether it is abnormal; when the temperature is abnormal, calculate the relative temperature difference to determine the defect level (normal, general defect, serious defect, critical defect); if the measured temperature is within the normal range, it is marked as no temperature abnormality.

[0064] Specifically, read the measured temperature of the hotspot area: from the standardized temperature matrix. In the process, extract the initial hotspot region mask. All temperature values ​​of the covered pixels. Assume this area contains... Each pixel has a temperature value recorded as... .calculate: Maximum temperature: ; Average temperature: .

[0065] Obtaining historical temperature statistical characteristics: from component-level structured temperature data In the middle, based on the obtained component identification Read the historical temperature statistics of this component: historical average temperature. Historical temperature standard deviation These statistics are calculated based on historical data from multiple tests of the same part of the device.

[0066] Temperature anomaly determination: The 2-standard-deviation criterion is used to determine whether an anomaly exists. If the conditions are met, then a temperature anomaly flag is set. Otherwise, mark And record "No abnormal temperature".

[0067] Defect level determination (only performed under abnormal temperature conditions): Calculate the relative temperature difference according to the power industry standard DL / T 664 "Application Specification for Infrared Diagnosis of Live Equipment". : ;in: Temperature at the same location during normal equipment operation (historical average can be used) approximate), The ambient temperature (obtained from the multimodal data acquisition module), and then based on... Value determines defect level: δ <20% is normal, 20% ≤ δ <40% are general defects, 40% ≤ δ <80% are serious defects; δ ≥80% are considered critical defects.

[0068] Topology verification: Extract the topology connection relationships of component identifiers from structured temperature data (e.g., "A-phase clamp" connects "A-phase conductor" and "A-phase bushing"); determine whether the initial hot spot area is located in a reasonable topology position in the current path or voltage-heated area: for current-heated equipment, the component where the hot spot is located must be at the contact interface of the current loop; for voltage-heated equipment, the component where the hot spot is located must be in the electric field concentration area; if the conditions are met, the topology verification passes; otherwise, it is marked as a topology anomaly.

[0069] Specifically, extracting topological connectivity: from component-level structured temperature data. In, according to component identification Read the topological connection relationship of the component. Determine the device's heat generation type: Query the knowledge graph to find the corresponding heat generation mechanism for the current device type. Current-induced heating type: The heating is caused by the current passing through the contact resistance, and the hot spot should be located at the contact interface of the current loop (such as the crimping point between the wire clamp and the conductor, the moving / stationary contact of the circuit breaker). Voltage-induced heating type: The heating is caused by dielectric loss or electric field concentration. The hot spot should be located in the area of ​​high electric field strength (such as the root of the bushing skirt or the contaminated area on the surface of the insulator).

[0070] Topology rationality assessment: For current-heated devices: the components where the hot spot is located must be considered. It must be located on the path of a current loop, meaning the component should be adjacent to at least two current-carrying components (forming a current path). For example, clamp → conductor, clamp → bushing both satisfy this condition. It is unreasonable for a hot spot to be located on an isolated component (such as a bracket not connected to any current-carrying component).

[0071] For voltage-heated devices: the hot spot should be located in an area of ​​concentrated electric field, typically at a high potential gradient (such as the base of the bushing skirt or near the equalizing ring). The topology should show the insulation between the component and the high-voltage or grounding terminal.

[0072] Specific judgment rules (taking current-heated type as an example): If the conditions are met, the topology check passes, and the flag is recorded. Otherwise, mark it as a topology anomaly. Record the cause of the abnormality (e.g., "the hot spot component is outside the current loop").

[0073] Comprehensive confidence calculation: The intelligent annotation and verification module obtains the classification confidence of the few-sample segmentation network output; it calculates the physical verification score based on the results of three verifications: component spatial location verification, temperature threshold verification, and topological relationship verification. The final annotation confidence is obtained by weighted fusion of the classification confidence and the physical verification score, and verification description is generated at the same time.

[0074] Specifically, obtaining classification confidence: Classification confidence is obtained from the output of the few-shot segmentation network. This value represents the network's confidence in the pre-labeled results, ranging from [0, 1]. The physical verification score is calculated based on three verification indicators: component spatial location verification, temperature threshold verification, and topological relationship verification. , , Calculate the physical verification score: ; Each item is assigned a value of 1 for passing and 0 for failing. Example: If all three items pass, then... If only two items pass, then .

[0075] Weighted fusion yields labeled confidence: weights are set. and satisfy ,and The formula for calculating the confidence level is as follows: The verification results and judgment criteria for each step are integrated into natural language text as verification instructions. Example format: Spatial location verification: The hot spot is located in phase A clamp, which is a standard heating component; pass. Temperature threshold verification: The highest measured temperature was 78.5℃, the historical average was 65.2℃, and the relative temperature difference was 20.4%, reaching the general defect level; pass. Topological relationship verification: Phase A clamp connects phase A conductor and phase A bushing, and is on the current path; pass. Overall confidence level. (like ,Pick , (emphasizing the importance of physical verification).

[0076] The automatic output splitting includes the following steps: The calculated annotation confidence level is compared with a preset confidence threshold (e.g., Th=0.85). If the annotation confidence level is greater than or equal to the confidence threshold Th, the initial equipment contour mask, initial hot spot area mask, defect level, component identification, measured temperature value, and verification description are packaged into a structured annotation result and directly output to the human-machine collaboration and data management module for archiving. If the annotation confidence level is less than the confidence threshold Th, the initial equipment contour mask, initial hot spot area mask, defect level (if any), verification description, and reasons for low confidence (such as "abnormal spatial location" or "no abnormal temperature but abnormal topology") are packaged into data awaiting manual review.

[0077] The human-machine collaboration and data management module is connected to the intelligent annotation and verification module. It is used to receive the results output by the intelligent annotation and verification module, perform manual review and correction on the data to be manually reviewed, and is responsible for the storage and management of all data. It also triggers incremental training of the intelligent annotation and verification module based on the manually corrected data.

[0078] The human-machine collaboration and data management module includes a data storage unit, a manual review unit, a data synchronization unit, and an incremental learning unit, wherein: The data storage unit is used to store equipment file information and annotation results. The equipment file information includes equipment CAD model, historical annotation records, historical temperature curves, equipment model and rated parameters, equipment operating years, equipment maintenance records, and equipment family defect information. The annotation result information includes original image identifiers, equipment outline masks, hotspot area masks, defect levels, annotation time, and annotation source.

[0079] The manual review unit provides a visual interactive interface for manual reviewers to adjust the outline, hotspots, and defect levels of the data to be manually reviewed, and outputs the corrected annotation results as the final annotation results to the data storage unit.

[0080] The data synchronization unit is used to synchronize the annotation results corrected by the manual review unit to the equipment file information in the data storage unit, and update the historical annotation records and historical temperature curves of the corresponding equipment.

[0081] The incremental learning unit is used to monitor the updates of device file information in the data storage unit. When the accumulated correction data reaches a preset threshold, it triggers the incremental training of the few-shot segmentation network in the intelligent annotation and verification module.

[0082] Specifically, incremental training of few-shot segmentation networks employs parameter-efficient fine-tuning methods (such as LoRA) to incrementally fine-tune the network. During training, the corrected data is... As key learning samples, a weighted loss function is used to improve the model's learning performance on the corrected samples: ; in, For the first The weight of each corrected sample (higher than that of a normal sample); The segmentation loss is calculated using cross-entropy loss, which measures the model's predicted mask. With manually corrected mask (Differences between them) The physical consistency loss is calculated based on the heat conduction law corresponding to the device type and is used to penalize pixels where the predicted hotspot is located in non-heat-generating areas. This is a balancing coefficient used to control the weight of physical constraints in the total loss.

[0083] After incremental training is completed, the updated model is validated. If the validation set accuracy is better than the current model, the updated model parameters are deployed to the intelligent annotation and validation module to replace the original model parameters, thus completing the model update.

[0084] Example 2 This embodiment also provides a labeling server, including: The data interface unit is used to receive the multimodal raw data uploaded by the multimodal data acquisition module; Storage unit, used to deploy data storage units in the human-machine collaboration and data management module; The computing unit is used to run the image preprocessing module, the intelligent annotation and verification module, and the manual review unit, data synchronization unit, and incremental learning unit in the human-computer collaboration and data management module. A network communication unit is used to interact with a human review terminal (when the human review unit is deployed on a remote terminal).

[0085] like Figure 2 As shown, the annotation server is configured to perform the following operations: S1. Receive the multimodal raw data uploaded by the multimodal data acquisition module through the data interface unit; S2. The image preprocessing module is run through the computing unit to enhance, fuse, normalize the working conditions, and construct a three-dimensional temperature field on the received multimodal raw data, generating multimodal feature maps, standardized temperature matrices, and structured temperature data. S3. The intelligent annotation and verification module is run through the computing unit. It receives multimodal feature maps, standardized temperature matrices, and structured temperature data. By generating component attention priors, it projects the 3D spatial bounding boxes in the structured temperature data onto the 2D image plane to generate attention masks, which are then concatenated with the multimodal feature maps to obtain enhanced multimodal feature maps. It outputs initial device contour masks and initial hotspot region masks through few-shot segmentation network inference, and uses temperature statistical features in the structured temperature data to assist in hotspot detection. It packages the initial device contour mask, initial hotspot region mask, component labels, and hotspot temperature information into pre-annotation results through temporary storage of pre-annotation results. It performs physical law verification on the pre-annotation results through component spatial position verification, temperature threshold verification, and topological relationship verification, and calculates the physical verification score. It calculates and fuses the classification confidence score and physical verification score to obtain the annotation confidence score and generates verification descriptions. It automatically splits the output, compares the annotation confidence score with the preset threshold, and outputs high-confidence annotation results and pre-annotation results awaiting manual review, along with corresponding verification descriptions. S4. The human-machine collaboration and data management module is run through the computing unit to receive the high-confidence annotation results and pre-annotation results awaiting manual review and corresponding verification instructions output by the intelligent annotation and verification module. For the high-confidence annotation results, they are stored in the storage unit through the data synchronization unit. For the pre-annotation results awaiting manual review and corresponding verification instructions, they are pushed to the manual review terminal through the network communication unit. The manual correction annotation results returned by the manual review terminal are received and stored in the storage unit through the data synchronization unit. The historical annotation records and historical temperature curves of the corresponding equipment in the equipment file information are updated synchronously. S5. The incremental learning unit in the human-machine collaboration and data management module is run through the computing unit to monitor the update of device file information in the storage unit. When the accumulated manually corrected annotation results reach the preset threshold, the incremental training of the few-sample segmentation network in the intelligent annotation and verification module is triggered. The parameter efficient fine-tuning method is used to perform incremental fine-tuning and update the model parameters.

[0086] This invention simultaneously acquires infrared images, visible light images, 3D point clouds, and operating condition parameters through a multimodal data acquisition module, providing a complete data foundation for subsequent processing. The operating condition normalization unit in the image preprocessing module standardizes and corrects the infrared temperature matrix using load current and ambient temperature, eliminating the interference of operating condition fluctuations on heat generation determination and ensuring that the annotation results reflect the true state of the equipment. The cross-modal feature fusion unit deeply integrates texture, color, and geometric features, while the 3D temperature field construction unit achieves dimensionality enhancement from 2D images to 3D component-level temperature semantics, providing an accurate spatial benchmark for intelligent annotation. The intelligent annotation and verification module achieves rapid pre-annotation of equipment contours and hotspots with a small number of annotation samples using a few-sample segmentation network, while also introducing component spatial location verification, temperature threshold verification, and... The topological relationship verification employs triple physical constraints, integrating heat conduction patterns from the knowledge graph with deep learning to avoid "physically impossible" errors in AI annotation. It calculates annotation confidence by weighting and fusing classification confidence with physical verification scores, enabling automatic distribution of high-confidence results and targeted delivery of low-confidence results. The human-machine collaboration and data management module manually reviews and corrects low-confidence results, synchronizing the corrected data to the equipment archive database. The incremental learning unit uses an efficient parameter fine-tuning method to incrementally train the few-sample segmentation network, constructing a complete closed loop of "annotation-review-correction-learning." This effectively solves the problem of scarce power equipment fault samples, significantly reducing manual annotation costs while ensuring the physical interpretability of the annotations, thus improving both annotation quality and efficiency.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An intelligent annotation system for infrared images of power equipment, characterized in that: include: The multimodal data acquisition module is used to simultaneously acquire infrared images, visible light images, 3D point cloud data, and equipment operating parameters of power equipment to construct multimodal raw data. The image preprocessing module is used to enhance, fuse, normalize, and construct a three-dimensional temperature field for multimodal raw data, and output multimodal feature maps, standardized temperature matrices, and structured temperature data. The intelligent annotation and verification module is used to pre-annotate the device outline and hotspot areas based on multimodal feature maps, standardized temperature matrices, and structured temperature data using a few-shot learning network; at the same time, it uses structured temperature data to verify the physical laws of the pre-annotation results and calculate the annotation confidence level, thereby achieving automatic output distribution. The human-machine collaboration and data management module is used to manually review and correct data to be manually reviewed. It is also responsible for the storage and management of all data and triggers incremental training of the intelligent annotation and verification module based on the manually corrected data. The pre-annotation of the device outline and hotspot areas includes the following steps: Extract the 3D spatial bounding boxes of each component from the structured temperature data, backproject them onto the 2D image plane through perspective projection, and generate the corresponding component attention mask; The component attention mask and the multimodal feature map are concatenated along the channel dimension to obtain an enhanced multimodal feature map; The enhanced multimodal feature map is input into the few-sample segmentation network, which outputs an initial device contour mask and an initial hot spot region mask. The generation of the initial hot spot region mask also takes into account the temperature statistical characteristics of each component in the structured temperature data. The initial device outline mask, initial hotspot region mask, corresponding component labels, hotspot temperature information, and original image identifiers are packaged into a pre-annotated result.

2. The intelligent annotation system for infrared images of power equipment according to claim 1, characterized in that: The equipment operating parameters include load current and ambient temperature; the image preprocessing module includes: An image enhancement unit is used to progressively enhance and fuse infrared and visible light images to obtain an enhanced image. The cross-modal feature fusion unit is used to fuse enhanced infrared images, visible light images and 3D point cloud data at the feature level to generate a multimodal feature map. The operating condition normalization unit is used to standardize the temperature matrix of the infrared image according to the operating condition parameters of the equipment, convert the measured temperature to the equivalent temperature under standard operating conditions, and output the standardized temperature matrix. The 3D temperature field construction unit is used to map a standardized temperature matrix onto 3D point cloud data, generate the 3D temperature field of the device, register the 3D temperature field with the device CAD model, and output component-level structured temperature data.

3. The intelligent annotation system for infrared images of power equipment according to claim 2, characterized in that: The output normalized temperature matrix includes the following steps: Read the measured load current from the multimodal raw data. and ambient temperature ; Read the rated load current from the equipment file database. and standard ambient temperature ; Based on the equipment's historical temperature curves, historical load current sequences, and historical ambient temperature sequences, a load-temperature coefficient is fitted. and ambient temperature coefficient ; For temperature matrix Each temperature value in Calculate the equivalent temperature under standard operating conditions: ； Obtain the standardized temperature matrix .

4. The intelligent annotation system for infrared images of power equipment according to claim 3, characterized in that: The structured temperature data at the output component level includes the following steps: The pixel coordinates of the standardized temperature matrix are spatially aligned with the world coordinates of the 3D point cloud data. An interpolation algorithm is used to assign a temperature value to each point cloud coordinate to generate an initial 3D temperature field. The CAD model of the currently inspected equipment is read from the equipment archive database. The iterative nearest point algorithm is used to rigidly register the three-dimensional temperature field with the equipment CAD model to establish the mapping relationship between point cloud coordinates and CAD component identifiers. Based on the registration results, temperature statistical characteristics are calculated by dividing the regions according to the CAD model components, and component-level structured temperature data is generated, including equipment identification, component name, temperature statistical characteristics, three-dimensional bounding box, and topological connection relationship between components.

5. The intelligent annotation system for infrared images of power equipment according to claim 1, characterized in that: The physical laws are verified and the labeled confidence level is calculated, including the following steps: Component spatial location verification: Calculate the geometric center pixel coordinates based on the initial hotspot region mask, and back-project them into 3D space using the known camera projection matrix to obtain the hotspot's 3D coordinates; perform spatial inclusion judgment between the hotspot's 3D coordinates and the 3D bounding boxes of each component in the structured temperature data to determine the component identifier to which the hotspot belongs; according to the device type-standard heating component mapping rules stored in the internal knowledge graph, determine whether the component identifier belongs to a component that is allowed to generate heat: if so, the spatial verification passes; otherwise, it is marked as an abnormal spatial location. Temperature threshold verification: Read the measured temperature values ​​within the area covered by the initial hot spot mask from the standardized temperature matrix; Historical temperature statistics corresponding to component identifiers are obtained from structured temperature data; Calculate the deviation between the measured temperature and the historical average, and use the two-standard-deviation criterion to determine whether it is abnormal; when the temperature is abnormal, calculate the relative temperature difference to determine the defect level; if the measured temperature is within the normal range, it is marked as no temperature abnormality. Topology verification: Extract the topology connection relationships of component identifiers from structured temperature data; determine whether the initial hot spot region is located in a reasonable topology position in the current path or voltage-heated region: for current-heated devices, the component where the hot spot is located must be at the contact interface of the current loop; for voltage-heated devices, the component where the hot spot is located must be in the electric field concentration region; if the conditions are met, the topology verification passes; otherwise, it is marked as a topology anomaly. Comprehensive confidence calculation: Obtain the classification confidence score output by the few-sample segmentation network, calculate the physical verification score based on the results of three verifications: component spatial location verification, temperature threshold verification, and topological relationship verification, and finally obtain the label confidence score by weighted fusion of classification confidence score and physical verification score, while generating verification description.

6. The intelligent annotation system for infrared images of power equipment according to claim 1, characterized in that: The automatic output splitting includes the following steps: The calculated label confidence level is compared with the preset confidence level threshold. If the annotation confidence level is greater than or equal to the confidence level threshold, the initial equipment contour mask, initial hot spot area mask, defect level, component identification, measured temperature value, and verification description are packaged into a structured annotation result and directly output to the human-machine collaboration and data management module for archiving; if the annotation confidence level is less than the confidence level threshold, the initial equipment contour mask, initial hot spot area mask, defect level, verification description, and reasons for low confidence level are packaged into data awaiting manual review.

7. The intelligent annotation system for infrared images of power equipment according to claim 1, characterized in that: The human-machine collaboration and data management module includes a data storage unit, a manual review unit, a data synchronization unit, and an incremental learning unit; The data storage unit is used to store equipment file information and annotation results; the equipment file information includes equipment CAD model, historical annotation records, historical temperature curves, equipment model and rated parameters, equipment operating years, equipment maintenance records, and equipment family defect information; the annotation results include original image identifiers, equipment outline masks, hotspot area masks, defect levels, annotation time, and annotation source. The manual review unit is used to provide a visual interactive interface for manual reviewers to adjust the outline, hotspots and defect levels of the data to be manually reviewed, and output the corrected annotation results as the final annotation results to the data storage unit.

8. The intelligent annotation system for infrared images of power equipment according to claim 7, characterized in that: The data synchronization unit is used to synchronize the annotation results corrected by the manual review unit to the equipment file information in the data storage unit, and update the historical annotation records and historical temperature curves of the corresponding equipment. The incremental learning unit is used to monitor the updates of device file information in the data storage unit. When the accumulated correction data reaches a preset threshold, it triggers the incremental training of the few-sample segmentation network in the intelligent annotation and verification module.

9. A labeling server, applied to an intelligent labeling system for infrared images of power equipment as described in any one of claims 1-8, characterized in that: include: The data interface unit is used to receive the multimodal raw data uploaded by the multimodal data acquisition module; Storage unit, used to deploy data storage units in the human-machine collaboration and data management module; The computing unit is used to run the image preprocessing module, the intelligent annotation and verification module, and the manual review unit, data synchronization unit, and incremental learning unit in the human-computer collaboration and data management module. The network communication unit is used for data interaction with the manual review terminal.

10. A labeling server according to claim 9, characterized in that: The annotation server is configured to perform the following operations: S1. Receive the multimodal raw data uploaded by the multimodal data acquisition module through the data interface unit; S2. The image preprocessing module is run through the computing unit to enhance, fuse, normalize the working conditions, and construct a three-dimensional temperature field for the multimodal raw data, generating multimodal feature maps, standardized temperature matrices, and structured temperature data. S3. The intelligent annotation and verification module is run through the computing unit. Based on the multimodal feature map, standardized temperature matrix and structured temperature data, the device outline and hot spot area are pre-annotated according to the few-shot learning network. The pre-annotation results are verified based on physical laws. The high-confidence annotation results and the pre-annotation results awaiting manual review and the corresponding verification instructions are output. S4. The human-machine collaboration and data management module is run through the computing unit. The high-confidence annotation results are directly stored in the storage unit. The pre-annotation results and verification instructions to be manually reviewed are pushed to the manual review terminal through the network communication unit. The manually corrected annotation results returned by the manual review terminal are received and stored in the storage unit. At the same time, the device file information is updated. S5. The incremental learning unit in the human-machine collaboration and data management module is run through the computing unit, and the incremental training of the intelligent annotation and verification module is triggered based on the manually corrected annotation results.

Images