SAM semantic segmentation-based fine labeling method, system and equipment for air and space images of power grid, and medium

Abstract

The invention discloses a power grid space image fine labeling method, system and device based on SAM semantic segmentation and a medium, and belongs to the technical field of power system maintenance, and the method comprises the steps: obtaining an initial device label through semantic segmentation model labeling; generating probability distribution of power grid equipment categories through the satellite image, and obtaining a final equipment label through the probability distribution of the power grid equipment categories; collecting defect information of the power grid equipment to generate a defect report, and marking the machine patrol image through the semantic segmentation model in combination with the defect report to obtain a defect mark; correcting the final equipment label and the defect label, and training the semantic segmentation model; according to the method, the semantic segmentation model is combined with the probability distribution of the power grid equipment category, the equipment characteristics in the satellite image under the complex background are extracted, and the image processing capability and the marking accuracy are improved; manual intervention is reduced through entropy value screening data calculation and self-training iteration logic, the labeling and maintenance cost is reduced, and the requirements of a power grid for the image recognition accuracy and the automation degree are met.

Description

Technical Field

[0001] This invention relates to the field of power system maintenance technology, specifically to a method, system, equipment, and medium for fine annotation of power grid aerospace images based on SAM semantic segmentation. Background Technology

[0002] With the deepening of the digital transformation of the power grid, image data based on aerospace platforms such as satellites and drones has become a core data source for power grid inspection, asset inventory, and risk monitoring. Its processing efficiency and recognition accuracy directly determine the level of intelligent operation and maintenance of the power grid.

[0003] Traditional annotation methods rely heavily on manual drawing or basic automated tools, making it difficult to cope with the morphological variations and occlusion interference of targets such as power transmission towers and conductors in complex backgrounds such as mountainous areas and urban building clusters. Manual annotation of a single high-resolution image can take several hours, and the annotation consistency is insufficient, which cannot meet the needs of large-scale processing of massive amounts of aerospace images.

[0004] Existing image processing and machine learning algorithms are inaccurate in extracting features from complex backgrounds and diverse structures in power grid satellite imagery, and have limited processing capabilities for high-resolution images. They cannot meet the requirements of image recognition accuracy and automation for the digital transformation of the power grid. They are also difficult to adapt to the diversity and complexity of power grid aerospace imagery data, resulting in high costs for manual annotation and model maintenance. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] Therefore, the technical problem solved by this invention is that existing image processing and machine learning algorithms are inaccurate in extracting features from complex backgrounds and diverse structures in power grid satellite imagery, have limited processing capabilities for high-resolution images, and cannot meet the requirements of power grid digital transformation for image recognition accuracy and automation; they are also difficult to adapt to the diversity and complexity of power grid aerospace imagery data, resulting in high costs for manual annotation and maintenance.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a method for fine annotation of power grid aerospace images based on SAM semantic segmentation, which includes the following steps: acquiring satellite images and airborne inspection images of the power grid system;

[0008] Initial equipment annotations are obtained by annotating satellite imagery using a semantic segmentation model;

[0009] The probability distribution of power grid equipment categories is generated by satellite imagery, and the initial equipment labels are optimized based on the probability distribution of power grid equipment categories to obtain the final equipment labels;

[0010] Defect information of the power grid system is collected to generate a defect report. The defect labels are obtained by annotating the machine inspection images using a semantic segmentation model combined with the defect report.

[0011] The semantic segmentation model is trained using final device and defect annotations.

[0012] As a preferred embodiment of the SAM semantic segmentation-based fine annotation method for power grid aerospace images of the present invention, the step of obtaining initial equipment annotations includes:

[0013] The satellite imagery is segmented using a semantic segmentation model to obtain the segmentation mask;

[0014] The initial device label is obtained by filling the internal holes of the segmentation mask with morphological closing operations and then smoothing the boundaries of the segmentation mask with morphological opening operations.

[0015] As a preferred embodiment of the SAM semantic segmentation-based fine annotation method for power grid aerospace images of the present invention, the method for generating the probability distribution of power grid equipment categories through feature data of satellite images includes the following steps:

[0016] Extract feature data from satellite imagery;

[0017] Weighted fusion of feature data generates comprehensive features;

[0018] The probability distribution of power grid equipment categories in the power grid system is generated based on comprehensive features.

[0019] As a preferred embodiment of the SAM semantic segmentation-based fine annotation method for power grid aerospace images of the present invention, the step of optimizing the initial equipment annotation based on the probability distribution of power grid equipment categories includes:

[0020] Obtain the probability distribution of equipment features and corresponding power grid equipment categories in satellite imagery;

[0021] Calculate the matching degree of device features, and assign corresponding feature weights to the corresponding device features based on the matching degree of device features;

[0022] The matching score of the initial device annotation is calculated based on the feature weights, and the valid annotation data in the initial device annotation is filtered based on the matching score.

[0023] The effective labeled data is corrected based on the distribution characteristics to obtain the final equipment labeling.

[0024] As a preferred embodiment of the SAM semantic segmentation-based fine annotation method for power grid aerospace images of the present invention, the step of annotating the machine inspection images by combining the semantic segmentation model with defect reports includes:

[0025] Obtain the report location information from the defect reports, classify the defect reports according to the report location information, and obtain the classified defects;

[0026] Acquire the image location information of the aerial patrol images, and register the defect report with the aerial patrol images by matching the report location information with the image location information;

[0027] The classified defects are associated with the registered machine-controlled images to obtain defect images;

[0028] Defect annotations are obtained by annotating defect images using a semantic segmentation model.

[0029] As a preferred embodiment of the SAM semantic segmentation-based fine annotation method for power grid aerospace images according to the present invention, the step of annotating defect images using a semantic segmentation model includes:

[0030] The defective image is divided into segments, and one frame is extracted every set number of frames as a key frame. The remaining frames that are not extracted are non-key frames.

[0031] The keyframes are input into the semantic segmentation model for annotation to obtain the keyframe mask;

[0032] The pixel displacement vector between keyframes and non-keyframes is calculated using optical flow.

[0033] The keyframe mask is passed along the pixel displacement vector to the non-keyframe position to generate the initial mask for the non-keyframe.

[0034] The initial mask of non-critical frames is corrected by timing consistency verification, and a mask of non-critical frames is generated.

[0035] Defect annotations are generated by matching key frames and non-key frames according to time sequence.

[0036] The beneficial effects of this preferred technical solution are as follows: by dividing key frames into key frames and non-key frames, the key frames are directly labeled using a semantic segmentation model, and then the displacement vector is calculated using optical flow. The key frame mask is then passed to the non-key frame position along the pixel displacement vector to generate the initial mask for the non-key frames. This not only allows the key frames to be labeled using a semantic segmentation model, but also reduces the redundant calculation of non-key frame labeling by passing the mask using optical flow, thereby improving labeling efficiency.

[0037] As a preferred embodiment of the SAM-based semantic segmentation method for fine-grained annotation of power grid aerospace images according to the present invention, the step of training the semantic segmentation model through final equipment annotation and defect annotation includes:

[0038] Using satellite imagery as input and final device annotations as the supervision target, the semantic segmentation model is trained.

[0039] Devices are labeled using a semantic segmentation model to obtain predicted device labels. The label entropy value of the predicted device labels is calculated, and the predicted device labels are classified based on the label entropy value.

[0040] The classified predicted equipment labels are manually sampled and verified to obtain the verified equipment labels.

[0041] The proofreading equipment annotations and the final equipment annotations were combined and the semantic segmentation model was trained again.

[0042] The annotation performance of the semantic segmentation model is evaluated, and the semantic segmentation model is saved if the evaluation result meets the set evaluation threshold.

[0043] If the evaluation results do not meet the set evaluation threshold, manual sampling and verification will be performed again, and the training will be repeated.

[0044] The semantic segmentation model is trained by using machine-surveyed images as input and defect annotations as supervision targets.

[0045] Defects are labeled using a semantic segmentation model to obtain predicted defect labels. The label entropy value of the predicted defect labels is calculated, and the predicted defect labels are classified according to the label entropy value.

[0046] The predicted defect labels after classification are manually sampled and checked to obtain the checked defect labels;

[0047] The proofreading defect annotations and defect annotations were combined and the semantic segmentation model was retrained.

[0048] The annotation performance of the semantic segmentation model is evaluated, and the semantic segmentation model is saved if the evaluation result meets the set evaluation threshold.

[0049] If the evaluation results do not meet the set evaluation threshold, manual sampling and verification will be performed again, and the training will be repeated.

[0050] The beneficial effects of this preferred technical solution are: by calculating entropy values ​​to filter data and using self-training iterative logic, manual intervention is reduced, labeling and maintenance costs are lowered, and the requirements of the power grid for image recognition accuracy and automation are met.

[0051] This invention provides a fine annotation system for power grid aerospace images based on SAM semantic segmentation.

[0052] To solve the above-mentioned technical problems, the present invention further provides the following technical solution: a power grid aerospace image fine annotation system based on SAM semantic segmentation, comprising: a data acquisition module: acquiring satellite images and air patrol images of the power grid system, and acquiring defect information of the power grid system;

[0053] First equipment annotation module: Annotates satellite images using a semantic segmentation model to obtain initial equipment annotations;

[0054] Data processing module: Generates probability distributions of power grid equipment categories using feature data from satellite imagery;

[0055] The second equipment labeling module optimizes the initial equipment labels by using the probability distribution of power grid equipment categories to obtain the final equipment labels;

[0056] Defect annotation module: Generates defect reports from defect information, and annotates the aerial patrol images using a semantic segmentation model in conjunction with the defect reports to obtain defect annotations;

[0057] Self-training module: Corrects the final device and defect labels to obtain corrected device and defect labels; trains the semantic segmentation model using the corrected device and defect labels.

[0058] This invention provides a computer device, including a memory and a processor. The memory stores a computer program. The feature is that when the processor executes the computer program, it implements the steps of a method for fine annotation of power grid aerospace images based on SAM semantic segmentation.

[0059] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of a method for fine annotation of power grid aerospace images based on SAM semantic segmentation.

[0060] The beneficial effects of this invention are as follows: This invention accurately extracts equipment features in complex backgrounds in satellite imagery by combining a semantic segmentation model with the probability distribution of power grid equipment categories, thereby improving image processing capabilities and annotation accuracy; by dividing key frames into key frames and non-key frames, key frames are directly annotated using a semantic segmentation model, and then the displacement vector is calculated using optical flow to transmit the key frame mask and generate the initial mask for non-key frames, reducing redundant calculations for non-key frames and improving annotation efficiency; by calculating entropy values ​​to filter data and using self-training iteration logic, manual intervention is reduced, lowering annotation and maintenance costs and meeting the power grid's requirements for image recognition accuracy and automation. Attached Figure Description

[0061] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating a method for fine annotation of power grid aerospace images based on SAM semantic segmentation, as provided in one embodiment of the present invention. Detailed Implementation

[0063] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0064] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a method for fine annotation of power grid aerospace images based on SAM semantic segmentation, including:

[0065] S100: Acquires satellite and machine-generated images of the power grid system.

[0066] S200: Satellite imagery is annotated using a semantic segmentation model to obtain initial equipment annotations.

[0067] S300: Generates a probability distribution of power grid equipment categories from satellite imagery, optimizes the initial equipment labels based on the probability distribution of power grid equipment categories, and obtains the final equipment labels.

[0068] S400: Collects defect information of the power grid system and generates defect reports. It then uses a semantic segmentation model to annotate the machine inspection images in conjunction with the defect reports to obtain defect annotations.

[0069] S500: The semantic segmentation model is trained using final device annotations and defect annotations.

[0070] It should be noted that manual annotation of power grid aerospace images is costly and difficult to maintain, and the personal safety of annotators in complex environments may also be affected. If image processing and machine learning algorithms are used for annotation, there are still technical problems such as insufficient accuracy of automatic annotation, weak model generalization ability, inaccurate feature extraction of complex backgrounds and diverse structures in power grid satellite images, and limited processing capabilities for high-resolution images. These methods cannot meet the requirements of image recognition accuracy and automation for power grid digital transformation, and are difficult to adapt to the diversity and complexity of power grid aerospace image data. Therefore, it is very important to propose a fine annotation method for power grid aerospace images based on SAM semantic segmentation.

[0071] Therefore, to address the aforementioned issues in the annotation of power grid aerospace images, steps S100-S600 are used to accurately extract equipment features in complex backgrounds in satellite images through a semantic segmentation model combined with the probability distribution of power grid equipment categories, thereby improving image processing capabilities and annotation accuracy. By calculating entropy values ​​to filter data and using self-training iterative logic, manual intervention is reduced, lowering annotation and maintenance costs and meeting the power grid's requirements for image recognition accuracy and automation.

[0072] Example 2, refer to Figure 1 This is the second embodiment of the present invention, which provides a method for fine annotation of power grid aerospace images based on SAM semantic segmentation.

[0073] In this embodiment of the application, the acquisition of satellite images and drone inspection images of the power grid system in step S100 includes: acquiring satellite images through remote sensing satellites and acquiring drone inspection images through drones;

[0074] Specifically, acquiring satellite imagery through remote sensing satellites includes obtaining satellite imagery of the power grid coverage area through remote sensing satellite service platforms (such as the Gaofen series satellites and Sentinel satellites) or professional geographic information service providers. This requires coverage of 220kV and above transmission line corridors and a 1km radius around 500kV and above substations.

[0075] The acquired satellite images have a spatial resolution of no less than 1m and a spectral range that includes both visible and near-infrared bands, making it easy to distinguish between vegetation and metal equipment. The shooting time should be a clear day with less than 10% cloud cover to avoid obstruction affecting subsequent annotation.

[0076] The acquired satellite images are preprocessed by removing invalid images with cloud cover exceeding 10% or severe tilt through data cleaning. Gaussian filtering is then used to denoise the remaining images, smoothing out minor noise caused by atmospheric particle scattering. Subsequently, contrast stretching is used for enhancement to improve the grayscale differences between metal equipment, vegetation, and the sky. Finally, using the latitude and longitude coordinates of the power grid equipment as reference control points, a polynomial fitting algorithm is used to establish the mapping relationship between the pixel coordinates of the satellite images and the actual geographic coordinates, eliminating geometric distortions caused by satellite attitude deviations or Earth curvature. Ultimately, the geographic coordinate error of the satellite images is controlled within 3 pixels, completing the image preprocessing.

[0077] Specifically, drone-based imagery collection includes using multi-rotor drones, such as the DJI Matrice 350RTK, equipped with high-definition visible light cameras and infrared thermal imagers with a temperature measurement range of -20℃ to 150℃. Flight routes are planned based on the distribution of power grid equipment: for transmission lines, routes are planned along the line direction and parallel to the conductors, with a flight altitude of 50-80m above the top of the towers; for substations, routes are planned in a circular and vertical overhead pattern, covering key equipment such as main transformers, circuit breakers, and insulators within the substation.

[0078] During the drone's flight, the real-time positioning function is enabled, the positioning accuracy is set to ≤1m, and visible light video stream and infrared video stream are collected in real time, with the video frame rate set to 25fps; when encountering strong winds ≥6m / s or severe weather such as thunderstorms, the collection is suspended to avoid blurry drone patrol images and safety risks to the collection equipment.

[0079] In this embodiment of the application, obtaining the initial device label in step S200 includes the following steps A1-A2:

[0080] A1: Segment satellite images using a semantic segmentation model to obtain a segmentation mask;

[0081] In this embodiment, the semantic segmentation model adopts the SAM model. Satellite imagery covering a 10km transmission line corridor is selected, with a resolution of 0.5m. The satellite imagery is cropped into 512×512 pixel sub-images. The SAM model is used to segment the satellite imagery: for each sub-image, three types of text prompts are input: transmission tower, substation, and line corridor. One to two center points of each potential equipment area are marked as spatial prompts. The SAM model generates an initial mask based on the prompt information, and then removes overlapping masks through non-maximum suppression, retaining segmentation results with a confidence level ≥0.85. Segmentation masks for transmission tower, substation, and line corridor are obtained. The segmentation mask for transmission tower outlines the tower body and crossarm contours, the segmentation mask for substation covers the perimeter wall of the substation area, and the segmentation mask for line corridor forms a 15m wide area along the conductor direction. The segmentation mask is accompanied by corresponding equipment category labels and pixel coordinate information.

[0082] In one alternative implementation, the semantic segmentation model can also employ the U-Net model. Satellite imagery covering a 10km transmission line corridor with a resolution of 0.5m is selected and cropped into 512×512 pixel sub-images. When segmenting using the U-Net model, for each sub-image, 1-2 points are manually marked within the approximate area of ​​each of the three target types: transmission towers, substations, and line corridors, as location guidance. The U-Net model has a symmetrical encoder-decoder structure. The encoder gradually extracts deep image features through convolution and pooling operations, while the decoder utilizes deconvolution and skipping operations. The leap connection fuses features from different levels of the encoder to restore image resolution. During segmentation, the model classifies each pixel in the image based on the input markers and the feature patterns it has learned. After initial segmentation, morphological operations, such as opening operations to remove small noise and closing operations to fill small holes, are used to optimize the segmentation results. Finally, clear segmentation masks for transmission towers, substations, and line corridors are obtained. The transmission tower mask outlines the tower body and crossarm contours, the substation mask completely covers the perimeter wall of the substation, and the line corridor mask forms a 15m wide strip along the conductor direction.

[0083] In another alternative implementation, the semantic segmentation model can also employ the YOLOv5 model. Satellite imagery covering a 10km transmission line corridor with a resolution of 0.5m is selected and cropped into 512×512 pixel sub-images. To adapt to the semantic segmentation task, the YOLOv5 model is improved by adding a convolutional layer for pixel-level classification in the network header. For each sub-image, three types of textual cues—transmission tower, substation, and line corridor—are input, and 1-2 center points for each potential equipment area are marked as spatial cues. The YOLOv5 model uses Mosaic data augmentation at the input end, scaling and stitching multiple randomly cropped sub-images to enrich the training data. By training on data features, the model's learning ability for devices in different scenarios is improved. In the network backbone, the CSP structure reduces the amount of computation while enhancing the feature extraction capability, effectively capturing the unique structure of transmission towers, the layout of substations, and the linear features of line corridors. In the detection head, the improved structure of the YOLOv5 model performs category prediction for each pixel, generating preliminary segmentation results. Non-maximum suppression is used to remove overlapping masks, and segmentation results with a confidence level ≥0.85 are retained. The final transmission tower segmentation mask can outline the tower body and crossarm contours, the substation segmentation mask covers the perimeter of the substation area, and the line corridor segmentation mask forms a 15m wide area along the conductor direction.

[0084] A2: Fill the internal holes of the segmentation mask with morphological closing operations, and then smooth the boundaries of the segmentation mask with morphological opening operations to obtain the initial device annotation.

[0085] In this embodiment, a 5×5 pixel rectangular structural element is used to adapt to the columnar features of the transmission tower. Morphological closing operations are iterated on the segmentation mask. When performing morphological closing operations on the segmentation mask of the transmission tower, the foreground pixels at the edges of the internal holes are first expanded outward through dilation to cover the hole area. Then, the erosion operation is used to shrink it to a reasonable range of the original segmentation mask, filling the holes with a diameter ≤ 8 pixels inside the tower body, such as the small gaps at the connection between the crossarm and the tower body, so that the main body area of ​​the transmission tower forms a complete connected domain. Subsequently, morphological opening operations are performed: a 3×3 pixel circular structure is used. First, the erosion operation is used to eliminate the jagged protrusions with a width ≤ 3 pixels on the edge of the tower base. Then, the dilation operation is used to smooth the boundary and restore it to a reasonable range of the original segmentation mask outline, smoothing the outline of the segmentation mask. The resulting initial equipment annotation can outline the complete outline of the transmission tower from the tower base to the top.

[0086] In one optional implementation, the initial device label obtained by segmentation mask can be filled with internal holes in the segmentation mask using a region growing algorithm, and the boundaries of the segmentation mask can be smoothed using a Gaussian filtering algorithm. For internal holes in the segmentation mask caused by occlusion, the foreground pixels around the holes are used as seed points, and a gray-level difference threshold of ≤5 is set. The holes are gradually filled using a region growing algorithm. Then, a 5×5 Gaussian filter is applied to the mask boundary, with a standard deviation of σ = 1.5. The smoothing properties of the Gaussian function are used to eliminate edge jaggedness. For example, after processing the mask of a certain transmission tower, holes with a diameter of ≤8 pixels inside the tower body are completely filled, and the gray-level transition of the boundary pixels is continuous.

[0087] In another alternative implementation, the initial device label obtained by segmenting the mask can be filled with holes inside the segmenting mask through connected component analysis. The boundary of the segmenting mask is smoothed by a bilateral filtering algorithm. Connected component analysis is performed on the segmenting mask, and connected components with an area of ​​less than 10 pixels are identified as holes and filled with the pixel value of the largest connected component around them, i.e., the main body of the device. Then, bilateral filtering is used with a spatial sigma of 3 and a grayscale sigma of 20 to process the boundary. While preserving the overall outline of the boundary, small protrusions are removed. For example, when processing the substation mask, small holes inside the substation wall are successfully filled, and the smoothness of the wall boundary is improved with no obvious pixel jumps.

[0088] In this embodiment of the application, the step S300 of generating the probability distribution of power grid equipment categories using feature data from satellite imagery includes the following steps B1-B3:

[0089] B1: Extract feature data from satellite imagery;

[0090] In this embodiment of the application, spectral features, spatial features, and location features of satellite imagery are extracted;

[0091] Specifically, multispectral band analysis is used to extract the reflectance characteristics of transmission towers and substation areas in satellite imagery. The average gray value of metal equipment such as poles and substation structures is calculated in the visible light band of 400-760nm, with the average gray value set in the range of 180-220, which is higher than the gray value range of 60-100 for vegetation. The vegetation coverage parameter is calculated in the near-infrared band of 760-2500nm, and the normalized vegetation index of vegetation in the transmission line corridor is calculated. When the vegetation index is ≤0.3, it is judged as a low vegetation interference area, and when the vegetation index is >0.3, it is judged as a high vegetation interference area, forming a spectral feature vector.

[0092] Spatial features of power grid equipment are extracted through edge detection and morphological analysis. For the transmission tower area, the coordinates of the connection point between the crossarm and the tower body and the continuity of the vertical edge are obtained, where a continuous tower body is defined as having ≥50 consecutive pixels. For the substation area, the rectangular outline parameters of the station area wall (such as length, width, and corner coordinates) and the density of buildings are extracted to form spatial features.

[0093] By combining image geographic coordinates to extract spatial correlation features, the straight-line distance between the transmission tower and adjacent towers, and the relative position with roads or rivers are obtained. The typical straight-line distance of a 220kV transmission tower is 300-500m, and the relative position with roads or rivers is about 50m from the edge of the road and about 100m from the edge of the river. The connection relationship between the substation and the surrounding transmission towers (e.g., the number of towers and their corresponding coordinates within 1km of the substation area) and the shortest path with the main road (e.g., the substation is 2km away from the main road) are calculated to form location features.

[0094] B2: Weighted fusion of feature data to generate comprehensive features;

[0095] In this embodiment of the application, the obtained spectral features, spatial features, and location features are assigned different weights based on the contribution analysis of power grid equipment classification. The spectral features, spatial features, and location features are then weighted and fused according to the assigned weights to generate a comprehensive feature.

[0096] The weighted fusion formula for spectral features, spatial features, and location features based on the assigned weights is as follows:

[0097] F = w1·F1 + w2·F2 + w3·F3

[0098] Where F is the comprehensive feature, F1 is the spectral feature vector, F2 is the spatial feature vector, F3 is the position feature vector, w1 is the weight of the spectral feature vector, w2 is the weight of the spatial feature vector, and w3 is the weight of the position feature vector.

[0099] In the embodiments of this application, taking the comprehensive characteristics of a substation as an example, the spectral feature vector is [0.82, 0.65, 0.3], where 0.82 is the visible light reflectance, 0.65 is the near-infrared reflectance, and 0.3 is the vegetation disturbance degree; the spatial feature vector is [0.9, 0.85, 0.78], where 0.9 is the triangular topological integrity, 0.85 is the vertical edge continuity, and 0.78 is the accuracy of the connection point between the crossarm and the tower; the positional feature vector is [0.8, 0.7, 0.95], where 0.8 is the distance matching degree with adjacent towers, 0.7 is the distance from the road, and 0.95 is the distribution degree along the line corridor.

[0100] Based on the contribution analysis of power grid equipment classification, spectral features can quickly distinguish building materials from vegetation and farmland, excluding most non-equipment areas, and are the core screening basis with the highest contribution. Spatial structure features can identify equipment types, with the second highest contribution. Geographic topology features only assist in verification, so their contribution is the lowest. Finally, the weight of spectral features was set to 0.4, the weight of spatial features to 0.35, and the weight of location features to 0.25. The comprehensive features were calculated using a weighted fusion formula as [0.843, 0.7325, 0.6305].

[0101] B3: Generate the probability distribution of power grid equipment categories in the power grid system based on comprehensive features.

[0102] In this embodiment of the application, a 100×100 pixel grid cell in the image is used as the analysis unit, corresponding to an actual geographical area of ​​50m×50m. Based on comprehensive features, the probability distribution of "transmission tower", "substation" and "non-grid features" is calculated and output:

[0103] The calculation based on comprehensive characteristics includes:

[0104] The formula used to calculate the linear prediction value is as follows:

[0105]

[0106] in, Category C k The linear prediction value, w k1 w k2 w k3 The model weights for spectral features, spatial features, and location features are respectively, b k x1, x2, and x3 are the quantized values ​​of spectral features, spatial features, and location features, respectively, obtained through comprehensive feature analysis.

[0107] The probability distribution of power grid equipment categories is then calculated using the following probability distribution calculation formula:

[0108]

[0109] Among them, P(C k |x) represents the comprehensive feature vector x, belonging to C k The probability of the class, C k The weights corresponding to the target category;

[0110] In this embodiment, the comprehensive feature vector of a certain region is [0.843, 0.7325, 0.6305]. Category C1 is a transmission tower, with weights of spectral features, spatial features, and location features of 1.8, 1.5, and 0.6, respectively, and a bias term of -1.2; Category C2 is a substation, with weights of spectral features, spatial features, and location features of 1.2, 0.8, and 1, respectively, and a bias term of -1.8; Category C3 is a non-grid target, with weights of spectral features, spatial features, and location features of 0.3, 0.2, and 0.4, respectively, and a bias term of -0.5.

[0111] Substituting into the linear prediction formula, the linear prediction value for the transmission tower is 1.79445, the linear prediction value for the substation is 0.4281, and the linear prediction value for the non-grid target is 0.1516.

[0112] Substituting these values ​​into the probability distribution calculation formula, we obtain the probability distribution for transmission towers as 0.92, the probability distribution for substations as 0.05, and the probability distribution for non-grid targets as 0.03.

[0113] In this embodiment of the application, the optimization of the initial equipment labeling in step S300 based on the probability distribution of power grid equipment categories includes the following steps C1-C3:

[0114] C1: Obtain the probability distribution of equipment features and corresponding power grid equipment categories in satellite imagery;

[0115] In this embodiment of the application, features are extracted from the rectangular substation area with a substation probability of 0.9 and the transmission tower grid with a transmission tower probability of 0.7-0.8. The equipment features include the extraction of the average grayscale value of concrete buildings with a value of 170 and the rectangular outline axis ratio of 1.5 from the substation area, and the extraction of the average metal reflectivity value of 190 and the triangular topological vertex spacing of 80m from the transmission tower grid.

[0116] Distribution characteristics include 4-5 transmission tower grids within 1km of each substation, with a grid spacing of 320-480m along the corridor. The substation and transmission tower areas do not overlap with non-grid targets (farmland, grayscale 50-70).

[0117] C2: Calculate the matching degree of device features, and assign corresponding feature weights to the corresponding device features based on the matching degree of device features;

[0118] In this embodiment of the application, the characteristic template of the substation is set as follows: the average gray value of the concrete building is ∈ [160, 180] and the axial ratio of the rectangular outline is ∈ [1.2, 1.8].

[0119] The characteristic template of the transmission tower is set as follows: the mean metal reflectivity ∈ [180, 200], and the vertex spacing of the triangular topology ∈ [70m, 90m].

[0120] The matching degree between the actual equipment features and the template is calculated using the following formula:

[0121]

[0122] Where S is the matching degree, A is the actual feature value, M is the median value of the template feature, X is the maximum value of the template feature, and Y is the minimum value of the template feature;

[0123] Feature weights are calculated using the following formula:

[0124]

[0125] Among them, S i,k W represents the matching degree of the i-th feature of the k-th type of device. i,k Let be the weight of the i-th feature of the k-th type of device, where k is the device category identifier, i and j are the feature numbers within the same type of device, and n is the total number of features of the k-th type of device;

[0126] During a certain substation feature matching process, the actual grayscale value of the concrete building was 173, and the median value of the template was (160+180) / 2=170. Substituting these values ​​into the matching degree calculation formula, the grayscale matching degree of the substation was found to be 0.85.

[0127] During a substation feature matching process, the actual value of the rectangular profile axis ratio was 1.65, and the median value of the template was (1.2+1.8) / 2=1.5. Substituting these values ​​into the matching degree calculation formula, the substation profile axis ratio matching degree was found to be 0.75.

[0128] Substituting into the feature weight calculation formula, the grayscale weight of the concrete building is 0.85 / (0.85+0.75)=0.53; the aspect ratio weight of the rectangular outline is 0.75 / (0.85+0.75)=0.47;

[0129] During a certain transmission tower feature matching process, the true value of the metal reflectivity was 185, and the median value of the template was (180+200) / 2=190. Substituting these values ​​into the matching degree calculation formula, the transmission tower reflectivity matching degree was found to be 0.75.

[0130] During a certain transmission tower feature matching, the true value of the vertex spacing of the triangular topology was 86, and the median value of the template was (70+90) / 2=80. Substituting these values ​​into the matching degree calculation formula, the vertex matching degree of the transmission tower topology was found to be 0.7.

[0131] Substituting into the feature weight calculation formula, the weight of metallic reflectivity is 0.75 / (0.75+0.7)=0.52; the weight of rectangular profile axis ratio is 0.7 / (0.85+0.75)=0.48.

[0132] C3: Calculate the matching score of the initial device annotations based on the feature weights, and filter the valid annotation data in the initial device annotations based on the matching score;

[0133] In this embodiment of the application, the initial equipment markings include 2 substation markings and 15 transmission tower markings. The concrete grayscale matching degree of the substation markings is 0.85 and 0.68, and the rectangular axis ratio matching degree is 0.75 and 0.52, respectively. Three of the 15 transmission tower markings are extracted, and the metal reflectivity matching degree of these three markings is 0.88, 0.52 and 0.76, respectively, and the triangle vertex spacing matching degree is 0.78, 0.45 and 0.69, respectively.

[0134] The matching score is calculated using the following formula:

[0135] G=Σ(M i ×W i )

[0136] Where G is the matching score, M i W represents the matching degree of the i-th feature. i The weight of the i-th feature;

[0137] The matching scores were calculated using the matching score calculation formula. The matching scores for the two substations were 0.85×0.53+0.75×0.47=0.803 and 0.68×0.53+0.52×0.47=0.6048, respectively. The matching scores for the three transmission towers at the sampling points were 0.88×0.52+0.78×0.48=0.832, 0.52×0.52+0.45×0.48=0.4846 and 0.76×0.52+0.69×0.48=0.7264, respectively.

[0138] If the matching score is lower than 0.74, the substation label with a matching score of 0.6048 and the transmission tower label with a matching score of 0.4846 will be removed.

[0139] C4: Correct the valid labeled data according to the distribution characteristics to obtain the final equipment label.

[0140] In this embodiment, the distribution characteristics are set as follows: there are 4-5 transmission towers within 1 kilometer of the substation; the distance between the transmission towers is 320-480 meters. In the valid labeling data, there are only 3 transmission tower labels near the substation. The distance between a transmission tower 1 kilometer away from the substation is 1050 meters, which is only 1.50 meters more than 1 kilometer. Therefore, the labeling of the substation should be adjusted according to the location of the transmission tower 1 kilometer away to make it conform to the distribution characteristics of 4-5 transmission towers within 1 kilometer of the substation. In the valid labeling data, the distance between two transmission towers is 510 meters. Moving the label of one of the transmission towers 40 meters along the corridor will make the distance between the transmission towers meet the requirement of 320-480 meters.

[0141] In this embodiment of the application, step S400, which involves collecting defect information of the power grid system to generate a defect report, includes collecting defect information of equipment such as transmission lines and substations through channels such as manual inspection record filling of the power grid operation and maintenance system, data feedback from drone inspections, and alarms from online monitoring devices. This information includes insulator damage, conductor strand breakage, abnormal equipment noise, etc. The occurrence time, line, equipment number, specific location, defect type, and severity description of each defect are recorded and included in a standardized format, containing a location field and a defect details field. The defect report is then integrated and generated.

[0142] In this embodiment of the application, the annotation of the air patrol image by combining the semantic segmentation model with the defect report in step S400 includes the following steps D1-D4:

[0143] D1: Obtain the report location information from the defect report, classify the defect report according to the report location information, and obtain the classified defects;

[0144] In this application embodiment, the precise location coordinates corresponding to each defect are extracted from the defect report, and the coordinates are used as the core classification basis to group defects within a similar coordinate range, such as the same tower or adjacent 2-meter range, into one category, forming a classification defect aggregated by spatial location, so that the scattered defect information is grouped in an orderly manner according to geographical coordinates, laying the foundation for subsequent labeling.

[0145] Specifically, the defect information obtained from the defect report includes: Defect 1: The third insulator from the top in the insulator string of the tower with coordinates (118.2325°E, 31.5618°N), voltage of 220kV, and number 12, is damaged; Defect 2: The fifth insulator from the top in the insulator string of the tower with coordinates (118.2687°E, 31.5892°N), voltage of 220kV, and number 28, is damaged; Defect 3: The second insulator from the top in the insulator string of the tower with coordinates (118.3051°E, 31.6176°N), voltage of 220kV, and number 45, is damaged.

[0146] D2: Obtain the image location information of the aerial patrol images, and register the defect report with the aerial patrol images by matching the report location information with the image location information;

[0147] In this embodiment of the application, the precise image location information corresponding to each frame of the drone-captured inspection image metadata is extracted, including the latitude and longitude coordinates and shooting range at the time of shooting. Then, the spatial coordinates of the location information of each defect report in the defect report are compared with the image location information. When the location coordinates of the defect report fall within the shooting range of a certain frame of drone inspection image and the coordinate deviation is ≤2 meters, the registration of the defect report with the corresponding drone inspection image is completed, ensuring that each defect can be accurately associated with the drone inspection image containing its defect area.

[0148] Specifically, the 5-minute machine patrol video is broken down into 9000 still images, named according to timestamp and frame number. The position information of each frame is extracted, and the coordinates of each defect are checked against the coverage area of ​​the video frame one by one. The target frame containing the defect coordinates is found. The coordinates of defect 1 are (118.2327°E, 31.5620°N). The coverage area of ​​a certain target frame in the video is (118.2320°-118.2332°E, 31.5613°-31.5625°N). The coordinates of defect 1 fall completely within the coverage area of ​​a certain target frame, thus confirming that defect 1 is aligned with a certain target frame. In the aligned target frame, the specific area corresponding to the defect is found, and the defect information is bound to the corresponding specific area.

[0149] D3: Associate the classified defects with the registered machine-guided images to obtain defect images;

[0150] In this embodiment of the application, after completing the registration of the classified defects and the inspection images, the detailed information of each type of defect is further associated with the corresponding registered inspection images: First, the core attributes of each defect are extracted from the classified defect data (e.g., the voltage of defect 1 is 220kV and the third insulator on the number 12 tower is damaged; the voltage of defect 2 is 220kV and the fifth insulator on the number 28 tower is damaged). Then, these attribute information are bound to the metadata of the registered inspection images. The metadata includes frame number, timestamp, and shooting coordinates. The image area corresponding to the defect is marked in the registered image, and a defect image containing defect attributes and image location is generated. The defect image not only retains the original image details of the inspection images, but also includes defect association information.

[0151] Specifically, the registered defects 1, 5280, 5280, and 8130 are configured for registration. Core attributes are extracted from the categorized defect data: Defect 1 is the 3rd damaged insulator string on tower number 12 (220kV); Defect 2 is the 5th damaged insulator string on tower number 28 (220kV); and Defect 3 is the 2nd damaged insulator string on tower number 45 (220kV). These attribute information is then bound to the metadata of the corresponding registered images. Defect 1 is associated with the metadata of frame 1560, including frame number 1560, timestamp 00:01:42, and shooting coordinates 118.2326°E, 31.5619°N. Defect 2 is associated with the metadata of frame 5280, including frame number 5280, timestamp 00:02:56, and shooting coordinates 118.26°E, 31.5619°N. Metadata for frame 8130 associated with defect 3 (85°E, 31.5890°N): Frame number 8130, timestamp 00:04:24, shooting coordinates 118.3050°E, 31.6175°N. A defect image is generated. This image retains the original details of the tower insulators from the machine-generated image, while also including corresponding defect attributes and image location information. Defect attributes include voltage, tower number, component location, and defect type. Image location information includes frame number, timestamp, and shooting coordinates. For example, the defect image for defect 1, in addition to a clear image of the insulator string on tower number 12, also simultaneously displays the 3rd insulator from the top of tower number 12 (voltage 220kV) that is damaged, as well as relevant information from frame 1560 of the machine-generated image (timestamp 00:01:42, 118.2326°E, coordinates 31.5619°N).

[0152] D4: Defect annotations are obtained by annotating the defect images using a semantic segmentation model.

[0153] In this embodiment, the semantic segmentation model adopts the SAM model. The defect image is input into the SAM model for annotation. For the defect image A obtained after registration of defect 1, the irregular area where the crack is located is automatically segmented, a red pixel-level mask is generated, and the label "insulator damage (crack type)" is output. For the defect image B obtained after registration of defect 2, the incomplete area with edge damage is segmented and the label "insulator damage (edge ​​type)" is output. For the defect image C obtained after registration of defect 3, the hole area with through damage is completely segmented and the label "insulator damage (through type)" is output.

[0154] In this embodiment of the application, step S400, which involves labeling the defect image using a semantic segmentation model, includes the following steps E1-E5:

[0155] E1: Divide the defective image into segments, extract one frame every set number of frames as a key frame, and the remaining frames that are not extracted are non-key frames.

[0156] In this embodiment of the application, the generated defect image is divided into time dimensions, and key frames are selected according to the set rule of extracting 1 frame every 5 frames: starting from the first frame of the defect image, the first frame, the sixth frame, the eleventh frame, and so on are extracted. These extracted frames are key frames, which retain their complete defect details and related information; while the frames that are not extracted, such as the second to fifth frames, the seventh to tenth frames, and the twelfth to fourteenth frames, are non-key frames.

[0157] It should be noted that this division not only preserves representative core images in the defective image in the form of keyframes, but also reduces the amount of redundant data processing for non-keyframes.

[0158] In one optional implementation, the defect image can be segmented by extracting keyframes based on the clarity of the defect area. For the generated defect image, the clarity index of the defect area is calculated frame by frame. The gradient value of the defect area is calculated using the Laplacian operator. The higher the gradient value, the clearer the edge of the area and the more complete the defect details. A gradient value threshold of ≥80 is set as a high-resolution frame. All high-resolution frames are selected from the image sequence. Then, adjacent high-resolution frames are deduplicated by time interval to finally determine the keyframes. The rest are non-keyframes. The frame with the clearest defect details is retained first to avoid missing clear defect frames at fixed intervals.

[0159] In another optional implementation, the defect image can be segmented by extracting key frames based on changes in equipment position. Combining the positional characteristics of equipment in the power grid defect image, the displacement vector of equipment pixels between adjacent frames is calculated using the optical flow method. For each frame, the pixel coordinates of the core area of ​​the equipment are extracted, and the displacement distance with the previous frame is calculated. Frames with a displacement distance ≥ 5 pixels are defined as frames with significant changes in equipment position, and such frames are selected from the image sequence as key frames. If the displacement distance of multiple consecutive frames is < 5 pixels, only the first frame is retained as a key frame, and the rest are set as non-key frames. The key frame extraction rhythm is dynamically adjusted according to the equipment position to retain the complete defect image after the equipment position changes, avoid redundant extraction of frames with unchanged positions, and reduce displacement calculation errors of non-key frames.

[0160] E2: Input the keyframes into the semantic segmentation model for annotation to obtain the keyframe mask;

[0161] In this embodiment, key frames such as the 1st, 6th, and 11th frames of the defect image are input into the SAM model. The SAM model extracts the global features of the insulator in the key frames through an image encoder, quickly locates the main outline of the insulator and eliminates background interference. Then, for areas with abnormal grayscale and texture breaks in the image, a key frame mask is generated through a mask decoder. For the 1st key frame, a red mask adapted to the initial view of the crack is generated, covering the entire length of the crack. For the 6th key frame, due to the rightward shift of the shooting angle, the SAM model adjusts the mask boundary to ensure complete fit with the shifted crack area. When the angle of the 11th key frame shifts downward, the mask synchronously adapts to the vertical position change of the crack to obtain the key frame mask. The key frame mask includes the crack details in each key frame and also includes a category label for the crack-type damage, providing an accurate reference template for the generation of subsequent non-key frame masks.

[0162] E3: Calculate the pixel displacement vector between keyframes and non-keyframes using optical flow;

[0163] In this embodiment, the Farneback dense optical flow algorithm is used to calculate the pixel displacement vector between keyframes and non-keyframes. An image pyramid structure is constructed for keyframe 1 and non-keyframe 2 respectively. The image pyramid contains multiple layers of images with different resolutions, starting from the original image. The size of each layer is half that of the layer above (e.g., if the original image is 200×200 pixels, the next layer is 100×100 pixels). For example, when constructing the image pyramid of the first frame, the bottom layer is the original 200×200 pixel image, and the next layer is a 100×100 pixel image obtained through bilinear interpolation or other methods.

[0164] At each level of the image pyramid, for each pixel in keyframe 1 and non-keyframe 2, a quadratic polynomial is used to approximate its local neighborhood. By least-squares weighted fitting of the neighborhood pixel information, polynomial coefficients are obtained to describe the relationship between the pixel and its neighborhood. Taking the pixel with coordinates (x,y) in keyframe 1 as an example, the coefficients of each term in the quadratic polynomial are determined by weighted fitting of the gray values ​​of 25 pixels in its 5×5 neighborhood, thereby describing the variation pattern of the pixel in the neighborhood. The same operation is performed on the corresponding pixels in non-keyframe 2.

[0165] According to the optical flow constraint equation:

[0166] I(x,y,t)=I(x+Δx,y+Δy,t+Δt)

[0167] Where I(x,y,t) is the gray value of the pixel at coordinate (x,y) in keyframe 1 at time t, I(x+Δx,y+Δy,t+Δy) is the gray value of the corresponding pixel in non-keyframe 2 at time t+Δt, and Δx and Δy are the displacements of the pixel in the x and y directions, respectively.

[0168] The optical flow constraint equation is linearized, and the coefficients obtained by polynomial approximation are substituted into it. Δx and Δy are solved by iterative optimization. Assuming that the gray value of a certain pixel in keyframe 1 is 120, and the gray values ​​of the pixels around the corresponding position in non-keyframe 2 and the polynomial coefficients are known, after multiple iterations, it is found that the pixel has a displacement of +2 pixels in the x direction and a displacement of 0 pixels in the y direction, that is, the displacement vector (+2,0) of the pixel is obtained.

[0169] By traversing all pixels in each layer of the image pyramid of keyframe 1, the displacement vector (Δx, Δy) of each pixel relative to the corresponding pixel in non-keyframe 2 is calculated through the above steps. These displacement vectors constitute the dense optical flow field between keyframe 1 and non-keyframe 2, describing the motion information of each pixel between the two frames. For example, in a certain layer of keyframe 1, pixels at different positions may obtain different displacement vectors such as (+3, 0) and (0, -1), reflecting the differences in pixel motion caused by factors such as drone movement and slight swaying of insulators. Finally, the overall set of pixel displacement vectors of non-keyframe 2 relative to keyframe 1 is obtained.

[0170] E4: Pass the keyframe mask along the pixel displacement vector to the non-keyframe position to generate the initial mask for the non-keyframe.

[0171] In this embodiment, all pixel coordinates belonging to the defect region are selected from the keyframe mask and denoted as the keyframe mask pixel set. Based on the calculation results of the Farneback algorithm, the displacement vector of each pixel in the non-keyframe corresponding to the keyframe mask pixel set is obtained. The corresponding position of each pixel in the non-keyframe is calculated for each pixel in the keyframe mask pixel set. All target coordinates in the non-keyframe are marked as defect regions to generate the initial mask of the non-keyframe.

[0172] Specifically, taking the generation of non-keyframes 2-5 from keyframe 1 in the defect image sequence of tower No. 12 as an example, the known information includes: the basic information of keyframe 1 is a resolution of 200×200 pixels, the defect is "insulator crack type damage", the keyframe mask pixel set covers the crack area, the core coordinate range is X = 85-115 pixels, Y = 90-120 pixels, a total of 31*31 = 961 mask pixels, and the keyframe mask pixel set is denoted as S = {(85,90),(86,90),...,(1 15,120)}, The pixel displacement vectors of non-keyframes 2-5 relative to keyframe 1 are all horizontally shifted to the right. Because the drone slowly moves to the right to shoot, and the displacement vectors of all mask pixels in the same non-keyframe are consistent, the specific displacement data are as follows: The displacement vector of non-keyframe 2 relative to keyframe 1 is (+2, 0), the displacement vector of non-keyframe 3 relative to keyframe 1 is (+4, 0), the displacement vector of non-keyframe 4 relative to keyframe 1 is (+6, 0), and the displacement vector of non-keyframe 5 relative to keyframe 1 is (+8, 0).

[0173] The initial mask generation for non-keyframe 2 includes: extracting the mask pixel set S = {(85,90),(86,90),...,(115,120)} from keyframe 1; matching the displacement vector of non-keyframe 2: the displacement of all mask pixels is (+2, 0); calculating the target coordinates: taking the boundary pixels of the keyframe mask as an example, the target coordinates of (85,90) are (85+2,90+0), i.e., (87,90); similarly, the target coordinates of (115,120) are (117,120), and all mask pixels in the middle are translated according to this rule; marking all target coordinates in non-keyframe 2, obtaining the core range of the initial mask of non-keyframe as X = 87-117 pixels, Y = 90-120 pixels, retaining the length and shape of the crack in keyframe 1, only adjusting the position synchronously with the rightward movement of pixels;

[0174] Similarly, the displacement vector of non-keyframe 3 is (+4,0). After translating the keyframe mask (85, 90), it becomes (89, 90). After translating (115, 120), it becomes (119, 120). The core range of the initial mask of the non-keyframe is X = 89-119 pixels and Y = 90-120 pixels. The displacement vector of non-keyframe 4 is (+6,0). The core range of the initial mask of the non-keyframe is X = 91-121 pixels and Y = 90-120 pixels. The displacement vector of non-keyframe 5 is (+8,0). The core range of the initial mask of the non-keyframe is X = 93-123 pixels and Y = 90-120 pixels.

[0175] E5: Correct the initial mask of non-critical frames through temporal consistency verification and generate non-critical frame masks; match critical frames and non-critical frames according to temporal sequence to form defect annotations.

[0176] In this embodiment, when correcting the initial mask of a non-critical frame through temporal consistency verification, three simple criteria are set: position overlap, maximum distance of shape similarity, and grayscale matching brightness difference. Then, the initial mask of the non-critical frame is compared with the mask of the previous critical frame or the corrected non-critical frame one by one according to the shooting sequence. If all three criteria are met, the initial mask is directly used as the non-critical frame mask. If one criterion is not met, the problem area is corrected. If the position is offset, it is adjusted according to the movement range of the previous frame. If the bright area is marked incorrectly, the incorrect mark is deleted and the missing dark area is added to generate an accurate non-critical frame mask that is consistent with the previous and subsequent frames. All critical frames and non-critical frames are organized according to the shooting order to form defect annotations.

[0177] Specifically, the positional overlap requirement is that the overlap area between the non-keyframe mask and the previous keyframe or the corrected non-keyframe mask must be ≥90%; the maximum distance requirement for shape similarity is that the maximum distance between the non-keyframe mask and the previous keyframe or the corrected non-keyframe mask must be ≤3 pixels; the grayscale matching brightness difference requirement is that the difference between the non-keyframe mask and the previous keyframe or the corrected non-keyframe mask must be ≤10. If the positional overlap requirement is not met, the boundary of the previous frame mask should be referenced, and adjustments should be made according to the normal movement range (e.g., the previous...). If a frame shifts 2 pixels to the right, but the current frame shifts 4 pixels to the right, then shift 2 pixels to the left. If the maximum distance of shape similarity does not meet the requirements, first find the deformed edge and then refer to the shape ratio of the previous frame (for example, the previous frame is a thin strip 31 pixels long and 30 pixels wide), delete the redundant marks in the deformed area, or add missing marks (for example, if one side edge "shrinks in", add the mark according to the edge position of the previous frame). If the grayscale matching brightness difference does not meet the requirements, delete these erroneous marks and add the missing dark areas.

[0178] Taking the correction of the defect image of tower No. 12 as an example, the mask of keyframe 1 is known to be X = 85-115, Y = 90-120, with the thin stripe corresponding to the crack shape. The initial masks of non-keyframes 2-5 are incremented by shifting 2 pixels to the right. The brightness of a normal insulator is 135, and the brightness of a defect is 110. Non-keyframes 2-4 are directly compared with the previous frame. The overlap is ≥90%, the shape is not distorted, both are 31 pixels in the X direction and 30 pixels in the Y direction, and the brightness difference is ≤3. Therefore, the initial non-keyframe mask can be used directly. The mask of non-keyframe 2 is X = 87-117, Y = 90. -120; The initial mask of non-critical frame 5, X=93-123, Y=90-120, is compared with the previous non-critical frame 4, X=91-121. The brightness difference of 14 exceeds the grayscale matching brightness difference requirement. The problem is in the X=122-123 area, where the brightness is 132-135, the same as a normal insulator. Therefore, this is an incorrect mark. So delete the incorrect mark of X=122-123 and add the missing dark area of ​​X=92-93. After correction, the non-critical frame mask is X=92-121, Y=90-120. At this time, all three standards are met.

[0179] If the initial mask of a non-critical frame is X=95-125, Y=88-122, and compared with the previous frame X=92-121, Y=90-120, the shape similarity does not meet the standard. The width in the Y direction becomes 34 pixels, which is 4 pixels wider than the previous frame. The bottom Y=121-122 and top Y=88-89 areas are redundant deformed parts, and the brightness of these areas is 130-134, which is close to that of a normal insulator. Referring to the shape ratio of the previous frame Y=90-120, the redundant marks of Y=88-89 and Y=121-122 are deleted. At the same time, the X direction is adjusted to 94-124 by shifting 2 pixels to the right of the previous frame. After correction, the mask is X=94-124, Y=90-120. The shape returns to a thin strip, the maximum distance from the previous frame is 2 pixels, and the brightness difference is 2, which meets the requirements.

[0180] All keyframes and non-keyframes are organized in the order of shooting to form defect annotations.

[0181] In this embodiment of the application, step S500, training the semantic segmentation model through final device annotation, includes the following steps F1-F5:

[0182] F1: Using satellite imagery as input and final device annotations as the supervision target, the semantic segmentation model is trained.

[0183] In this embodiment, 5,000 satellite images and the final equipment annotations are divided into a training set and a validation set in an 8:2 ratio, input into the SAM model, with a learning rate of 0.001 and a batch size of 8, and trained for 50 rounds, with the parameters optimized using the cross-entropy loss function;

[0184] Specifically, taking the distinction between three types of equipment—towers, insulators, and conductors—as an example, towers, insulators, and conductors are designated as Category 1, Category 2, and Category 3, respectively. The SAM model output is converted into probability values ​​for the three types of equipment using the Softmax function, ensuring that the sum of the probability values ​​for all equipment is 1. The final equipment is then labeled and coded, indicating which category a pixel actually belongs to. The label for that category is 1, and for other categories it is 0.

[0185] The cross-entropy loss function used is as follows:

[0186]

[0187] in, For a single pixel loss, L is the average loss function of the current image, N is the total number of pixels in a single image, n is the total number of device categories, i is the pixel index, c is the device category index, and y is the pixel index. i,c For the final device label encoding of the i-th pixel in the c-th device category, p i,c The model predicts the probability of the i-th pixel in the c-th device category;

[0188] During model training, the average loss of the current batch of images is first calculated. If the average loss is greater than a set threshold, the gradient of the loss with respect to each model parameter is calculated through backpropagation. For example, if the convolutional kernel weights are w and the bias is b, the corresponding parameter gradient is... and The threshold is set to 0.4, which can be adjusted according to the training status of the SAM model. Gradient descent is used to adjust the corresponding parameters according to their gradients. For example, the formula for updating the parameters of the convolutional kernel weights is:

[0189]

[0190] Among them, w new Here are the updated convolutional kernel weights, and here are the original convolutional kernel weights. η is the gradient of the convolution kernel weights, and η is the learning rate;

[0191] Taking a 220kV satellite image of tower number 12 as an example, three types of equipment are labeled: tower = 1, insulator = 2, and conductor = 3. Each satellite image has a resolution of 200*200 pixels. Pixels A and B are selected for calculation. Pixel A is actually a tower, with true labels y1 = 1, y2 = 0, y3 = 0, and the SAM model's prediction probabilities are p1 = 0.7, p2 = 0.2, and p3 = 0.1. Pixel B is actually an insulator, with true labels y1 = 0, y2 = 1, y3 = 0, and the SAM model's prediction probabilities are p1 = 0.3, p2 = 0.5, and p3 = 0.2. The SAM model's training learning rate is set to 0.001.

[0192] The loss of a single pixel for pixel A is calculated as -log(0.7) = 0.1549 using the cross-entropy loss function formula; the loss of a single pixel for pixel B is calculated as -log(0.5) = 0.3010. Assuming that the average loss of a single pixel for other pixels is 0.2, the average loss function of the current image is calculated as (0.1549 + 0.3010 + 39998 * 0.2) / 40000 = 0.2 using the cross-entropy loss function formula.

[0193] The current weight of a certain convolutional kernel in the SAM model is set to 0.5, and the gradient of the loss with respect to that weight is 2. The updated convolutional kernel weight is calculated to be 0.5 - 0.001 * 2 = 0.498 using the convolutional kernel weight update formula.

[0194] F2: Device labeling is performed through a semantic segmentation model to obtain predicted device labels. The label entropy value of the predicted device labels is calculated, and the predicted device labels are classified according to the label entropy value.

[0195] In this embodiment, the model trained in step F1 is used to annotate 1000 new satellite images in the validation set, resulting in 1000 sets of predicted device annotations. These annotations include prediction masks for three types of devices and the class probability of each pixel. The pixel entropy value is calculated for the prediction mask of each image according to the pixel entropy value calculation formula. The pixel entropy value calculation formula is as follows:

[0196]

[0197] Where H is the pixel entropy value, p c The model predicts the probability value of the sample belonging to class c, where c is the class index. The average entropy value of the entire image is taken as the entropy value of the predicted label.

[0198] In this embodiment, the SAM model predicts the probability distribution of a pixel as follows: the probability of belonging to a tower is p1 = 0.8, the probability of belonging to an insulator is p2 = 0.1, and the probability of belonging to a conductor is p3 = 0.1; substituting these values ​​into the pixel entropy calculation formula, we get H = -((-0.2575) - (-0.3322) - (-0.3322)) = 0.92;

[0199] We define H > 0.8 as high-entropy prediction labels, 0.3 ≤ H ≤ 0.8 as medium-entropy prediction labels, and H < 0.3 as low-entropy prediction labels.

[0200] In an optional implementation, the classification of predicted equipment labels can also be achieved by calculating the mean of the maximum category probability. For each pixel in the predicted equipment label, the maximum predicted probability of it belonging to the three categories of pole, insulator, and conductor is extracted (e.g., the probability distribution of a certain pixel is 0.8, 0.1, and 0.1, with the maximum probability being 0.8). The mean of the maximum probability of all pixels in the entire image is calculated. A mean of ≥0.85 is set as a high-quality label, 0.6 ≤ mean <0.85 is set as a medium-quality label, and a mean <0.6 is set as a low-quality label. The label quality is directly measured by the confidence of the model prediction.

[0201] In another alternative implementation, the classification of predicted device labels can also be achieved by calculating the mask matching degree. For each image, the overlap degree is obtained by calculating the proportion of the number of overlapping pixels between the predicted mask and the final device label mask to the total number of pixels in the predicted mask. An overlap degree ≥ 0.8 is set as high-quality label, 0.5 ≤ overlap degree < 0.8 is set as medium-quality label, and overlap degree < 0.5 is set as low-quality label. Based on the existing final device labels as a benchmark, the deviation between the predicted results and the actual labels is directly quantified.

[0202] F3: Manually sample and verify the classified predicted equipment labels to obtain the verified equipment labels;

[0203] In this embodiment, sampling is carried out according to the principle of sampling more for high-entropy prediction labels and sampling less for low-entropy prediction labels, and the sampling prediction equipment labels are checked by power operation and maintenance personnel.

[0204] Specifically, if there are 250 high-entropy prediction labels, 450 medium-entropy prediction labels, and 300 low-entropy prediction labels in a certain annotation result, and 50%, 20%, and 5% are sampled from the high-entropy, medium-entropy, and low-entropy prediction labels respectively, then 125 high-entropy prediction labels, 90 medium-entropy prediction labels, and 15 low-entropy prediction labels will be sampled for verification.

[0205] F4: Merge the proofreading device annotations and the final device annotations to retrain the semantic segmentation model;

[0206] In this embodiment, 230 proofread equipment labels and corresponding satellite images obtained after proofreading are merged with 5,000 satellite images and corresponding final equipment labels, and then re-divided into training and validation sets according to an 8:2 ratio. These sets are then input into the SAM model for retraining.

[0207] F5: Evaluate the annotation performance of the semantic segmentation model. If the evaluation result meets the set evaluation threshold, save the semantic segmentation model.

[0208] If the evaluation results do not meet the set evaluation threshold, manual sampling and verification will be performed again, and the training will be repeated.

[0209] In this embodiment, 50 satellite images that were not used in training are selected for annotation, and the intersection-union ratio (IU / R) and accuracy of the device annotation are calculated based on the final device annotation. The formula for calculating the IU / R is as follows:

[0210]

[0211] Where IoU is the intersection-union ratio, |P∩T| represents the number of pixels correctly predicted by the SAM model, and |P∪T| represents the total number of pixels correctly predicted, incorrectly predicted, and actually existing but not predicted by the SAM model;

[0212] In a 200*200 pixel satellite image, the towers are labeled and evaluated. The final number of pixels belonging to towers in the equipment label is 1000, and the number of pixels predicted as towers is 1200. Among them, 800 pixels are correctly predicted. Then, the crossover ratio (IoU) can be calculated using the formula: IoU = 800 / (1000+1200-800) = 0.57.

[0213] The accuracy calculation formula is:

[0214]

[0215] Where AC is the accuracy, N is the total number of pixels in the satellite imagery, and p i t is the predicted category of the i-th pixel by the SAM model. i Let δ(p) be the true class of the i-th pixel. i ,t i ) is an indicator function, where p i =t i At that time, δ(p) i ,t i ) = 1, indicating correct labeling, p i ≠t i At that time, δ(p) i ,t i ) = 0 indicates an incorrect label.

[0216] Using a 200*200 pixel satellite image dataset, after pixel-by-pixel comparison model prediction and final device annotation, the number of correctly labeled pixels was found to be 35,000. The accuracy calculation formula AC = 35000 / 40000 = 87.5%.

[0217] In this embodiment, the evaluation threshold is set as follows: Intersection over Union (IoU) ≥ 0.85 and accuracy ≥ 90%. If the IoU and accuracy of the SAM prediction model trained earlier do not meet the requirements, the process returns to step F3, trains for another 20 rounds, and then evaluates again. The evaluation result is set as IoU ≥ 0.86 and accuracy ≥ 92%, which meets the set evaluation threshold. The semantic segmentation model trained in this round is then saved for subsequent automatic device annotation.

[0218] In this embodiment of the application, step S500, which trains the semantic segmentation model through defect annotation, includes the following steps G1-G5:

[0219] G1: Use machine-surveyed images as input and defect annotations as supervision targets to train the semantic segmentation model;

[0220] In this embodiment, a dataset of 3000 machine-inspection images of 220kV towers was selected, containing defects such as insulator damage and conductor strand breakage, with a resolution of 1920×1080 pixels. The machine-inspection images were used as input, and the corresponding defect labels were used as supervised targets for training the SAM model. The dataset was divided into a training set (2400 images) and a validation set (600 images) in an 8:2 ratio. A learning rate of 0.001 and a batch size of 8 were set, and the model was trained for 40 epochs, with parameters optimized using the cross-entropy loss function. Taking insulator damage as an example, "insulator damage" was designated as category 1, and "normal area" as category 2. The model output was converted into two probability values ​​(summing 1) using the Softmax function. Finally, the equipment labels used one-hot encoding (pixel labels for damaged areas were 1, and for normal areas, 0). In a tower crane inspection image, the model prediction probability of pixel C (actually a damaged insulator) is 0.7 (category 1) and 0.3 (category 2), and the loss of a single pixel is -log(0.7)≈0.1549; the average loss of the entire image is calculated to be 0.22. The initial weight of a certain convolution kernel in the model is 0.6, the gradient of the loss with respect to that weight is 1.8, and the updated weight is 0.6-0.001×1.8=0.5982.

[0221] G2: Defects are labeled using a semantic segmentation model to obtain predicted defect labels. The label entropy value of the predicted defect labels is calculated, and the predicted defect labels are classified according to the label entropy value.

[0222] In this embodiment, the model trained in step G1 is used to annotate 600 new aerial survey images in the validation set, resulting in 600 sets of predicted defect annotations (including defect prediction masks and pixel category probabilities). The average entropy value of each image is calculated using the pixel entropy formula, which is as follows: (c=1 for damaged, c=2 for normal). For example, if the predicted probability of a pixel is 0.8 (damaged) or 0.2 (normal), the entropy value H = -(0.8log20.8 + 0.2log20.2) ≈ 0.509. We set entropy values ​​> 0.6 as high entropy (e.g., defective images with vegetation obstruction, totaling 180 images), 0.3 ≤ entropy value ≤ 0.6 as medium entropy (e.g., slightly blurred defective images, totaling 270 images), and entropy values ​​< 0.3 as low entropy (e.g., clear, unobstructed defective images, totaling 150 images). This completes the classification.

[0223] G3: Manually sample and verify the classified predicted defect labels to obtain verified defect labels;

[0224] In this implementation, following the principle of sampling more high-entropy images and less low-entropy images, 50% (90 images) of high-entropy images, 20% (54 images) of medium-entropy images, and 5% (8 images) of low-entropy images were sampled, totaling 152 images, which were then handed over to two power maintenance personnel for verification. For example, in the high-entropy tower images, the originally predicted damaged area included some normal insulators. After verification, the erroneous mask was deleted and the missing damaged area was added. In the medium-entropy tower images, the blurred edges of the damaged mask boundaries were corrected, and finally 152 defect labels were obtained for verification.

[0225] G4: Merge the proofreading defect annotations and defect annotations and retrain the semantic segmentation model;

[0226] In this embodiment, 152 proofread defect annotations and corresponding machine patrol images are merged with the 3000 machine patrol images and defect annotations in step G1 to form a dataset of 3152 images. The dataset is then re-divided into a training set (2522 images) and a validation set (630 images) in an 8:2 ratio. The dataset is then input into the SAM model and trained again for 25 rounds with the learning rate adjusted to 0.0005. A hybrid loss function is used (new data weight 1.2, old data weight 0.8) to enhance the impact of the proofread data on the model.

[0227] G5: Evaluate the annotation performance of the semantic segmentation model. If the evaluation result meets the set evaluation threshold, save the semantic segmentation model.

[0228] In this embodiment, 40 untrained power grid inspection images are selected as a test set to calculate the Intersection over Union (IoU) and accuracy to evaluate the model's performance. In one tower image, 1200 pixels are marked as damaged, the model predicts 1350 damaged pixels, and 1080 pixels are correctly predicted, resulting in an IoU of 1080 / (1200+1350-1080) = 0.88. In the entire image of 40,000 pixels, 36,800 pixels are correctly marked, resulting in an accuracy of 36,800 / 40,000 = 92%. Evaluation thresholds are set as IoU ≥ 0.85 and accuracy ≥ 90%. If the evaluation results meet the thresholds, the SAM model is saved for automatic defect labeling in subsequent power grid inspection images.

[0229] Example 3, referring to Figure 1 This is the third embodiment of the present invention, which provides a fine-grained annotation system for power grid aerospace images based on SAM semantic segmentation, comprising:

[0230] Data acquisition module: Acquires satellite and machine inspection images of the power grid system, and collects defect information of power grid equipment;

[0231] First equipment annotation module: Annotates satellite images using a semantic segmentation model to obtain initial equipment annotations;

[0232] Data processing module: Generates probability distributions of power grid equipment categories using feature data from satellite imagery;

[0233] The second equipment labeling module optimizes the initial equipment labels by using the probability distribution of power grid equipment categories to obtain the final equipment labels;

[0234] Defect annotation module: Generates defect reports from defect information, and annotates the aerial patrol images using a semantic segmentation model in conjunction with the defect reports to obtain defect annotations;

[0235] Self-training module: Corrects the final device and defect labels to obtain corrected device and defect labels; trains the semantic segmentation model using the corrected device and defect labels.

[0236] Example 4, the fourth embodiment of the present invention, differs from the previous three embodiments in that: if the function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0237] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0238] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0239] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0240] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A power grid space image fine labeling method based on SAM semantic segmentation, characterized in that, The method comprises the following steps: collecting satellite images and machine patrol images of a power grid system; annotating the satellite images by using a semantic segmentation model to obtain initial device annotations; generating a probability distribution of power grid device categories from the satellite images, optimizing the initial device annotations based on the probability distribution of the power grid device categories, and obtaining final device annotations; collecting defect information of the power grid system to generate a defect report, annotating the machine patrol images by using the semantic segmentation model in combination with the defect report, and obtaining defect annotations; training the semantic segmentation model based on the final device annotations and the defect annotations.

2. The power grid space image fine labeling method based on SAM semantic segmentation of claim 1, wherein, The step of obtaining initial device annotations comprises the following steps: segmenting the satellite images by using the semantic segmentation model to obtain segmentation masks; filling internal holes of the segmentation masks by morphological closing operation, and then smoothing the boundaries of the segmentation masks by morphological opening operation to obtain initial device annotations.

3. The method of claim 2, wherein the method comprises: The step of generating a probability distribution of power grid device categories from the feature data of the satellite images comprises the following steps: extracting feature data from the satellite images; performing weighted fusion on the feature data to generate comprehensive features; generating a probability distribution of power grid device categories in the power grid system based on the comprehensive features.

4. The method of claim 3, wherein the method comprises: The step of optimizing the initial device annotations based on the probability distribution of the power grid device categories comprises the following steps: obtaining device features in the satellite images and a probability distribution of corresponding power grid device categories; calculating the matching degree of the device features, and assigning corresponding feature weights to the device features according to the matching degree of the device features; calculating the matching score of the initial device annotations according to the feature weights, and screening valid annotation data in the initial device annotations according to the matching score; correcting the valid annotation data based on the distribution features to obtain final device annotations.

5. The method of claim 4, wherein the method comprises: The step of annotating the machine patrol images by using the semantic segmentation model in combination with the defect report comprises the following steps: obtaining report position information in the defect report, classifying the defect report based on the report position information, and obtaining classified defects; obtaining image position information of the machine patrol images, registering the defect report with the machine patrol images by using the report position information and the image position information; associating the classified defects with the registered machine patrol images to obtain defect images; annotating the defect images by using the semantic segmentation model to obtain defect annotations.

6. The method of claim 5, wherein the method comprises: The step of annotating the defect images by using the semantic segmentation model comprises the following steps: dividing the defect images, extracting a frame as a key frame every certain number of frames, and the remaining frames that are not extracted are non-key frames; inputting the key frame into the semantic segmentation model for annotation to obtain a key frame mask; calculating the pixel displacement vector between the key frame and the non-key frame by using an optical flow method; transferring the key frame mask to the position of the non-key frame along the pixel displacement vector to generate a non-key frame initial mask; correcting the non-key frame initial mask by time sequence consistency verification to generate a non-key frame mask; matching the key frame and the non-key frame in time sequence to form defect annotations.

7. The method of claim 6, wherein the method comprises: The step of training the semantic segmentation model based on the final device annotations and the defect annotations comprises the following steps: The satellite image is taken as input, and the final device label is taken as a supervised target to train the semantic segmentation model; Device labeling is performed through the semantic segmentation model to obtain predicted device labeling, the labeling entropy value of the predicted device labeling is calculated, and the predicted device labeling is classified according to the labeling entropy value; The classified predicted device labeling is manually sampled and corrected to obtain corrected device labeling; The corrected device labeling and the final device labeling are combined to retrain the semantic segmentation model; The labeling effect of the semantic segmentation model is evaluated, and if the evaluation result meets the set evaluation threshold, the semantic segmentation model is saved; If the evaluation result does not meet the set evaluation threshold, manual sampling and correction are re-performed and the semantic segmentation model is retrained; The machine patrol image is taken as input, and the defect label is taken as a supervised target to train the semantic segmentation model; Defect labeling is performed through the semantic segmentation model to obtain predicted defect labeling, the labeling entropy value of the predicted defect labeling is calculated, and the predicted defect labeling is classified according to the labeling entropy value; The classified predicted defect labeling is manually sampled and corrected to obtain corrected defect labeling; The corrected defect labeling and the defect labeling are combined to retrain the semantic segmentation model; The labeling effect of the semantic segmentation model is evaluated, and if the evaluation result meets the set evaluation threshold, the semantic segmentation model is saved; If the evaluation result does not meet the set evaluation threshold, manual sampling and correction are re-performed and the semantic segmentation model is retrained.

8. A power grid space image fine labeling system based on SAM semantic segmentation, applying a power grid space image fine labeling method based on SAM semantic segmentation according to any one of claims 1-7, characterized in that, It comprises: A data acquisition module: acquiring satellite images and machine patrol images of the power grid system, and acquiring defect information of the power grid system; A first device labeling module: labeling the satellite images through a semantic segmentation model to obtain initial device labeling; A data processing module: generating a probability distribution of power grid device categories through feature data of the satellite images; A second device labeling module: optimizing the initial device labeling through the probability distribution of the power grid device categories to obtain final device labeling; A defect labeling module: generating a defect report through defect information, labeling the machine patrol images through the semantic segmentation model combined with the defect report to obtain defect labeling; A self-training module: correcting the final device labeling and the defect labeling to obtain corrected device labeling and corrected defect labeling; and training the semantic segmentation model through the corrected device labeling and the corrected defect labeling. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the steps of the power grid space image fine labeling method based on SAM semantic segmentation in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the power grid space image fine labeling method based on SAM semantic segmentation in any one of claims 1 to 7.

Images