Power equipment defect identification method and device based on pose fusion

Abstract

The invention provides an electrical equipment defect identification method and device based on pose fusion, the method is applied to an identification device comprising a data processing module, a model construction module and a defect identification module, and the method specifically comprises the following steps: obtaining historical image data and corresponding defect information, and aligning pose data with the historical image data according to a shooting timestamp; performing perspective distortion correction and deblurring processing on the historical image data according to the corresponding pose data to obtain a corresponding equipment area image, and training a YOLO model in combination with the corresponding pose data and defect information to obtain a power equipment defect identification model; and constructing a recognition sample according to the currently collected image data and pose data of the power equipment, and inputting the recognition sample into the power equipment defect recognition model to obtain a defect recognition result of the power equipment. According to the method, the pose data can be fused into the pre-training data preprocessing stage and the training stage of the model, the method can adapt to defect identification of power equipment in different pose scenes, and the accuracy is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of power equipment defect identification technology, and in particular to a power equipment defect identification method and apparatus based on pose fusion. Background Technology

[0002] In power system operation and maintenance scenarios, power equipment is widely distributed in complex terrains such as mountains, hills, and crossing rivers. Most of the equipment is located at high altitudes or in high-voltage environments. Traditional manual inspection methods not only require maintenance personnel to climb towers and come into close contact with high-voltage equipment, posing safety risks such as falls from heights and electric shocks, but are also limited by the coverage area of ​​manpower, resulting in extremely low inspection efficiency for wide-area lines. Furthermore, relying on human eyes to judge defects can easily lead to the omission of hidden problems such as minor cracks and slight oil leaks due to factors such as fatigue and lighting.

[0003] With the development of drone technology, drone inspection, with its safety advantages of non-contact operation and efficiency advantages of wide-area and rapid coverage, is gradually replacing traditional manual inspection and becoming the mainstream method of power operation and maintenance. Moreover, most drones are equipped with the YOLO target detection algorithm, which can locate power equipment defects in real time from the drone's images, without the need to send massive amounts of images back to the ground base station for processing, thus improving inspection efficiency.

[0004] However, in actual inspection scenarios, the shooting pose of inspection drones often changes due to environmental factors, leading to problems such as perspective distortion and motion blur in the captured images. The traditional YOLO object detection algorithm, which uses only the visual features of the image as input during training and inference, cannot adapt to changes in pose and the resulting changes in image content. In complex pose scenarios, it is prone to false positives and false negatives, resulting in low accuracy in defect identification. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies that use the YOLO target detection algorithm in conjunction with inspection drones for power equipment defect identification. These shortcomings include the inability to adapt to changes in drone pose and the resulting changes in image content, the susceptibility to false detections and missed detections in complex pose scenarios, and the low accuracy of defect identification. This invention provides a power equipment defect identification method and device based on pose fusion, which integrates pose data into the preprocessing and training stages of the YOLO model to obtain a power equipment defect identification model that adapts to different pose scenarios, thereby ensuring the accuracy of power equipment defect identification.

[0006] The objective of this invention is achieved through the following technical solution: A method for identifying defects in power equipment based on pose fusion includes: Acquire historical image data and corresponding defect information of power equipment collected by inspection drones, and align the pose data of the inspection drones with the historical image data according to the shooting timestamp; Based on the corresponding pose data, perspective distortion correction and deblurring are performed on historical image data to obtain the corresponding device area image. Based on the equipment area image and the corresponding pose data, combined with the corresponding defect information, a YOLO model is trained to obtain a power equipment defect recognition model. The system acquires image data of the power equipment currently collected by the inspection drone, combines the pose data to construct recognition samples, inputs the recognition samples into the power equipment defect recognition model, and obtains the defect recognition results of the power equipment.

[0007] By employing pose data-driven perspective distortion correction and deblurring, tilted and blurred images can be restored to regular and clear images of the equipment area. This improves the utilization of historical image data, addresses the small sample size problem in pose-based scenarios, and provides the model with sufficient training data that more closely resembles the actual defect morphology. During model training, normalized pose data is also used as input, enabling the YOLO model to learn the visual features of defects while simultaneously mastering the correlation between different poses and defect presentation morphologies. Ultimately, this results in a power equipment defect recognition model capable of adapting to various pose scenarios. When implementing defect recognition, the power equipment defect recognition model can quickly invoke the corresponding adaptation strategy learned during training based on the current pose, thereby improving the accuracy of defect recognition in complex pose scenarios.

[0008] Furthermore, the step of performing perspective distortion correction and deblurring on historical image data based on the corresponding pose data to obtain the corresponding device region image includes: Based on the corresponding pose data, obtain the downward angle parameters, device distance, and angular velocity during shooting; A perspective projection matrix is ​​constructed based on the tilt angle parameters and equipment distance. Combined with the physical dimensions of the power equipment, perspective distortion correction is performed on the corresponding historical image data. Obtain the image acquisition frame rate corresponding to historical image data, and calculate the blur kernel by combining it with angular velocity; The historical image data after perspective distortion correction is deblurred based on the fuzz kernel, and the corresponding device area image is obtained based on the deblurred historical image data.

[0009] Furthermore, the step of deblurring the historical image data after perspective distortion correction based on the blur kernel, and obtaining the corresponding device area image based on the deblurred historical image data, includes: The historical image data after perspective distortion correction is divided into regions using an edge detection algorithm to obtain the corresponding device region and background region. The device region is deblurred using a non-blind deconvolution algorithm based on a blur kernel, while the background region is low-pass filtered. Based on the region segmentation results, the blurred equipment area image is cropped out from the historical image data.

[0010] Furthermore, the step of training a YOLO model based on the equipment region image and corresponding pose data, combined with corresponding defect information, to obtain a power equipment defect identification model includes: The corresponding pose data is normalized, training samples are constructed by combining them with the device area image, and defect labels are added to each training sample by the corresponding defect information. Classify pose scenes according to the corresponding pose data, and obtain the training sample set for each pose scene. The learning rate iteration rule of the YOLO model is set according to the cosine annealing strategy, and the focus loss function is used as the loss function of the YOLO model. The model parameters of the YOLO model are iteratively optimized according to the training sample set corresponding to each pose scenario until the iteration stopping condition is met, so as to obtain the power equipment defect identification model.

[0011] Furthermore, the step of acquiring image data of the power equipment currently collected by the inspection drone and constructing recognition samples by combining pose data includes: Align image data and pose data according to the shooting timestamp, and perform perspective distortion correction and deblurring on the image data based on the corresponding pose data to obtain the corresponding device area image; The pose data is normalized and combined with the device area image to construct recognition samples; The validity of the identified samples is verified. If the validity verification fails, the identification samples are corrected by combining the image data and pose data of historical frames until the validity verification is passed.

[0012] Furthermore, the validity verification of the identified samples includes: Extract the device outline from the device region image, calculate the corresponding device region proportion and edge closure, and perform integrity verification on the device region image based on the device region proportion and edge closure. The pose data is validated for reasonableness based on the physical dimensions of the power equipment and the pixel size of the equipment in the equipment area image. Image quality verification of the device area image is performed based on the average gradient and contrast of the device area image. If any validation fails, the sample is deemed to have failed the validity validation. If each check passes, the sample is deemed to have passed the validity check.

[0013] Furthermore, the step of combining historical frame image and pose data for identification sample correction includes: The correction type for the identified sample is determined based on the validity verification results; When the correction type is device area image integrity correction, the corresponding frame image of the device area image and the previous valid image are aligned by feature matching, and the device area image is corrected according to the device area edge of the previous valid image. When the correction type is pose data rationality correction, the effective poses of the previous two frames are fused by linear interpolation, and the current pose data is replaced by the fused pose data. When the correction type is device area image quality correction, the corresponding frame image of the device area image and the previous valid frame image are aligned by feature matching, and the feature area and pixel brightness of the device area image are adjusted according to the feature area and illumination parameters of the previous valid frame image.

[0014] Furthermore, the defect identification results of the power equipment include defect type, defect bounding box, and confidence level.

[0015] Furthermore, after inputting the identified samples into the power equipment defect identification model and obtaining the defect identification results of the power equipment, the following steps are also performed: Candidate defects are selected based on confidence level, and the movement trajectory of the same candidate defect is matched between consecutive frames. A confidence moving average is performed on the movement trajectory of the same candidate defect, and the real defects are selected based on the confidence average and the movement trajectory.

[0016] A pose fusion-based power equipment defect identification device, used to perform any of the identification methods described above, includes: The data processing module is used to align pose data and image data, and to preprocess the image data based on the pose data. The model building module, connected to the data processing module, is used to train the YOLO model based on the preprocessed equipment area image, combined with the corresponding pose data and defect information, to obtain a power equipment defect identification model. The defect identification module is connected to the data processing module and the model building module respectively. It is used to construct identification samples based on the currently acquired image data and pose data, and to obtain the defect identification results of the power equipment through the power equipment defect identification model.

[0017] The beneficial effects of this invention are: By employing pose data-driven perspective distortion correction and deblurring, tilted and blurred images can be restored to regular and clear images of the equipment area. This improves the utilization of historical image data, addresses the small sample size problem in some pose scenarios, and provides the model with sufficient training data that more closely resembles the actual defect morphology. During model training, normalized pose data is also used as input, allowing the YOLO model to learn the visual features of defects while simultaneously mastering the correlation between different poses and defect presentation morphologies. Ultimately, this yields a power equipment defect recognition model capable of adapting to various pose scenarios. Furthermore, the validity of current recognition samples is validated to ensure reliability. The power equipment defect recognition model then quickly invokes the corresponding adaptation strategy learned during training based on the current pose, achieving final power equipment defect recognition and improving the accuracy of defect recognition in complex pose scenarios. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of a process of the present invention; Figure 2 This is a schematic diagram of a structure according to an embodiment of the present invention.

[0019] The module consists of: 1. Data processing module; 2. Model building module; and 3. Defect identification module. Detailed Implementation

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] Example: A method for identifying defects in power equipment based on pose fusion, such as Figure 1 As shown, it includes: Acquire historical image data and corresponding defect information of power equipment collected by inspection drones, and align the pose data of the inspection drones with the historical image data according to the shooting timestamp; Based on the corresponding pose data, perspective distortion correction and deblurring are performed on historical image data to obtain the corresponding device area image. Based on the equipment area image and the corresponding pose data, combined with the corresponding defect information, a YOLO model is trained to obtain a power equipment defect recognition model. The system acquires image data of the power equipment currently collected by the inspection drone, combines the pose data to construct recognition samples, inputs the recognition samples into the power equipment defect recognition model, and obtains the defect recognition results of the power equipment.

[0022] Traditional YOLO models rely solely on image visual features for training, which cannot adapt to image distortion and blurring caused by changes in drone pose. Therefore, it is necessary to obtain the pose parameters of the captured image and the defect information of the defects in the image in advance, and match the three together so that the subsequent YOLO model can learn the association between pose and defect features.

[0023] When inspection drones collect data, image data is generated by camera sensors, while pose data is generated by GPS and IMU inertial measurement units. These are two independent data streams with differences in sampling frequency or sensor delay. Therefore, historical image data and pose data are first aligned according to the shooting timestamp to ensure the spatiotemporal consistency of pose and image and avoid data misalignment.

[0024] Because pose changes can cause perspective distortion and motion blur in image data, and the original images often contain a large amount of irrelevant background, directly using such images to train the model will severely affect the accuracy of defect recognition. Therefore, after completing the corresponding image and pose alignment, it is necessary to further perform perspective distortion correction and deblurring on historical image data using pose data in order to convert invalid or low-quality images into valid samples.

[0025] Specifically, the step of performing perspective distortion correction and deblurring on historical image data based on the corresponding pose data to obtain the corresponding device region image includes: Based on the corresponding pose data, obtain the downward angle parameters, device distance, and angular velocity during shooting; A perspective projection matrix is ​​constructed based on the tilt angle parameters and equipment distance. Combined with the physical dimensions of the power equipment, perspective distortion correction is performed on the corresponding historical image data. Obtain the image acquisition frame rate corresponding to historical image data, and calculate the blur kernel by combining it with angular velocity; The historical image data after perspective distortion correction is deblurred based on the fuzz kernel, and the corresponding device area image is obtained based on the deblurred historical image data.

[0026] Changes in the tilt angle and distance to equipment during inspection drone photography can cause perspective distortion in power equipment. For example, at excessively large tilt angles, insulators may appear as trapezoids with narrower tops and wider bottoms than regular rectangular skirts. Conversely, at greater distances, the entire equipment may appear stretched and deformed. Sudden changes in angular velocity during drone photography can also cause motion blur. Both perspective distortion and motion blur prevent the subsequent YOLO model from learning the true morphology of defects, thus affecting its accuracy in defect identification.

[0027] Therefore, the first step is to obtain the tilt angle, device distance, and angular velocity at the time of shooting based on the corresponding pose data. The tilt angle determines the perspective tilt of the power equipment in the corresponding image, the device distance relates to the device's pixel scale, and the angular velocity directly affects the intensity of motion blur. When obtaining the pose parameters, it is also necessary to control the deviation between the extracted pose parameters and the timestamp of the corresponding historical image within a certain range. Furthermore, the reasonableness of the values ​​needs to be further verified. If the values ​​exceed the predetermined range, error compensation must be added to ensure the accuracy of subsequent processing.

[0028] After obtaining the corresponding pose parameters, the internal parameters of the visual camera carried by the inspection drone are first obtained, such as focal length and principal point coordinates. Then, the rotation matrix is ​​calculated based on the tilt angle parameters, such as rotating around the x-axis by the corresponding angle. Then, the translation vector is determined based on the device distance, that is, the position of the device center point in the camera coordinate system of the visual camera. The two are combined to generate a projection matrix that maps the three-dimensional world coordinates to two-dimensional image pixels.

[0029] Simultaneously, the theoretical pixel size corresponding to the physical size of the power equipment is calculated to achieve the corresponding image correction. By comparing the actual pixel size of the corrected image with the theoretical value, the projection matrix parameters are adjusted. Finally, the equipment shape, which can be distorted by trapezoids and stretching, is restored to a regular outline that conforms to the real physical size, thus completing the perspective distortion correction.

[0030] Then, the image acquisition frame rate of the vision camera mounted on the inspection drone is obtained, and the corresponding exposure time is obtained. The rotation angle during the exposure time, i.e., the blur angle, is then calculated based on the angular velocity. Next, the blur angle is converted into a blur length based on the image resolution of the corresponding historical image data. Finally, a blur kernel matrix of the corresponding size is generated based on the blur angle and blur length.

[0031] Then, based on the blur kernel matrix, the historical image data after perspective distortion correction is deblurred. The corresponding device area image is then obtained from the deblurred historical image data, including: The historical image data after perspective distortion correction is divided into regions using an edge detection algorithm to obtain the corresponding device region and background region. The device region is deblurred using a non-blind deconvolution algorithm based on a blur kernel, while the background region is low-pass filtered. Based on the region segmentation results, the blurred equipment area image is cropped out from the historical image data.

[0032] After perspective distortion correction, the geometry of the power equipment in the image has returned to normal, but it still contains a large amount of irrelevant background such as sky, trees, and ground, and the equipment area may be blurred due to changes in the drone's angular velocity. At this point, it is necessary to further partition the historical image data and perform targeted deblurring processing to improve processing efficiency while ensuring the deblurring effect.

[0033] Specifically, the Canny edge detection algorithm can be used to segment historical image data after perspective distortion correction. First, the historical image data after perspective distortion correction is smoothed using Gaussian filtering to suppress noise. Then, the gradient magnitude and direction are calculated, and corresponding dual thresholds are set to generate a binary edge map. The high threshold is used to preserve sharp edges, and the low threshold is used to connect weak edges. Because the equipment region has clear material boundaries, its edge gradient value is significantly higher than that of the background region. Based on this, morphological operations are used to optimize the contour continuity based on the edge map. Then, a contour extraction algorithm is used to obtain all closed contours, and the target equipment contour is selected by combining the prior shape features of the power equipment. Finally, the image is divided into equipment region and background region. Then, differential deblurring processing can be performed on the segmented equipment region and background region.

[0034] For the equipment area, which contains critical defects such as cracks and oil leaks, a non-blind deconvolution algorithm is used by inputting the calculated blur kernel matrix. The image of the equipment area after perspective distortion correction is used as the initial value. Through iterative optimization, the forward projection error between the deblurring result and the blurring process is minimized, thus completing the deblurring process of the equipment area.

[0035] For the background region, since it is free of defects and has no significance for model training, it only needs to be processed by low-pass filtering. This can eliminate high-frequency noise and slight blur in the background, and avoid artifacts introduced by complex deblurring operations. This reduces the computational cost and provides a clean background boundary for subsequent cropping.

[0036] Finally, based on the region segmentation results, the extracted equipment outline is obtained, and the minimum bounding rectangle that ensures complete coverage of the equipment and edge defects is calculated. A certain pixel redundancy must be reserved outside the outline of the rectangle to avoid cropping out minor edge defects such as fine cracks on insulator edges and corrosion at transformer corners. Then, based on the pixel coordinates of the rectangle, the corresponding region is cropped from the deblurred historical image data to obtain an equipment region image containing only the equipment and a few necessary surrounding areas.

[0037] Although perspective distortion correction, deblurring, and region extraction can obtain image data that preserves the complete shape and defect details of power equipment, solving the problem of visual distortion caused by pose changes in the original image and providing a high-quality visual data foundation for defect recognition, if the YOLO model only learns defect patterns through visual features, it will still cause false detections and missed detections if the pose of the inspection drone differs significantly from that of the training samples due to a lack of understanding of the relationship between pose and defect shape.

[0038] Therefore, based on the acquired equipment area images, it is also necessary to train the YOLO model by combining the corresponding pose information and defect information, so that while learning visual features, it can simultaneously master the correspondence between pose and defect shape, so as to obtain a power equipment defect recognition model that can be adapted to different pose scenarios.

[0039] Specifically, the step of training a YOLO model based on equipment region images and corresponding pose data, combined with corresponding defect information, to obtain a power equipment defect identification model includes: The corresponding pose data is normalized, training samples are constructed by combining them with the device area image, and defect labels are added to each training sample by the corresponding defect information. Classify pose scenes according to the corresponding pose data, and obtain the training sample set for each pose scene. The learning rate iteration rule of the YOLO model is set according to the cosine annealing strategy, and the focus loss function is used as the loss function of the YOLO model. The model parameters of the YOLO model are iteratively optimized according to the training sample set corresponding to each pose scenario until the iteration stopping condition is met, so as to obtain the power equipment defect identification model.

[0040] Since the values ​​and units of the depression angle, device distance, and angular velocity are different, directly inputting them into the model would lead to an imbalance in parameter weights. Therefore, a normalization algorithm is first used to map all pose parameters to the [0, 1] interval. After normalization, the device area image is scaled to a fixed size, converted into an image tensor, and pixel values ​​are normalized. At the same time, the normalized pose parameters are used to form a 1×3 feature vector. Then, corresponding defect labels are set according to the defect information, ultimately forming training samples of image tensor, pose vector, and defect labels.

[0041] Considering that during inspections by drones, samples at small depression angles and medium distances account for more than half of the total samples, while samples at large depression angles and long distances account for a smaller proportion, direct mixed training would lead to overfitting of the model to multiple sample scenarios. Therefore, the pose scenarios are classified using the depression angle parameter and device distance as classification dimensions. All samples are traversed and assigned to the corresponding scenarios according to the pose parameters, forming a scenario-specific sample set. For scenarios with insufficient sample size, data augmentation techniques such as horizontal flipping can be used to expand the sample size. At the same time, defect bounding box consistency verification is used to ensure that the augmented defect boxes still accurately select the target.

[0042] Finally, the model parameters were iteratively optimized by improving the training strategy. Regarding the learning rate setting, a cosine annealing strategy was used to set the learning rate iteration rules. Specifically, the learning rate iteration rules included: The initial learning rate is set to 0.001, and the learning rate is updated after each annealing cycle of training.

[0043] The formula for updating the learning rate is: ; in, For the updated learning rate, To minimize the learning rate, The maximum learning rate within the period. For the rounds within the current cycle, The total number of rounds in the cycle.

[0044] Restarting the cycle when the learning rate drops to its minimum ensures both rapid convergence in the early stages and fine-tuning of parameters in the later stages.

[0045] In terms of loss function selection, a focus loss function is introduced to address the imbalance between positive and negative samples. By setting the weight coefficient of positive samples and the hard sample mining coefficient, the model can focus on small defects and distortion defects, thus avoiding the problem of small defects and distortion defects being missed.

[0046] Then, during the specific training process, a scene-by-scene approach is adopted, loading the sample set in the order of different pose scenes. After each round of training, the training set loss value and the accuracy of the set validation set are calculated. When the increase in validation set accuracy is less than the preset accuracy increase threshold for 5 consecutive rounds, and the decrease in training set loss value is less than the preset loss decrease threshold, the iteration stopping condition is met, the model converges, and the model parameters with the highest validation set accuracy and the lowest loss value at this time are saved. Finally, a power equipment defect recognition model that can adapt to different pose scenes is obtained, which not only grasps the visual features of defects, but also adjusts the recognition strategy according to the pose.

[0047] During real-time inspections, the image data collected by the inspection drone will still exhibit perspective distortion and motion blur issues similar to those in historical data due to dynamic pose changes. Furthermore, the original images also contain a large amount of irrelevant background. In order to complete defect identification through the power equipment defect identification model, the collected image data needs to be processed and converted in the appropriate format to generate identification samples that meet the model's input requirements.

[0048] Specifically, acquiring image data of the power equipment currently collected by the inspection drone and constructing recognition samples by combining pose data includes: Align image data and pose data according to the shooting timestamp, and perform perspective distortion correction and deblurring on the image data based on the corresponding pose data to obtain the corresponding device area image; The pose data is normalized and combined with the device area image to construct recognition samples; The validity of the identified samples is verified. If the validity verification fails, the identification samples are corrected by combining the image data and pose data of historical frames until the validity verification is passed.

[0049] Following the processing methods for historical image data, perspective distortion correction and deblurring are performed on the currently acquired image data, and the equipment area image for defect identification is cropped out through edge detection.

[0050] After completing the corresponding image processing optimization, the pose data needs to be further normalized and integrated into recognition samples. Furthermore, the image processing optimization and pose data normalization processes must be consistent with the processing methods used for historical image data and corresponding pose data during model training to ensure that the constructed recognition sample format meets the input format requirements of the power equipment defect recognition model.

[0051] Considering that issues such as camera defocusing, sensor malfunction, and airflow interference may cause samples to become invalid during real-time inspection, the validity of the constructed recognition samples is further verified.

[0052] The validity of the recognition samples is verified from three dimensions: device integrity, pose rationality, and image quality. Extract the device outline from the device region image, calculate the corresponding device region proportion and edge closure, and perform integrity verification on the device region image based on the device region proportion and edge closure. The pose data is validated for reasonableness based on the physical dimensions of the power equipment and the pixel size of the equipment in the equipment area image. Image quality verification of the device area image is performed based on the average gradient and contrast of the device area image. If any validation fails, the sample is deemed to have failed the validity validation. If each check passes, the sample is deemed to have passed the validity check.

[0053] For integrity verification, the previous edge detection logic is used. Gaussian filtering is first applied to the processed equipment area image to suppress residual noise. Then, the equipment edges are extracted by Canny edge detection. Finally, the contour extraction algorithm is called to obtain all contours. Then, the target equipment contours are selected by combining the prior features of the power equipment.

[0054] Then, the proportion of the device region is calculated by comparing the total number of pixels enclosed by the selected target device outline with the total number of pixels in the device region image. At the same time, the total outline length of the device outline is counted, and closed outlines are selected. The pixel distance between the start and end points of the closed outline is determined, and the edge closure degree is obtained by the ratio of the sum of the pixel distances between the start and end points of the closed outline to the total outline length.

[0055] If the proportion of the device area is less than the preset area proportion threshold, or the edge closure is less than the preset closure threshold, then the current recognition sample is determined to fail the device integrity check.

[0056] For pose validity verification, since the ratio of the physical size of the device to its pixel size is the same as the ratio of the device distance in the pose data to the pixel size corresponding to the focal length of the visual camera, the theoretical device distance can be deduced from the physical size of the power equipment and the device pixel size in the image of the device area. If the theoretical device distance deviates significantly from the device distance in the pose data, the current recognition sample can be judged to have failed the pose validity verification.

[0057] For image quality verification, a device region image can be first converted to grayscale. Then, all non-derogatory pixels in the image are iterated through, and the absolute value of the grayscale difference between each pixel and its right and bottom adjacent pixels is calculated. The sum of these values ​​is then divided by twice the total number of valid pixels to obtain the average gradient. Simultaneously, based on the grayscale image, the maximum and minimum grayscale values ​​in the image are counted to obtain the contrast ratio.

[0058] If the average gradient is less than the preset gradient threshold, or the contrast exceeds the preset grayscale threshold, the image quality check is deemed unsuccessful.

[0059] Only when all three dimensions pass the corresponding verification is the sample considered to have passed the validity verification.

[0060] When a recognized sample fails the validity check, image data and pose data from historical frames are also used to correct the recognized sample, including: The correction type for the identified sample is determined based on the validity verification results; When the correction type is device area image integrity correction, the corresponding frame image of the device area image and the previous valid image are aligned by feature matching, and the device area image is corrected according to the device area edge of the previous valid image. When the correction type is pose data rationality correction, the effective poses of the previous two frames are fused by linear interpolation, and the current pose data is replaced by the fused pose data. When the correction type is device area image quality correction, the corresponding frame image of the device area image and the previous valid frame image are aligned by feature matching, and the feature area and pixel brightness of the device area image are adjusted according to the feature area and illumination parameters of the previous valid frame image.

[0061] To achieve targeted correction for identified samples, the dimensions that caused the failure to pass the validity check are first determined based on the validity check results, and then the corresponding correction type is determined. Differential correction is then performed based on the correction type.

[0062] When the correction type is device region image integrity correction, the device contour defects in the current frame need to be repaired based on the valid image from the previous frame that has passed verification. Specifically, feature matching is first performed on the device region image of the current frame and the valid image from the previous frame. Key device features are extracted from both frames, and the FLANN matcher is used to select the feature points with the highest matching degree. Then, the perspective transformation matrix between the two frames is calculated. Based on this matrix, the device edge contour of the valid image from the previous frame is mapped to the coordinate system of the current frame image, locating the missing areas or insufficiently proportioned edges of the device contour in the current frame. Subsequently, based on the mapped contour from the previous frame, the device region of the current frame is cropped a second time. If the device proportion in the current frame is insufficient, the cropping boundary is extended to the outer edge of the mapped contour. If the contour is broken, the gap is filled with the continuous part of the mapped contour.

[0063] When the correction type is pose data rationality correction, the reasonable pose parameters of the current frame need to be generated by linear interpolation of the effective poses of the previous two frames. Since the pose parameters of the inspection drone change according to the law of continuous motion during inspection, the effective poses of the previous two frames can serve as a reliable reference for the current frame. Specifically, the valid pose data of the previous two frames that have passed verification can be extracted first, the timestamp of the pose data of the current frame can be recorded, the time weight coefficient can be calculated, and the device distance, depression angle parameter, and angular velocity can be linearly interpolated according to the time weight coefficient to achieve pose data correction.

[0064] When the correction type is device area image quality correction, it is necessary to consider both feature sharpness and illumination consistency, adjusting the current frame's device area image based on the previous frame's effective image. Specifically, ORB feature matching can be used to locate the overlapping feature regions between the current frame's device area image and the previous frame. Using the sharp feature regions in the previous frame as references, a Poisson fusion algorithm is used to map them to the corresponding blurred regions in the current frame, replacing blurred pixel values ​​while preserving the overall structure of the current frame and avoiding the introduction of artifacts. For illumination parameter adjustment, the grayscale histogram of the previous frame's effective image is calculated, and the grayscale values ​​of the current frame's device area image are remapped according to the histogram distribution of the previous frame to achieve image quality correction.

[0065] After completing the corresponding corrections, the validity of the corrected identification samples must be verified again. If the validity verification still fails after multiple corrections, the inspection drone needs to be controlled to retake the relevant images until a valid identification sample is obtained.

[0066] After obtaining the identified samples that have passed the validity check, input them into the power equipment defect identification model to output the corresponding power equipment defect identification results.

[0067] The defect identification results of the power equipment include defect type, defect bounding box, and confidence level.

[0068] During real-time inspections by inspection drones, single-frame inference using the power equipment defect identification model may result in false detections due to transient interference, or low defect confidence levels due to fluctuations in single-frame image quality. Therefore, after the power equipment defect identification model outputs the defect identification results, the correlation between continuous frame data from the inspection drone is utilized to further distinguish between real defects and transient interference through trajectory matching and confidence smoothing, including: Candidate defects are selected based on confidence level, and the movement trajectory of the same candidate defect is matched between consecutive frames. A confidence moving average is performed on the movement trajectory of the same candidate defect, and the real defects are selected based on the confidence average and the movement trajectory.

[0069] In the single-frame output of the power equipment defect identification model, there are a large number of low-confidence candidates. These candidates mostly originate from non-defect interference. If all candidates are directly retained, the computational workload for subsequent trajectory matching will surge due to excessive invalid data, and false trajectories are prone to occur. Therefore, by setting a confidence threshold, candidate defects with a confidence level greater than the preset threshold are selected. This not only eliminates the vast majority of obvious false detections but also retains defects with slightly lower confidence levels in a single frame that may be genuine, reserving valid objects for subsequent multi-frame verification, thus balancing screening efficiency and the risk of missed detections.

[0070] During inspection, the drone moves steadily, and the coordinates of a real defect in consecutive frames show a continuous and smooth change. In contrast, the coordinate changes of instantaneous interference often exhibit discontinuous and abrupt characteristics. Therefore, a trajectory matching algorithm is used to find corresponding candidates with similar features and continuous positions for each candidate defect in consecutive frames. If a candidate defect can find a corresponding target that conforms to a pattern and forms a smooth trajectory in more than 3 consecutive frames, it is initially judged as a potential real defect. If it only appears in 1-2 frames and cannot match the trajectory of subsequent frames, it is judged as an instantaneous false detection and is eliminated.

[0071] Finally, since the confidence level of a single-frame defect is easily affected by fluctuations in image quality, relying solely on the confidence level of a single frame to determine the true defect may lead to the misjudgment of defects that have temporarily decreased in confidence but are actually present as invalid. In contrast, the confidence level of a true defect in consecutive frames generally shows a stable trend. Therefore, by using a moving average to eliminate the influence of single-frame fluctuations and reflect the true credibility of the defect, and by combining trajectory features for final screening, we can achieve this.

[0072] If the moving average confidence level is higher than the average confidence level threshold for true defects, the trajectory has ≥3 consecutive frames, and the standard deviation of the coordinate deviation between adjacent frames is ≤15 pixels, meaning the trajectory smoothness meets the standard, then the candidate defect is determined to be a true defect. Otherwise, it can be determined to be invalid.

[0073] It should also be noted that the various thresholds and judgment conditions mentioned in this embodiment can be set according to actual needs.

[0074] Another aspect of this embodiment also provides a power equipment defect identification device based on pose fusion, such as... Figure 2 As shown, it includes: Data processing module 1 is used to align pose data and image data, and to preprocess the image data based on the pose data; Model building module 2, connected to the data processing module, is used to train the YOLO model based on the preprocessed equipment area image, combined with the corresponding pose data and defect information, to obtain a power equipment defect recognition model. The defect identification module 3 is connected to the data processing module and the model building module respectively. It is used to construct identification samples based on the currently acquired image data and pose data, and to obtain the defect identification results of the power equipment through the power equipment defect identification model.

[0075] The data processing module, model building module, and defect identification module are all data processing components such as MCUs and microprocessors, directly integrated into the inspection drone, and carry corresponding algorithms for data processing, model training, and model application. Through the cooperation of the data processing module, model building module, and defect identification module, the inspection drone can effectively identify defects in power equipment during the inspection process.

[0076] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any way. Other variations and modifications are possible without departing from the technical solutions described in the claims.

Claims

1. A method for identifying defects in power equipment based on pose fusion, characterized in that, include: Acquire historical image data and corresponding defect information of power equipment collected by inspection drones, and align the pose data of the inspection drones with the historical image data according to the shooting timestamp; Based on the corresponding pose data, perspective distortion correction and deblurring are performed on historical image data to obtain the corresponding device area image. Based on the equipment area image and the corresponding pose data, combined with the corresponding defect information, a YOLO model is trained to obtain a power equipment defect recognition model. The system acquires image data of the power equipment currently collected by the inspection drone, combines the pose data to construct recognition samples, inputs the recognition samples into the power equipment defect recognition model, and obtains the defect recognition results of the power equipment.

2. The method for power equipment defect identification based on pose fusion according to claim 1, characterized in that, The step of performing perspective distortion correction and deblurring on historical image data based on corresponding pose data to obtain the corresponding device region image includes: Based on the corresponding pose data, obtain the downward angle parameters, device distance, and angular velocity during shooting; A perspective projection matrix is ​​constructed based on the tilt angle parameters and equipment distance. Combined with the physical dimensions of the power equipment, perspective distortion correction is performed on the corresponding historical image data. Obtain the image acquisition frame rate corresponding to historical image data, and calculate the blur kernel by combining it with angular velocity; The historical image data after perspective distortion correction is deblurred based on the fuzz kernel, and the corresponding device area image is obtained based on the deblurred historical image data.

3. The method for power equipment defect identification based on pose fusion according to claim 2, characterized in that, The step of deblurring the historical image data after perspective distortion correction based on the blur kernel, and obtaining the corresponding device area image based on the deblurred historical image data, includes: The historical image data after perspective distortion correction is divided into regions using an edge detection algorithm to obtain the corresponding device region and background region. The device region is deblurred using a non-blind deconvolution algorithm based on a blur kernel, while the background region is low-pass filtered. Based on the region segmentation results, the blurred equipment area image is cropped out from the historical image data.

4. The method for power equipment defect identification based on pose fusion according to claim 1, characterized in that, The process of training a YOLO model based on equipment region images and corresponding pose data, combined with corresponding defect information, to obtain a power equipment defect identification model includes: The corresponding pose data is normalized, training samples are constructed by combining them with the device area image, and defect labels are added to each training sample by the corresponding defect information. Classify pose scenes according to the corresponding pose data, and obtain the training sample set for each pose scene. The learning rate iteration rule of the YOLO model is set according to the cosine annealing strategy, and the focus loss function is used as the loss function of the YOLO model. The model parameters of the YOLO model are iteratively optimized according to the training sample set corresponding to each pose scenario until the iteration stopping condition is met, so as to obtain the power equipment defect identification model.

5. The method for power equipment defect identification based on pose fusion according to claim 1, characterized in that, The process of acquiring image data of power equipment currently collected by the inspection drone and constructing recognition samples by combining pose data includes: Align image data and pose data according to the shooting timestamp, and perform perspective distortion correction and deblurring on the image data based on the corresponding pose data to obtain the corresponding device area image; The pose data is normalized and combined with the device area image to construct recognition samples; The validity of the identified samples is verified. If the validity verification fails, the identification samples are corrected by combining the image data and pose data of historical frames until the validity verification is passed.

6. The method for power equipment defect identification based on pose fusion according to claim 5, characterized in that, The validity verification of the identified samples includes: Extract the device outline from the device region image, calculate the corresponding device region proportion and edge closure, and perform integrity verification on the device region image based on the device region proportion and edge closure. The pose data is validated for reasonableness based on the physical dimensions of the power equipment and the pixel size of the equipment in the equipment area image. Image quality verification of the device area image is performed based on the average gradient and contrast of the device area image. If any validation fails, the sample is deemed to have failed the validity validation. If each check passes, the sample is deemed to have passed the validity check.

7. The method for power equipment defect identification based on pose fusion according to claim 6, characterized in that, The step of combining image data and pose data from historical frames to correct recognition samples includes: The correction type for the identified sample is determined based on the validity verification results; When the correction type is device area image integrity correction, the corresponding frame image of the device area image and the previous valid image are aligned by feature matching, and the device area image is corrected according to the device area edge of the previous valid image. When the correction type is pose data rationality correction, the effective poses of the previous two frames are fused by linear interpolation, and the current pose data is replaced by the fused pose data. When the correction type is device area image quality correction, the corresponding frame image of the device area image and the previous valid frame image are aligned by feature matching, and the feature area and pixel brightness of the device area image are adjusted according to the feature area and illumination parameters of the previous valid frame image.

8. The method for power equipment defect identification based on pose fusion according to claim 1, characterized in that, The defect identification results of the power equipment include defect type, defect bounding box, and confidence level.

9. The method for power equipment defect identification based on pose fusion according to claim 8, characterized in that, After inputting the identified samples into the power equipment defect identification model and obtaining the defect identification results of the power equipment, the following steps are also performed: Candidate defects are selected based on confidence level, and the movement trajectory of the same candidate defect is matched between consecutive frames. A confidence moving average is performed on the movement trajectory of the same candidate defect, and the real defects are selected based on the confidence average and the movement trajectory.

10. A power equipment defect identification device based on pose fusion, used to execute the identification method according to any one of claims 1 to 9, characterized in that, include: The data processing module is used to align pose data and image data, and to preprocess the image data based on the pose data. The model building module, connected to the data processing module, is used to train the YOLO model based on the preprocessed equipment area image, combined with the corresponding pose data and defect information, to obtain a power equipment defect identification model. The defect identification module is connected to the data processing module and the model building module respectively. It is used to construct identification samples based on the currently acquired image data and pose data, and to obtain the defect identification results of the power equipment through the power equipment defect identification model.

Images