Power equipment detection method and device based on data acquisition of inspection unmanned aerial vehicle, electronic equipment and storage medium

Abstract

The invention discloses a power equipment detection method and device based on data acquisition of an inspection unmanned aerial vehicle, electronic equipment and a storage medium, and the method comprises the steps: obtaining a multi-view image of to-be-detected power equipment, and inputting the multi-view image into an abnormal region detection model, and generating an anomaly existence probability and a suspected anomaly region coordinate of the equipment under the current view angle. When it is detected that the abnormal probability at a certain visual angle exceeds a preset threshold value, a reinspection instruction is sent to the inspection unmanned aerial vehicle in an adjacent area, and image data of the equipment at a new visual angle are fed back. A second anomaly existence probability is calculated by re-inputting the new view angle image to the anomaly detection model. If the probability exceeds a preset threshold value, judging that the power equipment is abnormal, and sending an alarm signal; otherwise, judging that the equipment is normal. According to the invention, the abnormal accurate positioning and intelligent alarm of the power equipment can be realized, and the efficiency and accuracy of equipment inspection can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power equipment detection, and particularly relates to a power equipment detection method, device, electronic device and storage medium based on data collection by an inspection drone. Background Art

[0002] With the continuous expansion of the scale of the power system, the stable operation of power equipment is directly related to the safety of the power grid. Traditional manual inspection has problems such as low efficiency, high risk of working in dangerous environments, and difficulty in covering remote areas. The detection technology based on drones equipped with multi-spectral sensors can efficiently obtain key data such as surface deformation, temperature distribution, and partial discharge of equipment, providing an important basis for early fault warning. Through the collaborative analysis of visible light, infrared, and ultraviolet multi-modal data, the accuracy of equipment defect identification can be significantly improved, and it has become the core means of intelligent power grid operation and maintenance.

[0003] However, the current power equipment detection technology based on drones still has significant defects. The detection process highly depends on the reliability of single-shot collected data. If the drone sensor suddenly fails (such as lens fouling, infrared module failure, etc.), there is a lack of an effective cross-device collaborative verification mechanism, resulting in a sharp increase in the false positive rate; the multi-modal data fusion ability is weak, and visible light, infrared, and ultraviolet images are often processed independently, lacking a unified joint analysis framework, causing frequent false alarms. Summary of the Invention

[0004] Embodiments of the present invention provide a power equipment detection method, device, electronic device and storage medium based on data collection by an inspection drone. By implementing the present invention, single-shot detection errors can be effectively eliminated, the false positive risk caused by sensor failures or environmental interference can be significantly reduced, and the efficiency and accuracy of equipment inspection can be improved.

[0005] An embodiment of the present invention provides a power equipment detection method based on data collection by an inspection drone, including:

[0006] Obtaining first visible light images, first infrared temperature distribution images, and first partial discharge ultraviolet images of a power equipment to be detected from multiple perspectives;

[0007] Inputting the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective into a preset abnormal area detection model, so that the abnormal area detection model generates a first abnormal existence probability and suspected coordinates of an abnormal area of the power equipment to be detected under the current perspective according to the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective;

[0008] When the probability of the abnormal existence of the power equipment to be detected from a perspective exceeds a preset probability threshold, a re-inspection instruction is sent to a patrol unmanned aerial vehicle in an adjacent area of the patrol area where the power equipment to be detected is located, so that the patrol unmanned aerial vehicle in the adjacent area feeds back the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image of the power equipment to be detected from the target perspective; wherein, the target perspective includes the first abnormal area;

[0009] Input the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image into a preset abnormal area detection model, so that the abnormal area detection model generates the second probability of the abnormal existence of the power equipment to be detected from the current perspective according to the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image from the same perspective;

[0010] Judge whether the second probability of the abnormal existence exceeds the preset probability threshold. If so, it is determined that the power equipment to be detected is abnormal and an alarm signal is sent. If not, it is determined that the power equipment to be detected is normal.

[0011] Furthermore, the patrol unmanned aerial vehicle is provided with an image capturing device, a thermal sensing image device, and an ultraviolet corona imager;

[0012] The acquisition of the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives includes:

[0013] Collect a visible light image of the power equipment to be detected from multiple perspectives through the image capturing device;

[0014] Collect the first infrared temperature distribution image of the power equipment to be detected from multiple perspectives captured by the thermal sensing image device;

[0015] Collect the first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives captured by the ultraviolet corona imager.

[0016] Furthermore, the training of the abnormal area detection model includes;

[0017] Obtain a multi-perspective image dataset of power equipment; wherein, the multi-perspective image dataset of power equipment includes a number of power equipment image groups and corresponding abnormal existence labels; the power equipment image group includes a training visible light image, a training infrared temperature distribution image, and a training partial discharge ultraviolet image of the power equipment from the same perspective; the abnormal existence label includes an abnormal existence information label of the power equipment and a corresponding abnormal area label;

[0018] Randomly divide the multi-perspective image dataset of power equipment into several batches of training samples according to a preset quantity;

[0019] Input the training samples of each batch into the abnormal area detection model in sequence to train the abnormal area detection model until the preset number of training times is reached. Among them, when the abnormal area detection model receives each batch of training samples, it outputs the probability of the existence of abnormality corresponding to the training samples and the suspected coordinates of the abnormal area. Calculate the value of the first loss function through the first loss function according to the probability of the existence of abnormality and the corresponding label of the existence information of the abnormality. Calculate the value of the second loss function through the second loss function according to the suspected coordinates of the abnormal area and the corresponding abnormal area label. Use the optimizer to update the abnormal area detection model according to the value of the first loss function and the value of the second loss function.

[0020] Further, before the step of inputting the training samples of each batch into the abnormal area detection model in sequence to train the abnormal area detection model until the preset number of training times is reached, it further includes:

[0021] For each batch of training samples, perform data augmentation processing and normalization processing to generate updated training samples. Among them, the data augmentation processing includes any one or a combination of random horizontal flipping, random vertical flipping, random rotation, random adjustment of image brightness, random addition of Gaussian noise, and random cropping processing.

[0022] Further, the power equipment detection method based on inspection drone data collection further includes:

[0023] In the case of determining that the power equipment to be detected is abnormal, obtain the geographical information of the power equipment to be detected.

[0024] Generate an equipment abnormality report according to the geographical information, the first visible light image, the first infrared temperature distribution image, the first partial discharge ultraviolet image, the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image.

[0025] Send the equipment abnormality report to the dispatching center.

[0026] Further, the power equipment detection method based on inspection drone data collection further includes:

[0027] In the case of determining that the power equipment to be detected is abnormal, increase the drone allocation priority of the inspection area where the power equipment to be detected is located.

[0028] On the basis of the above method item embodiments, the present invention correspondingly provides device item embodiments.

[0029] An embodiment of the present invention provides a power equipment detection device based on inspection UAV data collection, including: an image acquisition module, an abnormal information generation module, a power equipment re-inspection image acquisition module, a power equipment re-inspection module, and a power equipment detection module;

[0030] The image acquisition module is used to acquire the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives;

[0031] The abnormal information generation module is used to input the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective into a preset abnormal area detection model, so that the abnormal area detection model generates the first abnormal existence probability and the suspected coordinates of the abnormal area of the power equipment to be detected under the current perspective according to the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective;

[0032] The power equipment re-inspection image acquisition module is used to send a re-inspection instruction to an inspection UAV in the adjacent area of the inspection area where the power equipment to be detected is located when the abnormal existence probability of the power equipment to be detected in a certain perspective exceeds a preset probability threshold, so that the inspection UAV in the adjacent area feeds back the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image of the power equipment to be detected in the target perspective; wherein, the target perspective includes the first abnormal area;

[0033] The power equipment re-inspection module is used to input the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image into a preset abnormal area detection model, so that the abnormal area detection model generates the second abnormal existence probability of the power equipment to be detected under the current perspective according to the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image of the same perspective;

[0034] The power equipment detection module is used to judge whether the second abnormal existence probability exceeds a preset probability threshold. If so, it is determined that the power equipment to be detected is abnormal and an alarm signal is sent. If not, it is determined that the power equipment to be detected is normal.

[0035] Further, the power equipment detection device based on inspection UAV data collection further includes: an abnormal area detection model training module;

[0036] The abnormal area detection model training module is used to obtain a multi-perspective image dataset of power equipment. Among them, the multi-perspective image dataset of power equipment includes several groups of power equipment images and corresponding abnormal existence labels. The group of power equipment images includes training visible light images, training infrared temperature distribution images, and training partial discharge ultraviolet images of power equipment under the same perspective. The abnormal existence labels include abnormal existence information labels of power equipment and corresponding abnormal area labels. Randomly divide the multi-perspective image dataset of power equipment into several batches of training samples according to a preset quantity. Input each batch of training samples into the abnormal area detection model in turn to train the abnormal area detection model until the preset number of training times is reached. Among them, when the abnormal area detection model receives each batch of training samples, it outputs the abnormal existence probability corresponding to the training samples and the suspected coordinates of the abnormal area. Calculate the first loss function value through the first loss function according to the abnormal existence probability and the corresponding abnormal existence information label. Calculate the second loss function value through the second loss function according to the suspected coordinates of the abnormal area and the corresponding abnormal area label. Use the optimizer to update the abnormal area detection model according to the first loss function value and the second loss function value.

[0037] Based on the above method item embodiments, the present invention correspondingly provides an electronic device item embodiment.

[0038] An embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it can implement the power equipment detection method based on inspection UAV data collection described in any one of the above method item embodiments.

[0039] Based on the above method item embodiments, the present invention correspondingly provides a storage medium item embodiment.

[0040] An embodiment of the present invention provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, it can implement the power equipment detection method based on inspection UAV data collection described in any one of the above method item embodiments.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] An embodiment of the present invention provides a method, device, electronic device, and storage medium for detecting power equipment based on data collection by inspection drones. The method comprehensively detects the power equipment to be detected through multi-perspective image acquisition, including visible light images, infrared temperature distribution images, and partial discharge ultraviolet images. The three images from the same perspective are input into an abnormal area detection model to generate the probability of the existence of an abnormality and the coordinates of the suspected abnormal area. When the probability of the existence of an abnormality detected exceeds the threshold, a re-inspection instruction is sent to the drones in adjacent areas to obtain a second set of images of the equipment for verification, and the second probability of the existence of an abnormality is generated again through the detection model. If the second probability still exceeds the standard, it is determined that the equipment is abnormal and an alarm is issued.

[0043] The present invention synchronously inputs visible light, infrared, and ultraviolet images from the same perspective into a fusion model, overcoming the false alarm problem caused by independent processing of multi-modal data; when the detected probability of an abnormality exceeds the threshold, adjacent drones are automatically scheduled to perform multi-angle re-inspection on the target area, and the single detection error is effectively eliminated through cross-verification of multiple devices, significantly reducing the risk of misjudgment caused by sensor failures or environmental interference. Description of the Drawings

[0044] Figure 1 It is a schematic flowchart of a method for detecting power equipment based on data collection by inspection drones provided by an embodiment of the present invention.

[0045] Figure 2 It is a schematic structural diagram of a device for detecting power equipment based on data collection by inspection drones provided by an embodiment of the present invention. Detailed Embodiments

[0046] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0047] As Figure 1 shown, an embodiment of the present invention provides a method for detecting power equipment based on data collection by inspection drones, which at least includes the following steps:

[0048] Step S1: Obtain the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives;

[0049] In a preferred embodiment, the inspection drone is provided with an image capturing device, a thermal imaging device, and an ultraviolet corona imager;

[0050] The acquisition of the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the power equipment to be detected includes:

[0051] Collect a visible light image of the power equipment to be detected from multiple perspectives through the image capture device;

[0052] Collect the first infrared temperature distribution image of the power equipment to be detected from multiple perspectives captured by the thermal induction imaging device;

[0053] Collect the first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives captured by the ultraviolet corona imager.

[0054] Specifically, the image capture device collects visible light images of the target device from multiple perspectives through a preset flight path. Within the flight altitude range of 10 - 30 meters, with parameters of resolution ≥ 20 million pixels, exposure time of 1 / 500 seconds, and aperture f / 2.8, uniformly capture visible light images of at least 6 orthogonal perspectives along the circumferential direction of the device. At the same time, enable the HDR mode and multi-frame synthesis technology to improve the dynamic range (≥ 14EV), and record the attitude angle data of the unmanned aerial vehicle and synchronously store the image timestamps through the on-board IMU; the thermal induction imaging device starts within 0.5 seconds after the visible light collection is completed, and synchronously obtains the infrared thermal map of the corresponding perspective with a thermal sensitivity ≤ 0.05°C and a temperature measurement range of -20°C to 500°C. Combine the laser rangefinder to calibrate the shooting distance in real time (spatial resolution ≤ 5cm / pixel) and overlay the ambient temperature and humidity compensation parameters to generate a standardized temperature gradient distribution matrix; the ultraviolet corona imager starts during the gap between visible light and infrared collection, with a working band of 240 - 280nm and a minimum detection intensity ≤ 5pC. After suppressing the ambient light interference through a narrowband filter, capture the discharge ultraviolet photon signal at a frame rate of 25 frames per second, generate a spatial density map of the discharge spot in the photon counting mode, and achieve pixel-level spatial registration with the visible light image through the feature point matching algorithm. The three sensors achieve millisecond-level synchronization of cross-modal data through hardware trigger signals (time deviation ≤ 50ms), and ensure the unity of spatial coordinates based on the GPS / Beidou positioning system and inertial navigation data. Finally, store the multi-source data in the encrypted storage module in the format of timestamp - attitude angle - environmental parameter association.

[0055] Step S2: Input the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective into a preset abnormal area detection model, so that the abnormal area detection model generates the first abnormal existence probability of the power equipment to be detected in the current perspective and the suspected coordinates of the abnormal area according to the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective;

[0056] Specifically, the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective are input into a preset abnormal area detection model through a spatio-temporal registration interface. The model adopts a multi-branch convolutional neural network architecture. Among them, the visible light branch uses a pre-trained ResNet-50 network to extract the surface deformation and corrosion features of the device. The infrared branch analyzes the abnormal temperature gradient distribution through a lightweight MobileNet-V3 network. The ultraviolet branch uses dilated convolution to capture the spatio-temporal correlation features of the discharge spots. After the three-way feature maps are aligned by a spatial registration algorithm based on SIFT feature points, cross-modal feature fusion is performed through a cross-attention mechanism, and a pixel-level abnormal heat map is calculated and a three-dimensional abnormal probability field is generated. Among them, the abnormal existence probability is represented by the Sigmoid activation value output by the global average pooling layer (the value range is 0-1). The suspected coordinates of the abnormal area are extracted by the non-maximum suppression algorithm to extract the connected areas with a probability value ≥ 0.5 in the heat map, and their centroids are mapped to the device geographic coordinate system (longitude, latitude, altitude). At the same time, the three-dimensional bounding box parameters (length × width × height ± 0.1m accuracy) of the abnormal area are output. When the model is trained, a multi-task joint loss function is used. Among them, the abnormal probability prediction branch uses a weighted cross-entropy loss function to strengthen the small target detection ability, and the coordinate regression branch uses a Smooth L1 loss function to optimize the positioning accuracy, and finally realizes the accurate positioning of abnormalities under the collaborative analysis of multi-source heterogeneous data.

[0057] In a preferred embodiment, the training of the abnormal area detection model includes;

[0058] Obtain a multi-perspective image dataset of power equipment; wherein, the multi-perspective image dataset of power equipment includes several groups of power equipment images and corresponding abnormal existence labels; the power equipment image group includes the training visible light image, the training infrared temperature distribution image, and the training partial discharge ultraviolet image of the power equipment under the same perspective; the abnormal existence labels include the abnormal existence information label of the power equipment and the corresponding abnormal area label;

[0059] Randomly divide the multi-perspective image dataset of power equipment into several batches of training samples according to a preset quantity;

[0060] Input the training samples of each batch into the abnormal area detection model in sequence to train the abnormal area detection model until the preset number of training times is reached; among them, when the abnormal area detection model receives each batch of training samples, it outputs the probability of the existence of abnormalities corresponding to the training samples and the suspected coordinates of the abnormal area; calculate the value of the first loss function through the first loss function according to the probability of the existence of abnormalities and the corresponding abnormality existence information label; calculate the value of the second loss function through the second loss function according to the suspected coordinates of the abnormal area and the corresponding abnormal area label; use the optimizer to update the abnormal area detection model according to the value of the first loss function and the value of the second loss function.

[0061] Specifically, the training method of the abnormal area detection model specifically includes: First, construct a multi-view image dataset of power equipment, which is collected by an inspection UAV group deployed in different geographical regions, covering multi-modal data of typical power equipment such as transformers, insulators, and cable joints in normal, aging, and fault states. Each group of data includes visible light images (resolution ≥ 20 million pixels), infrared thermal maps (temperature accuracy ±0.1°C), and ultraviolet discharge images (photon counting sensitivity ≤ 5 pC) of the same equipment under the same spatio-temporal reference. And generate high-precision abnormal labels through a combination of laser trackers and manual annotation. Among them, the abnormal existence label is a binary classification identifier (0 / 1), and the abnormal area label uses the bounding box parameters (longitude, latitude, altitude of the center point, length, width, and height dimensions) in a three-dimensional geographic coordinate system; In the dataset preprocessing stage, perform spatio-temporal alignment on the multi-modal data, use an image registration algorithm based on SIFT feature points to eliminate perspective deviation, and achieve cross-modal data standardization through histogram equalization, temperature-emissivity calibration, and discharge intensity normalization; Subsequently, divide the dataset into a training set, a validation set, and a test set according to a ratio of 7:2:1. In the training stage, adopt a dynamic batch processing strategy, randomly select 32 groups of data in each batch and apply combined augmentation operations, including random horizontal / vertical flipping (probability 0.5), ±15-degree rotation, brightness fluctuation (±20%), Gaussian noise injection (σ = 0.01), and multi-modal synchronous cropping (retention rate ≥ 80%). The augmented data is input into the abnormal area detection model for multi-task joint training; When the model propagates forward, the visible light, infrared, and ultraviolet branches respectively extract features through pre-trained ResNet-50, MobileNet-V3, and a custom dilated convolution network, and generate a joint feature map through a cross-modal attention fusion layer. Among them, the abnormal existence probability is output by a global average pooling layer connected to a Sigmoid activation function, and the abnormal area coordinates are obtained by regression through a spatial transformation network; During the training process, the first loss function uses weighted cross-entropy loss (positive sample weight 3.0) to calculate the classification error, and the second loss function uses Smooth L1 loss to measure the coordinate regression deviation. The two constitute a joint loss function with a weight ratio of 0.7:0.3, update the parameters through an AdamW optimizer (learning rate 1e-4, weight decay 0.01), and adopt a cosine annealing learning rate scheduling strategy. Evaluate the model performance on the validation set every 10 epochs of training. When the validation loss does not decrease for 3 consecutive epochs, trigger the early stopping mechanism, and finally save the model parameters with the highest F1-score on the validation set for deployment.

[0062] In a preferred embodiment, before inputting each batch of training samples into the abnormal area detection model in sequence and training the abnormal area detection model until the preset number of training times is reached, it further includes:

[0063] For each batch of training samples, data augmentation processing and normalization processing are performed to generate updated training samples; among them, the data augmentation processing includes any one or a combination of random horizontal flipping, random vertical flipping, random rotation, random adjustment of image brightness, random addition of Gaussian noise, and random cropping processing.

[0064] Specifically, for each batch of training samples, multi-modal data collaborative augmentation and standardized preprocessing are performed. Specifically, random horizontal flipping (probability 0.5), vertical flipping (probability 0.3), and ±15-degree rotation operations are applied to the visible light image, and at the same time, a matching spatial transformation is performed on the infrared thermal map to maintain the geometric consistency of the multi-modal data. At the same time, Poisson distribution-based photon counting noise injection (λ = 0.1) is performed on the ultraviolet discharge image; in the brightness adjustment stage, ±20% linear brightness fluctuations are used for the visible light image, and the emissivity calibration parameters of the infrared thermal map are corrected synchronously to ensure the authenticity of the temperature gradient distribution; subsequently, cross-channel normalization processing is performed on the three-modal data. Among them, the visible light image is standardized using the mean ([0.485, 0.456, 0.406]) and standard deviation ([0.229, 0.224, 0.225]) of the ImageNet dataset. The infrared thermal map is linearly mapped to the [0, 1] interval according to the rated temperature range [-20°C, 500°C] of the device, and the ultraviolet discharge image is constrained within the range of [0, 255] for the photon count value through maximum-minimum normalization; finally, random cropping (cropping ratio 85%-100%) and bicubic interpolation are performed on all modal data to scale to a unified size (512×512 pixels) to generate an updated training sample set with geometric alignment, radiation correction, and enhanced noise robustness, ensuring the spatial correspondence relationship and physical dimension interpretability of the multi-modal features.

[0065] Step S3, when the probability of the existence of an abnormality of the power equipment to be detected in a certain perspective exceeds a preset probability threshold, a re-inspection instruction is sent to an inspection UAV in the adjacent area of the inspection area where the power equipment to be detected is located, so that the inspection UAV in the adjacent area feeds back the second visible light image, the second infrared temperature distribution image, and the second local discharge ultraviolet image of the power equipment to be detected in the target perspective; wherein, the target perspective encompasses the first abnormal area.

[0066] It should be noted here that when the probability of the abnormality existence of the power equipment to be detected under at least one perspective exceeds the dynamically adjusted preset probability threshold θ (θ = 0.6, which can also be flexibly adjusted according to the actual situation), the cross-region collaborative re-inspection protocol is triggered: based on the device's geographical coordinates and the real-time position information of the unmanned aerial vehicle (UAV), the central control system calculates the optimal scheduling path through an improved ant colony algorithm, and sends an encrypted re-inspection instruction packet to the adjacent idle UAVs within the range of 300 - 800 meters from the target device. The instruction packet includes the three-dimensional geographical coordinates (longitude, latitude, altitude, accuracy ±0.1 m) of the abnormal area, the normal vector of the target perspective (pitch angle α ∈ [-30°, 30°], azimuth angle β ∈ [0°, 360°]), and the acquisition parameter configuration (visible light exposure time 1 / 800 s, infrared temperature measurement sensitivity 0.02 °C, ultraviolet sampling rate 30 fps); the UAV receiving the instruction automatically flies to the target point along the Bezier curve path, starts to adjust the multi-axis gimbal attitude at a distance of 5 - 15 meters from the device surface, and makes the shooting perspective cover the geometric circumscribed sphere (radius expansion coefficient 1.2 - 1.5 times) of the first abnormal area through lidar-assisted visual servo control, and synchronously acquires the second visible light image (resolution 8192×5464 pixels), the second infrared temperature distribution image (thermal sensitivity 0.03 °C), and the second partial discharge ultraviolet image (photon counting dynamic range 16 bit), and synchronously encapsulates the three-modal data with the high-precision IMU attitude data (pitch angle error ≤ 0.1°) and the GPS differential positioning data (horizontal accuracy 2 cm, elevation accuracy 5 cm) through the 5G private network channel, and real-time transmits them back to the edge computing node in the form of an encrypted data stream for cross-device data fusion analysis.

[0067] Step S4: Input the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image into a preset abnormal area detection model, so that the abnormal area detection model generates the second probability of the abnormality existence of the power equipment to be detected under the current perspective according to the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image from the same perspective;

[0068] Specifically, input the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image into a preset abnormal area detection model. The model generates the second abnormal existence probability through the following process: First, perform spatial alignment on the three-modal re-inspection data. Based on the abnormal area coordinates in the initial inspection stage, align the re-inspection images with the initial inspection data at the pixel level through an image registration algorithm. Subsequently, input the re-inspection visible light, infrared, and ultraviolet images from the same perspective as a whole into the model, reuse the multi-modal fusion analysis framework in the initial inspection stage, extract the device surface deformation characteristics, temperature gradient changes, and discharge intensity distribution in the re-inspection images, and compare the feature differences between the initial inspection and re-inspection data. Finally, output the second abnormal existence probability under the current perspective. This probability value synthesizes the consistency analysis results of multi-source data in the two detections and is used to verify the authenticity of the initial inspection abnormality, thereby eliminating the misjudgment risk caused by environmental interference or instantaneous errors in single detections.

[0069] Step S5: Determine whether the second abnormal existence probability exceeds a preset probability threshold. If it is, determine that the power device to be detected is abnormal and send an alarm signal. If not, determine that the power device to be detected is normal.

[0070] Specifically, when determining whether the second abnormal existence probability exceeds a preset probability threshold, if the probability value is greater than or equal to the threshold (default set to 0.65), determine that the power device to be detected is abnormal and trigger the alarm signal generation process. The alarm signal includes the geographical coordinates (longitude, latitude, altitude) of the abnormal device, the preliminary classification code of the abnormal type (generated based on the combined characteristics of visible light deformation, infrared temperature gradient, and ultraviolet discharge intensity), and the recommended disposal measure index number. At the same time, associate the abnormal detection result with the device historical maintenance database to match similar fault cases. If the probability value is lower than the threshold, determine that the device status is normal, update the detection cycle parameter in the device health record, and reduce the detection priority of this device within the next 30 days. During this process, the system automatically records the time series of the initial inspection and re-inspection data and the environmental condition parameters (temperature, humidity, wind speed) to optimize the dynamic learning weights of the abnormal area detection model.

[0071] In a preferred embodiment, the power device detection method based on inspection drone data collection further includes:

[0072] In the case of determining that the power device to be detected is abnormal, obtain the geographical information of the power device to be detected;

[0073] Generate a device abnormality report according to the geographical information, the first visible light image, the first infrared temperature distribution image, the first partial discharge ultraviolet image, the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image;

[0074] Send the abnormal report of the device to the dispatching center.

[0075] Specifically, when it is determined that the power equipment to be detected is abnormal, the system automatically retrieves the preset geographical coordinate information (including longitude, latitude and equipment installation height) in the equipment registration database, and combines the first visible light image, infrared temperature distribution image, partial discharge ultraviolet image in the initial inspection stage and the second visible light image, infrared temperature distribution image, partial discharge ultraviolet image in the re-inspection stage. Through the multi-modal data fusion engine, a structured abnormal report is generated. The report includes the relative position annotation of the abnormal area on the equipment body (based on the mapping relationship between the image coordinate system and the three-dimensional model of the equipment), the preliminary analysis result of the abnormal type (such as "connector overheating - discharge composite defect") and the environmental correlation parameters (temperature, humidity, wind speed at the time of collection). Finally, the report is compressed into a standardized data packet through a dedicated communication protocol and pushed to the operation and maintenance management platform of the power grid dispatching center in real time, and the position of the abnormal equipment and the associated line topology information are highlighted in the GIS map of the platform, and the work order dispatching system is triggered synchronously to generate a work order instruction including the equipment number, abnormal level and recommended maintenance time.

[0076] In a preferred embodiment, the power equipment detection method based on inspection drone data collection further includes:

[0077] When it is determined that the power equipment to be detected is abnormal, the allocation priority of the drone in the inspection area where the power equipment to be detected is located is increased.

[0078] Specifically, when it is determined that the power equipment to be detected is abnormal, the system automatically raises the drone task priority in the inspection area where the equipment is located to the highest level, which is specifically manifested as: immediately interrupting other non-emergency inspection tasks in this area, reallocating at least two spare drones to fly to the abnormal area in coordination, and increasing the inspection frequency from the regular once per hour to once every 15 minutes for intensive monitoring; at the same time, in the global task queue of the central dispatching system, double computing resources and communication bandwidth are dynamically allocated for the tasks related to this abnormal equipment to ensure that the high-definition images and sensor data collected by multiple drones are transmitted back to the analysis platform in real time; according to the growth trend of the abnormal probability value, if the abnormal probability exceeds the threshold in two consecutive re-inspections, the monitoring range is further extended to adjacent equipment, and a temporary warning airspace with a coverage radius of 500 meters is generated to prohibit unauthorized drones from entering and interfering with the detection operation.

[0079] As Figure 2 shown, an embodiment of the present invention provides a power equipment detection device based on inspection drone data collection, including: an image acquisition module, an abnormal information generation module, a power equipment re-inspection image acquisition module, a power equipment re-inspection module, and a power equipment detection module;

[0080] The image acquisition module is used to acquire the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives;

[0081] The abnormal information generation module is used to input the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective into a preset abnormal area detection model, so that the abnormal area detection model generates the first abnormal existence probability and the suspected coordinates of the abnormal area of the power equipment to be detected under the current perspective according to the first visible light image, the first infrared temperature distribution image, and the first partial discharge ultraviolet image of the same perspective;

[0082] The power equipment re-inspection image acquisition module is used to send a re-inspection instruction to a patrol unmanned aerial vehicle in the adjacent area of the patrol area where the power equipment to be detected is located when the abnormal existence probability of the power equipment to be detected in a certain perspective exceeds a preset probability threshold, so that the patrol unmanned aerial vehicle in the adjacent area feeds back the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image of the power equipment to be detected in the target perspective; wherein, the target perspective includes the first abnormal area;

[0083] The power equipment re-inspection module is used to input the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image into a preset abnormal area detection model, so that the abnormal area detection model generates the second abnormal existence probability of the power equipment to be detected under the current perspective according to the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image of the same perspective;

[0084] The power equipment detection module is used to determine whether the second abnormal existence probability exceeds a preset probability threshold. If so, it determines that the power equipment to be detected is abnormal and sends an alarm signal. If not, it determines that the power equipment to be detected is normal.

[0085] In a preferred embodiment, the power equipment detection device based on patrol unmanned aerial vehicle data acquisition further includes: an abnormal area detection model training module;

[0086] The abnormal area detection model training module is used to obtain a multi-view image dataset of power equipment; wherein, the multi-view image dataset of power equipment includes several groups of power equipment images and corresponding abnormal existence labels; each group of power equipment images includes training visible light images, training infrared temperature distribution images, and training partial discharge ultraviolet images of power equipment from the same perspective; the abnormal existence labels include abnormal existence information labels of power equipment and corresponding abnormal area labels; randomly divide the multi-view image dataset of power equipment into several batches of training samples according to a preset quantity; sequentially input each batch of training samples into the abnormal area detection model to train the abnormal area detection model until a preset number of training times is reached; wherein, when the abnormal area detection model receives each batch of training samples, it outputs the abnormal existence probability corresponding to the training samples and the suspected coordinates of the abnormal area; calculate the first loss function value through the first loss function according to the abnormal existence probability and the corresponding abnormal existence information label; calculate the second loss function value through the second loss function according to the suspected coordinates of the abnormal area and the corresponding abnormal area label; use the optimizer to update the abnormal area detection model according to the first loss function value and the second loss function value.

[0087] It should be noted that the embodiments of the device described above correspond to the above embodiments of the present invention, and can implement the power equipment detection method based on inspection drone data collection described in any one of the above of the present invention. In addition, the embodiments of the above device are merely illustrative. The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement without creative work.

[0088] Based on the above method embodiment of the present invention, an embodiment of an electronic device is correspondingly provided.

[0089] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the power equipment detection method based on inspection drone data collection described in any one of the present invention, or when the processor executes the computer program, it implements the functions of each module in the above device embodiments.

[0090] Exemplarily, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the terminal device.

[0091] The terminal device may be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0092] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device, and connects various parts of the entire terminal device through various interfaces and circuits.

[0093] The memory may be used to store the computer program and / or modules. The processor realizes various functions of the terminal device by running or executing the computer program and / or modules stored in the memory, and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0094] Based on the above method item embodiments, the present invention correspondingly provides storage medium item embodiments;

[0095] Another embodiment of the present invention provides a storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the storage medium is located to execute any one of the above-mentioned power equipment detection methods based on inspection UAV data collection of the present invention.

[0096] Among them, the above storage medium is a computer-readable storage medium. The computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0097] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without conflict, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0098] The above is the preferred embodiment of the present invention. It should be noted that for those of ordinary skill in the art of this technology, without departing from the principle of the present invention, several improvements and refinements can still be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A method for detecting electric power equipment based on data collection by inspection drones, characterized in that: include: Acquire a first visible light image, a first infrared temperature distribution image, and a first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives; Inputting the first visible light image, the first infrared temperature distribution image and the first partial discharge ultraviolet image at the same viewing angle into a preset abnormal area detection model, so that the abnormal area detection model generates the first abnormality existence probability of the power equipment to be detected at the current viewing angle and the suspected coordinates of the abnormal area according to the first visible light image, the first infrared temperature distribution image and the first partial discharge ultraviolet image at the same viewing angle; When the probability of the existence of an abnormality of the power equipment to be detected at a certain viewing angle exceeds a preset probability threshold, a re-inspection instruction is sent to an inspection drone in an adjacent area of ​​the inspection area where the power equipment to be detected is located, so that the inspection drone in the adjacent area feeds back a second visible light image, a second infrared temperature distribution image and a second partial discharge ultraviolet image of the power equipment to be detected at a target viewing angle; wherein the target viewing angle includes the first abnormal area; Inputting the second visible light image, the second infrared temperature distribution image and the second partial discharge ultraviolet image into a preset abnormal area detection model, so that the abnormal area detection model generates a second abnormality existence probability of the power equipment to be detected at the current viewing angle according to the second visible light image, the second infrared temperature distribution image and the second partial discharge ultraviolet image at the same viewing angle; It is determined whether the probability of the existence of the second abnormality exceeds a preset probability threshold. If so, it is determined that the power equipment to be detected is abnormal and an alarm signal is sent. If not, it is determined that the power equipment to be detected is normal.

2. The power equipment detection method based on inspection drone data collection according to claim 1 is characterized in that: The inspection drone is provided with an image capture device, a thermal imaging device and an ultraviolet corona imager; The method of acquiring a first visible light image, a first infrared temperature distribution image, and a first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives includes: The image capture device is used to collect a visible light image of the power equipment to be inspected under multiple viewing angles; Collecting a first infrared temperature distribution image of the power equipment to be inspected under multiple viewing angles captured by the thermal imaging device; The first partial discharge ultraviolet images of the power equipment to be inspected under multiple viewing angles captured by the ultraviolet corona imager are collected.

3. The power equipment detection method based on inspection drone data collection as claimed in claim 2 is characterized in that: The training of the abnormal region detection model includes: Acquire a multi-view image dataset of electric power equipment; wherein the multi-view image dataset of electric power equipment includes a plurality of electric power equipment image groups and corresponding abnormal existence labels; the electric power equipment image groups include training visible light images, training infrared temperature distribution images and training partial discharge ultraviolet images of electric power equipment at the same viewing angle; the abnormal existence labels include abnormal existence information labels of electric power equipment and corresponding abnormal area labels; Randomly dividing the multi-view image dataset of electric power equipment into a plurality of batches of training samples according to a preset number; Input each batch of training samples into the abnormal area detection model in turn, and train the abnormal area detection model until the preset number of training times is reached; wherein, when the abnormal area detection model receives each batch of training samples, it outputs the probability of abnormal existence and the suspected coordinates of the abnormal area corresponding to the training sample; calculates the first loss function value through the first loss function according to the probability of abnormal existence and the corresponding abnormal existence information label; calculates the second loss function value through the second loss function according to the suspected coordinates of the abnormal area and the corresponding abnormal area label; and uses the optimizer to update the abnormal area detection model according to the first loss function value and the second loss function value.

4. The power equipment detection method based on inspection drone data collection as claimed in claim 3 is characterized in that: In the step of sequentially inputting the training samples of each batch into the abnormal region detection model and training the abnormal region detection model until a preset number of training times is reached, the method further includes: For each batch of training samples, data enhancement processing and normalization processing are performed to generate updated training samples; wherein the data enhancement processing includes any one or a combination of random horizontal flipping, random vertical flipping, random rotation, random adjustment of image brightness, random addition of Gaussian noise and random cropping.

5. The power equipment detection method based on inspection drone data collection as claimed in claim 4 is characterized in that: Also includes: When it is determined that the power equipment to be detected is abnormal, obtaining geographic information of the power equipment to be detected; Generate a device abnormality report according to the geographic information, the first visible light image, the first infrared temperature distribution image, the first partial discharge ultraviolet image, the second visible light image, the second infrared temperature distribution image, and the second partial discharge ultraviolet image; The equipment abnormality report is sent to the dispatch center.

6. The power equipment detection method based on inspection drone data collection as claimed in claim 5 is characterized in that: Also includes: When it is determined that the power equipment to be inspected is abnormal, the priority of allocating drones to the inspection area where the power equipment to be inspected is located is increased.

7. A power equipment detection device based on inspection drone data collection, characterized in that: include: Image acquisition module, abnormal information generation module, power equipment re-inspection image acquisition module, power equipment re-inspection module and power equipment detection module; The image acquisition module is used to acquire a first visible light image, a first infrared temperature distribution image and a first partial discharge ultraviolet image of the power equipment to be detected from multiple perspectives; The abnormal information generation module is used to input the first visible light image, the first infrared temperature distribution image and the first partial discharge ultraviolet image at the same viewing angle into a preset abnormal area detection model, so that the abnormal area detection model generates the first abnormality existence probability of the power equipment to be detected at the current viewing angle and the suspected coordinates of the abnormal area according to the first visible light image, the first infrared temperature distribution image and the first partial discharge ultraviolet image at the same viewing angle; The electric power equipment re-inspection image acquisition module is used to send a re-inspection instruction to an inspection drone in an adjacent area of ​​the inspection area where the electric power equipment to be inspected is located when the probability of the existence of an abnormality of the electric power equipment to be inspected at a certain viewing angle exceeds a preset probability threshold, so that the inspection drone in the adjacent area can feed back a second visible light image, a second infrared temperature distribution image and a second partial discharge ultraviolet image of the electric power equipment to be inspected at a target viewing angle; wherein the target viewing angle includes the first abnormal area; The electric power equipment re-inspection module is used to input the second visible light image, the second infrared temperature distribution image and the second partial discharge ultraviolet image into a preset abnormal area detection model, so that the abnormal area detection model generates a second abnormality existence probability of the electric power equipment to be detected at a current viewing angle according to the second visible light image, the second infrared temperature distribution image and the second partial discharge ultraviolet image at the same viewing angle; The power equipment detection module is used to determine whether the probability of the existence of the second abnormality exceeds a preset probability threshold. If so, it is determined that the power equipment to be detected is abnormal and an alarm signal is sent. If not, it is determined that the power equipment to be detected is normal.

8. The power equipment detection device based on inspection drone data collection as claimed in claim 7 is characterized in that: Also includes: Abnormal area detection model training module; The abnormal area detection model training module is used to obtain a multi-view image data set of power equipment; wherein the multi-view image data set of power equipment includes several power equipment image groups and corresponding abnormal existence labels; the power equipment image groups include training visible light images, training infrared temperature distribution images and training partial discharge ultraviolet images of power equipment at the same viewing angle; the abnormal existence labels include abnormal existence information labels of power equipment and corresponding abnormal area labels; the multi-view image data set of power equipment is randomly divided into several batches of training samples according to a preset number; the training samples of each batch are sequentially input into the abnormal area detection model to train the abnormal area detection model until the preset number of training times is reached; wherein, when the abnormal area detection model receives each batch of training samples, it outputs the abnormal existence probability and the suspected coordinates of the abnormal area corresponding to the training sample; according to the abnormal existence probability and the corresponding abnormal existence information label, a first loss function value is calculated by a first loss function; according to the suspected coordinates of the abnormal area and the corresponding abnormal area label, a second loss function value is calculated by a second loss function; and the abnormal area detection model is updated by an optimizer according to the first loss function value and the second loss function value.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the power equipment detection method based on patrol drone data collection as described in any one of claims 1 to 6 can be implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it can implement the power equipment detection method based on patrol drone data collection as described in any one of claims 1 to 6.

Images