A power supply site inspection method and system based on a UAV

Abstract

The application provides a power supply site inspection method and system based on a UAV; wherein the method comprises: receiving first image data of a power supply site shot by a UAV, and performing first feature extraction on the first image data; inputting the first feature into a first abnormality judgment model, and judging whether the output result of the first abnormality judgment model is abnormal; if yes, determining and obtaining second image data according to the first image data, performing second feature extraction on the second image data, and inputting the second feature into a second abnormality judgment model; and determining whether the power supply site is abnormal according to the output result of the second abnormality judgment model. The application simultaneously performs abnormality judgment on the first feature of the power equipment itself and the second feature that may cause false "abnormality", so that the accuracy of the abnormality judgment of the power equipment in the UAV inspection process can be significantly improved, and the false alarm probability can be reduced.

Description

Technical Field

[0001] This invention relates to the field of power grid inspection technology, and more specifically, to a method, system, electronic device, and computer storage medium for on-site power grid inspection based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Intelligent inspection drones are an important product of the new century's artificial intelligence and virtual reality technologies. In power grid inspection practice, the use of intelligent inspection drones can record various data in a timely and complete manner, achieve comprehensive monitoring, promptly detect potential hazards in production, and ensure the safe and reliable operation of the power system.

[0003] Current drone inspection technologies typically utilize image recognition, thermal imaging, and other detection methods to extract features from power equipment and determine whether it is in an abnormal state based on preset judgment rules or deep recognition models. However, there are true and false "abnormal" states for power equipment. For example, when workers are maintaining power facilities, the state of the facilities may also be "abnormal." Current technologies have limited research on this situation and cannot effectively solve the problem of misjudgment in such cases. Summary of the Invention

[0004] In order to at least solve the technical problems existing in the background art, the present invention provides a method, system, electronic device and computer storage medium for on-site power supply inspection based on UAV.

[0005] The first aspect of the present invention provides a method for on-site power supply inspection based on unmanned aerial vehicles (UAVs), comprising the following steps:

[0006] Receive first image data of the power site captured by the drone, and perform first feature extraction on the first image data;

[0007] The first feature is input into the first anomaly detection model, and the output of the first anomaly detection model is determined to be an anomaly.

[0008] If so, then determine and obtain the second image data based on the first image data, perform second feature extraction on the second image data, and input the second feature into the second anomaly judgment model;

[0009] The presence of anomalies at the power site is determined based on the output of the second anomaly detection model.

[0010] Furthermore, the method also includes:

[0011] Determine whether a manual control command has been received, and / or determine whether the current date and / or time meets the first preset condition, and / or determine whether an alarm signal from the power alarm system has been received;

[0012] If so, an inspection control command is generated, wherein the inspection control command is used to trigger the UAV to inspect the power supply site.

[0013] Further, the first feature extraction of the first image data includes:

[0014] The first image data is subjected to power equipment object recognition processing. Based on the recognition processing, line features are extracted from the power equipment object. The inspection difficulty of the power equipment object is determined based on the line feature extraction.

[0015] The magnification factor of the first image data is determined based on the inspection difficulty, and the first image data is magnified based on the magnification factor to obtain the third image data;

[0016] The third image data is subjected to first feature extraction.

[0017] Further, the step of determining and obtaining the second image data based on the first image data includes:

[0018] The objects in the first image data are identified, and the contour features of each object are extracted based on the identification process.

[0019] Based on the outline features, determine whether each of the objects meets the second preset condition;

[0020] Image data of the area where the object is located that does not meet the second preset condition is used as the second image data.

[0021] Further, the step of inputting the first feature into the first anomaly detection model and determining whether the output of the first anomaly detection model is anomaly includes:

[0022] The first feature is input into the first anomaly detection model, and the first anomaly detection model outputs an anomaly evaluation value.

[0023] If the anomaly assessment value is greater than or equal to the first threshold, then the output result of the first anomaly judgment model is determined to be anomaly;

[0024] If the anomaly assessment value is less than the first threshold and greater than or equal to the second threshold, then the first feature is input into the third anomaly judgment model. If the matching result output by the third anomaly judgment model is not empty, then the output result of the first anomaly judgment model is determined to be anomaly.

[0025] Furthermore, after determining whether there is an anomaly at the power site based on the output of the second anomaly judgment model, the process further includes:

[0026] The anomaly determination result is output to the staff for confirmation, and the output object of the anomaly determination result is determined based on the confirmation result.

[0027] Furthermore, the first anomaly detection model, the second anomaly detection model, and the third anomaly detection model are all constructed using deep learning algorithms.

[0028] A second aspect of the present invention provides a power supply field inspection system based on a drone, comprising a receiving module, a processing module, and a storage module; the processing module is connected to the receiving module and the storage module.

[0029] The storage module is used to store executable computer program code;

[0030] The receiving module is used to receive the first image data of the power site captured by the drone and transmit it to the processing module;

[0031] The processing module is configured to execute the method described in the preceding one by invoking the executable computer program code in the storage module.

[0032] A third aspect of the present invention provides an electronic device comprising: a memory storing executable program code; a processor coupled to the memory; the processor invoking the executable program code stored in the memory to perform the method as described in any of the preceding claims.

[0033] A fourth aspect of the present invention provides a computer storage medium storing a computer program that, when executed by a processor, performs the method described in any of the preceding claims.

[0034] The present invention employs two anomaly detection models: a first anomaly detection model and a second anomaly detection model. The first anomaly detection model analyzes whether the power equipment itself exhibits an "anomaly" based on extracted first features. Then, it extracts other interfering features (second features) that might lead to false alarms, thereby verifying the anomaly detection result output by the first anomaly detection model. Therefore, by simultaneously performing anomaly detection on both the first features of the power equipment and the second features that might lead to false alarms, the present invention significantly improves the accuracy of power equipment anomaly detection during UAV inspections and reduces the probability of false alarms. Attached Figure Description

[0035] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is a flowchart illustrating a power supply field inspection method based on a drone, as disclosed in an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram of a power supply field inspection system based on a drone disclosed in an embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0040] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “said,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms, and “multiple” generally includes at least two unless the context clearly indicates otherwise.

[0041] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0042] It should be understood that although the terms first, second, third, etc., may be used to describe ... in the embodiments of this application, these ... should not be limited to these terms. These terms are only used to distinguish .... For example, without departing from the scope of the embodiments of this application, first ... can also be referred to as second ..., and similarly, second ... can also be referred to as first ....

[0043] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to detection.” Similarly, depending on the context, the phrases “if determination” or “if detection (of the stated condition or event)” can be interpreted as “when determination” or “in response to determination” or “when detection (of the stated condition or event)” or “in response to detection (of the stated condition or event).”

[0044] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0045] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0046] Example 1

[0047] Please see Figure 1 , Figure 1 This is a flowchart illustrating a method for on-site power supply inspection based on a drone, as disclosed in an embodiment of the present invention. Figure 1 As shown in the figure, an embodiment of the present invention provides a power supply field inspection method based on a drone, which includes the following steps:

[0048] Receive first image data of the power site captured by the drone, and perform first feature extraction on the first image data;

[0049] The first feature is input into the first anomaly detection model, and the output of the first anomaly detection model is determined to be an anomaly.

[0050] If so, then determine and obtain the second image data based on the first image data, perform second feature extraction on the second image data, and input the second feature into the second anomaly judgment model;

[0051] The presence of anomalies at the power site is determined based on the output of the second anomaly detection model.

[0052] In this embodiment of the invention, existing technologies for power equipment inspection mainly focus on improving the extraction of abnormal features, while neglecting the "anomalies" caused by normal maintenance operations, which can easily lead to false alarms. Therefore, this invention sets up two anomaly judgment models: a first anomaly judgment model and a second anomaly judgment model. The first anomaly judgment model analyzes whether the power equipment itself has an "anomaly" based on the extracted first feature of the power equipment, and then extracts other interfering features, i.e., second features, that may lead to false "anomalies." The anomaly judgment result output by the first anomaly judgment model is then verified. Thus, by simultaneously judging the "anomaly" of the power equipment itself and the second features that may lead to false "anomalies," this invention can significantly improve the accuracy of power equipment anomaly judgment during UAV inspections and reduce the probability of false alarms.

[0053] The implementing entity of this invention can be the processing equipment or server of an inspection drone. The processing equipment can be a processor or a mobile device. The processor can be a Central Processing Unit (CPU), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices. The mobile device can be a smartphone, tablet, computer, wearable device, virtual / augmented reality device, etc. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0054] Furthermore, the method also includes:

[0055] Determine whether a manual control command has been received, and / or determine whether the current date and / or time meets the first preset condition, and / or determine whether an alarm signal from the power alarm system has been received;

[0056] If so, an inspection control command is generated, wherein the inspection control command is used to trigger the UAV to inspect the power supply site.

[0057] In this embodiment of the invention, the invention provides the above three methods for triggering drones to inspect power sites: management personnel manually determine inspection needs, automatically control inspections based on single or periodically set inspection dates and / or times, and automatically control inspections in response to alarm signals from the power alarm system. These three inspection triggering methods can be used individually or in combination. When used in combination, any one of them needs to be met to trigger the drone to perform the inspection task. Of course, in addition to the above three methods, the invention does not exclude other triggering methods; however, since they cannot be exhaustively listed, they will not be elaborated upon here.

[0058] Further, the first feature extraction of the first image data includes:

[0059] The first image data is subjected to power equipment object recognition processing. Based on the recognition processing, line features are extracted from the power equipment object. The inspection difficulty of the power equipment object is determined based on the line feature extraction.

[0060] The magnification factor of the first image data is determined based on the inspection difficulty, and the first image data is magnified based on the magnification factor to obtain the third image data;

[0061] The third image data is subjected to first feature extraction.

[0062] In this embodiment of the invention, due to the wide variety of power equipment and their varying complexity and detail—for example, power cables and surge arresters are basically just simple straight lines, while transformers have many complex lines—this invention first extracts line features from the inspection objects in the first image data to analyze their inspection difficulty. Based on this, the first image data is magnified, and the magnification factor is positively correlated with the inspection difficulty. That is, the more complex the line features of the power equipment (more lines, more curves), the greater the inspection difficulty, and the higher the magnification factor is set accordingly. This results in third image data containing more details and fewer interferences, which significantly improves the accuracy of the first feature extraction. A correlation between the number of lines, the number of curves, and the inspection difficulty can be established in advance, and the relationship can be quickly determined by looking up a table during use.

[0063] It should be noted that the first image data in this embodiment of the present invention can be conventional image data, but the clarity will be compromised after magnification. Therefore, it is preferable to set the first image data as real-time video image data, so that a clearer close-up image can be obtained by adjusting the optical focal length of the drone.

[0064] Further, the step of determining and obtaining the second image data based on the first image data includes:

[0065] The objects in the first image data are identified, and the contour features of each object are extracted based on the identification process.

[0066] Based on the outline features, determine whether each of the objects meets the second preset condition;

[0067] Image data of the area where the object is located that does not meet the second preset condition is used as the second image data.

[0068] In this embodiment of the invention, when workers perform maintenance operations on electrical equipment, they generally need to place corresponding tools on-site and may also disassemble the electrical equipment. This results in the outline features of the maintained electrical equipment being significantly different from those of normal electrical equipment on-site. The second preset condition is mainly based on connection features (e.g., normal connection of power cables to transformers and other equipment) and routing features. These two features reflect that the layout of various electrical equipment on-site is complete and the connections are normal. Therefore, this invention quickly filters out the feature areas of potential electrical equipment being maintained by workers by analyzing the relationship between the outline features of the object and the second preset condition, thereby determining the second image data. Compared with traditional image recognition technologies such as worker identification and specific equipment identification, the solution of this invention is significantly easier to identify, reduces the computational load, and improves the determination rate of the second image data.

[0069] It should be noted that this embodiment of the present invention does not extract all the line features of the object, but only the outer contour features. This setting can reduce the amount of data processing on the one hand, and avoid the complex inner line features of the object itself from causing the power equipment to be mistakenly identified as the aforementioned tool or disassembled parts assembly.

[0070] Further, the step of inputting the first feature into the first anomaly detection model and determining whether the output of the first anomaly detection model is anomaly includes:

[0071] The first feature is input into the first anomaly detection model, and the first anomaly detection model outputs an anomaly evaluation value.

[0072] If the anomaly assessment value is greater than or equal to the first threshold, then the output result of the first anomaly judgment model is determined to be anomaly;

[0073] If the anomaly assessment value is less than the first threshold and greater than or equal to the second threshold, then the first feature is input into the third anomaly judgment model. If the matching result output by the third anomaly judgment model is not empty, then the output result of the first anomaly judgment model is determined to be anomaly.

[0074] In this embodiment of the invention, the output of the first anomaly detection model is analyzed in two aspects: if the anomaly evaluation value based on line feature analysis is greater than or equal to a first threshold, it is directly determined to be an anomaly; if the anomaly evaluation value is less than the first threshold but greater than or equal to a second threshold, which is an intermediate state that is neither sufficiently abnormal nor sufficiently normal, the first feature is further input into a third anomaly detection model for matching calculation. If a match is found, it indicates that the intermediate state is a preset anomaly. The third anomaly detection model is pre-trained and can calculate the matching degree between the first feature and several pre-stored anomaly templates, thereby determining whether an anomaly exists.

[0075] This embodiment of the invention adopts different judgment methods for different situations. Line feature analysis is relatively simple and quick, while the matching calculation of the third anomaly judgment model can ensure the accuracy of recognition, thus achieving a balance between recognition accuracy and recognition efficiency.

[0076] Furthermore, after determining whether there is an anomaly at the power site based on the output of the second anomaly judgment model, the process further includes:

[0077] The anomaly determination result is output to the staff for confirmation, and the output object of the anomaly determination result is determined based on the confirmation result.

[0078] In this embodiment of the invention, after obtaining the anomaly determination result through the aforementioned steps, the invention further outputs the anomaly determination result to the staff for confirmation. If the staff approves the anomaly result identified by the inspection drone, the anomaly determination result can be output to the corresponding object to facilitate timely handling of the anomaly. The output objects include various electrical control devices, staff, etc., specifically determined according to the electrical equipment experiencing the anomaly and the anomaly level, which will not be elaborated further here.

[0079] Furthermore, the first anomaly detection model, the second anomaly detection model, and the third anomaly detection model are all constructed using deep learning algorithms.

[0080] In the embodiments of this invention, the deep learning algorithms involved may be Convolutional Neural Networks (CNN), Feed Forward Neural Networks (FF), Deep Feed-forward Neural Networks (DFF), Radial Basis Networks (RBN), Recurrent Neural Networks (RNN), etc. This invention does not limit these algorithms, and the specific construction methods are mature existing technologies, which will not be elaborated upon in this invention.

[0081] Example 2

[0082] Please see Figure 2 , Figure 2 This is a schematic diagram of a power supply field inspection system based on a drone, as disclosed in an embodiment of the present invention. Figure 2 As shown, an embodiment of the present invention provides a power supply field inspection system based on a drone, comprising a receiving module (101), a processing module (102), and a storage module (103); the processing module (102) is connected to the receiving module (101) and the storage module (103);

[0083] The storage module (103) is used to store executable computer program code;

[0084] The receiving module (101) is used to receive the first image data of the power site captured by the UAV and transmit it to the processing module (102);

[0085] The processing module (102) is configured to execute the method described in the preceding one by invoking the executable computer program code in the storage module (103).

[0086] The specific functions of the power supply field inspection system based on UAV in this embodiment are the same as those in Embodiment 1 above. Since the system in this embodiment adopts all the technical solutions of the above embodiments, it has at least all the beneficial effects brought about by the technical solutions of the above embodiments, which will not be repeated here.

[0087] Example 3

[0088] Please see Figure 3 , Figure 3 This invention discloses an electronic device comprising: a memory storing executable program code; a processor coupled to the memory; the processor calling the executable program code stored in the memory to execute the method described in Embodiment 1.

[0089] Example 4

[0090] This invention also discloses a computer storage medium storing a computer program, which is executed by a processor as described in Embodiment 1.

[0091] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device.

[0092] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, system, or device.

[0093] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0094] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, all of which fall within the scope of protection of the present invention.

Claims

1. A method for on-site power supply inspection based on unmanned aerial vehicles (UAVs), characterized in that, The process includes the following steps: receiving first image data of a power site captured by a drone; extracting a first feature from the first image data; inputting the first feature into a first anomaly detection model; determining whether the output of the first anomaly detection model is an anomaly; if so, determining and acquiring second image data based on the first image data; extracting a second feature from the second image data; inputting the second feature into a second anomaly detection model; and determining whether an anomaly exists at the power site based on the output of the second anomaly detection model. The step of determining and acquiring second image data based on the first image data includes: performing identification processing on each object in the first image data; extracting contour features of each object based on the identification processing; determining whether each object meets a second preset condition based on the contour features; and using the image data of the area where the object that does not meet the second preset condition is located as the second image data; wherein the second image data corresponds to the feature area of ​​a potential worker maintaining power equipment. The first feature extraction of the first image data includes: performing power equipment object recognition processing on the first image data; performing line feature extraction on the power equipment object based on the recognition processing; determining the inspection difficulty of the power equipment object based on the line feature extraction; determining the magnification factor of the first image data based on the inspection difficulty; performing magnification processing on the first image data based on the magnification factor to obtain third image data; and performing the first feature extraction on the third image data. The step of inputting the first feature into the first anomaly detection model and determining whether the output result of the first anomaly detection model is abnormal includes: inputting the first feature into the first anomaly detection model, and the first anomaly detection model outputting an anomaly evaluation value; if the anomaly evaluation value is greater than or equal to a first threshold, then determining that the output result of the first anomaly detection model is abnormal; if the anomaly evaluation value is less than the first threshold but greater than or equal to a second threshold, then inputting the first feature into a third anomaly detection model, and if the matching result output by the third anomaly detection model is not empty, then determining that the output result of the first anomaly detection model is abnormal.

2. The method for on-site power supply inspection based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that... The method further includes: determining whether a manual control command has been received, and / or determining whether the current date and / or time meets a first preset condition, and / or determining whether an alarm signal from a power alarm system has been received; if so, generating an inspection control command, wherein the inspection control command is used to trigger the UAV to inspect the power supply site.

3. The method for on-site power supply inspection based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that... After determining whether there is an anomaly at the power site based on the output result of the second anomaly judgment model, the method further includes: outputting the anomaly judgment result to the staff for confirmation, and determining the output object of the anomaly judgment result based on the confirmation result.

4. The method for on-site power supply inspection based on unmanned aerial vehicles (UAVs) according to claim 1, characterized in that... Both the first anomaly detection model and the second anomaly detection model are constructed using deep learning algorithms.

5. An artificial intelligence-based inspection plan formulation system, comprising a receiving module, a processing module, and a storage module; the processing module is connected to the receiving module and the storage module; the storage module is used to store executable computer program code; the receiving module is used to receive first image data of the power site captured by a drone and transmit it to the processing module; characterized in that... The processing module is configured to execute the method as described in any one of claims 1-4 by calling the executable computer program code in the storage module.

6. An electronic device, comprising: a memory storing executable program code; a processor coupled to the memory; characterized in that... The processor calls the executable program code stored in the memory to execute the method as described in any one of claims 1-4.

7. A computer storage medium storing a computer program, characterized in that... The computer program is executed by the processor to perform the method as described in any one of claims 1-4.

Images