Unmanned aerial vehicle intelligent supervision method and system in electric power operation scene

Abstract

The embodiment of the invention provides an unmanned aerial vehicle intelligent supervision method and system in an electric power operation scene, and belongs to the technical field of electric power operation supervision. The method comprises the following steps: acquiring an on-site operation scene image; inputting the operation scene image into a preset intelligent identification model to obtain a judgment factor of the field facility; and performing combined judgment on the judgment factors to determine whether the field safety measures are in place or not. According to the method and the system, an intelligent identification model is adopted to carry out combined judgment according to images of an electric power operation site, and whether field safety measures are in place or not is determined by combining a combined judgment result. Compared with manual supervision and inspection in the prior art, the method and the system provided by the invention can overcome the defect that full-view supervision cannot be realized by manual supervision and inspection, and improve the efficiency of field supervision and inspection.

Description

Technical Field

[0001] The present invention relates to the technical field of power operation inspection, and in particular to an intelligent inspection method and system of unmanned aerial vehicle in a power operation scenario. Background Art

[0002] The number of power operation sites is large, the locations are wide, the environment is complex, and there are many construction workers, making safety management and control difficult. In traditional inspection technologies, manual inspections and ground monitoring systems are usually relied on. Although the implementation of remote monitoring through system platforms and various monitoring terminals has achieved initial results in on-site safety management and control, line maintenance involves many operations at high altitudes, which makes it impossible for inspectors to supervise the entire process without blind spots, and there are monitoring blind spots in high-risk operation scenarios. Summary of the invention

[0003] The purpose of the embodiments of the present invention is to provide a method and system for intelligent inspection of unmanned aerial vehicles in an electric power operation scenario, which can improve the inspection efficiency of the electric power operation site.

[0004] In order to achieve the above-mentioned purpose, an embodiment of the present invention provides a method for intelligent inspection of a UAV in a power operation scenario, comprising:

[0005] Acquire on-site work scene images;

[0006] Inputting the work scene image into a preset intelligent recognition model to obtain determination factors of on-site facilities;

[0007] A combined judgment is performed on the determination factors to determine whether the safety measures on site are in place.

[0008] Optionally, the intelligent recognition model includes a discriminator for determining a score of the sample input according to a similarity between the sample input and a feature distribution space of a defective sample.

[0009] Optionally, the method comprises:

[0010] Obtain a sample set of power images;

[0011] Determining the proportion of sample categories in the sample set;

[0012] Using a compensation generator to augment the sample set according to the sample category ratio;

[0013] The discriminator is trained using the augmented sample set.

[0014] Optionally, the compensation generator includes a pointer attention mechanism for calibrating local features in the power image.

[0015] Optionally, the method further comprises:

[0016] Dividing each sample in the sample set into a plurality of regions;

[0017] numbering said areas;

[0018] Performing clustering operation on the samples according to the number of the area where the important features of each sample are located;

[0019] Determining the distribution characteristics of the serial numbers of the areas where the important features are located according to the results of the clustering operation;

[0020] The weight of the pointer attention mechanism and the proportion of the sample categories are determined according to the distribution characteristics.

[0021] Optionally, using a compensation generator to augment the sample set according to the sample category ratio includes:

[0022] The sample set is augmented by fusing the Transformer diffusion model.

[0023] Optionally, the discriminator includes:

[0024] The multi-scale feature extraction layer includes a spatial feature extractor, a frequency domain feature extractor and a LIC feature extractor, wherein the spatial feature extractor is used to extract global features according to the input image by using a statistical method, the frequency domain feature extractor is used to extract frequency domain features according to the image by using a Fourier transform method or a wavelet transform method, and the LIC feature extractor is used to extract LIC features according to the associated information corresponding to the image;

[0025] A multi-scale feature fusion layer, used to fuse the global features, frequency domain features and LIC features to obtain fused features;

[0026] A low-dimensional feature extraction layer, used to generate a low-dimensional feature matrix according to the fused features;

[0027] A medium-dimensional feature extraction layer, used for further feature extraction based on the low-dimensional feature matrix to obtain a medium-dimensional feature matrix;

[0028] A high-dimensional feature extraction layer, used for further feature extraction based on the medium-dimensional feature matrix to obtain a high-dimensional feature matrix;

[0029] A first classifier, used for performing classification according to the low-dimensional feature matrix to obtain the first classification result;

[0030] A second classifier, used for performing classification according to the medium-dimensional feature matrix to obtain the second classification result;

[0031] A third classifier, used for performing classification according to the high-dimensional feature matrix to obtain the third classification result;

[0032] A fusion layer is used to generate the score according to the first classification result, the second classification result and the third classification result.

[0033] Optionally, a combined judgment is performed on the judgment factors to determine whether the safety measures on site are in place, including:

[0034] Determine whether the staff is wearing a safety belt;

[0035] In the case where it is determined that the worker is wearing a safety belt, determining whether the waist belt of the safety belt is fixed to the building;

[0036] In the case where it is determined that the waist belt is not fixed to the building, it is determined that the anti-fall measures are abnormal;

[0037] In the case where it is determined that the waist belt is fixed to a building, determining whether the backup rope of the safety belt is hung high and used low;

[0038] In the case where it is determined that the backup rope is not hung high and used low, it is determined that the anti-falling measures are abnormal;

[0039] In the case where it is judged that the backup rope is hung high and used low, it is determined that the anti-falling measures are effective;

[0040] When it is determined that the staff is not wearing a safety belt, it is determined that the anti-fall measures are ineffective.

[0041] On the other hand, the present invention also provides an intelligent drone inspection system in an electric power operation scenario, the system comprising a processor, and the processor is configured to execute any of the methods described above.

[0042] Through the above technical solution, the embodiment of the present invention provides a method and system for intelligent inspection of drones in power operation scenarios. The method and system use intelligent recognition models to make combined judgments based on images of the power operation site, and combine the results of the combined judgments to determine whether the safety measures on site are in place. Compared with the manual inspection in the prior art, the method and system provided by the present invention can overcome the defect that manual inspection cannot supervise from all angles, and improve the efficiency of on-site inspection.

[0043] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0045] Figure 1 This is a flow chart of a method for intelligent inspection of a UAV in a power operation scenario according to an embodiment of the present invention;

[0046] Figure 2 is a flowchart of a method for obtaining a discriminator according to an embodiment of the present invention;

[0047] Figure 3 is a flowchart of a method for determining a sample category ratio and a pointer attention mechanism according to an embodiment of the present invention;

[0048] Figure 4 is a structural block diagram of a discriminator according to an embodiment of the present invention; and

[0049] Figure 5 It is a flowchart of a determination factor and a method of combination determination according to one embodiment of the present invention. DETAILED DESCRIPTION

[0050] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0051] like Figure 1 The figure shows a flow chart of a method for intelligent inspection of unmanned aerial vehicles in a power operation scenario according to an embodiment of the present invention. Figure 1 In the method, the method may include the following steps:

[0052] In step S10, an image of the working scene at the site is obtained;

[0053] In step S11, the work scene image is input into a preset intelligent recognition model to obtain the determination factors of the on-site facilities;

[0054] In step S12, a combined judgment is performed on the judgment factors to determine whether the safety measures on site are in place.

[0055] In this Figure 1 In the method shown, step S10 can be used to obtain an on-site work scene image, wherein the work scene image can be captured by a drone or an image acquisition device set up on-site.

[0056] Step S11 can be used to input the work scene image into a preset intelligent recognition model to obtain the determination factors of the on-site facilities. Among them, for the intelligent recognition model, it can be a variety of network structures known to those skilled in the art. In one example of the present invention, the intelligent recognition model may include a discriminator. The discriminator can be used to determine the score of the sample input based on the similarity between the sample input and the feature distribution space of the defective sample. The specific method for obtaining the discriminator can be a neural network training method known to those skilled in the art. In one example of the present invention, the specific method for obtaining the discriminator can include the following: Figure 2 In the steps shown in Figure 2 In the method of obtaining the discriminator, the method may include the following steps:

[0057] In step S20, a sample set of power images is obtained. The sample set may be a small sample with a small number. In the sample set, each sample may include an image, a location of a corresponding calibrated important feature, and a score.

[0058] In step S21, the sample category ratio in the sample set is determined;

[0059] In step S22, a compensation generator is used to augment the sample set according to the sample category ratio. In addition, in this example, in order to ensure that the augmented sample set can still represent the feature distribution feature points of the original sample set, the compensation generator may further include a pointer attention mechanism. The pointer attention mechanism can be used to calibrate local features in power images. The specific method for augmenting the sample set may be in various forms known to those skilled in the art. In one example of the present invention, the sample set may be augmented by a method of fusing a Transformer diffusion model.

[0060] In step S23, the discriminator is trained using the augmented sample set.

[0061] In this example, the specific method for determining the sample category ratio and the pointer attention mechanism may be in various forms known to those skilled in the art, including but not limited to manual calibration, network identification, etc. In one example of the present invention, the method for determining the sample category ratio and the pointer attention mechanism may include: Figure 3 In the steps shown in Figure 3 In the method of determining the sample category ratio and the pointer attention mechanism, the method may include the following steps:

[0062] In step S30, each sample in the sample set is divided into a plurality of regions;

[0063] In step S31, the regions are numbered;

[0064] In step S32, clustering operations are performed on the samples according to the numbers of the regions where the important features of each sample are located;

[0065] In step S33, the distribution characteristics of the numbers of the areas where the important features are located are determined according to the results of the clustering operation;

[0066] In step S34, the weight of the pointer attention mechanism and the proportion of sample categories are determined according to the distribution characteristics.

[0067] In this embodiment, the specific structure of the discriminator can be in various forms known to those skilled in the art. In one example of the present invention, the structure of the discriminator can be as follows: Figure 4 As shown. Figure 4 In the discriminator, the discriminator may include a multi-scale feature extraction layer 10, a multi-scale feature fusion layer 20, a low-dimensional feature extraction layer 30, a medium-dimensional feature extraction layer 40, a high-dimensional feature extraction layer 50, a first classifier 60, a second classifier 70, a third classifier 80, and a fusion layer 90. The multi-scale feature extraction layer 10 may include a spatial feature extractor 11, a frequency domain feature extractor 12, and a LIC feature extractor 13. The spatial feature extractor 11 may be used to extract global features using a statistical method based on the input image, the frequency domain feature extractor 12 may be used to extract frequency domain features using a Fourier transform method or a wavelet transform method based on the image, and the LIC feature extractor 13 may be used to extract LIC features based on the associated information corresponding to the image. The multi-scale feature fusion layer 20 may be used to fuse global features, frequency domain features, and LIC features to obtain fused features. The low-dimensional feature extraction layer 30 may be used to generate a low-dimensional feature matrix based on the fused features. The medium-dimensional feature extraction layer 40 may be used to perform further feature extraction based on the low-dimensional feature matrix to obtain a medium-dimensional feature matrix. The high-dimensional feature extraction layer 50 can be used to perform further feature extraction according to the medium-dimensional feature matrix to obtain a high-dimensional feature matrix. The first classifier 60 can be used to classify according to the low-dimensional feature matrix to obtain a first classification result. The second classifier 70 is used to classify according to the medium-dimensional feature matrix to obtain a second classification result. The third classifier 80 can be used to classify according to the high-dimensional feature matrix to obtain a third classification result. The fusion layer 90 can be used to generate a score based on the first classification result, the second classification result, and the third classification result.

[0068] Step S12 can be used to perform a combined judgment on the judgment factors to determine whether the safety measures on site are in place. The specific method for the judgment factors and the combined judgment can be in various forms known to those skilled in the art. In one example of the present invention, the judgment factors and the combined judgment method are as follows: Figure 5 and as shown in Table 1. Specifically, in this Figure 5As shown in Table 1, the determination factors and the combined determination method may include the following steps:

[0069] In step S40, it is determined whether the worker is wearing a safety belt;

[0070] In step S41, when it is determined that the worker is wearing a safety belt, it is determined whether the waist belt of the safety belt is fixed to the building;

[0071] In step S42, when it is determined that the waist belt is not fixed to the building, it is determined that the anti-falling measures are abnormal;

[0072] In step S43, when it is determined that the waist belt is fixed to a building, it is determined whether the backup rope of the safety belt is hung high and used low;

[0073] In step S44, when it is determined that the backup rope is not hung high and used low, it is determined that the anti-falling measures are abnormal;

[0074] In step S45, when it is determined that the backup rope is hung high and used low, it is determined that the anti-falling measures are effective;

[0075] In step S46, when it is determined that the worker is not wearing a safety belt, it is determined that the anti-falling measures are ineffective.

[0076] Table 1

[0077]

[0078] On the other hand, the present invention also provides an intelligent drone inspection system in an electric power operation scenario, the system comprising a processor, and the processor is configured to execute any of the methods described above.

[0079] Through the above technical solution, the embodiment of the present invention provides a method and system for intelligent inspection of drones in power operation scenarios. The method and system use intelligent recognition models to make combined judgments based on images of the power operation site, and combine the results of the combined judgments to determine whether the safety measures on site are in place. Compared with the manual inspection in the prior art, the method and system provided by the present invention can overcome the defect that manual inspection cannot supervise from all angles, and improve the efficiency of on-site inspection.

[0080] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0081] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0082] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0083] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0084] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0085] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0086] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0087] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0088] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A method for intelligent inspection by drones in power operation scenarios, characterized in that: The method comprises: Acquire on-site work scene images; Inputting the work scene image into a preset intelligent recognition model to obtain determination factors of on-site facilities; A combined judgment is performed on the determination factors to determine whether the safety measures on site are in place.

2. The method according to claim 1, characterized in that The intelligent recognition model includes a discriminator for determining a score of the sample input according to a similarity between the sample input and a feature distribution space of a defective sample.

3. The method according to claim 2, characterized in that The method comprises: Obtain a sample set of power images; Determining the proportion of sample categories in the sample set; Using a compensation generator to augment the sample set according to the sample category ratio; The discriminator is trained using the augmented sample set.

4. The method according to claim 3, characterized in that The compensation generator includes a pointer attention mechanism for calibrating local features in the power image.

5. The method according to claim 4, characterized in that The method further comprises: Dividing each sample in the sample set into a plurality of regions; numbering the areas; Performing clustering operation on the samples according to the number of the area where the important features of each sample are located; Determining the distribution characteristics of the serial numbers of the areas where the important features are located according to the results of the clustering operation; The weight of the pointer attention mechanism and the proportion of the sample categories are determined according to the distribution characteristics.

6. The method according to claim 3, characterized in that The sample set is augmented according to the sample category ratio using a compensation generator, comprising: The sample set is augmented by fusing the Transformer diffusion model.

7. The method according to claim 2, characterized in that The discriminator comprises: The multi-scale feature extraction layer includes a spatial feature extractor, a frequency domain feature extractor and a LIC feature extractor, wherein the spatial feature extractor is used to extract global features according to the input image by using a statistical method, the frequency domain feature extractor is used to extract frequency domain features according to the image by using a Fourier transform method or a wavelet transform method, and the LIC feature extractor is used to extract LIC features according to the associated information corresponding to the image; A multi-scale feature fusion layer, used to fuse the global features, frequency domain features and LIC features to obtain fused features; A low-dimensional feature extraction layer, used to generate a low-dimensional feature matrix according to the fused features; A medium-dimensional feature extraction layer, used for further feature extraction based on the low-dimensional feature matrix to obtain a medium-dimensional feature matrix; A high-dimensional feature extraction layer, used for further feature extraction based on the medium-dimensional feature matrix to obtain a high-dimensional feature matrix; A first classifier, used for performing classification according to the low-dimensional feature matrix to obtain a first classification result; A second classifier, used for performing classification according to the medium-dimensional feature matrix to obtain a second classification result; A third classifier, used for performing classification according to the high-dimensional feature matrix to obtain a third classification result; A fusion layer is used to generate the score according to the first classification result, the second classification result and the third classification result.

8. The method according to claim 1, characterized in that Combined judgment is made based on the above-mentioned factors to determine whether the safety measures on site are in place, including: Determine whether the staff is wearing a safety belt; In the case where it is determined that the worker is wearing a safety belt, determining whether the waist belt of the safety belt is fixed to the building; In the case where it is determined that the waist belt is not fixed to the building, it is determined that the anti-fall measures are abnormal; In the case where it is determined that the waist belt is fixed to a building, determining whether the backup rope of the safety belt is hung high and used low; In the case where it is determined that the backup rope is not hung high and used low, it is determined that the anti-falling measures are abnormal; In the case where it is judged that the backup rope is hung high and used low, it is determined that the anti-falling measures are effective; When it is determined that the staff is not wearing a safety belt, it is determined that the anti-fall measures are ineffective.

9. An intelligent drone inspection system for power operation scenarios, characterized in that: The system comprises a processor configured to execute the method according to any one of claims 1 to 8.

Images