Unmanned aerial vehicle image distribution network element automatic identification method and system based on deep learning

Abstract

The invention relates to an unmanned aerial vehicle image distribution network element automatic identification method and system based on deep learning. The method comprises the following steps: collecting initial image data of a to-be-detected area of a distribution network line; marking distribution network elements in the initial image data, and performing enhancement processing on the marked initial image data to form a sample data set; constructing a distribution network element recognition model based on an improved YOLOv8 network, wherein the improved YOLOv8 network adds a hybrid module combining self-attention and convolution into an original YOLOv8 network; performing iterative training on the distribution network element recognition model by using the sample data set to obtain a trained distribution network element recognition model; and obtaining real-time image data of a to-be-detected area of the distribution network line, inputting the real-time image data into the trained distribution network element recognition model, and obtaining a distribution network element recognition result.

Description

Technical Field

[0001] The present invention relates to a method and system for automatically identifying distribution network elements using unmanned aerial vehicle images based on deep learning, and belongs to the technical field of distribution network inspection. Background Art

[0002] Currently, distribution network inspections often use drones to obtain orthophotos on the inspection route. However, these collected orthophotos need to be manually visually confirmed, which is labor-intensive and cannot ensure that the distribution network elements in each image are correctly identified.

[0003] The existing technology has methods for inspection target identification through visual algorithms, but the current visual algorithm models do not have high enough detection accuracy for distribution network elements, and most of them cannot meet the task requirements. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention proposes a method and system for automatic identification of network elements of drone images based on deep learning.

[0005] The technical solution of the present invention is as follows:

[0006] On the one hand, the present invention proposes a method for automatically identifying network elements of drone images based on deep learning, comprising the following steps:

[0007] Collect initial image data of the distribution network line area to be tested;

[0008] Annotate the distribution network elements in the initial image data, and enhance the annotated initial image data to form a sample data set;

[0009] Construct a distribution network element recognition model based on an improved YOLOv8 network, wherein the improved YOLOv8 network adds a hybrid module combining self-attention and convolution to the original YOLOv8 network; iteratively train the distribution network element recognition model using a sample data set to obtain a trained distribution network element recognition model;

[0010] Obtain real-time image data of the distribution network line area to be tested, input it into the trained distribution network element recognition model, and obtain the distribution network element recognition results.

[0011] As a preferred implementation, the method for marking the network distribution elements in the initial image data is specifically as follows:

[0012] Confirm the distribution network elements in the initial image data, and annotate each confirmed distribution network element using the image standard tool. The annotation information includes the category, location and shape information of the distribution network element;

[0013] Generate structured data from standard information.

[0014] As a preferred implementation, the enhancement processing of the annotated initial image data specifically includes:

[0015] The annotated initial image data is translated, rotated, cropped, scaled, color adjusted, and noise added.

[0016] As a preferred embodiment, the improved YOLOv8 network includes a backbone network, a neck network and a head network, wherein:

[0017] The neck network includes multiple Conv modules and C2f modules. A hybrid module combining self-attention and convolution is connected to multiple C2f modules to enhance features.

[0018] In the head network, the positioning loss function is set to the UIoU loss function.

[0019] On the other hand, the present invention also proposes a deep learning-based automatic recognition system for UAV image distribution network elements, comprising:

[0020] A data acquisition module is used to collect initial image data of the distribution network line area to be tested;

[0021] The sample processing module is used to annotate the distribution network elements in the initial image data and perform enhancement processing on the annotated initial image data to form a sample data set;

[0022] A model training module is used to construct a distribution network element recognition model based on an improved YOLOv8 network, wherein the improved YOLOv8 network adds a hybrid module combining self-attention and convolution to the original YOLOv8 network; the distribution network element recognition model is iteratively trained using a sample data set to obtain a trained distribution network element recognition model;

[0023] The distribution network element recognition module is used to obtain real-time image data of the distribution network line area to be tested, input it into the trained distribution network element recognition model, and obtain the distribution network element recognition results.

[0024] As a preferred implementation, the method for marking the network distribution elements in the initial image data is specifically as follows:

[0025] Confirm the distribution network elements in the initial image data, and annotate each confirmed distribution network element using the image standard tool. The annotation information includes the category, location and shape information of the distribution network element;

[0026] Generate structured data from standard information.

[0027] As a preferred implementation, the enhancement processing of the annotated initial image data specifically includes:

[0028] The annotated initial image data is translated, rotated, cropped, scaled, color adjusted, and noise added.

[0029] As a preferred embodiment, the improved YOLOv8 network includes a backbone network, a neck network and a head network, wherein:

[0030] The neck network includes multiple Conv modules and C2f modules. A hybrid module combining self-attention and convolution is connected to multiple C2f modules to enhance features.

[0031] In the head network, the positioning loss function is set to the UIoU loss function.

[0032] On the other hand, the present invention also proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for automatic identification of network distribution elements of drone images based on deep learning as described in any embodiment of the present invention is implemented.

[0033] On the other hand, the present invention further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatic identification of drone image distribution network elements based on deep learning as described in any embodiment of the present invention.

[0034] Additional aspects and advantages of the present invention will be set forth in the following description, and some of them will be apparent from the description, or may be understood by practicing the present invention. In addition, the various aspects and advantages of the present invention may be realized and obtained by the method steps and combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of a method flow chart of Embodiment 1 of the present invention;

[0036] Figure 2 It is a schematic diagram of the structure of the improved YOLOv8 network in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0039] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0040] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0041] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0042] Embodiment 1:

[0043] See also Figure 1 This embodiment provides a method for automatically identifying network elements of drone images based on deep learning, comprising the following steps:

[0044] S100, collecting initial image data of the distribution network line area to be tested; the initial image data is an orthophoto, which is obtained by planning a UAV flight route in the corresponding area to be tested according to the distribution network line project, and using the UAV to obtain UAV image data and other related data in the test area. The UAV photogrammetry method is used to obtain the orthophoto of the area to be tested from the image data and other related data collected by the UAV.

[0045] S200, use image annotation tools to annotate distribution network elements in the image, including different elements such as poles, transformers, and connecting lines, and record the element category, location, shape and other information of each distribution network element to form structured data; at the same time, in order to improve the generalization ability of the model, perform image enhancement processing on the annotated image, such as translation, rotation, cropping, scaling, color adjustment, noise addition, etc., to increase the diversity of samples. Put the enhanced and annotated initial image data into the sample data set.

[0046] S300, constructing a distribution network element recognition model based on an improved YOLOv8 network for distribution network elements, wherein the improved YOLOv8 network adds a self-attention and convolution mixed model (ACMix) to the original YOLOv8 network to increase the global perception ability of distribution network elements and improve the detection effect of distribution network elements; dividing the distribution network element sample data set into a training set, a test set, and a validation set according to an 8:1:1 ratio, iteratively training the distribution network element recognition model using the training set, and testing and verifying it through the test set and the validation set to obtain a trained distribution network element recognition model.

[0047] S400: Acquire real-time image data of the distribution network line area to be tested, input it into a trained distribution network element recognition model, and obtain distribution network element recognition results.

[0048] See also Figure 2 In this embodiment, the improved YOLOv8 network includes a Backbone network, a Neck network and a Head network, wherein:

[0049] The Backbone network includes multiple Conv modules, C2f modules and SPPF modules, the Neck network includes Conv modules, C2f modules, upsampling modules and Concat modules, and the Head network includes Conv modules, standard convolutions, classification loss calculation blocks and position loss calculation blocks. The improved YOLOv8 network proposed in this embodiment is connected to a hybrid module combining ACMix self-attention and convolution in the first two C2f modules in the Neck network for feature enhancement, which can effectively improve the detailed feature extraction of the model; and in the position loss calculation block of the Head network, the positioning loss function is set to the UIoU loss function.

[0050] Based on the above improved YOLOv8 network, the recognition process of the model is as follows:

[0051] A1. The sample data set of distribution network elements is passed into the Backbone network for feature extraction of distribution network elements. The Backbone network is divided into 0-9 layers. Specifically, the processed image is serially stacked 4 times through the Conv module and the C2f module, and the image and the convolution kernel are convolved. After batch normalization, the features are extracted through the activation function to obtain the tensor x. Then, through the C2f module, the input tensor x is divided into four parts along the channel dimension: P1, P2, P3, P4. Finally, the features of the extracted distribution network elements are passed through the spatial feature pooling layer for feature fusion.

[0052] A2. Input the distribution network element features obtained after feature fusion into the Neck network for feature fusion and enhancement processing. Input the outputs P3, P4, and P5 of the feature extraction network (Backbone) into the PAN-FPN network structure to fuse feature maps of multiple scales. F1 is obtained by fusing P5 with P4 after upsampling. F1 is fused with P3 after passing through the C2f module and upsampling once, and then passing through a hybrid module combining ACMix self-attention and convolution to obtain T1. F2 is obtained by fusing T1 with F1 after passing through a convolution layer. F2 is obtained by passing through a C2f module and then passing through a hybrid module combining ACMix self-attention and convolution to obtain T2. ​​T2 is fused with P5 after passing through a convolution layer to obtain F3. F3 is obtained by passing through a C2f layer and then passing through a hybrid module combining ACMix self-attention and convolution to obtain T3. Finally, the output branch features of T1, T2, and T3 are sent to the Head for result solution and loss function calculation.

[0053] A3. In the Head network, the enhanced and fused features obtained in the Neck network are separated from the classification and detection heads using the decoupled head knot; the UIoU loss function Loss_box formula is as follows:

[0054]

[0055] Among them, inter represents the result of intersection operation, union represents the result of union operation, and α and γ represent hyperparameters.

[0056]

[0057] ration=-0.005×epoch+2

[0058] Among them, ratio is the scaling ratio hyperparameter, epoch is the training round, w1 and h1 represent the width and height of the real box respectively, w2 and h2 represent the width and height of the predicted box respectively, w1′ and h1′ represent the width and height of the real box scaling ratio respectively, w2′ and h2′ represent the width and height of the predicted box scaling ratio respectively, b1_x1, b1_y1, b1_x2 and b1_y2 represent the coordinates of the four corner points of the real box respectively, b2_x1, b2_y1, b2_x2 and b2_y2 represent the coordinates of the four corner points of the predicted box respectively.

[0059] A4. The improved YOLOv8 network obtained above is used for distribution network element data recognition and detection to obtain prediction results. Specifically, the results obtained by the improved YOLOv8 network recognition are firstly filtered for confidence; then non-maximum suppression (NMS) is performed; finally, coordinate transformation is used to obtain the final distribution network element recognition and detection results.

[0060] Based on the above technical solution, this embodiment constructs a fully automated distribution network element automatic processing method based on a deep learning method to solve the low efficiency problem of manual visual inspection; constructs an improved YOLOV8 network, adopts a hybrid module (ACMix) combining self-attention and convolution, increases the global perception ability of distribution network elements, and effectively improves the detection effect of distribution network elements; uses the UIoU loss function to dynamically assign weights to prediction boxes of different qualities, effectively improving the performance of the model in high-quality target detection tasks.

[0061] Embodiment 2:

[0062] This embodiment proposes a deep learning-based automatic recognition system for drone image distribution network elements, including:

[0063] A data acquisition module, used to acquire initial image data of the distribution network line area to be tested; this module is used to implement the function of step S100 in the first embodiment, and will not be described in detail here;

[0064] A sample processing module, used to annotate the network elements in the initial image data, and perform enhancement processing on the annotated initial image data to form a sample data set; this module is used to implement the function of step S200 in the first embodiment, and will not be described in detail here;

[0065] A model training module is used to construct a distribution network element recognition model based on an improved YOLOv8 network, wherein the improved YOLOv8 network adds a hybrid module combining self-attention and convolution to the original YOLOv8 network; the distribution network element recognition model is iteratively trained using a sample data set to obtain a trained distribution network element recognition model; this module is used to implement the function of step S300 in the first embodiment, which will not be described in detail here;

[0066] The distribution network element identification module is used to obtain real-time image data of the distribution network line area to be tested, input it into the trained distribution network element identification model, and obtain the distribution network element identification result; this module is used to implement the function of step S400 in embodiment 1, and will not be repeated here.

[0067] As a preferred implementation of this embodiment, the method of marking the network distribution elements in the initial image data is specifically as follows:

[0068] Confirm the distribution network elements in the initial image data, and annotate each confirmed distribution network element using the image standard tool. The annotation information includes the category, location and shape information of the distribution network element;

[0069] Generate structured data from standard information.

[0070] As a preferred implementation of this embodiment, the enhancement processing of the annotated initial image data specifically includes:

[0071] The annotated initial image data is translated, rotated, cropped, scaled, color adjusted, and noise added.

[0072] As a preferred implementation of this embodiment, the improved YOLOv8 network includes a backbone network, a neck network and a head network, wherein:

[0073] The neck network includes multiple Conv modules and C2f modules. A hybrid module combining self-attention and convolution is connected to multiple C2f modules to enhance features.

[0074] In the head network, the positioning loss function is set to the UIoU loss function.

[0075] Embodiment three:

[0076] This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for automatic identification of network elements of drone images based on deep learning as described in any embodiment of the present invention is implemented.

[0077] Embodiment 4:

[0078] This embodiment proposes a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the method for automatic identification of drone image distribution network elements based on deep learning as described in any embodiment of the present invention is implemented.

[0079] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0080] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0081] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0082] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk, and other media that can store program codes.

[0083] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for automatic identification of network elements in drone images based on deep learning, characterized in that: The following steps are involved: Collect initial image data of the distribution network line area to be tested; Annotate the distribution network elements in the initial image data, and enhance the annotated initial image data to form a sample data set; Construct a distribution network element recognition model based on an improved YOLOv8 network, wherein the improved YOLOv8 network adds a hybrid module combining self-attention and convolution to the original YOLOv8 network; iteratively train the distribution network element recognition model using a sample data set to obtain a trained distribution network element recognition model; Obtain real-time image data of the distribution network line area to be tested, input it into the trained distribution network element recognition model, and obtain the distribution network element recognition results.

2. According to the method of automatic identification of network elements of drone images based on deep learning in claim 1, it is characterized in that: The method of marking the network distribution elements in the initial image data is specifically as follows: Confirm the distribution network elements in the initial image data, and annotate each confirmed distribution network element using the image standard tool. The annotation information includes the category, location and shape information of the distribution network element; Generate structured data from standard information.

3. According to the method of automatic identification of network elements of drone images based on deep learning in claim 1, it is characterized in that: The enhancement processing of the annotated initial image data specifically includes: The annotated initial image data is translated, rotated, cropped, scaled, color adjusted, and noise added.

4. According to the method of automatic identification of network elements of drone images based on deep learning in claim 1, it is characterized in that: The improved YOLOv8 network includes a backbone network, a neck network and a head network, wherein: The neck network includes multiple Conv modules and C2f modules. A hybrid module combining self-attention and convolution is connected to multiple C2f modules to enhance features. In the head network, the positioning loss function is set to the UIoU loss function.

5. A deep learning-based automatic recognition system for network elements of drone images, characterized in that: include: A data acquisition module is used to collect initial image data of the distribution network line area to be tested; The sample processing module is used to annotate the distribution network elements in the initial image data and perform enhancement processing on the annotated initial image data to form a sample data set; A model training module is used to construct a distribution network element recognition model based on an improved YOLOv8 network, wherein the improved YOLOv8 network adds a hybrid module combining self-attention and convolution to the original YOLOv8 network; the distribution network element recognition model is iteratively trained using a sample data set to obtain a trained distribution network element recognition model; The distribution network element recognition module is used to obtain real-time image data of the distribution network line area to be tested, input it into the trained distribution network element recognition model, and obtain the distribution network element recognition results.

6. The deep learning-based automatic recognition system for network elements of unmanned aerial vehicle images according to claim 5 is characterized in that: The method of marking the network distribution elements in the initial image data is specifically as follows: Confirm the distribution network elements in the initial image data, and annotate each confirmed distribution network element using the image standard tool. The annotation information includes the category, location and shape information of the distribution network element; Generate structured data from standard information.

7. The deep learning-based automatic recognition system for network elements of unmanned aerial vehicle images according to claim 5 is characterized in that: The enhancement processing of the annotated initial image data specifically includes: The annotated initial image data is translated, rotated, cropped, scaled, color adjusted, and noise added.

8. The deep learning-based automatic recognition system for network elements of unmanned aerial vehicle images according to claim 5 is characterized in that: The improved YOLOv8 network includes a backbone network, a neck network and a head network, wherein: The neck network includes multiple Conv modules and C2f modules. A hybrid module combining self-attention and convolution is connected to multiple C2f modules to enhance features. In the head network, the positioning loss function is set to the UIoU loss function.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for automatic identification of network elements of drone images based on deep learning as described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for automatic identification of network distribution elements of drone images based on deep learning as described in any one of claims 1 to 4 is implemented.

Images