Improved yolov5 unmanned aerial vehicle inspection image processing method and system

Through the improved YOLOv5 method, image downsampling and multi-scale inference are used to use deep networks and multi-head self-attention mechanisms, combined with Monte Carlo method to denoise, the problem of image resolution reduction and perspective during drone inspection is solved, and the problem of image resolution reduction and perspective in drone inspection is improved. The accuracy and efficiency of inspection ensure the safe and stable operation of power grid lines.

CN119007044APending Publication Date: 2024-11-22ZHEJIANG PROVINCE ELECTRIC POWER FUEL CORP
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411097460.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-12
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

When existing drone inspection image processing technology detects defects in transmission and distribution lines, the resolution reduction leads to loss of details, making it difficult to accurately locate small defects, and the viewing angle problem leads to low detection accuracy, resulting in missed reporting, affecting maintenance efficiency and accuracy. sex.

Method used

Using the improved YOLOv5 method, image downsampling and multi-scale inference are performed through a deep network feature extractor, combined with the multi-head self-attention mechanism and the Monte Carlo method, high-fidelity feature maps are generated, defect area detection and classification are performed, and the Predict results denoising, improving image quality and detection accuracy.

Benefits of technology

It improves the accuracy and efficiency of image defect detection of drone inspections, clarifies the defect types, reduces misjudgment and misreporting by maintenance personnel, and ensures the safe and stable operation of power grid lines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007044A_ABST
    Figure CN119007044A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of unmanned aerial vehicles, in particular to an improved yov5 unmanned aerial vehicle inspection image processing method and system, and the method comprises the following steps: S1, data collection: carrying out the image data collection of electrical equipment; s2, carrying out computer generation on missing image information in the electrical equipment graph by adopting image down-sampling network processing and multi-scale reasoning; S3, carrying out multi-scale feature fusion on the electrical equipment graph through a multi-scale feature fusion module, and outputting defect areas of the electrical equipment graph, the defect areas including a broken line area, a falling area and a foreign matter area; s4, performing defect classification or defect positioning on the electrical equipment graph through a detection head; and S5, denoising the electrical equipment graph through a prediction result denoising module, and repeatedly sampling the electrical equipment graph by adopting a Monte Carlo method. The method has the advantages that the detection accuracy and the detection efficiency of the unmanned aerial vehicle inspection image defect of the power transmission and distribution line are improved, support is provided for maintenance decision, and safe and stable operation of a power grid line is guaranteed.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to an unmanned aerial vehicle inspection image processing method and system for improving yolov5. [Background technology]

[0002] Drone technology is increasingly being used in the power industry, especially in the inspection of overhead transmission lines. The high-definition cameras on drones can capture high-definition images with a resolution of 5000×5000 pixels or more. With such high-definition images, detailed views of transmission lines can be effectively obtained, including key electrical equipment such as towers, insulators, conductors, line corridors, and hardware. These images are essential for quickly detecting common defects in transmission lines, such as broken wires, fallen insulators, and foreign objects hanging on the wires.

[0003] Due to the large size of drone inspection images, existing technologies will reduce the image resolution in the preprocessing stage to make the algorithm's time and resource overhead controllable. However, due to the loss of many details by directly reducing the resolution, when detecting smaller defects, such as rust or improper installation of small hardware such as bolts and nuts, traditional technologies can only roughly locate the detected defects to a certain tower.

[0004] This type of test results with unclear positioning information cannot help maintenance personnel identify specific defects in electrical components. In order to clarify the type of defect, maintenance personnel have to go to the tower to investigate, report, and prepare targeted maintenance equipment or measures. In addition, maintenance personnel are prone to misjudgment when visually judging the inspection images, which delays the progress of maintenance.

[0005] In addition, the captured images also have the problem of low accuracy because the viewing angle is too close or too far, which leads to certain underreporting.

[0006] Therefore, an improved UAV inspection image processing method and system of YOLOv5 are provided to improve the accuracy and efficiency of defect detection in UAV inspection images of power transmission and distribution lines, thereby providing support for maintenance decisions and ensuring the safe and stable operation of power grid lines. [Summary of the invention]

[0007] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide an improved UAV inspection image processing method and system of YOLOv5, so as to improve the accuracy and efficiency of defect detection in UAV inspection images of power transmission and distribution lines, thereby providing support for maintenance decisions and ensuring the safe and stable operation of power grid lines.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0009] S1: Data collection: assigning drones to take high-definition images of electrical equipment according to the patrol route to collect image data of electrical equipment;

[0010] S2: Generate missing image information through deep network feature extractor, and use image downsampling network processing and multi-scale reasoning to computer generate missing image information in electrical equipment diagrams:

[0011] S21: Perform feature extraction data preprocessing on high-definition image data, use the multi-head self-attention mechanism for image downsampling network processing, and use the multi-head attention mechanism module to quickly reduce the resolution of electrical equipment images, with a high degree of image retention and retain key information;

[0012] S22: Perform multi-scale reasoning on high-definition image data. Use a multi-scale reasoning method to send multiple feature maps of different sizes output by the image downsampling network to the same YOLOv5 model for reasoning. The reasoning processes of the feature maps are independent of each other and do not interfere with each other. The feature maps are merged to output the predicted image.

[0013] S3: Perform multi-scale feature fusion on the electrical equipment diagram through a multi-scale feature fusion module, and output the defect area of ​​the electrical equipment diagram. The output value is the defect area of ​​the electrical equipment diagram, and the defect area includes a broken wire area, a falling area, and a foreign body area.

[0014] S4: classify or locate defects on the electrical equipment diagram through the detection head;

[0015] S5: The electrical equipment diagram is denoised through the prediction result denoising module, and the Monte Carlo method is used to repeatedly sample the electrical equipment diagram to remove random noise and improve the quality of the electrical equipment diagram.

[0016] Preferably, S11: the electrical equipment includes a drone image or video dataset of small equipment targets, medium equipment targets, and large equipment targets, and the dataset is labeled.

[0017] Preferably, S211: the image downsampling network is composed of several layers, the downsampling ratio of each layer is n, the size of the input feature is C×H×W, that is, H pixels in height, W pixels in width, and C channels in total, then each layer of the network divides the input feature into n×n grids in the spatial dimension, for a total of Grid;

[0018] As a preferred embodiment, S212: the n contained in each n×n grid 2 The features of each pixel are fused and then passed to the multi-head self-attention mechanism module to calculate the attention information globally, and finally output the features of one pixel for each grid. Pixels, width Feature map of pixels.

[0019] Preferably, the image supplementation processing also includes: S221: the image downsampling network outputs a feature map of size m×m pixels, and an additional feature map with a width and height of 1.5m and 0.5m respectively. The three are independently inferred using the same YOLOv5 model, and finally the output results of the three are merged.

[0020] Preferably, S51: randomly perturb the detection head parameters of YOLOv5 through the prediction result denoising module, and use the detection head with random perturbations to perform K inferences on the same input sample, so as to output K different prediction results for the same input.

[0021] Preferably, the electrical equipment image data is coded and labeled, and importance parameters are assigned according to the impact of the electrical equipment image data on subsequent operation and maintenance. The electrical equipment image data includes geographical location, altitude, equipment height, and maintenance location.

[0022] An improved yolov5 unmanned aerial vehicle inspection image processing system, the system adopts an improved yolov5 unmanned aerial vehicle inspection image processing method, the system includes a data acquisition module, which is used to assign unmanned aerial vehicles to take high-definition images of electrical equipment according to the inspection route, and to collect electrical equipment image data;

[0023] A deep network feature extractor for generating missing image information, using image downsampling network processing and multi-scale reasoning to computer-generate missing image information from electrical equipment diagrams;

[0024] A multi-scale feature fusion module is used to perform multi-scale feature fusion on the electrical equipment diagram and output the defect area of ​​the electrical equipment diagram. The output value is the defect area of ​​the electrical equipment diagram. The defect area includes a broken wire area, a falling area, and a foreign body area.

[0025] Detection head, used for defect classification or defect location on electrical equipment diagrams;

[0026] The prediction result denoising module is used to denoise the electrical equipment diagram. The Monte Carlo method is used to repeatedly sample the electrical equipment diagram to remove random noise and improve the quality of the electrical equipment diagram.

[0027] A computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute an improved yolov5 unmanned aerial vehicle inspection image processing method as described above.

[0028] A processor is used to run a program, wherein when the program is running, an improved yolov5 unmanned aerial vehicle inspection image processing method as described above is executed.

[0029] The beneficial effects of the present invention are as follows:

[0030] 1.S2: Generate missing image information through deep network feature extractor, and use image downsampling network processing and multi-scale reasoning to computer generate missing image information in electrical equipment diagrams:

[0031] S21: Perform feature extraction data preprocessing on high-definition image data, use the multi-head self-attention mechanism for image downsampling network processing, and use the multi-head attention mechanism module to quickly reduce the resolution of electrical equipment images, with a high degree of image retention and retain key information;

[0032] S211: The image downsampling network consists of several layers, and the downsampling ratio of each layer is n. The size of the input feature is C×H×W, that is, H pixels in height, W pixels in width, and C channels in total. Then each layer of the network divides the input feature into n×n grids in the spatial dimension, for a total of Grid;

[0033] For example, the downsampling ratio of each layer is n, and the size of the input feature is C×H×W, that is, H pixels high, W pixels wide, and C channels in total. Then each layer of the network divides the input feature into a 2×2 grid in the spatial dimension, for a total of Grid.

[0034] S212: The n contained in each n×n grid 2 The features of each pixel are fused and then passed to the multi-head self-attention mechanism module to calculate the attention information globally, and finally output the features of one pixel for each grid. Pixels, width Feature map of pixels.

[0035] For example, the features of the four pixels contained in each 2×21 grid are fused and passed to the multi-head self-attention mechanism module, and the attention information is calculated globally. Finally, the features of one pixel are output for each grid, which are merged into a high Pixels, width The pixel feature map is simple to operate, speeds up the work efficiency of reducing image resolution, shortens the response time, and the image downsampling network has the function of parameter learning and has the advantage of high degree of output feature detail retention.

[0036] 2.S22: Perform multi-scale reasoning on high-definition image data. Use the multi-scale reasoning method to send multiple feature maps of different sizes output by the image downsampling network to the same YOLOv5 model for reasoning. The reasoning process of the feature maps is independent of each other and does not interfere with each other. The feature maps are merged to output the predicted image.

[0037] S221: The image downsampling network outputs a feature map of size m×m pixels, as well as an additional feature map with a width and height of 1.5m and 0.5m respectively. The three are independently inferred using the same YOLOv5 model, and the output results of the three are finally merged to improve the clarity of the blurred blocks of the image.

[0038] 3. The detection head parameters of YOLOv5 are randomly perturbed through the prediction result denoising module, and the detection head with random perturbations is used to perform K inferences on the same input sample to produce K different prediction results for the same input.

[0039] The K prediction results here can all be regarded as having an unknown random noise, so when K is large enough, such as K ≥ 10, the mean of the K prediction results can be used as the denoised prediction result. Reduce image noise and improve image clarity.

[0040] These features and advantages of the present invention will be disclosed in detail in the following specific embodiments and drawings.

Brief Description of the Drawings

[0041] The present invention will be further described below in conjunction with the accompanying drawings:

[0042] Figure 1 This is a schematic diagram of the process of the UAV inspection image processing method of the present invention;

[0043] Figure 2 A schematic diagram of the system. [Specific implementation method]

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] The following first describes the concepts involved in this application in conjunction with the accompanying drawings. It should be noted that the following description of each concept is only to make the content of this application easier to understand and does not limit the scope of protection of this application.

[0046] Embodiment 1:

[0047] YOLOv5 is an object detection algorithm whose main framework is based on a convolutional neural network (CNN). It extracts image features by performing multiple convolution and pooling operations on the input image, and outputs the detection results through a fully connected layer. It is different from the traditional CNN object detection method in that it uses a mechanism called anchor to generate multiple detection boxes of different sizes and shapes. This allows for better detection of objects of different sizes and shapes. In addition, YOLOv5 also provides many optimizations, such as data enhancement and visual decoders, to improve detection efficiency.

[0048] An improved yolov5 UAV inspection image processing method, such as Figure 1 As shown, the steps include

[0049] S1: Data collection: assigning drones to take high-definition images of electrical equipment according to the patrol route to collect image data of the electrical equipment. The high-definition images can be images with a resolution of 5000×5000 pixels or above.

[0050] S11: Electrical equipment includes drone image or video datasets of small equipment targets, medium equipment targets, and large equipment targets, and the datasets are annotated. Small equipment targets can be hardware such as screws and nuts, medium equipment targets can be electrical equipment such as insulators and wires, and medium equipment targets can be large electrical equipment such as poles, towers, and line corridors.

[0051] S2: Generate missing image information through deep network feature extractor, and use image downsampling network processing and multi-scale reasoning to computer generate missing image information in electrical equipment diagrams:

[0052] S21: Perform feature extraction data preprocessing on high-definition image data, use the multi-head self-attention mechanism for image downsampling network processing, and use the multi-head attention mechanism module to quickly reduce the resolution of electrical equipment images, with a high degree of image retention and retain key information;

[0053] S211: The image downsampling network consists of several layers, and the downsampling ratio of each layer is n. The size of the input feature is C×H×W, that is, H pixels in height, W pixels in width, and C channels in total. Then each layer of the network divides the input feature into n×n grids in the spatial dimension, for a total of Grid;

[0054] For example, the downsampling ratio of each layer is n, and the size of the input feature is C×H×W, that is, H pixels high, W pixels wide, and C channels in total. Then each layer of the network divides the input feature into a 2×2 grid in the spatial dimension, for a total of Grid.

[0055] S212: The n contained in each n×n grid 2 The features of each pixel are fused and then passed to the multi-head self-attention mechanism module to calculate the attention information globally, and finally output the features of one pixel for each grid. Pixels, width Feature map of pixels.

[0056] For example, the features of the four pixels contained in each 2×2 grid are fused and passed to the multi-head self-attention mechanism module, and the attention information is calculated globally. Finally, the features of one pixel are output for each grid, which are merged into a high Pixels, width The pixel feature map is simple to operate, speeds up the work efficiency of reducing image resolution, shortens the response time, and the image downsampling network has the function of parameter learning and has the advantage of high degree of output feature detail retention.

[0057] By adding a multi-head attention mechanism, a weight is calculated based on the importance of a specific part of the input and its relevance to the output. A relevant score can be assigned to the input element, and the noisy part is ignored. Through the multi-head attention mechanism, the pixel grid is mapped to several different dimensions. The pixel grid is mapped in QKV to achieve dimensionality reduction, reduce the difficulty of calculation, thereby retaining key information, better feature extraction and selection, and improving recognition accuracy, as shown in the following formula:

[0058]

[0059] Among them, Attention represents the attention mechanism, softmax represents the multi-classification activation function, W is the learnable weight matrix, q, k and v represent the projection matrices of query, key and value respectively, d k Indicates the dimension size of each head.

[0060] The neighbor information of the aggregated nodes is obtained through graph convolution operation, and the mean square error loss is used as the loss function:

[0061]

[0062] H (l+1) represents the feature vector of the node at the l+1 layer, H (l) represents the feature vector of the node in layer l, W (l) represents the weight matrix of the lth layer, and ReLU represents the activation function.

[0063] S22: Perform multi-scale reasoning on high-definition image data. Use a multi-scale reasoning method to send multiple feature maps of different sizes output by the image downsampling network to the same YOLOv5 model for reasoning. The reasoning processes of the feature maps are independent of each other and do not interfere with each other. The feature maps are merged to output the predicted image.

[0064] S221: The image downsampling network outputs a feature map of size m×m pixels, as well as an additional feature map with a width and height of 1.5m and 0.5m respectively. The three are independently inferred using the same YOLOv5 model, and the output results of the three are finally merged.

[0065] For example, if the width and height of the size feature map output by the image downsampling network are 1280×1280 pixels, then when performing multi-scale reasoning, one possible configuration is to let the image downsampling network simultaneously output a size feature map of 1280×1280 pixels, as well as an additional feature map with a width and height of 1.5 times and 0.5 times, respectively, that is, a 1920×1920 pixel feature map and a 640×640 pixel feature map. The three are independently reasoned using the same YOLOv5 model, and finally the output results of the three are merged. The reasoning process of these feature maps is independent of each other and does not interfere with each other. After all sizes of feature maps are inferred, this embodiment merges their output results as the final output result.

[0066] S3: Perform multi-scale feature fusion on the electrical equipment diagram through a multi-scale feature fusion module, and output the defect area of ​​the electrical equipment diagram. The output value is the defect area of ​​the electrical equipment diagram, and the defect area includes a broken wire area, a falling area, and a foreign body area.

[0067] The broken line area mainly records line breakage, etc., the falling off area mainly records the falling off of hardware and other assembly parts, and the foreign body area mainly records garbage and other unknown objects.

[0068] S4: Defect classification or defect location is performed on the electrical equipment diagram through the detection head.

[0069] S5: The electrical equipment diagram is denoised through the prediction result denoising module, and the Monte Carlo method is used to repeatedly sample the electrical equipment diagram to remove random noise and improve the quality of the electrical equipment diagram.

[0070] S51: The detection head parameters of YOLOv5 are randomly perturbed through the prediction result denoising module, and the detection head with random perturbations is used to perform K inferences on the same input sample to achieve K different prediction results for the same input.

[0071] The K prediction results here can all be regarded as having an unknown random noise. Therefore, when K is large enough, such as K ≥ 10, the most approximate optimal solution can be obtained, and the mean of the K prediction results can be used as the denoised prediction result. Reduce image noise and improve image clarity.

[0072] The advantage of the Monte Carlo algorithm is that it can directly use a computer to generate a large number of random numbers for calculation. The more samples are taken, the greater the probability of obtaining the most approximate optimal solution, and the higher the clarity of the image pixels, which solves many problems that are difficult to solve with traditional methods.

[0073] The specific Monte Carlo operation is as follows:

[0074] 1. Use a random number generator to generate a random pixel grid molecular configuration.

[0075] 2. Make irregular changes to the particle coordinates of this pixel grid molecular configuration to generate a new pixel grid molecular configuration.

[0076] 3. Calculate the energy of the new pixel grid molecular configuration.

[0077]

[0078] Among them, vout(k) represents the energy of the kth output pixel block in vout, vin(k) represents the energy of the kth input pixel block in vin, na is a fixed parameter, and k is a value variable.

[0079] 4. Compare the energy change of the new pixel grid molecular configuration with the pixel grid molecular configuration before the change, and determine whether to accept the configuration.

[0080] If the energy of the new pixel grid molecular configuration is lower than the energy of the original pixel grid molecular configuration, the new pixel grid configuration is accepted and the next iteration is repeated using this configuration.

[0081] If the energy of the new pixel grid molecular configuration is higher than the energy of the original pixel grid molecular configuration, the Boltzmann factor is calculated and a random number is generated.

[0082] If this random number is greater than the calculated Boltzmann factor, then abandon this configuration and recalculate.

[0083] If the random number is less than the calculated Boltzmann factor, the configuration is accepted and the next iteration is repeated using this configuration.

[0084] 5. Perform iterative calculations in this way, perform K inferences on the same input sample, and output K different prediction results for the same input. Finally, search for the pixel grid that is lower than the given energy condition, obtain the optimal pixel grid, and end.

[0085] The electrical equipment image data is coded and labeled, and importance parameters are assigned according to the degree of impact of the electrical equipment image data on subsequent operation and maintenance. The electrical equipment image data includes geographical location, altitude, equipment height, and maintenance part location, and is normalized.

[0086] Embodiment 2:

[0087] An improved yolov5 UAV inspection image processing system, such as Figure 2 As shown, the system adopts an improved yolov5 UAV inspection image processing method as described above, characterized in that the system includes a data acquisition module for assigning UAVs to take high-definition images of electrical equipment according to the inspection route to collect image data of electrical equipment;

[0088] A deep network feature extractor for generating missing image information, using image downsampling network processing and multi-scale reasoning to computer-generate missing image information from electrical equipment diagrams;

[0089] A multi-scale feature fusion module is used to perform multi-scale feature fusion on the electrical equipment diagram and output the defect area of ​​the electrical equipment diagram. The output value is the defect area of ​​the electrical equipment diagram. The defect area includes a broken wire area, a falling area, and a foreign body area.

[0090] Detection head, used for defect classification or defect location on electrical equipment diagrams;

[0091] The prediction result denoising module is used to denoise the electrical equipment diagram. The Monte Carlo method is used to repeatedly sample the electrical equipment diagram to remove random noise and improve the quality of the electrical equipment diagram.

[0092] A computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute an improved yolov5 unmanned aerial vehicle inspection image processing method as described above.

[0093] A processor is used to run a program, wherein when the program is running, an improved yolov5 unmanned aerial vehicle inspection image processing method as described above is executed.

[0094] Those of ordinary skill in the art will appreciate that the units (or modules, steps, etc., the same below) of the examples described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0095] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0096] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nly Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An improved yolov5 UAV inspection image processing method, characterized in that: Steps include S1: Data collection: assigning drones to take high-definition images of electrical equipment according to the patrol route to collect image data of electrical equipment; S2: generating missing image information through a deep network feature extractor, and computer-generating missing image information of the electrical equipment diagram using image downsampling network processing and multi-scale reasoning; S21: performing feature extraction data preprocessing on the high-definition image data, using a multi-head self-attention mechanism for image downsampling network processing, and using a multi-head self-attention mechanism module to quickly reduce the resolution of the electrical equipment image, with a high degree of image retention, retaining key information; S22: performing multi-scale reasoning on the high-definition image data, using a multi-scale reasoning method to send multiple feature maps of different sizes output by the image downsampling network to the same YOLOv5 model for reasoning, wherein the reasoning processes of the feature maps are independent of each other and do not interfere with each other, and the feature maps are merged to output a predicted image; S3: performing multi-scale feature fusion on the electrical equipment diagram through a multi-scale feature fusion module, and outputting a defective area of ​​the electrical equipment diagram, where the output value is the defective area of ​​the electrical equipment diagram, and the defective area includes a broken wire area, a falling-off area, and a foreign body area; S4: classifying or locating defects on the electrical equipment diagram through a detection head; S5: De-noising the electrical equipment diagram through a prediction result denoising module, and repeatedly sampling the electrical equipment diagram using a Monte Carlo method, thereby removing random noise and improving the quality of the electrical equipment diagram.

2. According to the improved yolov5 unmanned aerial vehicle inspection image processing method of claim 1, it is characterized in that: S11: Electrical equipment includes drone image or video datasets of small equipment targets, medium equipment targets, and large equipment targets, and the datasets are annotated.

3. The improved yolov5 unmanned aerial vehicle inspection image processing method according to claim 1 is characterized in that: S211: The image downsampling network is composed of several layers, and the downsampling ratio of each layer is n. The size of the input feature is C×H×W, that is, H pixels in height, W pixels in width, and C channels in total. Then each layer of the network divides the input feature into n×n grids in the spatial dimension, for a total of Grid; S212: The n contained in each n×n grid 2 The features of each pixel are fused and then passed to the multi-head self-attention mechanism module to calculate the attention information globally, and finally output the features of one pixel for each grid. Pixels, width Feature map of pixels.

4. The improved yolov5 unmanned aerial vehicle inspection image processing method according to claim 1 is characterized in that: The image supplementation process also includes: S221: The image downsampling network outputs a feature map of size m×m pixels, as well as an additional feature map with a width and height of 1.5m and 0.5m respectively. The three are independently inferred using the same YOLOv5 model, and the output results of the three are finally merged.

5. The improved yolov5 unmanned aerial vehicle inspection image processing method according to claim 2 is characterized in that: S51: The detection head parameters of YOLOv5 are randomly perturbed through the prediction result denoising module, and the detection head with random perturbations is used to perform K inferences on the same input sample to achieve K different prediction results for the same input.

6. The improved yolov5 unmanned aerial vehicle inspection image processing method according to claim 2 is characterized in that: The electrical equipment image data is coded and labeled, and importance parameters are assigned according to the impact of the electrical equipment image data on subsequent operation and maintenance. The electrical equipment image data includes geographical location, altitude, equipment height, and maintenance location.

7. An improved yolov5 unmanned aerial vehicle inspection image processing system, the system adopts an improved yolov5 unmanned aerial vehicle inspection image processing method according to any one of claims 1 to 6, characterized in that: The system includes a data acquisition module for assigning a drone to capture high-definition images of electrical equipment according to a patrol route and to collect image data of the electrical equipment; A deep network feature extractor for generating missing image information, using image downsampling network processing and multi-scale reasoning to computer-generate missing image information in the electrical equipment diagram; A multi-scale feature fusion module is used to perform multi-scale feature fusion on the electrical equipment diagram and output a defect area of ​​the electrical equipment diagram. The output value is the defect area of ​​the electrical equipment diagram. The defect area includes a broken wire area, a falling area, and a foreign body area. A detection head, used for classifying or locating defects in the electrical equipment diagram; The prediction result denoising module is used to denoise the electrical equipment diagram by repeatedly sampling the electrical equipment diagram using the Monte Carlo method, thereby removing random noise and improving the quality of the electrical equipment diagram.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute an improved yolov5 drone inspection image processing method according to any one of claims 1 to 6.

9. A processor, characterized in that: The processor is used to run the program, wherein when the program is running, an improved YOLOv5 unmanned aerial vehicle inspection image processing method is executed.

Citation Information

Cited By

  • Power grid fault point high-definition imaging and intelligent identification system and method based on unmanned aerial vehicle

    CN119693828A