Unmanned aerial vehicle image recognition method and device based on YOLOv8 lightweight network model

Abstract

The invention relates to an unmanned aerial vehicle image recognition method based on a YOLOv8 lightweight network model, and the method comprises the steps: collecting and preprocessing image data containing multiple visual angles, multiple illumination and multiple weather conditions, and forming a data set; constructing a lightweight unmanned aerial vehicle image recognition model; training is carried out, and an optimal unmanned aerial vehicle image recognition model obtained through training is stored; and obtaining an identification result. According to the method, the lightweight CPU network is introduced, so that the parameter quantity is greatly reduced, the calculation complexity is reduced by 80%-90%, the smoothness and the stability of the unmanned aerial vehicle during execution of a picture recognition task are ensured, and a key guarantee is provided for continuous operation of the unmanned aerial vehicle in actual operation; the method can accurately focus on key features in the image, effectively reduces redundant information interference, can achieve efficient and accurate image recognition under the condition of limited computing resources of the unmanned aerial vehicle, and provides powerful technical support for application of the unmanned aerial vehicle in multiple fields.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method and device for drone image recognition based on a lightweight network model of YOLOv8. Background Art

[0002] Today, drones, with their numerous advantages, such as flexibility and adaptability to terrain, have penetrated deeply into key sectors such as agriculture, geography, and security. In the field of power transmission line inspection, drones are playing an increasingly important role, making significant contributions to ensuring the stable operation of power systems. The use of drones equipped with high-definition cameras for transmission line inspections primarily focuses on data collection. Drones can traverse pre-defined inspection routes, navigating mountains and urban skyscrapers, and approaching transmission towers and lines at close range. Compared to traditional manual inspections, drones can easily reach difficult and inaccessible areas, capturing clear, all-round images of key transmission line components such as insulators, hardware, and conductors. For example, in complex mountainous terrain, drones fly low over towers every 10-15 minutes, capturing 8-12 images of component status from multiple angles and under varying lighting conditions. This allows them to accurately capture subtle changes in the line, such as surface contamination and cracks on insulators and signs of conductor wear and breakage.

[0003] However, drones have significant limitations. Due to their limited payload and energy reserves, the computing devices they carry often face insufficient storage space and weak computing power. Traditional image recognition algorithm models are often built on complex neural network architectures, containing millions or even more parameters, and the computational process is complex and time-consuming. Forcibly porting such "heavyweight" models to drone platforms would cause a series of problems. First, the model's storage requirements could easily exceed the drone's onboard storage capacity, causing system lag or even crashes. Second, the complex computational process makes the recognition process time-consuming, unable to meet the real-time operation requirements of drones, severely limiting the drone's potential. Therefore, the development of lightweight, high-precision image recognition methods suitable for drones is extremely urgent. Summary of the Invention

[0004] In order to solve the problems of complex and time-consuming calculation process of traditional models, insufficient ability to focus on the features of key targets, and susceptibility to background noise, the primary purpose of the present invention is to provide a drone image recognition method based on a lightweight network model of YOLOv8, which significantly reduces the number of parameters, can accurately focus on the key features in the image, and effectively reduce redundant information interference.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a drone image recognition method based on a lightweight network model of YOLOv8, the method comprising the following steps in order:

[0006] (1) According to the transmission line inspection task, the UAV flight route and shooting frequency are planned, and image data with multiple perspectives, multiple lighting conditions, and multiple weather conditions are collected and preprocessed. The preprocessed data form a dataset, which is divided into a training set, a validation set, and a test set;

[0007] (2) Construct a lightweight drone image recognition model, select the YOLOv8 network as the basic architecture of the lightweight network, use a lightweight CPU network to replace the backbone network of the YOLOv8 network, add a global attention mechanism module after each C2f module in the neck network of the YOLOv8 network, and use the F-CIOU loss function to replace the CIOU loss function;

[0008] (3) Input the training set into the lightweight UAV image recognition model for training, and save the optimal UAV image recognition model obtained through training;

[0009] (4) The power transmission inspection images taken by the drone to be identified are input into the optimal drone image recognition model to obtain the recognition results.

[0010] In step (1), the pretreatment comprises the following steps:

[0011] (1a) Adopting an adaptive filtering algorithm to eliminate image noise, wherein the adaptive filtering algorithm adopts salt and pepper noise removal based on median filtering or smoothing based on Gaussian weighting;

[0012] (1b) Convert the color image into grayscale image by weighted averaging method;

[0013] (1c) Perform normalization processing to map pixel values ​​to the interval [0, 1] to ensure data scale consistency.

[0014] In step (2), the lightweight CPU network includes a 3×3 ordinary convolution, 6 3×3 depth-separable convolutions, 3 5×5 degree-separable convolutions and a spatial pyramid pooling module, using 3×3 ordinary convolution to extract low-level features of the image; the depth-separable convolution includes depth-wise convolution and point-by-point convolution;

[0015] The depth convolution applies a 3×3 convolution kernel to each input channel independently, extracting local spatial features channel by channel, with a parameter value of K 2 ×C in , K is the convolution kernel size, C in is the number of input channels;

[0016] Point-by-point convolution uses a 1×1 convolution kernel to integrate inter-channel information, with a parameter value of 1 2 ×C in ×C out , C out is the number of output channels.

[0017] In step (2), the global attention mechanism module includes channel attention and spatial attention;

[0018] Channel attention performs global average pooling on the input feature map to generate a channel-level weight vector:

[0019] M c (F) = σ(W1·GAP(F)+W0)

[0020] Among them, σ is the activation function, W1 and W0 are the parameters of the fully connected layer, and GAP(F) is the global average pooling result;

[0021] Spatial attention performs a convolution operation on the feature map after channel attention to generate a spatial weight matrix:

[0022] M S (F′)=σ(Conv 7×7 (F′))

[0023] Where F′ is the feature map obtained by channel attention, Conv 7×7 The convolution operation is performed on the feature map F′ using a 7×7 convolution kernel;

[0024] The final output feature map is:

[0025] F″=F′⊙M S (F′)⊙M C (F).

[0026] In step (2), the F-CIOU loss function is:

[0027]

[0028] Among them, IOU is the intersection-union ratio of the predicted box and the real box, ρ(b,b gt ) is the center distance, c is the minimum bounding box diagonal length, v is the aspect ratio consistency measure, α is the balance factor, and β is an adjustable hyperparameter.

[0029] Step (3) specifically includes the following steps in order:

[0030] (3a) Set SGD stochastic gradient descent as the optimization function of the network, the number of training epochs is 300, the initial learning rate is 0.01, and the batch size is set to 16;

[0031] (3b) Inputting the training sample set into the lightweight UAV image recognition model for training;

[0032] (3c) The converged network model that achieves the training goal is obtained, which is the optimal drone image recognition model.

[0033] Another object of the present invention is to provide an electronic device, comprising:

[0034] processor; and

[0035] A memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the drone image recognition method based on the lightweight network model of YOLOv8 as described above.

[0036] The present invention also provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, causes the processor to execute the drone image recognition method based on the YOLOv8 lightweight network model as described above.

[0037] It can be seen from the above technical solution that the beneficial effects of the present invention are: First, by introducing a lightweight CPU network, namely PPLCNet, the number of parameters is greatly reduced, and the computational complexity is reduced by 80%-90%. This enables the model to effectively adapt to the limited onboard storage capacity and relatively weak computing power of the drone, avoiding system freezes or crashes caused by excessive model size, ensuring the fluency and stability of the drone when performing image recognition tasks, and providing a key guarantee for the continuous operation of the drone in actual operations; second, the integration of the global attention mechanism module can accurately focus on the key features in the image and effectively reduce redundant information interference. In the transmission line inspection image test, the detection accuracy of insulator defects is improved by about 12%, and the recall rate of wire wear detection is improved by about 10%; third, the F-CIOU loss function re-weights the CIOU loss by IoU, optimizes the regression performance of the target detection model, and makes the model training process more efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a flow chart of the method of the present invention;

[0039] Figure 2 Schematic diagram of the lightweight UAV image recognition model in the present invention. DETAILED DESCRIPTION

[0040] like Figure 1 As shown, a drone image recognition method based on a lightweight network model of YOLOv8 includes the following steps in sequence:

[0041] (1) According to the transmission line inspection task, the UAV flight route and shooting frequency are planned, and image data with multiple perspectives, multiple lighting conditions, and multiple weather conditions are collected and preprocessed. The preprocessed data form a dataset, which is divided into a training set, a validation set, and a test set;

[0042] (2) Build a lightweight drone image recognition model, such as Figure 2 As shown in the figure, the YOLOv8 network is selected as the basic architecture of the lightweight network, the backbone network of the YOLOv8 network is replaced by a lightweight CPU network, a global attention mechanism module is added after each C2f module in the neck network of the YOLOv8 network, and the F-CIOU loss function is used to replace the CIOU loss function;

[0043] (3) Input the training set into the lightweight UAV image recognition model for training, and save the optimal UAV image recognition model obtained through training;

[0044] (4) The power transmission inspection images taken by the drone to be identified are input into the optimal drone image recognition model to obtain the recognition results.

[0045] In step (1), the pretreatment comprises the following steps:

[0046] (1a) Adopting an adaptive filtering algorithm to eliminate image noise, wherein the adaptive filtering algorithm adopts salt and pepper noise removal based on median filtering or smoothing based on Gaussian weighting;

[0047] (1b) Convert the color image into grayscale image by weighted averaging method;

[0048] (1c) Perform normalization processing to map pixel values ​​to the interval [0, 1] to ensure data scale consistency.

[0049] In step (2), the lightweight CPU network includes a 3×3 ordinary convolution, 6 3×3 depth-separable convolutions, 3 5×5 degree-separable convolutions and a spatial pyramid pooling module, using 3×3 ordinary convolution to extract low-level features of the image; the depth-separable convolution includes depth-wise convolution and point-by-point convolution;

[0050] The depth convolution applies a 3×3 convolution kernel to each input channel independently, extracting local spatial features channel by channel, with a parameter value of K 2 ×C in , K is the convolution kernel size, C in is the number of input channels;

[0051] Point-by-point convolution uses a 1×1 convolution kernel to integrate inter-channel information, with a parameter value of 1 2 ×C in ×C out , Cout is the number of output channels.

[0052] In step (2), the global attention mechanism module includes channel attention and spatial attention;

[0053] Channel attention performs global average pooling on the input feature map to generate a channel-level weight vector:

[0054] M c (F) = σ(W1·GAP(F)+W0)

[0055] Among them, σ is the activation function, W1 and W0 are the parameters of the fully connected layer, and GAP(F) is the global average pooling result;

[0056] Spatial attention performs a convolution operation on the feature map after channel attention to generate a spatial weight matrix:

[0057] M S (F′)=σ(Conv 7×7 (F′))

[0058] Where F′ is the feature map obtained by channel attention, Conv 7×7 The convolution operation is performed on the feature map F′ using a 7×7 convolution kernel;

[0059] The final output feature map is:

[0060] F″=F′⊙M S (F′)⊙M C (F).

[0061] In step (2), the F-CIOU loss function is:

[0062]

[0063] Among them, IOU is the intersection-union ratio of the predicted box and the real box, ρ(b,b gt ) is the center distance, c is the minimum bounding box diagonal length, v is the aspect ratio consistency measure, α is the balance factor, and β is an adjustable hyperparameter.

[0064] Step (3) specifically includes the following steps in order:

[0065] (3a) Set SGD stochastic gradient descent as the optimization function of the network, the number of training epochs is 300, the initial learning rate is 0.01, and the batch size is set to 16;

[0066] (3b) Inputting the training sample set into the lightweight UAV image recognition model for training;

[0067] (3c) The converged network model that achieves the training goal is obtained, which is the optimal drone image recognition model.

[0068] In actual operations, for power transmission line inspections, drone flight paths are planned based on the geographical distribution of the transmission lines and historically high-fault areas. For example, for line sections crossing mountainous areas surrounded by dense vegetation, the flight altitude is appropriately lowered to 50-80 meters, and the shooting frequency is increased to once every 10 minutes. This ensures clear capture of details such as insulators, conductors, and the surrounding environment, while minimizing image quality issues caused by tree branches.

[0069] After image acquisition, denoising is performed on the drone's onboard processor. Taking the adaptive filtering algorithm as an example, this algorithm dynamically adjusts filter parameters through statistical analysis of image pixel neighborhoods. For salt and pepper noise, the adaptive filtering algorithm determines the size range of the neighborhood window (e.g., 3×3, 5×5, etc.), as well as the initial value and adjustment step size of the noise detection threshold, based on the characteristics of the drone's image sensor and common noise levels. Isolated noisy pixels are identified, and their grayscale values ​​are replaced with the median or mean of their neighboring pixels based on their grayscale difference from surrounding pixels, effectively removing noise interference. During Gaussian noise removal, weighted smoothing is performed on the image based on the probability distribution of the noise, reducing the blurring effect of noise on image details. Grayscaling uses a weighted averaging method, assigning appropriate weights (e.g., 0.299, 0.587, and 0.114) to the red, green, and blue channels based on the human eye's sensitivity to different colors. This converts the color image into grayscale, ensuring clarity of the image's primary texture features while reducing data volume. The normalization operation maps the image pixel values ​​to the [0, 1] interval, ensuring the consistency of different image data scales and facilitating subsequent model processing.

[0070] After selecting the YOLOv8 network as the infrastructure, the introduction of depthwise separable convolution is a key step. Within the network's convolutional layers, during the depthwise convolution stage, a separate convolution kernel is used for feature extraction for each input channel. For example, when processing images of power transmission lines captured by drones, for texture feature extraction of insulator images, the convolution kernel for each channel can focus on extracting the unique texture information of that channel, such as the enamel texture and stain texture on the insulator surface. During the pointwise convolution stage, information fusion between channels is achieved through a 1×1 convolution kernel, integrating the features extracted from each channel by the depthwise convolution to form a complete feature description. Actual testing has shown that when processing a dataset containing 1,000 images of power transmission line components, the use of depthwise separable convolution reduced the number of model parameters from millions to hundreds of thousands, and computation time was shortened by approximately 85%, significantly reducing computational complexity and improving processing efficiency.

[0071] When incorporating the Global Attention Mechanism (GAM) during training, for example, when processing images containing multiple insulators and conductors using transmission line inspection images, the channel attention module first analyzes the feature map. Key feature channels of insulators, such as those reflecting their shape and surface condition, are given higher weights, enhancing their influence in subsequent calculations; channels containing background or interference information are weighted lower. The intermediate feature map, corrected by the channel attention, is then further processed by the spatial attention mechanism. This mechanism focuses on the location of objects in the image, assigning higher attention to areas containing insulators and conductors, highlighting the features of these key objects and suppressing interference from irrelevant areas. This enables the model to more accurately identify and locate objects, improving detection accuracy. In a test of 500 transmission line images against complex backgrounds, incorporating the GAM improved the model's detection accuracy for insulator defects by approximately 12% and its recall for conductor wear by approximately 10%.

[0072] When improving the YOLOv8 loss function, the F-CIOU loss function is reweighted by using the Intersection over Union (IoU) loss during training. Taking a security inspection image dataset containing objects of varying sizes and positions as an example, during the initial training phase, high-quality anchor examples with high IoU are given a relatively large weight in the loss calculation, guiding the model to more quickly learn accurate object location and classification information. For low-quality anchor examples with low IoU, the weights are appropriately reduced, but some influence is retained to prevent the model from completely ignoring potentially useful information contained in these examples. By continuously adjusting the model parameters, after 100 epochs of training, the model's accuracy in identifying insulator defects improved by approximately 8% compared to the original YOLOv8 network, and its convergence speed was accelerated by approximately 20%.

[0073] In terms of data partitioning, a dataset containing 10,000 images captured by drones during power transmission inspections is used as an example. Of these, 7,000 images are used as a training set, 1,500 as a validation set, and 1,500 as a verification set. The training set selection ensures coverage of images from different transmission line regions (such as mountainous areas, urban areas, and cross-river lines), different component conditions (contamination, cracks, and spontaneous explosions of insulators, wear, broken strands, and loose strands of conductors, deformation and corrosion of hardware, etc.), and various weather conditions (sunny, cloudy, rainy, foggy, snowy, etc.), providing the model with a rich set of learning samples. The validation set is selected for its representativeness, reflecting the model's performance under different actual operating conditions, and serving as a timely monitoring and adjustment tool during the training process. The test set retains its independence and is used to objectively evaluate the model's generalization capabilities.

[0074] During training, a stochastic gradient descent algorithm is used, with an initial learning rate of 0.01. After every 10 epochs, the learning rate is reduced by a decay factor of 0.5. Initially, a high learning rate enables the model to rapidly explore the parameter space. For example, when processing transmission line inspection images, it can quickly adjust the convolution kernel parameters and attention mechanism weights, initially learning the general characteristics and common defect patterns of objects such as insulators and conductors. As training progresses, as the learning rate is gradually reduced, the model can make finer adjustments based on existing learning, such as more accurately locating small cracks in insulators or slightly worn areas on conductors, thereby improving recognition accuracy. Throughout training, the model's accuracy, recall, and F1 score are continuously monitored using a validation set. When the model's accuracy on the validation set stops improving for four consecutive epochs, the model is considered to have reached its optimal state. Training is then terminated and the optimal model is saved. For example, in a model training for transmission line inspection images, after 80 epochs of training, the model achieved an accuracy of 90%, a recall rate of 88%, and an F1 value of 89% on the validation set, meeting the performance requirements of transmission line inspection operations. The model was saved for subsequent transmission line inspection image recognition tasks.

[0075] In summary, the present invention significantly reduces the number of parameters and computational complexity by 80%-90% by introducing a lightweight CPU network, namely PPLCNet. This enables the model to effectively adapt to the limited onboard storage capacity and relatively weak computing power of drones, avoiding system lag or crashes caused by excessive model size, ensuring the smoothness and stability of drones when performing image recognition tasks, and providing a key guarantee for the continuous operation of drones in actual operations. The integration of a global attention mechanism module can accurately focus on key features in the image and effectively reduce redundant information interference. In the transmission line inspection image test, the detection accuracy of insulator defects is improved by approximately 12%, and the recall rate of wire wear detection is improved by approximately 10%. The F-CIOU loss function reweights the CIOU loss by IoU, optimizing the regression performance of the target detection model and making the model training process more efficient. The present invention can achieve efficient and accurate image recognition under the conditions of limited computing resources of drones, providing strong technical support for the application of drones in multiple fields.

[0076] The above shows and describes the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions merely illustrate the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A drone image recognition method based on a lightweight network model of YOLOv8, characterized by: The method comprises the following steps in sequence: (1) According to the transmission line inspection task, the UAV flight route and shooting frequency are planned, and image data with multiple perspectives, multiple lighting conditions, and multiple weather conditions are collected and preprocessed. The preprocessed data form a dataset, which is divided into a training set, a validation set, and a test set; (2) Construct a lightweight drone image recognition model, select the YOLOv8 network as the basic architecture of the lightweight network, use a lightweight CPU network to replace the backbone network of the YOLOv8 network, add a global attention mechanism module after each C2f module in the neck network of the YOLOv8 network, and use the F-CIOU loss function to replace the CIOU loss function; (3) Input the training set into the lightweight UAV image recognition model for training, and save the optimal UAV image recognition model obtained through training; (4) The power transmission inspection images taken by the drone to be identified are input into the optimal drone image recognition model to obtain the recognition results.

2. The drone image recognition method based on the YOLOv8 lightweight network model according to claim 1 is characterized in that: In step (1), the pretreatment includes the following steps: (1a) Adopting an adaptive filtering algorithm to eliminate image noise, wherein the adaptive filtering algorithm adopts salt and pepper noise removal based on median filtering or smoothing based on Gaussian weighting; (1b) Convert the color image into grayscale image by weighted averaging method; (1c) Perform normalization processing to map pixel values ​​to the interval [0, 1] to ensure data scale consistency.

3. The drone image recognition method based on the YOLOv8 lightweight network model according to claim 1 is characterized in that: In step (2), the lightweight CPU network includes a 3×3 ordinary convolution, 6 3×3 depth-separable convolutions, 3 5×5 degree-separable convolutions and a spatial pyramid pooling module, using 3×3 ordinary convolution to extract low-level features of the image; the depth-separable convolution includes depth-wise convolution and point-by-point convolution; The depth convolution applies a 3×3 convolution kernel to each input channel independently, extracting local spatial features channel by channel, with a parameter value of K 2 ×C in , K is the convolution kernel size, C in is the number of input channels; Point-by-point convolution uses a 1×1 convolution kernel to integrate inter-channel information, with a parameter value of 1 2 ×C in ×C out , C out is the number of output channels.

4. The drone image recognition method based on the YOLOv8 lightweight network model according to claim 1 is characterized in that: In step (2), the global attention mechanism module includes channel attention and spatial attention; Channel attention performs global average pooling on the input feature map to generate a channel-level weight vector: M c (F)=σ(W1·GAP(F)+W0) Among them, σ is the activation function, W1 and W0 are the parameters of the fully connected layer, and GAP(F) is the global average pooling result; Spatial attention performs a convolution operation on the feature map after channel attention to generate a spatial weight matrix: M S (F′)=σ(Conv 7×7 (F′)) Where F′ is the feature map obtained by channel attention, Conv 7×7 The convolution operation is performed on the feature map F′ using a 7×7 convolution kernel; The final output feature map is: F″=F′⊙M S (F′)⊙M C (F)。 5. The drone image recognition method based on the YOLOv8 lightweight network model according to claim 1, characterized in that: In step (2), the F-CIOU loss function is: Among them, IOU is the intersection-union ratio of the predicted box and the real box, ρ(b,b gt ) is the center distance, c is the minimum bounding box diagonal length, v is the aspect ratio consistency measure, α is the balance factor, and β is an adjustable hyperparameter.

6. The drone image recognition method based on the YOLOv8 lightweight network model according to claim 1, characterized in that: Step (3) specifically includes the following steps in order: (3a) Set SGD stochastic gradient descent as the optimization function of the network, the number of training epochs is 300, the initial learning rate is 0.01, and the batch size is set to 16; (3b) Inputting the training sample set into the lightweight UAV image recognition model for training; (3c) The converged network model that achieves the training goal is obtained, which is the optimal drone image recognition model.

7. An electronic device comprising: processor; as well as A memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor causes the processor to execute the drone image recognition method based on the lightweight network model of YOLOv8 according to any one of claims 1 to 6.

8. A computer-readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the processor executes the drone image recognition method based on the YOLOv8 lightweight network model according to any one of claims 1 to 6.

Images