Unmanned aerial vehicle image high-efficiency identification system

Abstract

The invention relates to an unmanned aerial vehicle image high-efficiency identification system. The system comprises an unmanned aerial vehicle, an image acquisition unit, an image preprocessing unit, a target detection unit, a target tracking unit and a control unit. The unmanned aerial vehicle has a high-speed stable flight capability, and the image acquisition unit consists of a high-resolution camera and is used for capturing image data in real time in a flight process; the image preprocessing unit carries out denoising and deblurring processing on the captured image data to generate preprocessed image data; the target detection unit adopts a pre-trained deep learning model to realize efficient recognition of a target object in the preprocessed image; the target tracking unit is combined with a Kalman filter and an optical flow method to lock and track a target object; the control unit adjusts the flight path of the unmanned aerial vehicle according to the target object position and motion information provided by the target tracking unit to ensure continuous tracking of the target object; according to the system, efficient and accurate identification and tracking of a target object are realized through cooperative work of all the units.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a high-efficiency unmanned aerial vehicle image recognition system. Background Art

[0002] Drones are widely used in various fields, including environmental monitoring, agriculture, logistics, disaster relief, and military. In order to improve the autonomous flight capabilities of drones in complex environments, image recognition and target tracking technologies have received increasing attention. Traditional drone image recognition systems usually rely on basic camera equipment and simple image processing algorithms to capture and analyze image data along the flight path.

[0003] However, these existing systems have many limitations in high-speed flight. First, traditional camera equipment has difficulty capturing clear images when moving at high speeds, resulting in reduced recognition accuracy. Second, existing image processing algorithms are inefficient when processing large amounts of high-resolution image data and cannot respond to dynamic changes during flight in real time. In addition, traditional target tracking methods cannot stably lock onto targets during high-speed flight, affecting the drone's autonomous flight and mission execution capabilities.

[0004] Therefore, it is very necessary to develop a new and efficient drone image recognition system. Summary of the invention

[0005] The present application provides a high-efficiency drone image recognition system to achieve efficient and accurate recognition and tracking of target objects.

[0006] The present application provides a high-efficiency drone image recognition system, comprising: UAV, for high-speed and stable flight; An image acquisition unit, including a high-resolution camera placed on the drone to capture image data in real time during flight; An image preprocessing unit, used for performing preprocessing operations including denoising and deblurring on the image data captured by the image acquisition unit to obtain preprocessed image data; A target detection unit, comprising a pre-trained deep learning model, for identifying a target object in the pre-processed image provided by the image pre-processing unit, wherein the deep learning model is optimized based on a convolutional neural network structure to improve recognition speed and accuracy; A target tracking unit, used for executing an adaptive target tracking algorithm to lock and track the target object identified by the target detection unit, wherein the adaptive target tracking algorithm is implemented based on the fusion of Kalman filter and optical flow method, and is used for providing stable target tracking under high-speed flight conditions; The control unit adjusts the flight trajectory of the UAV according to the position and motion information of the target object provided by the target tracking unit to maintain continuous tracking of the target object.

[0007] This application has the following beneficial technical effects: (1) Through the combination of a high-resolution camera and an image preprocessing unit, the system is able to capture and process image data in real time during high-speed flight. The preprocessing unit performs denoising and deblurring operations, improving the clarity and quality of the image, providing a more reliable data basis for subsequent target recognition and tracking.

[0008] (2) The target detection unit uses a pre-trained deep learning model and is optimized based on a convolutional neural network structure. This optimization significantly improves the speed and accuracy of target recognition, and can quickly identify and classify target objects in images in complex environments, thereby improving the intelligence level of drones when performing tasks.

[0009] (3) The target tracking unit combines the Kalman filter and the optical flow method to achieve an adaptive target tracking algorithm, ensuring that the target object can be stably locked and tracked even in high-speed flight. This fusion algorithm improves the tracking stability and accuracy, allowing the drone to continuously track the target and maintain tracking effectiveness even in high-speed movement and complex environments.

[0010] (4) The control unit adjusts the flight trajectory of the drone in real time based on the position and motion information of the target object provided by the target tracking unit. This intelligent adjustment mechanism enables the drone to flexibly respond to the dynamic changes of the target object, ensuring continuous and stable tracking, and enhancing the autonomy and reliability of the drone in various tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of a high-efficiency drone image recognition system provided in the first embodiment of the present application. DETAILED DESCRIPTION

[0012] Many specific details are described in the following description to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present application, so the present application is not limited by the specific implementation disclosed below.

[0013] The first embodiment of the present application provides a high-efficiency drone image recognition system. Figure 1 , which is a schematic diagram of the first embodiment of the present application. Figure 1 The first embodiment of the present application provides a high-efficiency drone image recognition system for a detailed description.

[0014] The high-efficiency drone image recognition system comprises a drone 101 , an image acquisition unit 102 , an image preprocessing unit 103 , a target detection unit 104 , a target tracking unit 105 and a control unit 106 .

[0015] Drone 101, for high-speed and stable flight.

[0016] The drone 101 plays a vital role in the present invention, being responsible for achieving high-speed stable flight and ensuring that the system can operate efficiently in various tasks. In order to ensure high-speed stable flight, the drone 101 needs to have a number of key technologies and design features.

[0017] First of all, the UAV 101 is equipped with a powerful power system, including high-efficiency electric motors and a lightweight but sturdy fuselage structure. The electric motors provide sufficient thrust to enable the UAV to remain stable during high-speed flight, while the lightweight fuselage design helps to improve flight efficiency and reduce energy consumption. The fuselage material is usually made of carbon fiber composite materials to ensure its durability and stability during high-speed flight.

[0018] Secondly, the UAV 101 is also equipped with an advanced flight control system that integrates multiple sensors and control algorithms to achieve precise flight control. The main sensors include GPS module, inertial measurement unit (IMU), barometer and laser altimeter. These sensors provide real-time position, altitude, speed and attitude information, and the flight control system dynamically adjusts according to this data to ensure that the UAV remains stable during high-speed flight.

[0019] The flight control system uses advanced control algorithms such as PID control, LQR control and adaptive control. These algorithms, combined with sensor data, can adjust the drone's thrust, rudder and motor speed in real time to keep it in optimal condition under different flight conditions. Especially when flying at high speeds, these control algorithms can effectively respond to sudden airflow changes and environmental interference to ensure flight stability and safety.

[0020] In order to adapt to different mission requirements, UAV 101 also has a certain degree of autonomous flight capabilities. The flight control system has built-in path planning and obstacle avoidance algorithms, which can automatically adjust the flight trajectory according to the preset flight route and real-time environmental information, avoid obstacles, and fly flexibly in complex environments. These algorithms are usually combined with devices such as visual sensors and lidar, and through image recognition and laser scanning, they can perceive the surrounding environment in real time and respond quickly.

[0021] The communication system of UAV 101 is also crucial. It is equipped with a high-performance wireless communication module that can achieve real-time data transmission and command reception with the ground control station. The communication system usually uses multi-band radio technology to ensure stable signal connection in different environments. Through this communication link, the ground control station monitors the status of the UAV in real time and issues control commands when necessary to further ensure the smooth completion of the flight mission.

[0022] In short, UAV 101 ensures its stability and reliability in high-speed flight through the coordinated work of the power system, flight control system, autonomous flight capability and communication system.

[0023] The image acquisition unit 102 includes a high-resolution camera placed on the drone, and is used to capture image data in real time during flight.

[0024] The image acquisition unit 102 is a key component in the UAV image high-efficiency recognition system, responsible for capturing high-quality image data in real time during the high-speed flight of the UAV. The unit is mainly composed of a high-resolution camera and a related image transmission system to ensure the accuracy and timeliness of the image data.

[0025] The high-resolution camera is the core component of the image acquisition unit, and its performance directly affects the quality of the image data and the effect of subsequent processing. The camera usually has a resolution of no less than 4K to ensure that it can capture detailed image data. A high frame rate is also required, usually requiring 30 frames per second or higher, to ensure that continuous and clear images can be captured when the drone moves at high speed. The optical part of the camera uses a wide-angle lens to expand the field of view, ensure that it can cover more areas, and capture more environmental information.

[0026] In actual operation, the camera is fixedly mounted on the bottom or front of the drone and supported stably by an anti-vibration bracket. The anti-vibration bracket uses shock-absorbing materials and mechanisms to effectively absorb and buffer the vibrations generated during the flight of the drone, ensuring that the camera can remain stable under various flight conditions. This design greatly reduces the risk of image blur during flight and improves image clarity and stability.

[0027] The image transmission system is an important part of the image acquisition unit, responsible for transmitting the image data captured by the camera to the image preprocessing unit in real time. To this end, the system usually uses a high-speed data interface to ensure that large-capacity image data can be transmitted quickly and stably. The wireless transmission module is also a key part, especially when the UAV flies over long distances, the image data needs to be transmitted to the ground control station through a wireless channel. The wireless transmission module uses multi-band radio technology to ensure signal strength and transmission speed in different environments, avoiding data loss or delay due to unstable signals.

[0028] In addition, the image acquisition unit also includes an intelligent control system that manages the camera's working status and data transmission process. The system can dynamically adjust the camera's parameters, such as exposure time, focal length, and frame rate, according to flight conditions and mission requirements to ensure that the best images can be captured under different lighting and speed conditions. The intelligent control system also has an image cache function, which temporarily stores the image data captured by the camera through a built-in high-speed cache module to cope with emergencies during data transmission and ensure the integrity and continuity of the image data.

[0029] Through these designs and configurations, the image acquisition unit 102 can stably and continuously capture high-quality image data during the high-speed flight of the UAV, providing reliable basic data for subsequent image preprocessing, target detection and tracking.

[0030] The image preprocessing unit 103 is used to perform preprocessing operations including denoising and deblurring on the image data captured by the image acquisition unit to obtain preprocessed image data.

[0031] The image preprocessing unit 103 plays a key role in the UAV image high-efficiency recognition system. It is responsible for performing preprocessing operations such as denoising and deblurring on the image data captured by the image acquisition unit 102 to generate high-quality preprocessed image data to ensure the accuracy and stability of subsequent target detection and tracking.

[0032] First, the image preprocessing unit 103 starts to perform preliminary denoising by receiving the original image data transmitted by the image acquisition unit. The purpose of denoising is to eliminate the noise introduced in the image due to factors such as ambient light changes and sensor noise. Common denoising algorithms include Gaussian filtering, median filtering and bilateral filtering, which can effectively reduce noise while retaining image details. In specific implementation, the image preprocessing unit can dynamically select and adjust the filtering parameters to adapt to different image qualities and noise levels to ensure the optimization of the denoising effect.

[0033] After the denoising process is completed, the image preprocessing unit then performs deblurring. Deblurring is essential because the drone may produce motion blur during high-speed flight. Deblurring generally uses inverse filtering, blind deconvolution or motion compensation algorithms. These algorithms reconstruct clear images by analyzing the motion trajectory and blur level in the image. Implementing these algorithms requires high computing power, so the image preprocessing unit usually integrates a dedicated image processing chip or uses a high-performance processor to ensure the efficiency and effect of real-time processing.

[0034] After completing the denoising and deblurring processes, the image preprocessing unit further performs image enhancement operations. Image enhancement aims to improve the contrast and clarity of the image and make the target object more prominent. Commonly used methods include histogram equalization, adaptive histogram equalization, and Laplace enhancement. These methods can effectively enhance the details and edge information in the image and improve the recognition accuracy of the subsequent target detection unit.

[0035] In addition, the image preprocessing unit is also responsible for geometric correction and standardization of the image. Geometric correction is used to correct image distortion caused by camera viewing angle, changes in the drone's flight posture, etc., to ensure the geometric consistency of the image. Standardization includes adjusting the image to a fixed size and format to match the input standards required by the target detection unit. This step is usually achieved through operations such as image resampling, rotation, cropping, and normalization.

[0036] The design and implementation of the image preprocessing unit 103 ensures an efficient and smooth transition from image acquisition to image processing, and the generated high-quality preprocessed image data provides a reliable basis for subsequent target detection and tracking.

[0037] Furthermore, the image preprocessing unit is specifically used for: Perform noise suppression on image data and use adaptive median filtering algorithm to remove random noise in the image; Using motion estimation algorithms, image data is deblurred to restore motion blur caused by high-speed flight; The histogram equalization technique is applied to enhance the contrast and details of the image to improve the recognition accuracy of the subsequent target detection unit.

[0038] The image preprocessing unit is specifically used to perform a series of processing on the captured image data to improve the image quality and the accuracy of subsequent target detection. The specific implementation includes the following steps: First, the image preprocessing unit performs noise suppression on the image data. The noise suppression process uses the adaptive median filter algorithm, which is a commonly used denoising method, especially suitable for removing random noise in images, such as salt noise and pepper noise. The adaptive median filter algorithm processes the noise in the image by adjusting the size of the filter window. Specifically, the algorithm defines a window around each pixel, calculates the median value of the pixels within the window, and replaces the center pixel value with the median value, thereby effectively removing noise without losing image details.

[0039] Secondly, the image preprocessing unit uses a motion estimation algorithm to deblur the image data. Motion blur is usually caused by the rapid movement of the camera during the high-speed flight of the drone. The motion estimation algorithm first estimates the motion blur parameters present in the image, such as the direction and distance of movement. Then, through inverse filtering technology or blind deconvolution technology, the details in the image are restored to reduce or eliminate motion blur. This step is crucial to ensure the clarity of the image and the accuracy of subsequent processing.

[0040] Next, the image preprocessing unit applies histogram equalization technology to enhance the contrast and details of the image. Histogram equalization is a commonly used image processing technology that adjusts the grayscale distribution of the image to enhance the brightness and contrast in the image, thereby highlighting the details in the image. The specific operation is to calculate the grayscale histogram of the image and redistribute the grayscale values ​​through the cumulative distribution function so that the overall contrast of the image is improved. The enhanced image is visually clearer and has richer details, which helps the subsequent target detection unit improve the recognition accuracy.

[0041] Through the above steps, the image preprocessing unit can effectively improve the image quality and provide better input data for subsequent target detection and recognition. The adaptive median filtering algorithm can effectively remove random noise in the image, making the image cleaner; the motion estimation algorithm can restore the motion blur caused by high-speed flight, making the image clearer; the histogram equalization technology can enhance the contrast and details of the image, making the target object more prominent. These preprocessing steps work together to ensure that the image data obtained by the drone during high-speed flight can achieve the best quality, thereby improving the recognition accuracy and efficiency of the subsequent target detection unit.

[0042] The target detection unit 104 includes a pre-trained deep learning model for identifying the target object in the pre-processed image provided by the image pre-processing unit, wherein the deep learning model is optimized based on a convolutional neural network structure to improve recognition speed and accuracy.

[0043] The target detection unit 104 plays a vital role in the high-efficiency recognition system of drone images. Its main task is to identify the target object in the pre-processed image provided by the image pre-processing unit 103 through the pre-trained deep learning model. This unit relies on the advanced convolutional neural network (CNN) structure, and the optimized model can significantly improve the speed and accuracy of target recognition.

[0044] First, the object detection unit receives high-quality image data from the image preprocessing unit. In order to ensure the efficiency and accuracy of the recognition process, these image data have usually been de-noised, deblurred, and enhanced. After receiving the image, the object detection unit inputs it into the deep learning model for processing.

[0045] The core of the deep learning model is the convolutional neural network (CNN), which has excellent performance in processing image data. CNN consists of multiple convolutional layers, pooling layers, and fully connected layers, which extract feature information from the image through layer-by-layer processing. Specifically, the convolutional layer uses a set of trainable filters to perform convolution operations on the input image to extract low-level features such as edges and textures. The pooling layer uses downsampling operations to reduce the amount of data while retaining important features, thereby enhancing the model's anti-interference ability. The fully connected layer comprehensively processes the extracted features and outputs the category probability and position of each target object in the image.

[0046] In the target detection unit, the convolutional neural network structure is specifically optimized to meet the real-time processing requirements of drones flying at high speeds. First, the model adopts a lightweight design, which reduces the number of parameters and the amount of calculation, allowing it to run at high speed with limited computing resources. Second, by introducing skip connections and residual blocks, the model can more effectively transmit feature information, prevent gradient disappearance, and improve training efficiency and recognition accuracy.

[0047] In order to improve the robustness of target detection, the deep learning model also combines multi-scale feature fusion technology. Multi-scale feature fusion allows the model to extract and fuse feature information at different levels, thereby better identifying target objects of different scales and complexities. For example, the model can simultaneously consider the global features of large objects and the local details of small objects, significantly improving the accuracy and adaptability of detection.

[0048] During the training phase, the model uses large-scale labeled data for supervised learning, and uses the back-propagation algorithm to continuously adjust network parameters to minimize the gap between the predicted results and the true labels. The training data set covers a variety of flight scenarios and target object types, ensuring that the model has wide adaptability and high accuracy in practical applications.

[0049] In actual operation, the target detection unit not only identifies the category of the target object, but also accurately locates its position in the image. Each detection result includes the category label and location information (such as the coordinates of the bounding box) of the target object. This information is passed to the target tracking unit and control unit for subsequent target tracking and flight trajectory adjustment.

[0050] Furthermore, the deep learning model includes a feature extraction part, a feature fusion part, a target detection part and a post-processing part; The feature extraction part is used to extract features from the preprocessed image data to obtain a feature map; wherein the feature extraction part is implemented by an improved convolutional neural network, the improved convolutional neural network includes multiple convolutional layers, pooling layers and batch normalization layers, and introduces a separable convolutional layer; The feature fusion part is used to fuse the feature map provided by the feature extraction part to obtain a fused feature map; wherein the feature fusion part is implemented by a multi-scale feature fusion network, and the multi-scale feature fusion network adopts a fusion mechanism of feature maps with different resolutions; The object detection part is used to process the fused feature map provided by the feature fusion part to obtain candidate regions and corresponding confidence scores; the object detection part is implemented by a region proposal network, and an adaptive learning rate adjustment mechanism is introduced in the bounding box regression process to improve the accuracy of bounding box positioning; The post-processing part is used to process the candidate areas and corresponding confidence scores provided by the target detection part to obtain the position and category information of the target object.

[0051] The feature extraction part is the basic part of the deep learning model. Its main task is to extract features from the preprocessed image data and generate feature maps. This part is implemented using an improved convolutional neural network (CNN), which includes multiple layers of convolutional layers, pooling layers, and batch normalization layers. Separable convolutional layers are introduced to improve computational efficiency and reduce the number of parameters. Specifically, the convolutional layer is responsible for extracting low-level features in the image, such as edges and textures; the pooling layer reduces the size of the feature map through downsampling operations while retaining important features; the batch normalization layer is used to standardize the output of the convolutional layer, stabilize the training process, and accelerate convergence. The separable convolutional layer significantly reduces the computational complexity while maintaining the performance of the model by decomposing the standard convolution into depthwise convolution and pointwise convolution.

[0052] The function of the feature fusion part is to fuse the feature maps generated by the feature extraction part to generate a fused feature map. This part is implemented using the multi-scale feature fusion network (MSFN), which enhances the feature representation capability by fusing feature maps of different resolutions. In the specific implementation, the multi-scale feature fusion network uses mechanisms such as skip connections and feature pyramid networks (FPN) so that feature maps from different convolutional layers can be fused with each other, thereby capturing multi-scale target information in the image. Skip connections allow low-level features to be directly combined with high-level features, retaining more spatial information and details. The feature pyramid network further processes these fused feature maps to generate multi-scale fused feature maps, which improves the model's ability to detect targets of different sizes.

[0053] The target detection part uses the region proposal network (RPN) to process the fused feature map to generate candidate regions and corresponding confidence scores. The region proposal network generates a series of candidate regions on the feature map by sliding windows, and uses a convolutional neural network to extract the features of these candidate regions. Then, the RPN classifies and regresses the bounding boxes of these features to determine whether each candidate region contains the target object and gives the corresponding confidence score. In order to improve the accuracy of bounding box positioning, the target detection part introduces an adaptive learning rate adjustment mechanism. During the bounding box regression process, the learning rate is dynamically adjusted according to the regression error, so that the model can converge faster and more accurately, improving the positioning accuracy of the bounding box.

[0054] The post-processing part receives the candidate regions and confidence scores from the target detection part and further processes them to finally obtain the location and category information of the target object. Post-processing usually includes non-maximum suppression (NMS) and classification confidence threshold filtering. Non-maximum suppression is used to eliminate highly overlapping candidate regions and retain the regions most likely to contain the target object; classification confidence threshold filtering is used to remove candidate regions with lower confidence. After post-processing, the system can output the final location and category information of the target object to ensure the accuracy and reliability of the detection results.

[0055] Furthermore, the improved convolutional neural network specifically includes: a) Input layer, used to receive preprocessed image data, size is ; b) The first convolutional layer uses a convolution kernel size of , a convolution operation with a stride of 1, the output size is ; c) The first batch normalization layer is used to batch normalize the output of the first convolutional layer to keep the output size ; d) The first separable convolution layer includes depthwise convolution and pointwise convolution. The depthwise convolution uses a convolution kernel size of , stride is 1, point-by-point convolution uses Convolution, the output size is ; e) The first maximum pooling layer uses a pooling size of , a maximum pooling operation with a stride of 2, and an output size of

[0056] f) Repeat the structures from b) to e), gradually increase the number of channels and reduce the size of feature maps, to form a multi-level feature map, the output size of each layer gradually decreases, and the number of feature maps gradually increases; g) Output layer, which outputs multi-level feature maps, where the feature maps of each layer are different in resolution and number of channels.

[0057] First, the input layer is used to receive the preprocessed image data, the size of which is H×W×C, where H represents the height of the image, W represents the width of the image, and C represents the number of channels. The input layer ensures that the image data can smoothly enter the convolutional neural network for subsequent processing.

[0058] Next, the first convolutional layer performs preliminary processing on the input image data by using a 3×3 convolution kernel and a convolution operation with a stride of 1. This convolution operation can effectively extract low-level features in the image, such as edges and textures. After the first convolutional layer, the output feature map size is H×W×32, which means that the original image size remains unchanged, but the number of channels is increased to 32 to capture more feature information.

[0059] In order to improve the stability of the network and speed up the training process, the output of the first convolutional layer will be batch normalized through the first batch normalization layer. Batch normalization can standardize the mean and variance of the output feature map to avoid the problem of internal covariate shift. After batch normalization, the output size remains H×W×32.

[0060] Next, the first separable convolutional layer further processes the batch normalized feature map. The separable convolutional layer consists of depthwise convolution and pointwise convolution. Depthwise convolution uses a 3×3 convolution kernel and a stride of 1, while pointwise convolution uses a 1×1 convolution operation. Depthwise convolution performs convolution operations on each channel separately to retain more spatial information, while pointwise convolution increases the richness of feature expression by linearly combining channels. The output size of the first separable convolutional layer is H×W×64, which further increases the number of channels and captures richer image features.

[0061] After that, the first max pooling layer performs max pooling on the output of the separable convolutional layer. This pooling layer uses a 2×2 pooling kernel and a stride of 2, and reduces the size of the feature map by half through downsampling. Max pooling can extract the most significant features in the local area, reduce the size of the feature map, and retain important information. After the first max pooling layer, the output feature map size is H / 2×W / 2×64.

[0062] In order to form a multi-level feature map, the above convolution, batch normalization, separable convolution and maximum pooling operations are repeated in the network structure. Each layer gradually increases the number of channels and reduces the size of the feature map. This hierarchical structure enables the network to extract image features from different scales and levels, abstracting more advanced feature information layer by layer.

[0063] Finally, the output layer outputs multi-level feature maps, each of which has different resolutions and number of channels. These multi-level feature maps contain image features from low-level to high-level, which can provide rich information for subsequent target detection and recognition.

[0064] Through this structural design, the improved convolutional neural network can efficiently extract important features in the image, providing a solid foundation for target detection and tracking in the efficient drone image recognition system.

[0065] Below is a reference implementation code to illustrate the convolutional neural network for the feature extraction part of the drone image efficient recognition system.

[0066] import tensorflow as tf from tensorflow.keras.layers import Input, Conv2D, DepthwiseConv2D,Conv2DTranspose, MaxPooling2D, BatchNormalization, Activation from tensorflow.keras.models import Model def build_feature_extractor(input_shape): # Input layer, used to receive preprocessed image data, size is H×W×C inputs = Input(shape=input_shape, name='input_layer') # The first convolutional layer uses a convolution operation with a kernel size of 3×3 and a stride of 1, and the output size is H×W×32 x = Conv2D(filters=32, kernel_size=(3, 3), strides=1, padding='same', name='conv1')(inputs) # The first batch normalization layer is used to batch normalize the output of the first convolutional layer and keep the output size as H×W×32 x = BatchNormalization(name='bn1')(x) # Activation function layer, using ReLU activation function x = Activation('relu', name='relu1')(x) # The first separable convolution layer, including depthwise convolution and pointwise convolution # The depth convolution uses a convolution kernel size of 3×3 and a stride of 1 x = DepthwiseConv2D(kernel_size=(3, 3), strides=1, padding='same', name='dw_conv1')(x) # Point-by-point convolution uses 1×1 convolution, and the output size is H×W×64 x = Conv2D(filters=64, kernel_size=(1, 1), strides=1, padding='same', name='pw_conv1')(x) # Batch Normalization and Activation Function x = BatchNormalization(name='bn2')(x) x = Activation('relu', name='relu2')(x) # The first maximum pooling layer uses a maximum pooling operation with a pooling kernel size of 2×2 and a stride of 2, and the output size is H / 2× W / 2×64 x = MaxPooling2D(pool_size=(2, 2), strides=2, padding='same',name='pool1')(x) # Repeat convolution, batch normalization, separable convolution and maximum pooling operations to form a multi-level feature map # The second convolutional layer has an output size of H / 2× W / 2×128 x = Conv2D(filters=128, kernel_size=(3, 3), strides=1, padding='same', name='conv2')(x) x = BatchNormalization(name='bn3')(x) x = Activation('relu', name='relu3')(x) x = DepthwiseConv2D(kernel_size=(3, 3), strides=1, padding='same',name='dw_conv2')(x) x = Conv2D(filters=128, kernel_size=(1, 1), strides=1, padding='same', name='pw_conv2')(x) x = BatchNormalization(name='bn4')(x) x = Activation('relu', name='relu4')(x) x = MaxPooling2D(pool_size=(2, 2), strides=2, padding='same', name='pool2')(x) # The third convolutional layer, the output size is H / 4× W / 4×256 x = Conv2D(filters=256, kernel_size=(3, 3), strides=1, padding='same', name='conv3')(x) x = BatchNormalization(name='bn5')(x) x = Activation('relu', name='relu5')(x) x = DepthwiseConv2D(kernel_size=(3, 3), strides=1, padding='same',name='dw_conv3')(x) x = Conv2D(filters=256, kernel_size=(1, 1), strides=1, padding='same', name='pw_conv3')(x) x = BatchNormalization(name='bn6')(x) x = Activation('relu', name='relu6')(x) x = MaxPooling2D(pool_size=(2, 2), strides=2, padding='same', name='pool3')(x) # Output layer, output multi-level feature map outputs = x # Build the model model = Model(inputs, outputs, name='feature_extractor') return model # Define input image size H, W, C input_shape = (256, 256, 3) # Build feature extraction model model = build_feature_extractor(input_shape) # Print model structure model.summary() Furthermore, the multi-scale feature fusion network is specifically used for: Obtaining a multi-level feature map provided by the feature extraction part; Introduce skip connections between feature maps of different resolutions; Apply attention mechanism to feature maps of different resolutions and calculate the importance weight of each feature map; The feature maps of different resolutions and their corresponding weights are weighted fused to output the fused feature map.

[0067] The multi-scale feature fusion network specifically implements the following steps. First, it obtains multi-level feature maps from the feature extraction part. These feature maps vary in resolution and number of channels, covering various image features from low-level to high-level. The acquisition of multi-level feature maps is based on the design of the convolutional neural network in the feature extraction part. The network generates multiple feature maps through multi-layer convolution, pooling, and batch normalization operations, and each feature map contains information at different levels.

[0068] Next, skip connections are introduced between feature maps of different resolutions. This skip connection is a common method in deep learning networks to directly connect non-adjacent layers. Specifically, the skip connection can combine low-resolution high-level features with high-resolution low-level features, thereby enhancing the richness and diversity of feature representation. By introducing skip connections, the model can retain more spatial information and details, thereby improving the effect of feature fusion and the performance of the overall network.

[0069] In order to further improve the effect of feature fusion, the multi-scale feature fusion network also applies the attention mechanism to feature maps of different resolutions. The attention mechanism is a dynamic weighting method used to calculate the importance weight of each feature map. In the specific implementation process, the attention mechanism will weight each feature map, and the size of the weight reflects the importance of the feature map in the current task. In this way, the model can automatically adjust the contribution of different feature maps, so that important features get higher weights during the fusion process, while unimportant features are weakened.

[0070] After calculating the importance weight of each feature map, the multi-scale feature fusion network performs weighted fusion on the feature maps of different resolutions and their corresponding weights. The weighted fusion process is to weight each feature map according to the calculated weights to generate a fused feature map. This fused feature map combines information of different resolutions and levels, has stronger representation ability and higher recognition, and provides rich feature data for subsequent target detection and recognition tasks.

[0071] Through the above steps, the multi-scale feature fusion network realizes the effective fusion of the multi-level feature graphs generated by the feature extraction part. The introduction of jump connection and attention mechanism not only enhances the diversity and richness of feature representation, but also improves the model's ability to capture important features, thereby improving the overall performance of the high-efficiency drone image recognition system. This design ensures that the system can fully utilize the feature information at all levels when processing complex image tasks to achieve higher recognition accuracy and efficiency.

[0072] The following is an implementation code to describe the multi-scale feature fusion part of the UAV image high-efficiency recognition system.

[0073] import tensorflow as tf from tensorflow.keras.layers import Input, Conv2D, DepthwiseConv2D,MaxPooling2D, BatchNormalization, Activation, Add, GlobalAveragePooling2D,Multiply, Reshape, Lambda from tensorflow.keras.models import Model def build_multi_scale_feature_fusion(input_shapes): # Create an input layer to receive feature maps of different resolutions inputs = [Input(shape=shape) for shape in input_shapes] # Define skip connection def skip_connection(input1, input2, filters): # Perform a 1x1 convolution on the input to match the number of channels conv1 = Conv2D(filters, kernel_size=(1, 1), strides=1,padding='same')(input1) conv2 = Conv2D(filters, kernel_size=(1, 1), strides=1,padding='same')(input2) # Add the two inputs return Add()([conv1, conv2]) # Define the attention mechanism def attention_mechanism(input_feature): # Use the global average pooling layer to aggregate the global spatial information of the feature map gap = GlobalAveragePooling2D()(input_feature) # Reshape the pooling result for point-wise convolution gap = Reshape((1, 1, gap.shape[-1]))(gap) # Generate attention weights using two point-wise convolutional layers dense1 = Conv2D(gap.shape[-1] / / 4, kernel_size=(1, 1), strides=1, padding='same', activation='relu')(gap) dense2 = Conv2D(gap.shape[-1], kernel_size=(1, 1), strides=1,padding='same', activation='sigmoid')(dense1) # Weight the input feature map return Multiply()([input_feature, dense2]) # Apply attention mechanism to each input feature map attention_outputs = [attention_mechanism(input_feature) forinput_feature in inputs] # Perform skip connections and fusion operations on feature maps of different resolutions fused_features = attention_outputs[0] for feature in attention_outputs[1:]: fused_features = skip_connection(fused_features, feature,filters=fused_features.shape[-1]) # Final fusion feature map output outputs = fused_features # Build the model model = Model(inputs, outputs, name='multi_scale_feature_fusion') return model Furthermore, the region proposal network is specifically used for: Generate candidate regions on the fused feature map using a sliding window method; Perform bounding box regression on each candidate region and use an adaptive learning rate adjustment mechanism to dynamically adjust the learning rate according to the regression error to improve the bounding box positioning accuracy; Calculate the confidence score for each candidate region to determine whether it contains the target object.

[0074] The Region Proposal Network (RPN) is specifically used to generate candidate regions and evaluate them in order to determine the location and existence probability of the target object on the fused feature map. The specific implementation includes the following steps: First, the region proposal network uses a sliding window approach to generate candidate regions on the fused feature map. This means sliding a fixed-size window pixel by pixel on the feature map to extract a series of overlapping local regions. These windows can have different scales and aspect ratios to better accommodate the diversity of objects in the image. The sliding window approach ensures that every potential target location is taken into account, resulting in a large number of candidate regions that cover every part of the entire feature map.

[0075] After generating candidate regions, the region proposal network performs bounding box regression on each candidate region. The purpose of bounding box regression is to accurately locate the position of the target object by adjusting the boundaries of the candidate regions. In this process, RPN extracts features from each candidate region through a small convolutional neural network and predicts a more accurate bounding box. In particular, an adaptive learning rate adjustment mechanism is adopted to dynamically adjust the learning rate according to the regression error. This mechanism continuously adjusts the learning rate during training to converge to the optimal solution faster and improve the accuracy of bounding box positioning. Specifically, when the regression error is large, the learning rate will automatically increase to speed up learning; when the regression error is small, the learning rate will decrease to avoid over-adjustment. Through this adaptive mechanism, bounding box regression can locate the target object more accurately.

[0076] Next, in order to determine whether each candidate region contains the target object, the region proposal network calculates a confidence score for each candidate region. This process involves a classification task, that is, classifying the features of each candidate region to determine whether it contains the target object. The confidence score is a value between 0 and 1, indicating the probability that the candidate region contains the target object. The network processes the features of each candidate region through a classifier and outputs a confidence score. If the confidence score of a candidate region is higher than a preset threshold, it is considered that the region may contain the target object; otherwise, it is considered that it does not contain the target object.

[0077] The region proposal network generates candidate regions through a sliding window, performs bounding box regression on these regions to improve positioning accuracy, and calculates confidence scores to determine whether they contain target objects. Through these steps, RPN provides high-quality candidate regions for subsequent target detection and recognition, enabling the entire drone image high-efficiency recognition system to more accurately and efficiently identify and track target objects.

[0078] The code does not provide an adaptive learning rate adjustment mechanism. The adaptive learning rate adjustment mechanism is usually implemented during model training. Common methods include using adaptive optimization algorithms (such as Adam, RMSprop) or using learning rate schedulers (such as ReduceLROnPlateau, LearningRateScheduler) to dynamically adjust the learning rate.

[0079] The following is the reference implementation code of the Region Proposal Network (RPN) in the object detection part, using TensorFlow and Keras libraries.

[0080] import tensorflow as tf from tensorflow.keras.layers import Conv2D, Input, Reshape from tensorflow.keras.models import Model from tensorflow.keras.optimizers import Adam from tensorflow.keras.callbacks import ReduceLROnPlateau def build_rpn_model(base_layers, num_anchors): """ Building a Region Proposal Network (RPN) model :param base_layers: The output of the feature extraction network, i.e. the fused feature map :param num_anchors: The number of anchors per sliding window (combinations of different scales and aspect ratios) :return: RPN model """ # Convolutional layer, used to generate candidate regions rpn_conv = Conv2D(512, (3, 3), padding='same', activation='relu',kernel_initializer='normal', name='rpn_conv')(base_layers) # Bounding box regression layer, outputs the bounding box adjustment value (dx, dy, dw, dh) of each candidate region rpn_cls_score = Conv2D(num_anchors 2, (1, 1), activation='linear', kernel_initializer='uniform', name='rpn_cls_score')(rpn_conv) # Confidence score layer, output the probability of target existence in each candidate area rpn_bbox_pred = Conv2D(num_anchors 4, (1, 1), activation='linear', kernel_initializer='zero', name='rpn_bbox_pred')(rpn_conv) # Reshape the classification results into the form of (num_anchors, 2) rpn_cls_score_reshape = Reshape((-1, 2))(rpn_cls_score) # Reshape the bounding box prediction result into the form of (num_anchors, 4) rpn_bbox_pred_reshape = Reshape((-1, 4))(rpn_bbox_pred) # Softmax layer, normalize the confidence scores rpn_cls_prob = tf.keras.layers.Activation('softmax')(rpn_cls_score_reshape) # Build RPN model model = Model(base_layers, [rpn_cls_prob, rpn_bbox_pred_reshape]) return model # Define the size of the input feature map (for example: 32x32x256) input_shape = (32, 32, 256) num_anchors = 9 # Assume that each sliding window has 9 anchor points # Build the input layer inputs = Input(shape=input_shape) # Build RPN model rpn_model = build_rpn_model(inputs, num_anchors) # Compile the model, using the Adam optimizer and cross entropy loss function rpn_model.compile(optimizer=Adam(lr=1e-4), loss=['categorical_crossentropy', 'mean_squared_error']) # Set up an adaptive learning rate adjustment mechanism, using ReduceLROnPlateau reduce_lr = ReduceLROnPlateau(monitor='loss', factor=0.1, patience=10, min_lr=1e-6) # Assume that there are X_train and Y_train as training data # X_train, Y_train_cls, Y_train_bbox are feature maps, classification labels and bounding box regression labels respectively # This is just an example, no actual data is provided X_train = ... Y_train_cls = ... Y_train_bbox = ... # Train the model rpn_model.fit(X_train, [Y_train_cls, Y_train_bbox], epochs=50, batch_size=32, callbacks=[reduce_lr]) Furthermore, the post-processing part is specifically used for: Perform non-maximum suppression on candidate regions to remove candidate regions with high overlap and low confidence, and adopt a dynamic threshold adjustment mechanism to dynamically adjust the suppression threshold according to the confidence score; According to the results of non-maximum suppression processing, the final position and category information of the target object are output.

[0081] The post-processing part is mainly used to perform non-maximum suppression on the candidate area and output the final position and category information of the target object according to the processing results. The specific implementation includes the following steps.

[0082] First, the post-processing part performs non-maximum suppression on the candidate regions. Non-maximum suppression (NMS) is a commonly used technique that aims to remove candidate regions with high overlap and low confidence. The specific operation process is that among all candidate regions, the region with the highest confidence is first selected as the benchmark, and then the overlap between this region and all other candidate regions is calculated. For those regions whose overlap exceeds the preset threshold, if their confidence is low, they are removed. This process is repeated until all regions with high overlap and low confidence are removed, thereby retaining the most representative candidate regions.

[0083] In order to further optimize the effect of non-maximum suppression, a dynamic threshold adjustment mechanism is adopted in the post-processing part. This mechanism dynamically adjusts the suppression threshold according to the confidence score of the candidate region. Specifically, when the confidence of a candidate region is high, the system will reduce the overlap threshold accordingly to ensure that the region can be retained; conversely, when the confidence of a candidate region is low, the system will increase the overlap threshold to increase the possibility of removal. Through this dynamic adjustment, the system can more effectively retain high-confidence candidate regions, while removing low-confidence and high-overlap regions, further improving the accuracy and stability of target detection.

[0084] After completing the non-maximum suppression process, the post-processing part will output the final position and category information of the target object based on the processing results. This step is mainly to further analyze and process the remaining candidate areas after the non-maximum suppression process to determine the specific location and category of the target object in each area. During the specific operation, the system will combine the bounding box and confidence information provided by the previous target detection part to accurately locate each candidate area, and determine the category of the target object based on the output of the classifier. Finally, the system will generate a detailed target detection report, containing the location information (such as bounding box coordinates) and category information (such as object category label) of each target object.

[0085] Through these steps, the post-processing part can not only effectively remove extra and redundant candidate areas, but also accurately determine the location and category of the target object, greatly improving the overall performance and practicality of the UAV image high-efficiency recognition system.

[0086] The following is a reference implementation of the post-processing part of the deep learning model, using TensorFlow and Keras libraries.

[0087] import tensorflow as tf import numpy as np def non_max_suppression(boxes, scores, max_output_size, iou_threshold): """ Non-maximum suppression (NMS) processing :param boxes: bounding box coordinates [num_boxes, 4] :param scores: confidence scores [num_boxes] :param max_output_size: The maximum number of candidate regions to retain :param iou_threshold: overlap threshold (IOU threshold) :return: index after NMS [max_output_size] """ return tf.image.non_max_suppression(boxes, scores, max_output_size, iou_threshold) def dynamic_threshold_nms(boxes, scores, base_iou_threshold=0.5): """ Dynamic threshold non-maximum suppression processing, dynamically adjust the suppression threshold according to the confidence level :param boxes: bounding box coordinates [num_boxes, 4] :param scores: confidence scores [num_boxes] :param base_iou_threshold: basic overlap threshold (IOU threshold) :return: Bounding box and confidence score after NMS """ sorted_indices = np.argsort(-scores) boxes = boxes[sorted_indices] scores = scores[sorted_indices] keep_boxes = [ ] keep_scores = [ ] while len(scores) > 0: current_box = boxes[0] current_score = scores[0] keep_boxes.append(current_box) keep_scores.append(current_score) if len(scores) == 1: break boxes = boxes[1:] scores = scores[1:] iou_threshold = base_iou_threshold (1 - current_score) ious = compute_iou(current_box, boxes) valid_indices = np.where(ious <= iou_threshold)[0] boxes = boxes[valid_indices] scores = scores[valid_indices] return np.array(keep_boxes), np.array(keep_scores) def compute_iou(box, boxes): """ Calculate the overlap (IOU) between a bounding box and multiple bounding boxes :param box: bounding box coordinates [4] :param boxes: multiple bounding box coordinates [num_boxes, 4] :return: overlap [num_boxes] """ y1 = np.maximum(box[0], boxes[:, 0]) x1 = np.maximum(box[1], boxes[:, 1]) y2 = np.minimum(box[2], boxes[:, 2]) x2 = np.minimum(box[3], boxes[:, 3]) inter_area = np.maximum(0, y2 - y1) np.maximum(0, x2 - x1) box_area = (box[2] - box[0]) (box[3] - box[1]) boxes_area = (boxes[:, 2] - boxes[:, 0]) (boxes[:, 3] - boxes[:, 1]) iou = inter_area / (box_area + boxes_area - inter_area) return def post_process(rpn_boxes, rpn_scores, class_predictions, max_output_size=100, base_iou_threshold=0.5): """ In the post-processing part, the candidate regions and confidence scores provided by RPN are processed to output the final target location and category information. :param rpn_boxes: candidate region bounding box generated by RPN [num_boxes, 4] :param rpn_scores: confidence scores of candidate regions generated by RPN [num_boxes] :param class_predictions: category information output by the classifier [num_boxes, num_classes] :param max_output_size: The maximum number of candidate regions to retain :param base_iou_threshold: basic overlap threshold (IOU threshold) :return: The final target object position and category information """ # Perform dynamic threshold non-maximum suppression on the candidate area nms_boxes, nms_scores = dynamic_threshold_nms(rpn_boxes, rpn_scores, base_iou_threshold) # Get the category information of each candidate region nms_class_predictions = class_predictions[:len(nms_boxes)] final_classes = np.argmax(nms_class_predictions, axis=1) # Output the final target object position and category information final_boxes = nms_boxes final_scores = nms_scores final_class_labels = final_classes return final_boxes, final_scores, final_class_labels # Example data (assumed to be obtained from RPN and classifier) rpn_boxes = np.array([[50, 50, 150, 150], [55, 60, 155, 160], [100,100, 200, 200]]) rpn_scores = np.array([0.9, 0.75, 0.6]) class_predictions = np.array([[0.1, 0.9], [0.2, 0.8], [0.8, 0.2]]) # Post-processing, get the final target location and category information final_boxes, final_scores, final_class_labels = post_process(rpn_boxes, rpn_scores, class_predictions) print("Final target location:", final_boxes) print("Final confidence score:", final_scores) print("Final category labels:", final_class_labels) In order to train the deep learning model in the UAV image efficient recognition system, specific steps need to be followed. The model includes feature extraction part, feature fusion part, target detection part and post-processing part. The following are the detailed training steps.

[0088] First, prepare the dataset. The dataset should contain drone images in various scenes, with bounding boxes and category information of the target objects annotated in the images. The dataset should be divided into training set and validation set to evaluate the performance of the model.

[0089] Next, we build the feature extraction part of the deep learning model. The feature extraction part uses an improved convolutional neural network, including multiple convolutional layers, pooling layers, and batch normalization layers, and introduces separable convolutional layers. This part can be implemented using deep learning frameworks such as TensorFlow or PyTorch. The purpose of the feature extraction part is to extract features from the input preprocessed image data and generate feature maps.

[0090] Then, the feature fusion part is constructed. The feature fusion part is implemented using a multi-scale feature fusion network, which fuses feature maps of different resolutions from the feature extraction part. By introducing skip connections and attention mechanisms, feature maps of different resolutions can be better fused to generate a fused feature map containing multi-level information.

[0091] On this basis, the object detection part is constructed. The object detection part uses the region proposal network (RPN) to process the fused feature map and generate candidate regions and corresponding confidence scores. RPN generates candidate regions on the feature map through the sliding window method, and performs bounding box regression and confidence score calculation for each candidate region. In order to improve the accuracy of bounding box positioning, an adaptive learning rate adjustment mechanism is introduced to dynamically adjust the learning rate according to the regression error.

[0092] Finally, the post-processing part is constructed. The post-processing part processes the candidate regions and confidence scores provided by the object detection part. First, non-maximum suppression is performed to remove candidate regions with high overlap and low confidence, and a dynamic threshold adjustment mechanism is adopted. According to the results of non-maximum suppression, the final position and category information of the target object are output.

[0093] The training process is as follows: 1. Data preprocessing: Standardize the image data, scale the images to a uniform size, and perform data augmentation such as random cropping, rotation, and flipping.

[0094] 2. Initialize model parameters: Use pre-trained models (such as ResNet, VGG, etc.) to initialize parameters to speed up training convergence.

[0095] 3. Define the loss function: including classification loss and regression loss. Classification loss is used to evaluate the confidence score of the object detection part, and regression loss is used to evaluate the accuracy of bounding box regression.

[0096] 4. Select optimizer: Use Adam or SGD optimizer and set the initial learning rate. Introduce adaptive learning rate adjustment mechanism to dynamically adjust the learning rate according to the loss changes during training.

[0097] 5. Iterative training: In each training iteration, perform the following steps: Forward propagation: The input image data passes through the feature extraction part, feature fusion part and target detection part to generate candidate regions and confidence scores.

[0098] Calculate loss: Calculate classification loss and regression loss based on the predicted candidate regions and confidence scores.

[0099] Back propagation: Update the gradient of model parameters according to the loss and optimize the model.

[0100] 6. Validate the model: After each training iteration, use the validation set to evaluate the performance of the model, including the accuracy of object detection and the precision of bounding box positioning. Adjust the model parameters and training strategy based on the validation results.

[0101] 7. Save the model: Save the model parameters regularly during the training process for subsequent use and further optimization.

[0102] The target tracking unit 105 is used to execute an adaptive target tracking algorithm to lock and track the target object identified by the target detection unit, wherein the adaptive target tracking algorithm is implemented based on the fusion of Kalman filter and optical flow method, and is used to provide stable target tracking under high-speed flight conditions.

[0103] The target tracking unit 105 plays a key role in the high-efficiency recognition system of drone images. Its main function is to lock and continuously track the target object identified by the target detection unit 104. The unit relies on an adaptive target tracking algorithm that combines Kalman filter and optical flow method to ensure stable target tracking when the drone is flying at high speed.

[0104] First, the target tracking unit receives the position and motion information of the target object identified by the target detection unit. As one of the core components, the Kalman filter is mainly used to predict and correct the state of the target object. Through a series of mathematical operations, the Kalman filter uses the state estimate of the previous moment and the current observation data to predict the position and speed of the target object at the next moment. This prediction process includes two steps: state prediction and update. In the state prediction step, the Kalman filter predicts the position and speed of the target object at the next moment based on its motion model. In the update step, the predicted value is corrected using the observation data at the current moment to obtain a more accurate state estimate.

[0105] At the same time, the optical flow method is used to calculate the motion vector of the target object on the image plane. The optical flow method determines the direction and speed of the target object's movement by comparing the pixel changes in the image sequence. This method is particularly suitable for processing continuous image data and can provide high-precision motion detection results. In specific implementation, the optical flow method calculates the displacement vector of the target object by comparing the pixel intensity changes in adjacent frames, thereby providing real-time motion information.

[0106] To enhance tracking stability and accuracy, the target tracking unit combines the prediction results of the Kalman filter with the motion vectors of the optical flow method. This fusion method uses the state estimation of the Kalman filter to provide macro predictions, while using the fine motion detection of the optical flow method to correct the prediction error, ensuring that the tracking algorithm can maintain efficient and stable tracking performance under various complex flight conditions.

[0107] In addition, the target tracking unit also has an adaptive adjustment function, which can dynamically adjust the weights of the Kalman filter and the optical flow method according to the actual situation. For example, when the lighting conditions are good and the image quality is high, the system can increase its reliance on the results of the optical flow method; when the image noise is large or the lighting conditions are poor, the system can rely more on the prediction results of the Kalman filter. In this way, the target tracking unit can be flexibly adjusted in various environments to ensure the best tracking effect.

[0108] Through the above process, the target tracking unit 105 can continuously and stably track the target object, regardless of whether the drone is flying at high speed or in complex environmental conditions. This design not only improves the overall performance of the drone system, but also significantly enhances its reliability and practicality in practical applications.

[0109] Furthermore, the adaptive target tracking algorithm in the target tracking unit includes the following steps: Receive the initial position information of the target object from the target detection unit ,in and represents the initial position of the target object, and Indicates the initial width and initial height of the target object; In each frame of the image, the motion vector of the target object on the image plane is calculated using the optical flow method based on the image data obtained by the image acquisition unit. ,in and Indicates the displacement information of the target object between the current frame and the previous frame; Using the initial position information and motion vector of the target object, an adaptive Kalman filter is used to predict and update the position information of the target object, wherein the state vector of the adaptive Kalman filter is and the observation vector Calculate according to the following formula 1 and formula 2 respectively: ; ; in, Indicates at time The state vector includes the position, velocity and acceleration information of the target object; is the state transfer matrix, which is used to describe the motion model of the target object; is the control matrix, which is used to describe the external control input Effects on the motion of the target object; is process noise, which follows a Gaussian distribution with a mean of zero; is the observation vector, including the displacement information of the target object obtained from the optical flow method; is the observation matrix, which is used to map the state vector to the observation space; is the observation noise, which follows a Gaussian distribution with a mean of zero; is the adaptive process noise adjustment factor, calculated using the following formula 3: ; in, is a predetermined weight coefficient; is the adaptive observation noise adjustment factor, which is calculated by the following formula 4: ; in, is a predetermined weight coefficient; Through the prediction and correction steps of the adaptive Kalman filter, the updated position information of the target object in the current frame is calculated and output ,in and represents the updated position of the target object, and Indicates the updated width and updated height of the target object; According to the state update result of the adaptive Kalman filter, the motion trajectory of the target object is generated, and the target object is locked and continuously tracked in the image.

[0110] First, the target tracking unit receives the initial position information of the target object from the target detection unit. ,in and Indicates the initial horizontal and vertical coordinate positions of the target object, and Represents the initial width and initial height of the target object. This information provides the algorithm with the initial position and size of the target object in the image.

[0111] In each frame of the image, the target tracking unit calculates the motion vector of the target object on the image plane using the optical flow method based on the image data obtained by the image acquisition unit. .in, and Indicates the displacement information of the target object between the current frame and the previous frame. Optical flow is a commonly used computer vision technology that calculates the motion vector of the target object by analyzing the movement of pixels in an image sequence.

[0112] Next, using the initial position information and motion vector of the target object, an adaptive Kalman filter is used to predict and update the position information of the target object. The Kalman filter is a recursive algorithm that provides an optimal estimate of the system state by combining predictions and observations. The adaptive Kalman filter performs predictions and updates through the following steps: 1. State vector and the observation vector Calculate according to formula (1) and formula (2) respectively: ; ; in, Represents the state vector at time k, including the position, velocity, and acceleration information of the target object. is the state transfer matrix, which is used to describe the motion model of the target object. is the control matrix, which is used to describe the external control input The impact on the motion of the target object, is process noise and has a Gaussian distribution with a mean of zero.

[0113] is the observation vector, including the displacement information of the target object obtained from the optical flow method. is the observation matrix, which is used to map the state vector to the observation space, is the observation noise, which follows a Gaussian distribution with a mean of zero.

[0114] 2. Adaptive process noise adjustment factor Calculated by formula (3): ; Among them, γ is the predetermined weight coefficient. This formula is based on the state vector at the previous moment and the state vector at the previous two moments The difference is used to dynamically adjust the size of the process noise.

[0115] 3. Adaptive Observation Noise Adjustment Factor Calculated by formula (4): ; in, is a predetermined weight coefficient. This formula is based on the observation vector at the previous moment and the state vector at the previous moment The difference in mapping dynamically adjusts the size of the observation noise.

[0116] Through the prediction and correction steps of the adaptive Kalman filter, the target tracking unit calculates and outputs the updated position information of the target object in the current frame .in and represents the updated position of the target object, and Indicates the updated width and updated height of the target object.

[0117] Based on the state update results of the adaptive Kalman filter, the target tracking unit generates the motion trajectory of the target object, locks and continuously tracks the target object in the image. This process ensures stable tracking of the target object when the drone is flying at high speed, improving the overall performance and reliability of the system.

[0118] The control unit 106 adjusts the flight trajectory of the UAV according to the position and motion information of the target object provided by the target tracking unit to maintain continuous tracking of the target object.

[0119] The control unit 106 plays a vital role in the high-efficiency UAV image recognition system. Its main function is to dynamically adjust the flight trajectory of the UAV according to the target object position and motion information provided by the target tracking unit 105 to ensure continuous tracking of the target object. The unit combines real-time data processing and intelligent control algorithms to achieve accurate trajectory adjustment and target tracking of the UAV under high-speed flight conditions.

[0120] First, the control unit receives real-time data from the target tracking unit, which includes detailed information such as the current position, direction of movement, and speed of the target object. In order to ensure instant adjustment of the flight trajectory, the control unit uses a high-speed processor and efficient algorithms to quickly process and analyze the received data. These algorithms include prediction algorithms and optimization algorithms, which can calculate the best flight path for the drone in advance based on the movement trend of the target object.

[0121] The control unit uses prediction algorithms to estimate the future position of the target object. These algorithms are usually based on Kalman filtering or particle filtering technology, combining the historical motion data and current motion state of the target object to predict its position in the future. This prediction function enables the control unit to make flight trajectory adjustments in advance, improving the drone's response speed and tracking accuracy.

[0122] After determining the predicted position of the target object, the control unit applies an optimization algorithm to calculate the best flight trajectory for the drone. The optimization algorithm takes into account the drone's flight performance, speed, acceleration, and environmental factors to ensure that the calculated trajectory not only accurately tracks the target object, but also maximizes energy and time savings. Typical optimization algorithms include Algorithms, dynamic programming algorithms and genetic algorithms, etc., which can efficiently find the optimal path in complex environments.

[0123] After calculating the optimal flight trajectory, the control unit adjusts the attitude and motion parameters of the drone through the flight control system. The flight control system includes multiple actuators, such as motors, servos, and thrusters, which receive instructions from the control unit to adjust the heading, speed, and altitude of the drone. Specifically, the control unit adjusts the power of the motors and the angle of the servos to make the drone fly along the calculated optimal trajectory. In order to ensure the smoothness and accuracy of the adjustment process, the flight control system is also equipped with sensors such as inertial measurement units (IMUs) and GPS, which monitor the attitude and position of the drone in real time and feed back to the control unit for closed-loop control.

[0124] In addition, the control unit also has an adaptive adjustment function, which can dynamically adjust the control parameters according to the real-time flight status and environmental changes. For example, when the drone encounters sudden strong winds or other obstacles, the control unit can immediately recalculate the flight trajectory and adjust the drone's flight attitude to avoid collisions and other unexpected situations. This adaptive capability enables the drone to maintain efficient target tracking capabilities in a variety of complex and dynamic environments.

[0125] Through the above design and implementation, the control unit 106 ensures that the UAV can continuously and stably track the target object during high-speed flight, thereby improving the overall performance and reliability of the UAV system.

[0126] Furthermore, the control unit is specifically used for: Receiving the target object position and motion information provided by the target tracking unit; Based on the received target object position and motion information, the optimal flight path of the drone is calculated in real time. Algorithms to ensure optimality and safety of paths; Based on the calculated optimal flight path, corresponding attitude adjustment instructions are generated to control the heading, speed and altitude of the UAV.

[0127] The control unit is specifically used for the following tasks. First, the control unit receives the target object position and motion information provided by the target tracking unit. The target tracking unit continuously monitors the dynamics of the target object and updates its position and motion data in real time. After the control unit obtains this data, it provides basic information for subsequent path calculation and flight control.

[0128] Next, the control unit calculates the optimal flight path of the drone in real time based on the received target object position and motion information. To ensure the optimality and safety of the path, the control unit uses Algorithm for path planning. The algorithm is a graph search algorithm that is widely used in path planning and navigation problems. It finds the path with the lowest cost by moving between nodes and evaluating the cost of each node (including the actual cost from the starting point to the current node and the estimated cost from the current node to the target node). Specifically, the control unit first uses the current position of the drone and the position of the target object as the starting point and the end point to build a graph containing all possible flight paths. During the search process, the control unit evaluates the cost of each path, selects the path with the lowest cost and avoids obstacles, and ensures that the drone approaches the target object in the best way.

[0129] After calculating the optimal flight path, the control unit generates corresponding attitude adjustment instructions based on the path. These instructions are used to control the heading, speed and altitude of the drone. In the specific implementation process, the control unit will decompose the optimal path into a series of detailed flight steps and calculate the required attitude adjustment according to each step. For example, in order to adjust the heading, the control unit will generate corresponding steering instructions; in order to adjust the speed, the control unit will generate acceleration or deceleration instructions; in order to adjust the altitude, the control unit will generate ascent or descent instructions. All these instructions will be transmitted to the drone's flight control system to adjust the drone's flight attitude in real time.

[0130] Through these steps, the control unit ensures that the drone can approach and continuously track the target object in the optimal path. Receiving the target object's position and motion information, calculating the optimal flight path, and generating attitude adjustment instructions are all linked together, allowing the drone to perform tasks stably and efficiently in complex and dynamic environments.

[0131] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. A high-efficiency drone image recognition system, characterized in that: include: UAV, for high-speed and stable flight; An image acquisition unit, including a high-resolution camera placed on the drone to capture image data in real time during flight; An image preprocessing unit, used for performing preprocessing operations including denoising and deblurring on the image data captured by the image acquisition unit to obtain preprocessed image data; A target detection unit, comprising a pre-trained deep learning model, for identifying a target object in the pre-processed image provided by the image pre-processing unit, wherein the deep learning model is optimized based on a convolutional neural network structure to improve recognition speed and accuracy; A target tracking unit, used for executing an adaptive target tracking algorithm to lock and track the target object identified by the target detection unit, wherein the adaptive target tracking algorithm is implemented based on the fusion of Kalman filter and optical flow method, and is used for providing stable target tracking under high-speed flight conditions; The control unit adjusts the flight trajectory of the UAV according to the position and motion information of the target object provided by the target tracking unit to maintain continuous tracking of the target object.

2. The high-efficiency drone image recognition system according to claim 1 is characterized in that: The deep learning model includes a feature extraction part, a feature fusion part, a target detection part and a post-processing part; The feature extraction part is used to extract features from the preprocessed image data to obtain a feature map; wherein the feature extraction part is implemented by an improved convolutional neural network, the improved convolutional neural network includes multiple convolutional layers, pooling layers and batch normalization layers, and introduces a separable convolutional layer; The feature fusion part is used to fuse the feature map provided by the feature extraction part to obtain a fused feature map; wherein the feature fusion part is implemented by a multi-scale feature fusion network, and the multi-scale feature fusion network adopts a fusion mechanism of feature maps with different resolutions; The object detection part is used to process the fused feature map provided by the feature fusion part to obtain candidate regions and corresponding confidence scores; the object detection part is implemented by a region proposal network, and an adaptive learning rate adjustment mechanism is introduced in the bounding box regression process to improve the accuracy of bounding box positioning; The post-processing part is used to process the candidate areas and corresponding confidence scores provided by the target detection part to obtain the position and category information of the target object.

3. The high-efficiency drone image recognition system according to claim 2 is characterized in that: The improved convolutional neural network specifically includes: a) Input layer, used to receive preprocessed image data, size is ; b) The first convolutional layer uses a convolution kernel size of , a convolution operation with a stride of 1, the output size is ; c) The first batch normalization layer is used to batch normalize the output of the first convolutional layer to keep the output size ; d) The first separable convolution layer includes depthwise convolution and pointwise convolution. The depthwise convolution uses a convolution kernel size of , stride is 1, point-by-point convolution uses Convolution, the output size is ; e) The first maximum pooling layer uses a pooling size of , a maximum pooling operation with a stride of 2, and an output size of ; f) Repeat the structures from b) to e), gradually increase the number of channels and reduce the size of feature maps, to form a multi-level feature map, the output size of each layer gradually decreases, and the number of feature maps gradually increases; g) Output layer, which outputs multi-level feature maps, where the feature maps of each layer are different in resolution and number of channels.

4. The high-efficiency drone image recognition system according to claim 2, characterized in that: The multi-scale feature fusion network is specifically used for: Obtaining a multi-level feature map provided by the feature extraction part; Introduce skip connections between feature maps of different resolutions; Apply attention mechanism to feature maps of different resolutions and calculate the importance weight of each feature map; The feature maps of different resolutions and their corresponding weights are weighted fused to output the fused feature map.

5. The high-efficiency drone image recognition system according to claim 2, characterized in that: The region proposal network is specifically used for: Generate candidate regions on the fused feature map using a sliding window method; Perform bounding box regression on each candidate region and use an adaptive learning rate adjustment mechanism to dynamically adjust the learning rate according to the regression error to improve the bounding box positioning accuracy; Calculate the confidence score for each candidate region to determine whether it contains the target object.

6. The high-efficiency drone image recognition system according to claim 2, characterized in that: The post-processing part is specifically used for: Perform non-maximum suppression on candidate regions to remove candidate regions with high overlap and low confidence, and adopt a dynamic threshold adjustment mechanism to dynamically adjust the suppression threshold according to the confidence score; According to the results of non-maximum suppression processing, the final position and category information of the target object are output.

7. The high-efficiency drone image recognition system according to claim 1, characterized in that: The adaptive target tracking algorithm in the target tracking unit comprises the following steps: Receive the initial position information of the target object from the target detection unit ,in and represents the initial position of the target object, and Indicates the initial width and initial height of the target object; In each frame of the image, the motion vector of the target object on the image plane is calculated using the optical flow method based on the image data obtained by the image acquisition unit. ,in and Indicates the displacement information of the target object between the current frame and the previous frame; Using the initial position information and motion vector of the target object, an adaptive Kalman filter is used to predict and update the position information of the target object, wherein the state vector of the adaptive Kalman filter is and the observation vector Calculate according to the following formula 1 and formula 2 respectively: ； ； in, Indicates at time The state vector includes the position, velocity and acceleration information of the target object; is the state transfer matrix, which is used to describe the motion model of the target object; is the control matrix, which is used to describe the external control input Effects on the motion of the target object; is process noise, which follows a Gaussian distribution with a mean of zero; is the observation vector, including the displacement information of the target object obtained from the optical flow method; is the observation matrix, which is used to map the state vector to the observation space; is the observation noise, which follows a Gaussian distribution with a mean of zero; is the adaptive process noise adjustment factor, calculated using the following formula 3: ； in, is a predetermined weight coefficient; is the adaptive observation noise adjustment factor, which is calculated by the following formula 4: ； in, is a predetermined weight coefficient; Through the prediction and correction steps of the adaptive Kalman filter, the updated position information of the target object in the current frame is calculated and output ,in and represents the updated position of the target object, and Indicates the updated width and updated height of the target object; According to the state update result of the adaptive Kalman filter, the motion trajectory of the target object is generated, and the target object is locked and continuously tracked in the image.

8. The high-efficiency drone image recognition system according to claim 1, characterized in that: The image acquisition unit uses a high-resolution camera to capture continuous image frames in high-speed flight. The resolution of the high-resolution camera is not less than 4K, and the frame rate is not less than 30 frames per second.

9. The high-efficiency drone image recognition system according to claim 1, characterized in that: The image preprocessing unit is specifically used for: Perform noise suppression on image data and use adaptive median filtering algorithm to remove random noise in the image; Using motion estimation algorithms, image data is deblurred to restore motion blur caused by high-speed flight; The histogram equalization technique is applied to enhance the contrast and details of the image to improve the recognition accuracy of the subsequent target detection unit.

10. The high-efficiency drone image recognition system according to claim 1, characterized in that: The control unit is specifically used for: Receiving the target object position and motion information provided by the target tracking unit; Based on the received target object position and motion information, the optimal flight path of the drone is calculated in real time. Algorithms to ensure optimality and safety of paths; Based on the calculated optimal flight path, corresponding attitude adjustment instructions are generated to control the heading, speed and altitude of the UAV.

Images