An unmanned aerial vehicle part automatic identification method based on corner detection

Abstract

The application relates to the technical field of image recognition, and particularly provides an unmanned aerial vehicle part automatic recognition method based on corner point detection, which comprises the following steps: constructing a neural network model comprising a channel attention mechanism, acquiring multiple unmanned aerial vehicle images, and pre-processing and unmanned aerial vehicle part labeling the unmanned aerial vehicle images; acquiring corner point information in the unmanned aerial vehicle images after the unmanned aerial vehicle parts are labeled through a corner point detection method, and generating a corner point feature map; fusing the corner point feature map and the unmanned aerial vehicle images after the unmanned aerial vehicle parts are labeled to obtain a corner point image; and training the neural network model again by using the corner point image, inputting a to-be-detected target image into the trained neural network model, and recognizing unmanned aerial vehicle parts in the to-be-detected target image. The application enhances the robustness of the neural network model to complex scenes and noise interference, and realizes accurate and efficient unmanned aerial vehicle part automatic recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and specifically provides a method for automatically identifying parts of a UAV based on corner point detection. BACKGROUND

[0002] Since the 21st century, under the impetus of the wave of science and technology, unmanned aerial vehicles (UAVs) have been increasingly widely used in the fields of logistics, agriculture, disaster relief, and the like, thanks to their continuous expansion and optimization. The UAVs are small, flexible, highly agile, and low in cost, and can easily shuttle in various complex environments, perform diversified tasks, and be deployed on a large scale.

[0003] In the early days, the detection and identification of UAVs mainly relied on traditional technologies such as radar, infrared sensors, radio frequency detection, and acoustic sensors. Although these methods can achieve remote detection of UAVs to some extent, they often have high false alarm rates, limited detection distances, and weak anti-interference capabilities when facing complex environments or low-altitude flying UAVs. With the rapid development of computer vision and artificial intelligence technologies, image recognition methods based on deep learning have gradually been applied to the field of UAV detection and identification.

[0004] In recent years, computer vision and deep learning technologies have made significant progress in the field of image recognition, and in particular, convolutional neural network models (CNNs) and deep neural network models (DNNs) have shown great capabilities in target detection and classification. Using these deep learning models, many researchers have successfully used artificial intelligence to detect and identify UAVs in real time. Through deep analysis of images, deep learning models can better extract the appearance features of UAVs in images, and then perform UAV identification and classification, and still show good robustness in the face of complex environments.

[0005] Although deep learning technologies have shown great potential in UAV identification, they still face several important challenges, such as the need for large amounts of high-quality labeled data to train deep learning models, and the need for specialized data for UAV identification to include diverse environments involving different flight altitudes, angles, lighting conditions, and backgrounds, making it difficult to obtain such data sets. Data scarcity can lead to model overfitting, resulting in unstable recognition results in real-world scenarios. Moreover, the cost of data collection is high, and many scenarios involved in the data are difficult to simulate or obtain, making model training difficult. In addition, the training process of high-precision deep neural network models requires high-performance computing resources, and even on modern computing platforms, training high-precision models requires several days or even weeks.

[0006] In view of this, it is urgent to develop a high-efficiency, reliable, and practical UAV identification technology system. SUMMARY

[0007] To address the aforementioned problems, this invention provides an automatic identification method for drone parts based on corner detection. This method combines a neural network model with a channel attention mechanism and trains the neural network model using corner images containing corner information, thereby enhancing the neural network model's ability to identify drone parts and improving identification accuracy.

[0008] The automatic identification method for drone parts based on corner detection provided by this invention includes S1: constructing a neural network model with a channel attention mechanism, the neural network being used to automatically identify drone parts; and further includes:

[0009] S2: Acquire multiple drone images and label the drone parts in the drone images;

[0010] Corner detection methods are used to obtain corner information in drone images after marking drone parts, and corner feature maps are generated based on the corner information;

[0011] A corner image is obtained by fusing the corner feature map with the drone image after annotating the drone parts;

[0012] S3: Train a neural network model using corner images;

[0013] S4: Input the target image to be detected into the trained neural network model to identify the drone parts in the target image.

[0014] Preferably, the neural network model is the YOLO model with an SE channel attention mechanism.

[0015] Preferably, the SE channel attention mechanism includes global information compression operation, activation operation, and reweighting operation;

[0016] Global information compression operations include: encoding the RGB channels of the image to be recognized by the neural network model into a format with a scale of [scale value missing]. The global features are encoded using the following formula:

[0017] ,

[0018] in, This indicates the number of compressed images. Global eigenvalues ​​of each channel Represents the global feature vector. This represents the squeeze operation function. Represents the first... One channel, Indicates the height of the input feature map. This indicates the width of the input feature map. an index variable representing traversal in the feature map height direction, an index variable representing traversal in the feature map width direction, representing a real number set;

[0019] The excitation operation includes: for each RGB channel generating a weight of an interval, and the weight generation formula is as follows:

[0020] ,

[0021] wherein, represents a weight, represents an excitation operation function, represents a parameter to be learned in the excitation operation, which is specifically or , represents a Sigmoid activation function, represents a linear activation function, represents a first fully connected layer, , represents a second fully connected layer, ;

[0022] The reweighting operation includes: for each channel assigning different weights:

[0023] ,

[0024] wherein, represents a result of the channel of the input feature map after being weighted by the SE channel attention mechanism, represents a scaling function, represents a weight coefficient.

[0025] Preferably, the obtained multiple UAV images are UAV images with different shooting angles, different lighting conditions, different backgrounds, and different occlusion conditions.

[0026] Preferably, after obtaining the multiple UAV images, before labeling the UAV parts in the UAV images, the method further includes preprocessing the UAV images.

[0027] The preprocessing method is: cropping the UAV images to a predetermined size, performing pixel value normalization processing on the cropped UAV images, and performing image data enhancement on the normalized UAV images.

[0028] Preferably, the corner point information in the UAV image is detected by a SIFT corner point detection method, and the SIFT corner point detection method includes:

[0029] Q1: converting the unmanned aerial vehicle image after marking the unmanned aerial vehicle parts into an unmanned aerial vehicle grayscale image, normalizing the unmanned aerial vehicle grayscale image, and obtaining an unmanned aerial vehicle normalized image;

[0030] Q2: detecting extreme value points in a scale space of the unmanned aerial vehicle normalized image by using a Gaussian difference pyramid;

[0031] Q3: positioning the positions and scales of the extreme value points by using a quadratic function fitting method, removing extreme value points with a contrast lower than a preset threshold and edge response points, and taking the remaining extreme value points as scale stable feature points;

[0032] Q4: counting the direction distribution of gradients in a neighborhood of the scale stable feature points, and assigning a main direction to each scale stable feature point;

[0033] Q5: rotating the coordinate system of the neighborhood of the scale stable feature points to the main direction according to the main direction, and obtaining corner point information.

[0034] Preferably, the expression form of the corner point feature map is a corner point mask image or a heat map.

[0035] Preferably, the pixel value of a non-corner point information region in the corner point mask image is 0, and the pixel value of a corner point information region is 255 or 1.

[0036] Preferably, in S2, the fusion method is: taking the corner point feature map as a fourth channel, channel splicing the fourth channel with the RGB three channels of the unmanned aerial vehicle image after marking the unmanned aerial vehicle parts, and obtaining a corner point image.

[0037] Preferably, it further comprises S5: calculating the position coordinates of the unmanned aerial vehicle parts in the actual space according to the recognized unmanned aerial vehicle parts:

[0038] ,

[0039] ,

[0040] wherein, the unmanned aerial vehicle part actual space center of mass coordinates, the top left corner coordinates of the detection frame, the right bottom corner coordinates of the detection frame.

[0041] Compared with the prior art, the present application can achieve the following beneficial effects:

[0042] In the method, an innovative multi-channel fusion strategy is adopted for processing of the training unmanned aerial vehicle image, that is, a fourth channel formed by fusing an RGB three-channel and corner information forms a corner image. The corner image innovatively introduces a fourth channel constructed by the corner information on the basis of the original RGB three-channel, and the addition of the corner information channel improves the performance of the neural network model, so that the neural network model can focus on the corner area in the unmanned aerial vehicle image with higher accuracy. As a key geometric element in the image, the corner can clearly highlight important structures such as edge turning and connection points, and when facing the recognition and classification tasks, the neural network model can distinguish the unmanned aerial vehicle parts more accurately according to these in-depth understood features, greatly improving the accuracy and reliability of recognizing the unmanned aerial vehicle parts.

[0043] From the perspective of feature representation, the RGB three-channel mainly carries the color and basic texture information of the image, providing an intuitive description of the appearance of the unmanned aerial vehicle for the model, and the fourth channel composed of the corner information supplements the features from the unique perspective of geometric structure, which complement each other. This multi-channel fusion mode enables the neural network model to learn more rich and hierarchical feature representations, so that it can distinguish the unmanned aerial vehicle parts and background objects more easily by means of color and geometric features when processing unmanned aerial vehicle images in complex backgrounds, and significantly improves the detection and recognition ability of the unmanned aerial vehicle parts.

[0044] In terms of neural network model training efficiency, the fourth channel composed of the corner information provides clear guidance information for the neural network model, so that it can quickly locate the key area in the target image to be detected, reducing the calculation and search time of the neural network model in irrelevant areas, thereby accelerating the convergence speed, shortening the overall training time, and improving the training efficiency.

[0045] In addition, image noise has always been a factor affecting the performance of the neural network model. However, the corner detection algorithm itself usually has certain anti-noise ability and can filter out noise interference to a certain extent to extract accurate corner information and provide low-noise feature input for the model, effectively reducing the influence of noise on the recognition result and further improving the reliability of recognition. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 is a flow chart of an unmanned aerial vehicle part automatic recognition method based on corner detection according to an embodiment of the application;

[0047] Figure 2 is a principle diagram of an SE channel attention mechanism according to an embodiment of the application;

[0048] Figure 3 is a neural network model structure diagram according to an embodiment of the application;

[0049] Figure 4 is a flow chart of a SIFT corner detection method according to an embodiment of the present application;

[0050] Figure 5 is an effect diagram of recognizing a target image to be detected by using the neural network model according to an embodiment of the present application;

[0051] Figure 6 is an effect diagram of a neural network model according to an embodiment of the present application;

[0052] Figure 7 is an effect diagram of a neural network model according to an embodiment of the present application;

[0053] Figure 8 is an effect diagram of recognizing a target image to be detected by using the neural network model according to an embodiment of the present application;

[0054] Figure 9 is an effect diagram of recognizing a target image to be detected by using the neural network model according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not constitute a limitation on the present application. In different embodiments, similar elements are associated with similar element labels. In the following embodiments, many details are described in order to make the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different cases, or can be replaced by other elements, materials or methods. In some cases, some operations related to the present application are not shown or described in the specification in order to avoid the core part of the present application being overwhelmed by too much description, and it is not necessary to describe these related operations in detail for those skilled in the art based on the description in the specification and general technical knowledge in the art.

[0056] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other to form various embodiments without conflict. Meanwhile, the steps or actions in the method description can also be sequentially adjusted or adjusted in a manner that is obvious to those skilled in the art. Therefore, the order in the specification and the drawings is only for the purpose of clearly describing a certain embodiment, and does not mean a necessary order, unless otherwise stated that a certain order must be followed.

[0057] In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can be explicitly or implicitly included one or more. In the description of the present application, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0058] In the description of the present application, it should be noted that unless otherwise specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be connected inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0059] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0060] As Figure 1 shown, the embodiment of the present application provides an unmanned aerial vehicle part automatic identification method based on corner point detection, which is as follows:

[0061] S1: Construct a neural network model with channel attention mechanism.

[0062] In order to identify the unmanned aerial vehicle parts contained in the unmanned aerial vehicle image, the embodiment of the present application constructs a neural network model containing channel attention mechanism. By setting channel attention mechanism in the neural network model, the neural network model can pay more attention to channel weight.

[0063] As Figure 2 shown, it is the principle diagram of SE channel attention mechanism. The embodiment of the present application adds SE channel attention mechanism in YOLO model to construct a neural network model, as Figure 3 shown, it is the structure diagram of the neural network model of the embodiment of the present application.

[0064] The SE channel attention mechanism consists of three steps: Squeeze, Excitation, and Reweight, which are respectively global information compression, excitation, and reweighting operations. Details are as follows:

[0065] Since convolution extracts features from fixed regions, the images recognized by the neural network model (including drone images used for training and images of targets to be detected after training) cannot capture information across all channels. Therefore, a global information compression operation is used to compress the spatial features of each channel into global statistics, capturing the overall response between channels. This global information compression operation uses global average pooling to encode the RGB channels of the image recognized by the neural network model into a scale of [missing information]. The global features are defined by the following formula:

[0066] ,

[0067] in, This indicates the number of compressed images. The global eigenvalues ​​of each channel, where each eigenvalue is a scalar. This represents the global feature vector, composed of all global feature values, with dimension 1. , This represents the squeeze operation function. Represents the first... Each channel is essentially a two-dimensional matrix. This represents the height of the input feature map, in pixels. This represents the width of the input feature map, in pixels. This represents the index variable used for traversal along the height direction of the feature map. This represents the index variable used for traversal along the width of the feature map. Represent the set of real numbers, and explain It is a real number variable.

[0068] The core of the excitation operation is to use a fully connected layer (FC) and a sigmoid function to excite each RGB channel. Generate a The weights of the intervals are used to dynamically adjust the importance of the channels. The weight generation formula is as follows:

[0069] ,

[0070] in, This represents the weight, which is the final weight (importance coefficient) calculated for each channel. It is a number between 0 and 1. This represents the activation function. denotes the parameters that need to be learned in the whole excitation operation, here specifically or , denotes the Sigmoid activation function, which is a common activation function, denotes the linear activation function, which is also a commonly used activation function, denotes the first fully connected layer, , the role of the first fully connected layer is to reduce the dimension to reduce the calculation of the fully connected layer, denotes the second fully connected layer, , the role of the second fully connected layer is to restore to the original channel weight calculation.

[0071] The reweighting operation refers to channel weighting, that is, different weights are given to each channel:

[0072] ,

[0073] wherein, denotes the result of the first channel of the input feature map after the SE channel attention mechanism weighting, denotes the scaling function, which is used to refer to the reweighting (scaling) operation itself, denotes the first channel of the input feature map, which is the original feature that has not been weighted by the SE channel attention mechanism, denotes the weight coefficient, which comes from the output of the "excitation operation" step, and is a scalar value (between 0 and 1), representing the importance of the first channel.

[0074] Adding SE channel attention mechanism in neural network model, realizing dynamic adjustment of feature importance of each channel, through the dependence between channels, improving the perception ability of neural network model to key features, while suppressing redundant or noise channels.

[0075] S2: Obtain multiple UAV images, label the UAV parts in the preprocessed UAV images, obtain the corner information in the UAV images after labeling the UAV parts by using a corner detection method, generate a corner feature map according to the corner information, and obtain a corner image by fusing the corner feature map and the UAV image after labeling the UAV parts.

[0076] ​After the neural network model is constructed through S1, the constructed neural network model needs to be trained by using the unmanned aerial vehicle images. A plurality of unmanned aerial vehicle images are acquired for training the neural network model. Since the embodiment of the present application is aimed at detecting the rotor and battery parts of the rotor unmanned aerial vehicle, the plurality of unmanned aerial vehicle images acquired should contain the rotor and battery parts of the rotor unmanned aerial vehicle. At the same time, the plurality of unmanned aerial vehicle images acquired should have diversity, such as different shooting angles, different lighting conditions, different backgrounds, different occlusion conditions, etc. The unmanned aerial vehicle images can be acquired by self-shooting, or can be directly selected from the public unmanned aerial vehicle target detection dataset.

[0077] In order to meet the requirements of the neural network model for the input unmanned aerial vehicle images, before labeling the unmanned aerial vehicle parts in the unmanned aerial vehicle images, the unmanned aerial vehicle images in the dataset need to be preprocessed, and the preprocessing content includes cropping, normalization processing and image enhancement. In order to meet the requirements of the neural network model for the input images, first, the acquired unmanned aerial vehicle images are cropped according to the predetermined size. The embodiment of the present application uniformly crops the size of the unmanned aerial vehicle images to 640x640 pixels, to ensure that the size of the unmanned aerial vehicle images input into the neural network model has consistency; then, the pixel values of the cropped unmanned aerial vehicle images are normalized, for example, the pixel values of the unmanned aerial vehicle images are scaled from 0 to 255 to the range of 0 to 1; next, the number of unmanned aerial vehicle images is expanded using data enhancement technology to improve the generalization ability of the neural network model, and the data enhancement technology includes random cropping, flipping, rotation, adjusting brightness, etc. In order to optimize the performance of the neural network model and speed up the training process, the preprocessed unmanned aerial vehicle images are converted into the format required by the deep learning framework (such as from to , represents the dimension, represents the height of the unmanned aerial vehicle image, represents the width of the unmanned aerial vehicle image), and is organized into batches for neural network model training.

[0078] The unmanned aerial vehicle parts in the preprocessed unmanned aerial vehicle images are labeled. In order to ensure the accuracy of labeling, the embodiment of the present application adopts a manual labeling method to complete the labeling of the unmanned aerial vehicle parts, and the labeled unmanned aerial vehicle parts are rotors and batteries.

[0079] For unmanned aerial vehicle images, the present application provides another embodiment, which classifies the unmanned aerial vehicle images labeled with unmanned aerial vehicle parts into a dataset, and can divide the dataset into a training set, a validation set and a test set, and any unmanned aerial vehicle image belongs to and only belongs to a certain set in the training set, the validation set or the test set, to ensure the independence of data division and the reliability of evaluation results. The training set, the validation set or the test set is used to train, verify and test the constructed neural network model, and the embodiment of the present application divides the proportion of the training set, the validation set and the test set in the dataset into 7:2:1.

[0080] To make fuller use of the geometric features in UAV images, this embodiment of the invention utilizes a corner detection method to obtain corner information contained in the UAV image after marking UAV parts. Corner information refers to pixels in the UAV image that exhibit significant geometric changes, such as edge intersections and texture abrupt changes. Typically, corner information is discretely distributed in UAV images. Since discrete corner information cannot meet the input requirements of a neural network model, the acquired corner information is encoded to transform it into a corner feature map. That is, the representation of corner information is a corner feature map, such as a corner mask image or heatmap, to satisfy the input requirements of the neural network model.

[0081] The corner detection method selected in this embodiment of the invention is the SIFT corner detection method. In the field of image recognition, the image to be recognized often needs to be processed by rotation, scaling, brightness changes, etc. Some feature points in the image remain stable after processing; these stable feature points are the corner information. The SIFT corner detection method can detect these stable feature points in the image to be recognized. Therefore, the SIFT corner detection method is widely used in image matching, target recognition, 3D reconstruction, and other fields.

[0082] Corner detection requires first processing the drone image, and then using the SIFT corner detection method to detect corner information in the drone image, such as... Figure 4 As shown, the main steps of the SIFT corner detection method include detecting scale-space extrema, locating and filtering extrema, assigning orientation values, and generating feature point descriptors. The specific process for detecting corner information is as follows:

[0083] Q1: What is the image of the drone after marking its parts? Convert the image to a grayscale image of the drone, and then normalize the grayscale image to obtain a normalized drone image.

[0084] Annotated drone images Convert to drone grayscale image :

[0085] ,

[0086] in, Represents drone images The coordinates of the middle pixel, These represent images from drones. of Channels (i.e., red channel, green channel, and blue channel).

[0087] drone grayscale image Normalization is performed to adjust its grayscale value range to [0,1]:

[0088] ,

[0089] wherein, represents a normalized image of the UAV, represents a UAV grayscale image, which is equivalent to ,

[0090] The normalized image of the UAV is taken as input, and a SIFT corner point detection method is used to obtain feature points that remain stable in the normalized image of the UAV.

[0091] Q2: Use a Gaussian difference pyramid to detect extreme points in the scale space of the normalized image of the UAV.

[0092] In a real scene, the size of the same object in an image is uncertain. For example, a car far away and a car close by have a large size difference in an image. If feature points are detected only at a single resolution (scale), the details that can be detected at a small scale will be "smoothed out" at a large scale, and the overall features that can be detected at a large scale will be disturbed by "noise" at a small scale, and cannot be stably identified. Therefore, the core idea of the SIFT corner point detection method is to construct a scale space to simulate the "from far to near" scale change when a person observes an object, and to analyze the normalized image of the UAV at different scales at the same time, so as to ensure that the detected feature points "remain stable" when facing scale changes. The present embodiment refers to these feature points that "remain stable" when facing scale changes as scale-stable feature points.

[0093] The SIFT corner point detection method uses a Gaussian difference (Difference of Gaussian, DoG) pyramid to detect extreme points in the scale space of the normalized image of the UAV, that is, each pixel needs to be compared with multiple pixels in the scale space layer where the pixel is located and adjacent upper and lower scale space layers. If it is a local maximum or minimum (extreme point), it is considered as a potential scale-stable feature point.

[0094] A Gaussian function is used to generate the scale space of the normalized image of the UAV, and a Gaussian difference pyramid is obtained by calculating the difference between adjacent scale spaces. The extreme points of each pixel point in its neighborhood (including the same layer and the upper and lower layers) in the Gaussian difference pyramid are calculated as follows:

[0095] ,

[0096] wherein, represents a Gaussian function, represents a scale parameter, represents the spatial coordinates of a pixel point on the normalized image of the UAV.

[0097] The formula of the Gaussian difference is as follows:

[0098] ,

[0099] where, and denote the standard deviation of the Gaussian kernel of the adjacent scale.

[0100] Q3: The position and scale of the extreme value point are located by using the method of quadratic function fitting, and the extreme value points with contrast lower than a preset threshold and the edge response points are removed, and the remaining extreme value points are taken as the scale stable feature points.

[0101] Since the position and scale of the extreme value point detected initially are discrete, and may be affected by noise or discrete sampling, leading to inaccurate positioning, it is necessary to correct the position and scale of the extreme value point obtained by calculation. The embodiment of the application uses the method of quadratic function fitting to accurately locate the position and scale of the extreme value point, and more accurately determines the position and scale of the extreme value point in continuous space. The extreme value points after accurate positioning are screened, and the low-contrast points (extreme value points with contrast lower than a preset threshold) and the edge response points are removed, and the scale stable feature points are obtained.

[0102] Q4: The direction distribution of the scale gradient in the neighborhood of the scale stable feature point is counted, and the main direction is assigned to each scale stable feature point.

[0103] The obtained scale stable feature points are direction-assigned, to ensure that the scale stable feature points have "rotation invariance". The main direction and auxiliary direction are assigned to each scale stable feature point by counting the direction distribution of the gradient in the neighborhood of the scale stable feature point, so that the subsequent descriptor calculation can be rotationally aligned based on a unified coordinate system. The role of the main direction is to realize rotation invariance, and the main direction can be the most prominent direction of the scale stable feature point. The role of the auxiliary direction is to correct the main direction, and the actual direction is determined by the main direction and the auxiliary direction.

[0104] The gradient amplitude of each pixel in the neighborhood of the scale stable feature point is calculated and the gradient direction :

[0105] ,

[0106] ,

[0107] where, denotes the gray value or brightness value of the pixel point at a certain scale (determined by the degree of Gaussian blur), more accurately, the pixel value of the scale space image obtained after Gaussian blur processing of the original image at the point . The blur processing is to stably detect the features at different scales.

[0108] Centered on a scale-stable feature point, the gradient direction distribution within its neighborhood (usually within a prototype region with a radius of 4.5 times the scale) is statistically analyzed to form a histogram. The direction corresponding to the highest bar in the histogram is the primary direction. If the amplitude of other bars exceeds 80% of that of the highest bar, they are used as secondary directions. The secondary directions are not unique, but a scale-stable feature point can be assigned a maximum of 3 secondary directions.

[0109] Q5: Based on the main direction, rotate the coordinate system of the neighborhood of the scale-stable feature point to the main direction to obtain corner information.

[0110] Next, a descriptor is generated for each scale-stable feature point. The descriptor is generated by quantizing the gradient information of the neighborhood of the scale-stable feature point to construct a high-dimensional vector that is robust to image distortion (geometric transformation) and illumination changes. Specifically, based on the assigned principal and secondary directions, the coordinate system of the neighborhood of the scale-stable feature point is rotated to the principal direction so that subsequent scale-stable feature points are not affected by the rotation.

[0111] After generating descriptors for scale-stable feature points, stable feature points, i.e., corner information, are obtained. This corner information is then converted into a corner feature map for use in neural network model training. Typically, corner feature maps take the form of corner mask images or heatmaps; this embodiment of the invention uses a corner mask image as the corner feature map for the corner information.

[0112] Generate a corner mask image from the corner information obtained by the SIFT corner detection method:

[0113] Create a mask image And the mask image The size is the same as the preprocessed drone image, and the mask image is... All pixel values ​​are initialized to 0:

[0114] .

[0115] Then the mask image All corner information corresponding to the pixel value is set to 255 (or set to 1, in this embodiment of the invention it is set to 255):

[0116] ,

[0117] in, Keypoints refer to the set of all feature points identified and extracted from preprocessed UAV images using the SIFT corner detection algorithm. Indicates the first The coordinates of key points are used to determine the mask image. The binary image obtained after all the pixel values in the image are processed is the final corner mask image. The pixel value of 0 in the corner mask image is the non-corner information area, and the pixel value of 255 (or 1) is the corner information area. The corner mask image is the corner feature map.

[0118] The corner feature map (i.e., the corner mask image) is fused with the drone image labeled with the drone parts to obtain a corner image. The corner image contains both the labeled drone parts and the corner information. The fusion method is as follows: the corner feature map is taken as the fourth channel, and the RGB three channels of the drone image labeled with the drone parts are channel spliced to obtain the corner image.

[0119] S3: training a neural network model using the corner image.

[0120] The corner image is input into the created neural network model to train the neural network model, so that the neural network model learns to identify the drone parts in the to-be-detected image. The present embodiment sets learning rate, training batch, and other hyperparameters during the training process, and adopts a training model of forward inference and back propagation.

[0121] To verify the performance of the trained neural network model, the present embodiment compares the original YOLO model and the neural network model trained by adding the SE channel attention mechanism according to the method of the present embodiment. The verification performance includes precision, average precision, and recall rate, etc. The verification platform is as follows:

[0122] CPU: AMD Ryzen 7 7745HX with Radeon Graphics@3.60 GHz;

[0123] GPU: NVIDIA GeForce RTX 4060;

[0124] Operating system: Windows10 64bit RAM: DDRA 2400 32G;

[0125] Programming language: Python3.7;

[0126] Experimental platform: Pytorch;

[0127] As shown in Figs. Figure 5 , Figure 6 and Figure 7 , the simulation experiment verifies that the average precision of the original YOLO model is 52.7%, the P value (and the precision) is 0.949, the average precision of the neural network model of the present embodiment is 55.9%, and the P value is 0.957. It can be seen that the precision of the neural network model of the present embodiment increases by 2.2%. Figure 5In the above table, "all classes" means all classes, "mAP" is the mean average precision, i.e., the average of the average precisions of all classes; "@0.5" means that the intersection over union (IOU) threshold used in the calculation is 0.5. all classes 0.957 mAP@0.5 means that the mean average precision of all classes is 0.957 when the IOU threshold is 0.5, which reflects that the overall detection performance of the model on all classes is very good. Rotors 0.930 means the PR curve (precision-recall curve) of the neural network model on the "rotor" class, and the value 0.930 means the average precision of the class when the IOU (intersection over union) threshold is 0.5. The value is the area under the PR curve, which is the core indicator for measuring the performance of the class. The higher the value (closer to 1), the better the overall detection performance of the class. battery 0.985 means the PR curve of the neural network model on the "battery" class.

[0128] S4: inputting the to-be-detected target image into the trained neural network model to identify the UAV part in the to-be-detected target image.

[0129] The trained neural network model has the ability to identify the UAV part, as shown in FIGS. Figure 8 and Figure 9 The trained neural network model has the ability to identify the UAV part, as shown in FIGS.

[0130] S5: calculating the position coordinates of the UAV part in the actual space according to the identified UAV part.

[0131] In some fields, it is not only necessary to identify the UAV part in the UAV image, but also necessary to calculate the position of the UAV part in the actual space according to the identified UAV part. The embodiment of the present application also provides a method for calculating the position of the UAV part in the actual space: in physics, the center of mass is the average position of the mass distribution of an object. Assuming that the mass of the object is uniformly distributed and the detection frame of the YOLO model can be approximated as the envelope of the object boundary, the geometric center point of the detection frame can be approximated as the center of mass of the object. In image processing, since the object detected by the YOLO model does not provide mass distribution information, it is assumed that the "mass" of each pixel is consistent, and it is assumed that the center of mass is the geometric center of the position distribution and is approximately equal to the center of mass. Since the real boundary frame of the target in the training process of the YOLO model is usually manually labeled or automatically generated, it is considered that the real boundary frame closely surrounds the UAV part, so it can be assumed that the UAV part is located at the center of the detection frame during the detection process, and the distribution of the to-be-detected target in the detection frame is relatively uniform and symmetrical, and accordingly the center of mass coordinates of the UAV part in the actual space can be obtained :

[0132] ,

[0133] ,

[0134] wherein, represents the left upper corner coordinate of the detection frame, represents the right lower corner coordinate of the detection frame. The centroid coordinate obtained by calculation is mapped from the image coordinate system to the real space coordinate system, and the actual position of the unmanned aerial vehicle can be obtained.

[0135] Although the embodiments of the present application have been shown and described above, it should be understood that the above-described embodiments are exemplary and should not be construed as limiting the present application. Those skilled in the art can make changes, modifications, replacements and variations to the above-described embodiments within the scope of the present application.

[0136] The specific embodiments of the present application described above do not constitute a limitation on the scope of protection of the present application. Any various other corresponding changes and modifications made according to the technical concept of the present application shall be included in the scope of protection of the claims of the present application.

Claims

1. A method for automatic identification of drone parts based on corner detection, comprising S1: constructing a neural network model with a channel attention mechanism, wherein the neural network is used to automatically identify drone parts; characterized in that, Also includes: S2: Acquire multiple drone images and label the drone parts in the drone images; Using a corner detection method, corner information in the drone image after marking drone parts is obtained, and a corner feature map is generated based on the corner information; A corner image is obtained by fusing the corner feature map with the drone image after labeling the drone parts; the fusing method is as follows: the corner feature map is used as the fourth channel and is spliced ​​with the RGB three channels of the drone image after labeling the drone parts to obtain the corner image; Corner information in UAV images is detected using the SIFT corner detection method, which includes: Q1: Convert the drone image after marking the drone parts into a drone grayscale image, and normalize the drone grayscale image to obtain a normalized drone image. Q2: Use the difference of Gaussian pyramid to detect the extreme points in the scale space of the normalized image of the UAV; calculate the extreme points of each pixel in its neighborhood in the difference of Gaussian pyramid as follows: ， in, Represents the Gaussian function. Indicates the scale parameter. Represents the spatial coordinates of pixels in a normalized image of a drone; The formula for the difference of Gaussians is as follows: ， in, and This represents the Gaussian kernel standard deviation of adjacent scales; Q3: The position and scale of the extreme points are located by using a quadratic function fitting method, and extreme points with contrast below a preset threshold and edge response points are removed. The remaining extreme points are used as scale-stable feature points. Q4: Statistically analyze the directional distribution of the scale gradient within the neighborhood of the scale-stable feature point, and assign a main direction and a secondary direction to each scale-stable feature point. The secondary direction is not unique, and a scale-stable feature point can be assigned a maximum of 3 secondary directions. Centered on scale-stable feature points, the gradient direction distribution within their neighborhood is statistically analyzed to form a histogram. The direction corresponding to the highest bar in the histogram is the primary direction. If the amplitude of other bars exceeds 80% of that of the highest bar, they are used as secondary directions. The primary direction is the most significant direction of the scale-stable feature point, and the secondary direction is a correction of the primary direction. The actual direction is determined by both the primary and secondary directions. Q5: Based on the main direction, rotate the coordinate system of the neighborhood of the scale-stable feature point to the main direction to obtain corner information; The method for generating the corner feature map is as follows: Create a mask image And the mask image The size is the same as the preprocessed drone image, and the mask image is... All pixel values ​​are initialized to 0: ； Then the mask image All corner information is set to a pixel value of 255: ， in, Indicate key points, Indicates the first Coordinates of key points; determine the mask image. The binary image obtained after taking all pixel values ​​is the final corner mask image. In the corner mask image, the pixel value of 0 is the non-corner information area, and the pixel value of 255 is the corner information area. This corner mask image is the corner feature map. S3: Train the neural network model using the corner image; S4: Input the target image to be detected into the trained neural network model to identify the drone parts in the target image.

2. The automatic identification method for UAV parts based on corner detection as described in claim 1, characterized in that, The neural network model is a YOLO model with an SE channel attention mechanism.

3. The automatic identification method for UAV parts based on corner detection as described in claim 2, characterized in that, The SE channel attention mechanism includes global information compression operation, stimulus operation, and reweighting operation; The global information compression operation includes: encoding the RGB channels of the image to be recognized by the neural network model into a format with a scale of [scale value missing]. The global features are encoded using the following formula: ， in, This indicates the number of compressed images. Global eigenvalues ​​of each channel Represents the global feature vector. This represents the squeeze operation function. Represents the first... One channel, Indicates the height of the input feature map. This indicates the width of the input feature map. This represents the index variable used for traversal along the height direction of the feature map. This represents the index variable used for traversal along the width of the feature map. Represents the set of real numbers; The activation operation includes: for each RGB channel Generate a The weights of the intervals are calculated using the following formula: ， in, Indicates weight, This represents the activation function. This refers to the parameters that need to be learned during the stimulus operation; specifically, it refers to... or , This represents the Sigmoid activation function. Represents a linear activation function. This represents the first fully connected layer. , This indicates the second fully connected layer. ; The reweighting operation includes: for each channel Assign different weights: ， in, Represents the first... The result after weighting each channel using the SE channel attention mechanism. This represents the scaling function. This represents the weighting coefficient.

4. The automatic identification method for UAV parts based on corner detection as described in claim 1, characterized in that, The acquired drone images are from drones with different shooting angles, lighting conditions, backgrounds, and occlusions.

5. The automatic identification method for UAV parts based on corner detection as described in claim 1, characterized in that, After acquiring multiple drone images, before labeling the drone parts in the drone images, the process also includes preprocessing the drone images. The preprocessing method is as follows: cropping the drone image to a predetermined size, normalizing the pixel values ​​of the cropped drone image, and enhancing the image data of the normalized drone image.

6. The automatic identification method for UAV parts based on corner detection as described in claim 1, characterized in that, It also includes S5: calculating the position coordinates of the identified drone parts in actual space. ， ， in, This represents the centroid coordinates of the actual space of the drone component. This indicates the coordinates of the top-left corner of the detection box. This indicates the coordinates of the bottom right corner of the detection box.

Images