Unmanned aerial vehicle image recognition network training method, application method and electronic device
By constructing a UAV image recognition network and optimizing network parameters using feature extraction and cross-sample attention modules, the problem of high inter-class similarity in UAV image recognition was solved, and the recognition accuracy was improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SANYA SCI & EDUCATION INNOVATION PARK WUHAN UNIV OF TECH
- Filing Date
- 2023-07-27
- Publication Date
- 2026-06-23
Smart Images
Figure CN117079160B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer vision, and specifically to a method for training an image recognition network for unmanned aerial vehicles (UAVs), an application method, an electronic device, and a computer-readable storage medium. Background Technology
[0002] With the widespread use of drones, a large number of aerial images taken from the drone's perspective have been generated. Identifying these drone images and assigning a specific label to each unlabeled drone image based on its semantic information is a fundamental and crucial task in the field of drone image interpretation. This has important applications in environmental monitoring, traffic control, and disaster management.
[0003] Due to the different shooting angles of drone images, drone images have complex spatial distribution structures. When performing drone image recognition, the complex spatial object distribution makes it difficult to focus on the main body of the class. At the same time, the complex backgrounds of drone images are also very similar, resulting in high inter-class similarity between different classes of images. These problems all contribute to the low recognition accuracy of existing neural network models for drone images.
[0004] Existing technologies, such as peak response-based methods, address the aforementioned issues by describing the peak responses of key local regions and their corresponding contextual information, assigning different weights to different regions to highlight key areas of interest. However, in complex contexts, it is difficult to focus on the main subject, and therefore the problem of high inter-class similarity has not been well resolved. Summary of the Invention
[0005] In view of this, it is necessary to provide a training method, application method, electronic device and computer-readable storage medium for UAV image recognition network, in order to solve the technical problems in the prior art that it is difficult to focus on important features and cannot solve the problem of high inter-class similarity, which leads to low accuracy of UAV image recognition.
[0006] To address the above problems, this invention provides a method for training an image recognition network for unmanned aerial vehicles (UAVs), comprising:
[0007] Acquire a drone image dataset, which includes drone images and corresponding label information;
[0008] An initial UAV image recognition network is constructed, which includes a feature extraction module, a cross-sample attention module, and a classifier module.
[0009] Using the drone image as the input to the initial drone image recognition network, the drone image is subjected to feature extraction and feature optimization weighting based on the feature extraction module to obtain optimized feature data. The optimized feature data is subjected to cross-sample attention learning based on the cross-sample attention module to obtain enhanced feature data. The enhanced feature data is then input into the classifier module for decoding to obtain the classification prediction result.
[0010] Based on the classification prediction results and the label information, the label smoothing loss of the initial UAV image recognition network is determined, and the parameters of the initial UAV image recognition network are updated until the loss no longer decreases. The fully trained UAV image recognition network is then obtained based on the feature extraction module and the classifier module.
[0011] Furthermore, obtain the drone image dataset, including:
[0012] Acquire drone images and corresponding tag information;
[0013] The drone images are resized and cropped to ensure consistent image size. A drone image dataset is then constructed based on the drone images and their corresponding label information.
[0014] Furthermore, the feature extraction module includes four basic extraction modules, a feature optimization module, and a global average pooling layer. The basic extraction module consists of a downsampling layer and a pure convolutional block. Using the UAV image as the input to the initial UAV image recognition network, the feature extraction module performs feature extraction and feature optimization weighting on the UAV image to obtain optimized feature data, including:
[0015] The UAV image is input into the feature extraction module, and the initial feature data is obtained by sequentially extracting features through three basic extraction modules.
[0016] The initial optimized data is obtained by performing pixel feature optimization and weighting on the initial feature data based on the feature optimization module;
[0017] The initial optimized data is weighted by channel feature optimization based on the feature optimization module to obtain the optimized data;
[0018] The preferred data is processed by a basic extraction module to extract features and a global average pooling layer to obtain preferred feature data.
[0019] Furthermore, based on the feature optimization module, the initial feature data is weighted by pixel feature optimization to obtain initial optimized data, including:
[0020] The initial feature data is convolutionally reduced to obtain an embedded representation of pixel features;
[0021] The pixel response weight matrix is obtained by calculating the autocorrelation of features in the embedded representation of the pixel features based on the embedded Gaussian algorithm.
[0022] The global response map is obtained by multiplying the response weight matrix with the initial feature data;
[0023] The global response map is convolved to increase its dimensionality to obtain spatial context features, and the spatial context features are residually concatenated with the initial feature data to obtain the initial optimized data.
[0024] Furthermore, based on the feature optimization module, the initial optimized data is weighted by channel feature optimization to obtain optimized data, including:
[0025] The initial optimized data is subjected to global average pooling to obtain an embedded representation of channel features.
[0026] The embedded representation of the channel features is input into a multilayer perceptron network, and channel weights are generated for each channel feature based on the network parameter matrix of the multilayer perceptron network.
[0027] Based on the activation function gating mechanism, the reweighted matrix is obtained by adjusting the channel weights of each channel.
[0028] The preferred data is obtained by performing a Hadamard product operation on the initial preferred data and the reweighted matrix.
[0029] Furthermore, enhanced feature data is obtained by performing cross-sample attention learning on the preferred feature data based on the cross-sample attention module, including:
[0030] The preferred feature data is subjected to self-attention calculation and grouped according to the label information to obtain grouped self-attention results;
[0031] The grouped self-attention results are concatenated and then subjected to linear transformation, followed by residual connection and layer normalization to obtain the initial cross-sample attention features.
[0032] The cross-sample attention is nonlinearized based on a multilayer perceptron, and then residual connection and layer normalization operations are performed to obtain cross-sample attention features.
[0033] The enhanced feature data is obtained by concatenating the cross-sample attention features with the preferred feature data.
[0034] Further, based on the classification prediction results and the label information, the label smoothing loss of the initial UAV image recognition network is determined, and the parameters of the initial UAV image recognition network are updated until the loss no longer decreases. A fully trained UAV image recognition network is then obtained based on the feature extraction module and the classifier module, including:
[0035] The label information is smoothed to obtain label data;
[0036] The label smoothing loss of the initial UAV image recognition network is determined based on the classification prediction results and the label data;
[0037] The network parameters of the initial UAV image recognition network are updated using the backpropagation algorithm gradient until the loss no longer decreases.
[0038] The feature extraction module and the classifier module are combined to form a fully trained UAV image recognition network.
[0039] This invention also provides a method for network application of image recognition for unmanned aerial vehicles, comprising:
[0040] Acquire images of the drone to be retrieved;
[0041] The drone image to be retrieved is input into a fully trained drone image recognition network to identify the drone image category, wherein the fully trained drone image recognition network is determined according to any of the above-described drone image recognition network training methods.
[0042] The drone image category is obtained based on the output of the drone image recognition network.
[0043] The present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the UAV image recognition network training method described above, and / or the UAV image recognition network application method described above.
[0044] The present invention also provides a computer-storable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the UAV image recognition network training method described above, and / or the UAV image recognition network application method described above.
[0045] Compared with existing technologies, the beneficial effects of sampling the above embodiments are as follows: In the UAV image recognition network training method provided by the present invention, a UAV image dataset is first acquired, which includes UAV images and corresponding label information; then, an initial UAV image recognition network is constructed, which includes a feature extraction module, a cross-sample attention module, and a classifier module; the UAV images are used as input to the initial UAV image recognition network, and the UAV images are subjected to feature extraction and feature optimization weighting based on the feature extraction module to obtain optimized feature data, and the optimized feature data is subjected to cross-sample attention learning based on the cross-sample attention module to obtain enhanced feature data, and the enhanced feature data is input to the classifier module for decoding to obtain a classification prediction result; finally, the label smoothing loss of the initial UAV image recognition network is determined according to the classification prediction result and the label information, and the parameters of the initial UAV image recognition network are updated until the loss no longer decreases, and a fully trained UAV image recognition network is obtained according to the feature extraction module and the classifier module. In summary, this invention uses a feature extraction module to perform feature optimization and weighting on UAV images, extracting more discriminative features under the high spatial complexity of UAV images and improving classification performance. By using a cross-sample attention module to establish dependencies between different samples in each category, it can mine the similarity between samples of the same class and better distinguish similar classes. This solves the technical problems in the prior art, which is difficult to focus on important features and cannot solve the problem of high inter-class similarity. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0047] Figure 1 A schematic flowchart of an embodiment of the UAV image recognition network training method provided by the present invention;
[0048] Figure 2 A schematic flowchart of an embodiment of the UAV image recognition network application method provided by the present invention;
[0049] Figure 3 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation
[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0051] It should be understood that the accompanying drawings are not drawn to scale. The flowcharts used in this invention illustrate operations implemented according to some embodiments of the invention. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or performed simultaneously. Furthermore, those skilled in the art, guided by the content of this invention, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.
[0052] Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor systems and / or microcontroller systems.
[0053] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.
[0054] Figure 1 This is a schematic flowchart of an embodiment of the UAV image recognition network training method provided by the present invention, as shown below. Figure 1 As shown, the training method for UAV image recognition networks includes:
[0055] S101. Obtain a drone image dataset, wherein the drone image dataset includes drone images and corresponding label information;
[0056] S102. Construct an initial UAV image recognition network, which includes a feature extraction module, a cross-sample attention module, and a classifier module.
[0057] S103. Using the UAV image as the input of the initial UAV image recognition network, the UAV image is subjected to feature extraction and feature optimization weighting based on the feature extraction module to obtain optimized feature data. The optimized feature data is subjected to cross-sample attention learning based on the cross-sample attention module to obtain enhanced feature data. The enhanced feature data is input into the classifier module for decoding to obtain the classification prediction result.
[0058] S104. Determine the label smoothing loss of the initial UAV image recognition network based on the classification prediction result and the label information, update the parameters of the initial UAV image recognition network until the loss no longer decreases, and obtain a fully trained UAV image recognition network based on the feature extraction module and the classifier module.
[0059] Specifically, in the UAV image recognition network training method provided by this invention, a UAV image dataset is first acquired, which includes UAV images and corresponding label information. Then, an initial UAV image recognition network is constructed, comprising a feature extraction module, a cross-sample attention module, and a classifier module. Using the UAV images as input to the initial UAV image recognition network, the feature extraction module performs feature extraction and feature optimization weighting on the UAV images to obtain optimized feature data. The cross-sample attention module performs cross-sample attention learning on the optimized feature data to obtain enhanced feature data. The enhanced feature data is then input into the classifier module for decoding to obtain a classification prediction result. Finally, based on the classification prediction result and the label information, the label smoothing loss of the initial UAV image recognition network is determined, and the parameters of the initial UAV image recognition network are updated until the loss no longer decreases. A fully trained UAV image recognition network is then obtained based on the feature extraction module and the classifier module. In summary, this invention uses a feature extraction module to perform feature optimization and weighting on UAV images, extracting more discriminative features under the high spatial complexity of UAV images and improving classification performance. By using a cross-sample attention module to establish dependencies between different samples in each category, it can mine the similarity between samples of the same class and better distinguish similar classes. This solves the technical problems in the prior art, which is difficult to focus on important features and cannot solve the problem of high inter-class similarity.
[0060] In a specific embodiment of the present invention, obtaining a drone image dataset includes:
[0061] Acquire drone images and corresponding tag information;
[0062] The drone images are resized and cropped to ensure consistent image size. A drone image dataset is then constructed based on the drone images and their corresponding label information.
[0063] Specifically, drone images obtained by drones through aerial photography are acquired for network training. The images are resized and cropped to ensure that all images are the same size. The drone images are then classified and labeled to obtain corresponding label information. The drone images of the same size and the label information are combined to form a drone image dataset.
[0064] In a specific embodiment of the present invention, the feature extraction module includes four basic extraction modules, a feature optimization module, and a global average pooling layer. The basic extraction module consists of a downsampling layer and a pure convolutional block. Using the UAV image as the input to the initial UAV image recognition network, the feature extraction module performs feature extraction and feature optimization weighting on the UAV image to obtain optimized feature data, including:
[0065] The UAV image is input into the feature extraction module, and the initial feature data is obtained by sequentially extracting features through three basic extraction modules.
[0066] The initial optimized data is obtained by performing pixel feature optimization and weighting on the initial feature data based on the feature optimization module;
[0067] The initial optimized data is weighted by channel feature optimization based on the feature optimization module to obtain the optimized data;
[0068] The preferred data is processed by a basic extraction module to extract features and a global average pooling layer to obtain preferred feature data.
[0069] Specifically, in this embodiment of the invention, the feature extraction module consists of four basic extraction modules and a feature optimization module. The basic extraction modules are used to extract features from image data, while the feature optimization module is used to perform feature optimization and weighting on the features to increase the weight of important features. For the UAV image input to the feature extraction module, initial feature data is first obtained by feature extraction through three basic extraction modules; then, the initial feature data is obtained by pixel feature optimization and weighting through the feature optimization module to obtain initial optimized data, thereby establishing the relationship between each pixel and the entire image, obtaining global information in the image space, and ensuring that each channel has sufficient information; then, the feature optimization module performs channel feature optimization and weighting on the initial optimized data to obtain optimized data, thereby selecting more discriminative channel features and learning better feature representations; finally, the optimized feature data is obtained by a basic extraction module and global average pooling.
[0070] In a specific embodiment of the present invention, the initial optimized data is obtained by performing pixel feature optimization weighting on the initial feature data based on the feature optimization module, including:
[0071] The initial feature data is convolutionally reduced to obtain an embedded representation of pixel features;
[0072] The pixel response weight matrix is obtained by calculating the autocorrelation of features in the embedded representation of the pixel features based on the embedded Gaussian algorithm.
[0073] The global response map is obtained by multiplying the response weight matrix with the initial feature data;
[0074] The global response map is convolved to increase its dimensionality to obtain spatial context features, and the spatial context features are residually concatenated with the initial feature data to obtain the initial optimized data.
[0075] Specifically, for the initial feature data input First, a 1×1 convolution is used to obtain the pixel feature embedded representation of the feature. Simultaneously, to reduce the number of parameters, the number of feature channels is reduced, resulting in... After flattening the features, the autocorrelation of the features is calculated based on the embedded Gaussian algorithm. This autocorrelation is the sum of the feature weights of all other pixels in the current image space, representing the response of each pixel. The formula is as follows:
[0076]
[0077] in Represents the initial feature data. These represent the number of channels, height, and width of the feature data, respectively. and This represents the weight matrix of the convolutional network. It is the pixel response weight matrix for each pixel after normalization.
[0078] The resulting pixel response weight matrix is then multiplied back into the feature matrix as weights, using the following formula:
[0079]
[0080] in, This represents the weight matrix of the convolutional network. This represents the embedded representation of pixel features after convolution of the initial feature data. This represents the obtained global response graph.
[0081] At this point, we obtain features enriched with global context information. After flattening the state and transforming it back to its original shape, a 1×1 convolution is performed to restore the original number of channels, obtaining spatial context features. These features are then residually connected to the initial feature data, adding these features to the initial feature data to obtain the initial optimized data.
[0082] In a specific embodiment of the present invention, the preferred data is obtained by performing channel feature optimization weighting on the initial preferred data based on the feature optimization module, including:
[0083] The initial optimized data is subjected to global average pooling to obtain an embedded representation of channel features.
[0084] The embedded representation of the channel features is input into a multilayer perceptron network, and channel weights are generated for each channel feature based on the network parameter matrix of the multilayer perceptron network.
[0085] Based on the activation function gating mechanism, the reweighted matrix is obtained by adjusting the channel weights of each channel.
[0086] The preferred data is obtained by performing a Hadamard product operation on the initial preferred data and the reweighted matrix.
[0087] Specifically, after the pixel feature optimization and weighting described above, the features in each channel now possess more comprehensive spatial context information on top of the original information, requiring further feature optimization and weighting. First, a global average pooling operation is used to generate a channel feature embedded representation based on the channel's feature response, preserving overall characteristics and more high-dimensional information, as shown in the following formula:
[0088]
[0089] in, Indicates the first Image data for each channel, This represents the coordinates of each pixel in the image data.
[0090] Then, a feedforward neural network, a multilayer perceptron, is used to generate different channel weights for each channel feature using its network matrix; finally, the gating mechanism of the sigmoid activation function is used to adjust each channel according to its weights to obtain a reweighted matrix. This indicates the importance of each feature channel after feature selection. Finally, the initial selected data and the reweighted matrix are subjected to the Hadamard product operation to reweight the initial selected data in the channel dimension, effectively selecting important features and suppressing unimportant features, thus achieving feature optimization.
[0091] Compared with existing technologies, this invention captures comprehensive global spatial information through pixel feature optimization and weighting, so that each channel has enough information, thereby achieving more accurate channel selection. By optimizing and weighting channel features, more discriminative features are selected, better feature representations are learned, and classification performance is improved.
[0092] In a specific embodiment of the present invention, enhanced feature data is obtained by performing cross-sample attention learning on the preferred feature data based on a cross-sample attention module, including:
[0093] The preferred feature data is subjected to self-attention calculation and grouped according to the label information to obtain grouped self-attention results;
[0094] The grouped self-attention results are concatenated and then subjected to linear transformation, followed by residual connection and layer normalization to obtain the initial cross-sample attention features.
[0095] The cross-sample attention is nonlinearized based on a multilayer perceptron, and then residual connection and layer normalization operations are performed to obtain cross-sample attention features.
[0096] The enhanced feature data is obtained by concatenating the cross-sample attention features with the preferred feature data.
[0097] Specifically, this invention also incorporates a cross-sample attention module to learn the relationships between different samples within the same category. During the training phase, the cross-sample attention module of this invention first divides the preferred feature data as the Q, K, and V of self-attention according to a predetermined number of heads, and performs self-attention calculations. This allows for parallel calculation of multiple pieces of information selected from the input information, with each attention focusing on a different part of the input information. This enables the model to collectively focus on information from different representation subspaces at different locations, as shown in the following formula:
[0098]
[0099] in, This is the scaling factor.
[0100] The results of each self-attention group are then concatenated and linearly transformed, followed by residual connection and normalization to obtain the initial cross-sample attention features, as shown in the following formula:
[0101]
[0102] in, This represents the weight matrix of the fully connected layer used for the linear transformation. Indicates residual connection, This indicates a normalization operation.
[0103] The initial cross-sample attention features are then input into a multilayer perceptron for nonlinearization, followed by residual connection and normalization operations to obtain the final cross-sample attention features. The formula is as follows:
[0104]
[0105] in This represents a multilayer perceptron.
[0106] Finally, the obtained cross-sample attention features are concatenated into the optimized feature data to obtain the enhanced feature data.
[0107] Compared with existing technologies, the cross-sample attention module of the present invention further refines the attention layer by using a multi-head attention mechanism, establishes the dependency relationship between different samples in each category, mines the similarity between different samples in each category, realizes the promotion learning between different samples in the same category, and better distinguishes similar classes.
[0108] In a specific embodiment of the present invention, the label smoothing loss of the initial UAV image recognition network is determined based on the classification prediction result and the label information, the parameters of the initial UAV image recognition network are updated until the loss no longer decreases, and a fully trained UAV image recognition network is obtained based on the feature extraction module and the classifier module, including:
[0109] The label information is smoothed to obtain label data;
[0110] The label smoothing loss of the initial UAV image recognition network is determined based on the classification prediction results and the label data;
[0111] The network parameters of the initial UAV image recognition network are updated using the backpropagation algorithm gradient until the loss no longer decreases.
[0112] The feature extraction module and the classifier module are combined to form a fully trained UAV image recognition network.
[0113] Specifically, when calculating the loss, instead of converting the traditional encoding of label information into 0s and 1s, the label information is converted into label data using the following formula:
[0114]
[0115] in, This is a very small constant; in this experiment, it is set to 0.1. For class number, when That is, the classification probability corresponding to the category. for In other cases, the remaining low probabilities are divided equally.
[0116] The above method ensures that the probability optimization objective is no longer 1 or 0, and the optimal solution for the predicted value is no longer positive infinity, but a specific numerical value. This avoids overfitting to some extent and mitigates the impact of incorrect labels. The label smoothing loss is calculated based on the classification prediction results and the modified label data, using the following formula:
[0117]
[0118] in, This represents the probability of belonging to each category in the classification prediction results. This represents the current label data.
[0119] Then, the network parameters of the initial UAV image recognition network are updated based on the gradient of the backpropagation algorithm until the loss no longer decreases. The feature extraction module and the classifier module are then combined to obtain a fully trained UAV image recognition network. It should be noted that the cross-sample attention module is only used during training to learn sample relationships; it is removed in the testing and fully trained UAV image recognition networks.
[0120] In summary, this invention captures comprehensive global spatial information through pixel feature optimization and weighting, ensuring each channel has sufficient information for more accurate channel selection. Channel feature optimization and weighting selects more discriminative features, leading to better feature representation and improved classification performance. The attention layer is further refined using a multi-head attention mechanism through a cross-sample attention module, establishing dependencies between different samples in each category and mining similarities between samples within each category. This facilitates learning between different samples within the same category, better distinguishing similar classes. The calculation of label smoothing loss avoids network overfitting and mitigates the impact of incorrect labels.
[0121] This invention also provides a method for network applications of image recognition for unmanned aerial vehicles, such as... Figure 2 As shown, Figure 2 A flowchart illustrating an embodiment of the UAV image recognition network application method provided by the present invention includes:
[0122] S201. Obtain the image of the drone to be retrieved;
[0123] S202. Input the drone image to be retrieved into a fully trained drone image recognition network to identify the drone image category, wherein the fully trained drone image recognition network is determined according to any of the above-mentioned drone image recognition network training methods.
[0124] S203. Obtain the drone image category based on the output of the drone image recognition network.
[0125] In this embodiment of the invention, firstly, an image of the drone to be identified is acquired; then, the drone image to be identified is effectively identified using the well-trained drone image recognition network described above, and the drone image category is output.
[0126] The present invention also provides an electronic device 300, such as Figure 3 As shown, Figure 3This is a schematic diagram of an embodiment of the electronic device provided by the present invention. The electronic device 300 includes a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the program, it implements the UAV image recognition network training method and / or the UAV image recognition network application method as described above.
[0127] In a preferred embodiment, the above-described electronic device further includes a display 303 for displaying the process by which the processor 301 executes the UAV image recognition network training method and / or the UAV image recognition network application method as described above.
[0128] The processor 301 may be an integrated circuit chip with signal processing capabilities. The processor 301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP) or an application-specific integrated circuit (ASIC). It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can also be a microprocessor or any conventional processor.
[0129] The memory 302 may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Secure Digital (SD card), Flash Card, etc. The memory 302 stores programs, and the processor 301 executes these programs upon receiving execution instructions. The process definition methods disclosed in any of the foregoing embodiments of the present invention can be applied to the processor 301, or implemented by the processor 301.
[0130] The display 303 can be an LED display, an LCD display, or a touch display, etc. The display 303 is used to display various information from the electronic device 300.
[0131] Understandable, Figure 3 The structure shown is only a schematic diagram of one possible structure of electronic device 300. Electronic device 300 may also include more than one of the following: Figure 3 Show more or fewer components. Figure 3 The components shown can be implemented using hardware, software, or a combination thereof.
[0132] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the UAV image recognition network training method and / or the UAV image recognition network application method as described above.
[0133] Generally, computer instructions for implementing the methods of the present invention can be carried on any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media can include any computer-readable medium except for signals themselves that are temporarily propagating.
[0134] Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
[0135] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for training an image recognition network for unmanned aerial vehicles (UAVs), characterized in that, The method includes: Acquire a drone image dataset, which includes drone images and corresponding label information; An initial UAV image recognition network is constructed, which includes a feature extraction module, a cross-sample attention module, and a classifier module. The feature extraction module includes four basic extraction modules, a feature optimization module, and a global average pooling layer. The basic extraction module consists of a downsampling layer and a pure convolutional block. Using the drone image as the input to the initial drone image recognition network, the drone image is subjected to feature extraction and feature optimization weighting based on the feature extraction module to obtain optimized feature data. The optimized feature data is subjected to cross-sample attention learning based on the cross-sample attention module to obtain enhanced feature data. The enhanced feature data is then input into the classifier module for decoding to obtain the classification prediction result. Based on the classification prediction results and the label information, the label smoothing loss of the initial UAV image recognition network is determined, and the parameters of the initial UAV image recognition network are updated until the loss no longer decreases. Based on the feature extraction module and the classifier module, a fully trained UAV image recognition network is obtained. Using the drone image as input to the initial drone image recognition network, the feature extraction module performs feature extraction and feature optimization weighting on the drone image to obtain optimized feature data, including: The UAV image is input into the feature extraction module, and the initial feature data is obtained by sequentially extracting features through three basic extraction modules. The initial optimized data is obtained by performing pixel feature optimization and weighting on the initial feature data based on the feature optimization module; The initial optimized data is weighted by channel feature optimization based on the feature optimization module to obtain the optimized data; The preferred data is processed by a basic extraction module to extract features and a global average pooling layer to obtain preferred feature data.
2. The UAV image recognition network training method according to claim 1, characterized in that, The acquisition of the UAV image dataset includes: Acquire drone images and corresponding tag information; The drone images are resized and cropped to ensure consistent image size. A drone image dataset is then constructed based on the drone images and their corresponding label information.
3. The UAV image recognition network training method according to claim 1, characterized in that, The initial optimized data is obtained by performing pixel feature optimization and weighting on the initial feature data based on the feature optimization module, including: The initial feature data is convolutionally reduced to obtain an embedded representation of pixel features; The pixel response weight matrix is obtained by calculating the autocorrelation of features in the embedded representation of the pixel features based on the embedded Gaussian algorithm. The global response map is obtained by multiplying the response weight matrix with the initial feature data; The global response map is convolved to increase its dimensionality to obtain spatial context features, and the spatial context features are residually concatenated with the initial feature data to obtain the initial optimized data.
4. The UAV image recognition network training method according to claim 1, characterized in that, The process of obtaining optimized data by performing channel feature optimization weighting on the initial optimized data based on the feature optimization module includes: The initial optimized data is subjected to global average pooling to obtain an embedded representation of channel features. The embedded representation of the channel features is input into a multilayer perceptron network, and channel weights are generated for each channel feature based on the network parameter matrix of the multilayer perceptron network. Based on the activation function gating mechanism, the reweighted matrix is obtained by adjusting the channel weights of each channel. The preferred data is obtained by performing a Hadamard product operation on the initial preferred data and the reweighted matrix.
5. The UAV image recognition network training method according to claim 1, characterized in that, The enhanced feature data obtained by performing cross-sample attention learning on the preferred feature data based on the cross-sample attention module includes: The preferred feature data is subjected to self-attention calculation and grouped according to the label information to obtain grouped self-attention results; The grouped self-attention results are concatenated and then subjected to linear transformation, followed by residual connection and layer normalization to obtain the initial cross-sample attention features. The cross-sample attention is nonlinearized based on a multilayer perceptron, and then residual connection and layer normalization operations are performed to obtain cross-sample attention features. The enhanced feature data is obtained by concatenating the cross-sample attention features with the preferred feature data.
6. The UAV image recognition network training method according to claim 1, characterized in that, The step of determining the label smoothing loss of the initial UAV image recognition network based on the classification prediction result and the label information, updating the parameters of the initial UAV image recognition network until the loss no longer decreases, and obtaining a fully trained UAV image recognition network based on the feature extraction module and the classifier module includes: The label information is smoothed to obtain label data; The label smoothing loss of the initial UAV image recognition network is determined based on the classification prediction results and the label data; The network parameters of the initial UAV image recognition network are updated using the backpropagation algorithm gradient until the loss no longer decreases. The feature extraction module and the classifier module are combined to form a fully trained UAV image recognition network.
7. A method for network application of image recognition for unmanned aerial vehicles (UAVs), characterized in that, The method includes: Acquire images of the drone to be retrieved; The drone image to be retrieved is input into a fully trained drone image recognition network to identify the drone image category, wherein the fully trained drone image recognition network is determined according to the drone image recognition network training method according to any one of claims 1 to 6. The drone image category is obtained based on the output of the drone image recognition network.
8. An electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the UAV image recognition network training method according to any one of claims 1 to 6, and / or the UAV image recognition network application method according to claim 7.
9. A computer-storable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the UAV image recognition network training method according to any one of claims 1 to 6, and / or the UAV image recognition network application method according to claim 7.