Aircraft classification method and system based on fuzzy direction prior

Abstract

The invention discloses an aircraft classification method and system based on fuzzy direction prior, and relates to the technical field of image processing, and the method comprises the steps: obtaining a target image of a to-be-classified aircraft; performing blurring direction estimation and adjustment on the target image to obtain a target blurring direction; based on the target blurring direction, performing deblurring processing on the target image through maximum entropy regularization to obtain a deblurred image; generating a to-be-recognized image based on the deblurred image; identifying the to-be-identified image through a pre-trained ViT neural network model to obtain a classification result of the to-be-classified aircraft; wherein the position code of the ViT neural network model is generated based on the direction information of the image block; the self-attention mechanism of the ViT neural network model is generated based on the similarity between the direction angles of the image blocks. According to the invention, high-fidelity restoration of the degraded image can be realized; and the ViT neural network model is forced to pay attention to the features consistent with the fuzzy direction, so that the aircraft can be classified more accurately.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, more particularly, to an aerial vehicle classification method and system based on fuzzy direction prior. BACKGROUND

[0002] Aerial vehicle classification is a technology for accurately distinguishing flying vehicle targets in images. Its core is to identify the model by analyzing the local structural features, painting marks and dynamic behavior of the vehicle.

[0003] However, the differences between aircraft classes may only exist in local subtle features, and traditional models are easily disturbed by the main features of the fuselage, resulting in loss of key discriminative information. Moreover, aerial targets are often in motion, and the dynamic blur and complex background interference of the obtained aircraft images can seriously damage the recognizability of the features, resulting in low accuracy of aircraft classification.

[0004] To sum up, how to improve the classification accuracy of the aircraft is a problem that needs to be solved by the technical personnel in the field. SUMMARY

[0005] The purpose of the present application is to provide an aerial vehicle classification method based on fuzzy direction prior, which can solve the technical problem of how to improve the classification accuracy of the aircraft to some extent. The present application also provides an aerial vehicle classification system based on fuzzy direction prior.

[0006] In order to achieve the above purpose, the present application provides the following technical scheme:

[0007] An aerial vehicle classification method based on fuzzy direction prior, comprising:

[0008] obtaining a target image of a vehicle to be classified;

[0009] estimating and adjusting the fuzzy direction of the target image to obtain a target fuzzy direction;

[0010] based on the target fuzzy direction, performing deblurring processing on the target image through maximum entropy regularization to obtain a deblurred image;

[0011] generating a to-be-identified image based on the deblurred image;

[0012] identifying the to-be-identified image through a pre-trained ViT neural network model to obtain a classification result of the vehicle to be classified;

[0013] wherein the position encoding of the ViT neural network model is generated based on the direction information of the image block; and the self-attention mechanism of the ViT neural network model is generated based on the similarity between the direction angles of the image blocks.

[0014] In an example embodiment, the blurring direction estimation and adjustment on the target image to obtain a target blurring direction comprises:

[0015] an aircraft orientation angle estimation on the target image to obtain an initial blurring direction and a confidence of the initial blurring direction;

[0016] generating an angle search range based on the initial blurring direction;

[0017] processing the target image to obtain a target spectrum map;

[0018] determining a directional energy distribution information of the target spectrum map according to the angle search range, and generating an optimized angle based on the directional energy distribution information;

[0019] weighting and fusing the initial blurring direction and the optimized angle based on the confidence to obtain a target blurring direction.

[0020] In an example embodiment, the determining a directional energy distribution information of the target spectrum map according to the angle search range, and generating an optimized angle based on the directional energy distribution information comprises:

[0021] performing Radon projection on the target spectrum map at different angles within the angle search range, and performing line integration on the target spectrum map along the angle to generate a projection curve reflecting the directional energy distribution;

[0022] taking an angle corresponding to an energy peak in the projection curve as the optimized angle.

[0023] In an example embodiment, the processing the target image to obtain a target spectrum map comprises:

[0024] performing fast Fourier transform on the target image to obtain an initial spectrum map;

[0025] generating a logarithmic power spectrum of the initial spectrum map;

[0026] performing Gaussian filtering on the logarithmic power spectrum to obtain a target spectrum map.

[0027] In an example embodiment, the deblurring processing on the target image based on the target blurring direction through maximum entropy regularization to obtain a deblurred image comprises:

[0028] generating a linear motion blur kernel along the target blurring direction;

[0029] generating a smoothing constraint corresponding to the target blur direction, the smoothing constraint being used for weak smoothing along the target blur direction and strong smoothing along a direction perpendicular to the target blur direction;

[0030] generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization;

[0031] determining the deblurred image according to the deblurring energy function.

[0032] In an exemplary embodiment, the generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization comprises:

[0033] generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization;

[0034] the deblurring energy function comprises:

[0035] ;

[0036] ;

[0037] ;

[0038] wherein, represents the target image; represents the deblurred image; represents taking minimum value; represents norm; represents the linear motion blur kernel; represents tensor product; represents setting parameter; represents the smoothing constraint; represents setting parameter; represents the maximum entropy regularization; represents parameter emphasizing edge gradient; represents parameter suppressing noise and background interference; represents taking absolute value; represents gradient parallel to the target blur direction; represents gradient perpendicular to the target blur direction.

[0039] In an exemplary embodiment, the generating a deblurred image based on the deblurred image comprises:

[0040] determining a target slice of the deblurred image;

[0041] extracting a blur kernel length from the linear motion blur kernel;

[0042] generating an expansion parameter based on the blur kernel length and the expansion coefficient;

[0043] Expanding the target slice according to the expansion parameter to obtain an expanded slice;

[0044] Slicing the deblurred image according to the extended slices to obtain a sliced ​​image;

[0045] An image to be recognized is generated according to the deblurred image and the slice image.

[0046] In an exemplary embodiment, the generation formula of the position encoding of the ViT neural network model includes:

[0047] ;

[0048] ;

[0049] in, Indicates the The final position encoding of the image blocks; Indicates the number of the image block; Expressed as The standard position vector assigned to each image block; Indicates the The extraction direction is obtained by extracting the center position of each image block; Indicates the The vector obtained by encoding the extraction direction of each image block; Indicates setting parameters; Represents cosine operation; Represents sine operation; Indicates setting weight.

[0050] In an exemplary embodiment, the generation formula of the self-attention mechanism of the ViT neural network model includes:

[0051] ;

[0052] ;

[0053] in, Indicates the image blocks and the The self-attention mechanism value of each image patch; Indicates the number of the image block; represents the normalized exponential function; represents the query vector; represents the key vector; denotes a scaling factor for stabilizing gradient propagation; is dimensional; denotes a direction penalty coefficient for balancing semantic similarity and direction consistency; denotes a similarity score of direction angle difference; denotes a sensitivity coefficient; denotes an exponential function with a natural constant as base; denotes an extracted direction after extracting the center position of the image block; denotes taking an absolute value.

[0054] An aerial vehicle classification system based on fuzzy direction prior, comprising:

[0055] a target image acquisition module configured to acquire a target image of an aerial vehicle to be classified;

[0056] a target fuzzy direction generation module configured to estimate and adjust a fuzzy direction of the target image to obtain a target fuzzy direction;

[0057] a deblurring module configured to perform deblurring processing on the target image based on the target fuzzy direction through maximum entropy regularization to obtain a deblurred image;

[0058] a to-be-recognized image generation module configured to generate a to-be-recognized image based on the deblurred image;

[0059] a recognition module configured to recognize the to-be-recognized image through a pre-trained ViT neural network model to obtain a classification result of the aerial vehicle to be classified;

[0060] wherein a position encoding of the ViT neural network model is generated based on direction information of an image block; and a self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of image blocks.

[0061] The application provides an aerial vehicle classification method based on fuzzy direction prior, obtains a target image of a to-be-classified aerial vehicle; performs fuzzy direction estimation and adjustment on the target image to obtain a target fuzzy direction; performs deblurring processing on the target image through maximum entropy regularization based on the target fuzzy direction to obtain a deblurring image; generates a to-be-recognized image based on the deblurring image; and performs recognition on the to-be-recognized image through a pre-trained ViT neural network model to obtain a classification result of the to-be-classified aerial vehicle; wherein position encoding of the ViT neural network model is generated based on direction information of an image block; and a self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of image blocks. In the application, through fuzzy direction estimation, adjustment and maximum entropy regularization processing on the target image, edge sharpening and noise suppression can be balanced, high-fidelity restoration of a degraded image can be realized, high-frequency details of the target image can be recovered, a clearer image can be provided for the ViT neural network model for classification, and classification accuracy can be improved from the perspective of image quality; and the ViT neural network model integrates direction similarity scores into the self-attention mechanism, which can force the ViT neural network model to pay attention to features consistent with the fuzzy direction and suppress interference regions inconsistent with the fuzzy direction, so that the aerial vehicle can be classified more accurately. The aerial vehicle classification system based on fuzzy direction prior provided by the application also solves the corresponding technical problems. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0063] Figure 1 A flowchart of the aerial vehicle classification method based on fuzzy direction prior provided by the embodiments of the present application;

[0064] Figure 2 A Radon transform direction extraction effect diagram;

[0065] Figure 3 A fuzzy direction estimation flowchart;

[0066] Figure 4 A ViT neural network model overall framework diagram;

[0067] Figure 5 A structure schematic diagram of the aerial vehicle classification system based on fuzzy direction prior provided by the embodiments of the present application;

[0068] Figure 6A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1.

[0069] Figure 7 Another structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0071] Please refer to Figure 1 , Figure 1 A flowchart of an aircraft classification method provided by an embodiment of the present application is shown in FIG. 3.

[0072] The aircraft classification method provided by an embodiment of the present application can include the following steps:

[0073] Step S101: Obtain a target image of an aircraft to be classified.

[0074] In actual application, the target image of the aircraft to be classified can be obtained first. The target image can be an image obtained by photographing an aircraft flying in the air, etc. The format, size of the target image, and the type of the aircraft can be determined according to the application scenario, such as the aircraft can be an airplane, a drone, etc.

[0075] Step S102: Estimate and adjust the blur direction of the target image to obtain a target blur direction.

[0076] Step S103: Based on the target blur direction, perform deblurring processing on the target image through maximum entropy regularization to obtain a deblurred image.

[0077] In actual application, after obtaining the target image, the high-frequency details in the target image can be restored through blur direction estimation and maximum entropy regularization. That is, the blur direction of the target image can be estimated and adjusted to obtain a target blur direction, such as initializing the blur direction based on aircraft orientation estimation, optimizing the angle parameter through Radon transform spectrum analysis, and improving the estimation accuracy through confidence weighted fusion; and then based on the target blur direction, the target image is deblurred through maximum entropy regularization to obtain a deblurred image, such as constructing an energy function based on anisotropic total variation constraint and maximum entropy regularization, modeling a linear motion blur kernel along the blur direction, balancing edge sharpening and noise suppression, and realizing high-fidelity restoration of the degraded image.

[0078] In the example embodiment, in the process of blur direction estimation and adjustment of the target image, the aircraft heading angle estimation of the target image can be performed to obtain the initial blur direction and the confidence of the initial blur direction, such as the output angle estimation of the visible light aerial vehicle heading estimation method based on the cyclic angle coding as the initial blur direction and the confidence thereof; and then based on the initial blur direction, the angle search range is generated, such as setting the angle search range as and the like, represents the initial blur direction, represents the range influence parameter, such as and the like; the target image is processed to obtain a target spectrum; the direction energy distribution information of the target spectrum is determined according to the angle search range, and the optimized angle is generated based on the direction energy distribution information; the initial blur direction and the optimized angle are weighted and fused based on the confidence to obtain the target blur direction, such as through obtaining the target blur direction and the like, represents the target blur direction, represents the confidence, represents the optimized angle.

[0079] In a specific application scenario, the projection curve can be used to reflect the direction energy distribution information, that is, in the process of determining the direction energy distribution information of the target spectrum according to the angle search range and generating the optimized angle based on the direction energy distribution information, the Radon projection of the target spectrum at different angles can be performed in the angle search range, and the line integral of the target spectrum along the angle is performed to generate the projection curve for reflecting the direction energy distribution; the angle corresponding to the energy peak value in the projection curve is taken as the optimized angle.

[0080] It should be noted that the Radon transform is an integral transform method, and its core idea is to project a two-dimensional image along a straight line in different directions to generate a set of projection data. Mathematically, the definition of Radon transform is:

[0081] ;

[0082] wherein, represents the original image; represents the projection angle; represents the distance from the projection straight line to the origin; the physical meaning of the Radon transform is that the integral of the image along all parallel straight lines at the angle generates the projection value at the angle. In the frequency domain, the Radon transform is closely related to the Fourier slice theorem. The theorem points out that the image at a certain angle The one-dimensional Fourier transform of the Radon projection of the original image along the angle direction is equal to the slice of the two-dimensional Fourier transform of the original image along the angle direction. This characteristic enables the Radon transform to effectively capture the directional structural features in the image, and the direction extraction effect is as shown in Figure 2

[0083] In a specific application scenario, in the blur direction estimation, the Radon transform is mainly used to extract the main direction information of the motion blur from the frequency domain, and fine-tune the initial blur direction obtained by the aircraft heading estimation. That is, in the process of processing the target image to obtain the target spectrum diagram, the target image can be subjected to a fast Fourier transform to obtain an initial spectrum diagram; a logarithmic power spectrum of the initial spectrum diagram is generated; and a Gaussian filter is applied to the logarithmic power spectrum to obtain a target spectrum diagram. At this time, the generation process of the target blur direction is as shown in Figure 3 In this way, the Radon transform can accurately extract linear or directional features in the image through multi-angle projection. In the frequency domain, motion blur appears as stripes perpendicular to the blur direction. The Radon transform can efficiently locate the main direction of the blur by analyzing the energy distribution of these stripes. Compared with schemes that rely on a single data source, such as sensors or spatial edge detection, the present embodiment is less susceptible to noise interference. In other words, the present embodiment combines frequency domain analysis and spatial prior, i.e., aircraft heading estimation, to improve the robustness of direction estimation through multi-source data fusion. While suppressing noise interference, the present embodiment can accurately capture the main direction features of motion blur, providing key parameter support for subsequent deblurring and feature enhancement.

[0084] In an example embodiment, based on the target blur direction, the target image is deblurred by maximum entropy regularization to obtain a deblurred image, in which a linear motion blur kernel is generated along the target blur direction, where the blur length in the linear motion blur kernel can be estimated by the autocorrelation function method, which locates the first negative peak position to determine by analyzing the autocorrelation decay curve of the image along the blur direction; a smoothing constraint corresponding to the target blur direction is generated, the smoothing constraint is used for weak smoothing along the target blur direction and strong smoothing along the direction perpendicular to the target blur direction, and the degree of weak and strong smoothing can be flexibly determined according to the application scenario, such as weak smoothing according to a value less than or equal to a set intensity value, strong smoothing according to a value greater than the set intensity value, etc.; a deblurring energy function of the target image is generated based on the linear motion blur kernel, the smoothing constraint, and the maximum entropy regularization; and the deblurred image is determined according to the deblurring energy function.

[0085] ​In a specific application scenario, the maximum entropy regularization originates from the maximum entropy principle in information theory, and the core is to select the probability distribution with the maximum entropy as the optimal solution under the satisfaction of the known constraint condition. In the image restoration of the present application, the maximum entropy regularization forces the restoration result to tend to a natural smooth distribution by maximizing the information entropy of the image, avoiding artifacts such as ringing effect caused by local overfitting. Mathematically, the entropy is defined as wherein is the pixel intensity, the process of maximizing the entropy is to minimize the negative entropy term, thereby prompting the pixel intensity distribution to be more uniform and the information to be maximized. The total variation (TV) regularization realizes denoising and smoothing by minimizing the L1 norm of the image gradient. The traditional TV is isotropic, which applies the same penalty to the gradients of all directions, and is easy to cause excessive smoothing of the edge. The anisotropic total variation introduces a directional differentiation constraint, and weakens the gradient penalty along the blur direction and strengthens the penalty in the vertical direction, so as to retain the direction-sensitive structural features. The dynamic blur causes the gradient to diffuse along the motion direction, and the anisotropic TV retains the edge information of the blur direction by the directional differentiation constraint. Combined with the blur direction estimation result, the anisotropic TV can closely fit the motion degradation model and improve the physical interpretability of the restoration process. Therefore, in the process of generating the deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization, the present application adopts the joint optimization of the maximum entropy regularization and the anisotropic total variation constraint to realize the balance between the degradation modeling and the detail recovery, that is, to generate the deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization;

[0086] The deblurring energy function comprises:

[0087] ;

[0088] ;

[0089] ;

[0090] wherein, denotes the target image; denotes the deblurring image; denotes the minimum value; denotes the norm; denotes the linear motion blur kernel; denotes the tensor product; denotes the set parameter; denotes the smoothing constraint; denotes the set parameter; denotes the maximum entropy regularization; denotes the parameter for emphasizing the edge gradient; denotes the parameter for suppressing the noise and background interference; denotes taking an absolute value; denotes a gradient parallel to the target blur direction; denotes a gradient perpendicular to the target blur direction.

[0091] In this way, the maximum entropy regularization term forces the restored image to tend to be strongly smoothed in the perpendicular direction, avoiding local overfitting. In dynamic blur restoration, the ringing artifact caused by inaccurate blur kernel estimation is reduced by global smoothing constraint, thereby reducing the ringing effect. In addition, the maximum entropy maximizes the pixel intensity distribution to be closer to the statistical characteristics of natural scenes, thereby improving the visual rationality and separability of the restored image. Finally, combined with the directional constraint of anisotropic total variation, the maximum entropy regularization preserves key discriminative details while suppressing background interference.

[0092] Step S104: generating a to-be-identified image based on the deblurred image.

[0093] In actual applications, considering that the image information relied on in the classification and identification process of the aircraft may be different, after obtaining the deblurred image, a to-be-identified image can be generated based on the deblurred image, so as to subsequently apply the to-be-identified image to classify the aircraft.

[0094] In an example embodiment, in the process of generating the to-be-identified image based on the deblurred image, a target slice of the deblurred image can be determined, such as taking the wake area of the aircraft as the target slice; a blur kernel length is extracted from the linear motion blur kernel; an expansion parameter is generated based on the blur kernel length and an expansion coefficient, such as taking the product of the blur kernel length and the expansion coefficient as the expansion parameter; the target slice is expanded according to the expansion parameter to obtain an expanded slice, such as adding the expansion parameter to the target slice to obtain the expanded slice; the deblurred image is sliced according to the expanded slice to obtain a sliced image; and the to-be-identified image is generated according to the deblurred image and the sliced image. In this way, the expansion direction is along the main direction of the blur, ensuring that the linear structure of the slice is preserved to be continuous with the blur trail, and the slice can be decoupled from the background interference to become an auxiliary feature for classification rather than an interference source. For example, the wake diffusion patterns of aircrafts are different, and dynamic slice adjustment can capture such physical identifiers to improve the discriminability of fine-grained classification.

[0095] Step S105: identifying the to-be-identified image through a pre-trained ViT neural network model to obtain a classification result of the to-be-classified aircraft; wherein the position encoding of the ViT neural network model is generated based on the directional information of the image block; and the self-attention mechanism of the ViT neural network model is generated based on the similarity between the directional angles of the image blocks.

[0096] In practical applications, after obtaining the image to be identified, the pre-trained ViT neural network model can be used to identify the image to be identified and obtain the classification result of the aircraft to be classified. The overall framework of the ViT neural network model can be as follows: Figure 4 shown.

[0097] It should be noted that the standard ViT position encoding assigns fixed spatial position information to each image block via a learnable vector, but it lacks modeling of local directional characteristics. In scenarios such as motion blur and trail interference, features such as the linear structure of the trail and the directional continuity of the blurred smear have strong spatial directional correlations. Therefore, this application incorporates directional angle information of local blocks to embed physical priors, such as the blur direction and even the trail diffusion direction, into the position representation, enhancing the model's ability to perceive direction-sensitive structures. In other words, the position encoding of the ViT neural network model in this application is generated based on the directional information of the image blocks. Meanwhile, the standard self-attention mechanism achieves global feature interaction by calculating similarity weights between the query and the key. However, in scenarios with motion blur and cloud interference, high-contrast interference areas may be misclassified as key features. To avoid this, this application introduces directional similarity constraints in the direction-guided self-attention, forcing the model to focus on features consistent with the blur direction, such as the trail and the fuselage outline, while suppressing interference areas with inconsistent directions. In other words, the self-attention mechanism of the ViT neural network model in this application is generated based on the similarity between the directional angles of image blocks.

[0098] In an exemplary embodiment, the fuzzy direction prior can be embedded into the global modeling of ViT through direction-sensitive position encoding and direction-guided self-attention, thereby guiding the model to focus on the trail area with strong directional characteristics. First, ViT uses a learnable position encoding to assign a standard position vector to each image block, which is then mapped to the same dimension as the standard position vector and fused to obtain the final position encoding. That is, the generation formula of the position encoding of the ViT neural network model includes:

[0099] ;

[0100] ;

[0101] in, Indicates the The final position encoding of the image blocks; Indicates the number of the image block; Expressed as The standard position vector assigned to each image block; Indicates the The extraction direction is obtained by extracting the center position of each image block; Indicates the The vector obtained by encoding the extraction direction of each image block; Indicates setting parameters; Represents cosine operation; Represents sine operation; Indicates setting weight;

[0102] In this way, the linear structure and blurred shadows of the objects in the slice extend along a specific direction. Directional encoding makes adjacent image blocks with the same direction have similar position representations, enhancing their semantic consistency. However, the directional angle distribution of background areas, such as clouds, is irregular. Directional encoding can reduce their similarity with the target area and reduce false activation, thereby distinguishing the fuselage contour with consistent direction from the background interference with random wind direction, and improving the separability of the feature space.

[0103] In an exemplary embodiment, a generation formula of the self-attention mechanism of the ViT neural network model may include:

[0104] ;

[0105] ;

[0106] in, Indicates the image blocks and the The self-attention mechanism value of each image block gives adjacent tokens with the same direction a higher attention weight, and the model pays more attention to continuous structures along the fuzzy direction, such as airplane trails; Indicates the number of the image block; represents the normalized exponential function; represents the query vector; represents the key vector; represents the scaling factor used to stabilize gradient propagation; for Dimensions; Represents the direction penalty coefficient used to balance semantic similarity and direction consistency, controlling the strength of the direction prior; Represents the similarity score of the direction angle difference. When the directions of the two image blocks are consistent, that is, , the score approaches 1, and the greater the direction difference, the lower the score; represents the sensitivity coefficient, which is used to control the penalty intensity of direction difference; represents an exponential function with a natural constant as its base; Indicates the The extraction direction is obtained by extracting the center position of each image block; Indicates taking the absolute value;

[0107] In this way, the ViT neural network model aggregates features along consistent directions, for example, image patches in the tail region enhance interaction through high directional scores, forming a coherent response pattern, while low scores in the background region suppress their participation in feature updates. In other words, by introducing a directional similarity constraint in the self-attention mechanism, the model is forced to focus on structures consistent with the ambiguous direction and tail diffusion direction. High-contrast regions in the background, such as cloud edges, are weakened in attention weights due to low directional scores. The directional similarity score explicitly embeds the motion degradation prior, giving the attention mechanism a physical meaning rather than being purely data-driven.

[0108] It should be noted that Vision Transformer (ViT) is a model that innovatively applies the Transformer architecture in natural language processing to computer vision tasks. Its core principle is to replace traditional convolution operations with a global self-attention mechanism to achieve global modeling of image features. The introduction of ViT marks the unification of visual and language model architectures, and subsequent work continues to promote its application in detection, segmentation, and other tasks through hierarchical design, hybrid architecture (such as CNN+ViT), and efficient attention mechanisms. The following analyzes its principles in five aspects:

[0109] 1) Image patching and embedding (Patch Embedding): ViT divides the input image into fixed-size non-overlapping blocks (such as 16x16 pixels), and each block is flattened and mapped to a low-dimensional vector (e.g., 768 dimensions) through a linear projection, forming a sequence of word embeddings similar to NLP. For example, a 224x224 RGB image is divided into 196 blocks, each block is flattened to 16x16x3=768, and then compressed to an embedding vector through a fully connected layer. This process converts the image into sequential data, laying the foundation for subsequent Transformer processing.

[0110] 2) Positional encoding (Positional Encoding): Since the Transformer itself does not have spatial perception capabilities, ViT introduces a learnable one-dimensional position encoding to encode the two-dimensional spatial relationship between image blocks into a vector and add it to the embedding vector. Positional encoding automatically learns the topological structure between blocks, such as the distance between adjacent blocks and the global layout, through training. Although images are inherently two-dimensional, experiments have shown that one-dimensional encoding is sufficient.

[0111] 3) Transformer encoder architecture: The core of ViT is stacked with multiple layers of Transformer encoder, each of which contains two key modules: Multi-head self-attention (MSA): The inter-block association weights are calculated by multiple sets of parallel attention heads (such as 12 heads), which can capture global dependencies. For example, some attention heads at the bottom can focus on most areas of the image, while other heads remain locally focused, achieving multi-scale feature fusion. Feed-forward network (FFN): composed of two fully connected layers and GELU activation function, which performs nonlinear transformation on the attention output. Layer normalization (LayerNorm) and residual connection are applied after each module to accelerate training convergence.

[0112] 4) Classification head and special token: ViT adds a learnable classification token ([class]token) before the input sequence, and the output vector after passing through all the encoding layers is used as the overall representation of the image. Finally, the MLP classification head is used to output the prediction result. This design is inspired by the [CLS] token in BERT, which aggregates global information for classification and avoids the information loss of traditional pooling operations.

[0113] 5) Training strategy and improvement direction: Supervised pre-training: ViT needs to be pre-trained on a large-scale dataset (such as JFT-300M) before migrating to downstream tasks. Studies have shown that when the pre-training data is sufficient, the accuracy of ViT on ImageNet can reach 88.55%, surpassing the CNN models of the same period. Self-supervised learning: Subsequent research introduces mask image modeling (such as MAE, BEiT), which improves the model's generalization ability by reconstructing the blocked blocks. For example, MAE randomly masks 75% of the blocks and uses the remaining blocks to reconstruct the pixels, achieving an accuracy of 83.6% on ImageNet. Efficiency optimization: To address the high computational complexity (O(N2)) of ViT, improved models such as Swin Transformer introduce local window attention, reducing computational complexity while maintaining multi-scale features.

[0114] It is also worth mentioning that the Transformer is a deep learning architecture based on self-attention mechanisms, which has achieved great success in natural language processing (NLP) and other fields. The Transformer has high parallel computing capability, as the attention mechanism calculation can be performed in parallel, making the Transformer model highly efficient and capable of parallel processing. It also has strong representation ability, as the Transformer model can effectively capture global information of input data, thus achieving significant performance improvement in natural language processing tasks such as language modeling and translation. The Transformer model has been widely applied to various tasks, including but not limited to: language translation, text generation, dialogue systems, language models, image processing, recommendation systems, etc. In summary, the Transformer is a powerful deep learning model that has greatly promoted the development of natural language processing and other fields. The core concepts of the Transformer include attention mechanisms, encoder-decoder architecture, and multi-head attention. Here is an introduction to the Transformer:

[0115] 1) Attention mechanism is an important concept in neural networks that allows the model to focus on certain parts of the input sequence while ignoring others. In the Transformer, self-attention mechanism, also known as internal attention, is used. Self-attention mechanism enables the model to automatically highlight important words or phrases to understand the meaning of a sentence.

[0116] 2) The Transformer consists of two parts: the encoder (Encoder) and the decoder (Decoder). The encoder is responsible for processing the input sequence, while the decoder generates the target sequence based on the output of the encoder. Each encoder and decoder is stacked with multiple identical layers (blocks), usually 6 layers.

[0117] 3) Multi-head attention is a key innovation in the Transformer, which allows the model to learn attention distributions in different representation subspaces. This means that the model can focus on different positions of the input sequence simultaneously, capturing richer contextual information.

[0118] The application provides an aerial vehicle classification method based on fuzzy direction prior. The target image of the vehicle to be classified is obtained. The fuzzy direction of the target image is estimated and adjusted to obtain the target fuzzy direction. Based on the target fuzzy direction, the target image is deblurred by maximum entropy regularization to obtain a deblurred image. The deblurred image is used to generate an image to be recognized. The ViT neural network model is pre-trained to recognize the image to be recognized, and the classification result of the vehicle to be classified is obtained. The position encoding of the ViT neural network model is generated based on the direction information of the image block. The self-attention mechanism of the ViT neural network model is generated based on the similarity between the direction angles of the image blocks. In the application, the fuzzy direction of the target image is estimated, adjusted and processed by maximum entropy regularization, which can balance edge sharpening and noise suppression, realize high-fidelity restoration of the degraded image, and restore the high-frequency details of the target image, so as to provide a clearer image for the ViT neural network model for classification, which can improve the classification accuracy from the image quality angle. The ViT neural network model integrates the direction similarity score into the self-attention mechanism, which can force the ViT neural network model to focus on the features consistent with the fuzzy direction and suppress the interference area inconsistent with the fuzzy direction, so that the aerial vehicle can be classified more accurately.

[0119] In order to facilitate understanding of the effect of the aerial vehicle classification method based on fuzzy direction prior of the application, it is assumed that the aerial vehicle is an airplane, and the effect of the application is reflected by experimental comparison. The experimental data set is composed of visible light aerial vehicle images, covering 64 fine-grained sub-classes of passenger aircraft. Among them, the civil aviation aircraft includes Boeing 737-800, Airbus A320 and other modifications. The experimental data set is divided into a training set (60%), a validation set (20%) and a test set (20%) in the ratio of 6:2:2 by using a random division strategy. The batch size is set to 16. The experiment is run under a Linux-based operating system, and the algorithm is realized by relying on the PyTorch deep learning framework.

[0120] To verify the effectiveness of the aircraft fine-grained classification method based on the fuzzy direction prior of the present application, a systematic comparison is made with various state-of-the-art classification frameworks in the field of fine-grained classification. Including NTS-Net, S3N, FDL, LIO, PMG, DP-Net, GaRD, SPS, P2P-Net, ViT, TransFG, DCAL and other methods. These comparison models not only cover methods based on region positioning and attention mechanism, methods based on double path and feature decoupling, but also include Transformer-based architecture. And the selected comparison algorithms have been verified on the fine-grained classification public dataset, as much as possible to ensure the persuasiveness of the comparison experiment. Table 1 shows the classification accuracy of different fine-grained classification methods on the visible light aerial vehicle classification dataset. Table 2 shows the performance comparison of different deblurring methods combined with the ViT architecture in the classification task.

[0121] Table 1 Performance comparison experiment result table

[0122]

[0123] Table 2 Comparison experiment result table of different deblurring methods

[0124]

[0125] From Table 1, the method proposed in the present application is significantly better than all comparison methods with an accuracy of 91.8%, which is 6.9 percentage points higher than the second place DCAL (84.9%). From Table 2, the method of the present application has an accuracy of 91.8%, an inference time of 36ms and an FLOPs of 81.4G, which is better than other schemes in terms of accuracy and efficiency balance.

[0126] Please refer to Figure 5 , Figure 5 A structure diagram of an aerial vehicle classification system based on a fuzzy direction prior provided by an embodiment of the present application.

[0127] An aerial vehicle classification system based on a fuzzy direction prior provided by an embodiment of the present application can include:

[0128] A target image acquisition module 101 is configured to acquire a target image of an aircraft to be classified;

[0129] A target blur direction generation module 102 is configured to estimate and adjust the blur direction of the target image to obtain a target blur direction;

[0130] A deblurring module 103 is configured to perform deblurring processing on the target image based on the target blur direction through maximum entropy regularization to obtain a deblurred image;

[0131] The to-be-identified image generation module 104 is configured to generate a to-be-identified image based on the deblurred image;

[0132] The identification module 105 is configured to identify the to-be-identified image by using the pre-trained ViT neural network model, and obtain a classification result of the to-be-classified aircraft.

[0133] The position encoding of the ViT neural network model is generated based on direction information of the image block; and the self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of the image blocks.

[0134] The aerial vehicle classification system based on the blurred direction prior provided in the embodiments of the present application can include:

[0135] The estimation unit is configured to estimate an aircraft heading angle of the target image, and obtain an initial blurred direction and a confidence degree of the initial blurred direction.

[0136] The range generation unit is configured to generate an angle search range based on the initial blurred direction.

[0137] The processing unit is configured to process the target image, and obtain a target spectrum.

[0138] The angle determination unit is configured to determine direction energy distribution information of the target spectrum according to the angle search range, and generate an optimized angle based on the direction energy distribution information.

[0139] The fusion unit is configured to perform weighted fusion on the initial blurred direction and the optimized angle based on the confidence degree, and obtain a target blurred direction.

[0140] The aerial vehicle classification system based on the blurred direction prior provided in the embodiments of the present application can include:

[0141] The processing unit can be configured to perform fast Fourier transform on the target image, and obtain an initial spectrum; generate a logarithmic power spectrum of the initial spectrum; and perform Gaussian filtering on the logarithmic power spectrum, and obtain the target spectrum.

[0142] The aerial vehicle classification system provided in the embodiments of the present application can include:

[0143] The blurred kernel generation unit is configured to generate a linear motion blurred kernel along the target blurred direction.

[0144] The constraint unit is configured to generate a smoothing constraint corresponding to the target blur direction, and the smoothing constraint is configured to perform weak smoothing along the target blur direction and strong smoothing along a direction perpendicular to the target blur direction.

[0145] The function generation unit is configured to generate a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint, and maximum entropy regularization.

[0146] The deblurring unit is configured to determine a deblurred image according to the deblurring energy function.

[0147] The function generation unit can be configured to:

[0148] generate a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint, and maximum entropy regularization.

[0149] The deblurring energy function comprises:

[0150]

[0151]

[0152]

[0153] wherein, represents the target image; represents the deblurred image; represents taking a minimum value; represents a norm; represents the linear motion blur kernel; represents a tensor product; represents a set parameter; represents the smoothing constraint; represents a set parameter; represents the maximum entropy regularization; represents a parameter for emphasizing an edge gradient; represents a parameter for suppressing noise and background interference; represents taking an absolute value; represents a gradient parallel to the target blur direction; represents a gradient perpendicular to the target blur direction.

[0154] The function generation unit can be configured to:

[0155] The slice determination unit is configured to determine a target slice of the deblurred image;

[0156] The extraction unit is configured to extract a blur kernel length from the linear motion blur kernel.​​​

[0157] an expansion parameter generation unit, configured to generate an expansion parameter based on the blur kernel length and the expansion coefficient;

[0158] a slice expansion unit, configured to expand the target slice according to the expansion parameter to obtain an expanded slice;

[0159] a slice unit, configured to slice the deblurred image according to the expanded slice to obtain a sliced image;

[0160] a to-be-recognized image generation unit, configured to generate a to-be-recognized image according to the deblurred image and the sliced image.

[0161] The position encoding generation formula of the ViT neural network model provided in the embodiment of the application includes:

[0162]

[0163]

[0164] The self-attention mechanism generation formula of the ViT neural network model includes:

[0165]

[0166]

[0167] wherein, denotes the final position encoding of an i-th image block; denotes the number of the image block; denotes a standard position vector assigned to the i-th image block; denotes an extraction direction obtained by extracting the center position of the i-th image block; denotes a vector obtained by encoding the extraction direction of the i-th image block; denotes a set parameter; denotes a cosine operation; denotes a sine operation; denotes a set weight; denotes a self-attention mechanism value of an i-th image block and a j-th image block; denotes the number of the image block; denotes a normalized exponential function; denotes a query vector; denotes a key vector; ​​​​​​​​​​denotes a scaling factor for stabilizing gradient propagation; denotes a dimension of the image block; denotes a dimension of the image block; denotes a direction penalty coefficient for balancing semantic similarity and direction consistency; denotes a similarity score of the difference of direction angles; denotes a sensitivity coefficient; denotes an exponential function with a natural constant as a base; denotes an extracted direction after extracting a center position of the i-th image block; denotes taking an absolute value.

[0168] The application further provides an electronic device and a computer readable storage medium, both of which have the corresponding effects of the aerial vehicle classification method based on the fuzzy direction prior provided in the embodiments of the application. Please refer to Figure 6 , Figure 6 which is a structural schematic diagram of an electronic device provided in the embodiments of the application.

[0169] The electronic device provided in the embodiments of the application comprises a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the steps of the aerial vehicle classification method based on the fuzzy direction prior when executing the computer program.

[0170] Please refer to Figure 7 , the other electronic device provided in the embodiments of the application can further comprise: an input port 203 connected with the processor 202, used for transmitting an input command from the outside to the processor 202; a display unit 204 connected with the processor 202, used for displaying the processing result of the processor 202 to the outside; and a communication module 205 connected with the processor 202, used for realizing the communication between the electronic device and the outside. The display unit 204 can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module 205 comprises but is not limited to a mobile high-definition link technology (MHL), a universal serial bus (USB), a high-definition multimedia interface (HDMI), wireless connection: wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE 802.11s.

[0171] ​The computer readable storage medium provided in the embodiment of the present application stores a computer program, and the computer program is executed by a processor to implement the steps of the method for classifying aerial vehicles based on fuzzy direction priori described in any of the above embodiments.

[0172] The computer readable storage medium involved in the present application includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium known in the technical field.

[0173] The system for classifying aerial vehicles based on fuzzy direction priori, the electronic device, and the related part of the computer readable storage medium provided in the embodiment of the present application are described in detail in the corresponding part of the method for classifying aerial vehicles based on fuzzy direction priori provided in the embodiment of the present application, which will not be described here. In addition, the part of the above technical solution provided in the embodiment of the present application which is consistent with the implementation principle of the corresponding technical solution in the prior art is not described in detail, so as not to be too verbose.

[0174] It should also be noted that, in this document, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms “include”, “contain” or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement “including a…” does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0175] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for classifying aerial vehicles based on fuzzy direction priors, characterized in that: include: Acquire a target image of the aircraft to be classified; performing blur direction estimation and adjustment on the target image to obtain the target blur direction; Based on the target blur direction, deblurring the target image by maximum entropy regularization to obtain a deblurred image; generating an image to be recognized based on the deblurred image; Identify the image to be identified using a pre-trained ViT neural network model to obtain a classification result of the aircraft to be classified; Among them, the position encoding of the ViT neural network model is generated based on the directional information of the image block; the self-attention mechanism of the ViT neural network model is generated based on the similarity between the directional angles of the image blocks.

2. The method according to claim 1, characterized in that The step of estimating and adjusting the blur direction of the target image to obtain the target blur direction includes: Estimating the aircraft heading angle of the target image to obtain an initial blur direction and a confidence level of the initial blur direction; Based on the initial blur direction, generating an angle search range; Processing the target image to obtain a target spectrum graph; Determining directional energy distribution information of the target spectrum according to the angle search range, and generating an optimized angle based on the directional energy distribution information; Based on the confidence level, the initial blur direction and the optimized angle are weightedly fused to obtain a target blur direction.

3. The method according to claim 2, characterized in that Determining the directional energy distribution information of the target spectrum according to the angle search range, and generating an optimized angle based on the directional energy distribution information, includes: Within the angle search range, performing Radon projection on the target spectrum at different angles, and performing line integration on the target spectrum along the angle to generate a projection curve reflecting directional energy distribution; The angle corresponding to the energy peak in the projection curve is used as the optimization angle.

4. The method according to claim 2, characterized in that The processing of the target image to obtain a target spectrum graph includes: Performing a fast Fourier transform on the target image to obtain an initial spectrum graph; generating a logarithmic power spectrum of the initial spectrogram; Gaussian filtering is performed on the logarithmic power spectrum to obtain a target spectrum graph.

5. The method according to claim 2, characterized in that The step of performing a deblurring process on the target image by maximum entropy regularization based on the target blur direction to obtain a deblurred image includes: generating a linear motion blur kernel along the target blur direction; generating a smoothing constraint corresponding to the target blur direction, wherein the smoothing constraint is used to perform weak smoothing along the target blur direction and strong smoothing along a direction perpendicular to the target blur direction; generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothness constraint, and the maximum entropy regularization; The deblurred image is determined according to the deblurring energy function.

6. The method according to claim 5, characterized in that The generating of the deblurring energy function of the target image based on the linear motion blur kernel, the smoothness constraint and the maximum entropy regularization comprises: generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothness constraint, and the maximum entropy regularization; The deblurring energy function includes: ； ； ； in, represents the target image; represents the deblurred image; Indicates taking the minimum value; represents the norm; represents the linear motion blur kernel; represents the tensor product; Indicates setting parameters; represents the smoothness constraint; Indicates setting parameters; represents the maximum entropy regularization; Indicates the parameter for emphasizing edge gradient; Represents the parameters for suppressing noise and background interference; Indicates taking the absolute value; represents the gradient parallel to the target blur direction; Represents the gradient perpendicular to the target blur direction.

7. The method according to claim 6, characterized in that Generating an image to be recognized based on the deblurred image includes: determining a target slice of the deblurred image; extracting a blur kernel length from the linear motion blur kernel; generating an expansion parameter based on the blur kernel length and the expansion coefficient; Expanding the target slice according to the expansion parameter to obtain an expanded slice; Slicing the deblurred image according to the extended slices to obtain a sliced ​​image; An image to be recognized is generated according to the deblurred image and the slice image.

8. The method according to any one of claims 1 to 7, characterized in that The generation formula of the position encoding of the ViT neural network model includes: ； ； in, Indicates the The final position encoding of the image blocks; Indicates the number of the image block; Expressed as The standard position vector assigned to each image block; Indicates the The extraction direction is obtained by extracting the center position of each image block; Indicates the The vector obtained by encoding the extraction direction of each image block; Indicates setting parameters; Represents cosine operation; Represents sine operation; Indicates setting weight.

9. The method according to claim 8, characterized in that The generation formula of the self-attention mechanism of the ViT neural network model includes: ； ； in, Indicates the image blocks and the The self-attention mechanism value of each image patch; Indicates the number of the image block; represents the normalized exponential function; represents the query vector; represents the key vector; represents the scaling factor used to stabilize gradient propagation; for Dimensions; represents the direction penalty coefficient used to balance semantic similarity and direction consistency; Similarity score representing the difference in direction angles; represents the sensitivity coefficient; represents an exponential function with a natural constant as its base; Indicates the The extraction direction is obtained by extracting the center position of each image block; Indicates taking the absolute value.

10. A system for classifying aerial vehicles based on fuzzy direction priors, characterized in that: include: A target image acquisition module is used to acquire a target image of the aircraft to be classified; A target blur direction generating module is used to estimate and adjust the blur direction of the target image to obtain the target blur direction; a deblurring module, configured to perform a deblurring process on the target image by maximum entropy regularization based on the target blur direction to obtain a deblurred image; a module for generating an image to be identified, configured to generate an image to be identified based on the deblurred image; A recognition module is used to recognize the image to be recognized using a pre-trained ViT neural network model to obtain a classification result of the aircraft to be classified; Among them, the position encoding of the ViT neural network model is generated based on the directional information of the image block; the self-attention mechanism of the ViT neural network model is generated based on the similarity between the directional angles of the image blocks.

Images