An aerial vehicle classification method and system based on fuzzy direction prior

Abstract

The application discloses an aerial vehicle classification method and system based on a fuzzy direction prior, relates to the technical field of image processing, and obtains a target image of a to-be-classified aerial vehicle; fuzzy direction estimation and adjustment are performed on the target image to obtain a target fuzzy direction; based on the target fuzzy direction, the target image is deblurred through maximum entropy regularization to obtain a deblurred image; a to-be-recognized image is generated based on the deblurred image; the to-be-recognized image is recognized through a pre-trained ViT neural network model to obtain a classification result of the to-be-classified aerial vehicle; wherein, position coding of the ViT neural network model is generated based on direction information of an image block; and a self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of image blocks. The application can realize high-fidelity restoration of degraded images; and forces the ViT neural network model to pay attention to features consistent with the fuzzy direction, so that the aerial vehicle is more accurately classified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, more particularly to an aerial vehicle classification method and system based on fuzzy direction prior. BACKGROUND

[0002] Aerial vehicle classification is a technology for accurately distinguishing flying vehicle targets in images. Its core is to identify the model by analyzing the local structural features, painting marks and dynamic behavior of the vehicle.

[0003] However, the differences between aircraft classes may only exist in local subtle features, and traditional models are easily disturbed by the main features of the fuselage, resulting in loss of key discriminative information. Moreover, aerial targets are often in motion, and the dynamic blur and complex background interference of the obtained aircraft images can seriously damage the recognizability of the features, resulting in low accuracy of aircraft classification.

[0004] In summary, how to improve the classification accuracy of aircraft is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] The purpose of the present application is to provide an aerial vehicle classification method based on fuzzy direction prior, which can solve the technical problem of how to improve the classification accuracy of aircraft to some extent. The present application also provides an aerial vehicle classification system based on fuzzy direction prior.

[0006] In order to achieve the above purpose, the present application provides the following technical solution:

[0007] An aerial vehicle classification method based on fuzzy direction prior, comprising:

[0008] obtaining a target image of a vehicle to be classified;

[0009] estimating and adjusting the fuzzy direction of the target image to obtain a target fuzzy direction;

[0010] Based on the target fuzzy direction, the target image is deblurred by maximum entropy regularization to obtain a deblurred image;

[0011] Generating a to-be-identified image based on the deblurred image;

[0012] Identifying the to-be-identified image through a pre-trained ViT neural network model to obtain a classification result of the vehicle to be classified;

[0013] Wherein, the position encoding of the ViT neural network model is generated based on the direction information of the image block; and the self-attention mechanism of the ViT neural network model is generated based on the similarity between the direction angles of the image blocks.

[0014] In an example embodiment, the blurring direction estimation and adjustment on the target image to obtain a target blurring direction comprises:

[0015] an aircraft orientation angle estimation on the target image to obtain an initial blurring direction and a confidence of the initial blurring direction;

[0016] generating an angle search range based on the initial blurring direction;

[0017] processing the target image to obtain a target spectrum map;

[0018] determining a directional energy distribution information of the target spectrum map according to the angle search range, and generating an optimized angle based on the directional energy distribution information;

[0019] weighting and fusing the initial blurring direction and the optimized angle based on the confidence to obtain a target blurring direction.

[0020] In an example embodiment, the determining a directional energy distribution information of the target spectrum map according to the angle search range, and generating an optimized angle based on the directional energy distribution information comprises:

[0021] performing Radon projection on the target spectrum map at different angles within the angle search range, and performing line integration on the target spectrum map along the angle to generate a projection curve reflecting the directional energy distribution;

[0022] taking an angle corresponding to an energy peak in the projection curve as the optimized angle.

[0023] In an example embodiment, the processing the target image to obtain a target spectrum map comprises:

[0024] performing fast Fourier transform on the target image to obtain an initial spectrum map;

[0025] generating a logarithmic power spectrum of the initial spectrum map;

[0026] performing Gaussian filtering on the logarithmic power spectrum to obtain a target spectrum map.

[0027] In an example embodiment, the deblurring processing on the target image based on the target blurring direction through maximum entropy regularization to obtain a deblurred image comprises:

[0028] generating a linear motion blur kernel along the target blurring direction;

[0029] generating a smoothing constraint corresponding to the target blur direction, the smoothing constraint being used for weak smoothing along the target blur direction and strong smoothing along a direction perpendicular to the target blur direction;

[0030] generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization;

[0031] determining the deblurred image according to the deblurring energy function.

[0032] In an exemplary embodiment, the generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization comprises:

[0033] generating a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization;

[0034] the deblurring energy function comprises:

[0035] ;

[0036] ;

[0037] ;

[0038] wherein, represents the target image; represents the deblurred image; represents taking minimum value; represents norm; represents the linear motion blur kernel; represents tensor product; represents setting parameter; represents the smoothing constraint; represents setting parameter; represents the maximum entropy regularization; represents parameter emphasizing edge gradient; represents parameter suppressing noise and background interference; represents taking absolute value; represents gradient parallel to the target blur direction; represents gradient perpendicular to the target blur direction.

[0039] In an exemplary embodiment, the generating a deblurred image based on the deblurred image comprises:

[0040] determining a target slice of the deblurred image;

[0041] Extract the fuzzy kernel length from the linear motion fuzzy kernel;

[0042] Based on the fuzzy kernel length and the expansion coefficient, the expansion parameters are generated;

[0043] The target slice is expanded according to the expansion parameters to obtain an expanded slice;

[0044] The deblurred image is sliced ​​according to the extended slice to obtain a sliced ​​image;

[0045] An image to be identified is generated based on the deblurred image and the sliced ​​image.

[0046] In one exemplary embodiment, the formula for generating the positional encoding of the ViT neural network model includes:

[0047] ;

[0048] ;

[0049] in, Indicates the first The final position encoding of each image patch; Indicates the number of the image patch; Represented as the first Standard location vectors assigned to each image patch; Indicates the first The extraction direction is obtained by extracting the center position of each image patch; Indicates the first The vector obtained by encoding the extraction direction of each image patch; Indicates setting parameters; Represents the cosine operation; Represents the sine operation; This indicates that a weight is set.

[0050] In one exemplary embodiment, the formula for generating the self-attention mechanism of the ViT neural network model includes:

[0051] ;

[0052] ;

[0053] in, Indicates the first The image patch and the first The self-attention mechanism value of each image patch; Indicates the number of the image patch; This represents the normalized exponential function; Represents the query vector; Represents the key vector; denotes a scaling factor for stabilizing gradient propagation; is dimensional; denotes a direction penalty coefficient for balancing semantic similarity and direction consistency; denotes a similarity score of direction angle difference; denotes a sensitivity coefficient; denotes an exponential function with a natural constant as base; denotes an extracted direction after extracting the center position of the image block; denotes taking an absolute value.

[0054] An aerial vehicle classification system based on fuzzy direction prior, comprising:

[0055] a target image acquisition module, configured to acquire a target image of a vehicle to be classified;

[0056] a target fuzzy direction generation module, configured to perform fuzzy direction estimation and adjustment on the target image to obtain a target fuzzy direction;

[0057] a deblurring module, configured to perform deblurring processing on the target image based on the target fuzzy direction through maximum entropy regularization to obtain a deblurred image;

[0058] a to-be-recognized image generation module, configured to generate a to-be-recognized image based on the deblurred image;

[0059] a recognition module, configured to recognize the to-be-recognized image through a pre-trained ViT neural network model to obtain a classification result of the vehicle to be classified;

[0060] wherein, the position encoding of the ViT neural network model is generated based on direction information of an image block; and the self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of image blocks.

[0061] The application provides an aerial vehicle classification method based on fuzzy direction prior, obtains a target image of a to-be-classified aerial vehicle; performs fuzzy direction estimation and adjustment on the target image to obtain a target fuzzy direction; performs deblurring processing on the target image through maximum entropy regularization based on the target fuzzy direction to obtain a deblurring image; generates a to-be-recognized image based on the deblurring image; and performs recognition on the to-be-recognized image through a pre-trained ViT neural network model to obtain a classification result of the to-be-classified aerial vehicle; wherein position encoding of the ViT neural network model is generated based on direction information of an image block; and a self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of image blocks. In the application, through fuzzy direction estimation, adjustment and maximum entropy regularization processing on the target image, edge sharpening and noise suppression can be balanced, high-fidelity restoration of a degraded image can be realized, high-frequency details of the target image can be recovered, a clearer image can be provided for the ViT neural network model for classification, and classification accuracy can be improved from the perspective of image quality; and the ViT neural network model integrates direction similarity scores into the self-attention mechanism, which can force the ViT neural network model to pay attention to features consistent with the fuzzy direction and suppress interference regions inconsistent with the fuzzy direction, so that the aerial vehicle can be classified more accurately. The aerial vehicle classification system based on fuzzy direction prior provided by the application also solves the corresponding technical problems. BRIEF DESCRIPTION OF DRAWINGS

[0062] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0063] Figure 1 A flowchart of the aerial vehicle classification method based on fuzzy direction prior provided by the embodiments of the present application;

[0064] Figure 2 A Radon transform direction extraction effect diagram;

[0065] Figure 3 A fuzzy direction estimation flowchart;

[0066] Figure 4 A ViT neural network model overall framework diagram;

[0067] Figure 5 A structure schematic diagram of the aerial vehicle classification system based on fuzzy direction prior provided by the embodiments of the present application;

[0068] Figure 6A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1.

[0069] Figure 7 Another structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 2. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0071] Please refer to Figure 1 , Figure 1 A flowchart of an aircraft classification method provided by an embodiment of the present application is shown in FIG. 3.

[0072] The aircraft classification method provided by an embodiment of the present application can include the following steps:

[0073] Step S101: Obtain a target image of an aircraft to be classified.

[0074] In actual application, the target image of the aircraft to be classified can be obtained first. The target image can be an image obtained by photographing an aircraft flying in the air, etc. The format, size of the target image, and the type of the aircraft can be determined according to the application scenario, such as the aircraft can be an airplane, a drone, etc.

[0075] Step S102: Estimate and adjust the blur direction of the target image to obtain a target blur direction.

[0076] Step S103: Based on the target blur direction, perform deblurring processing on the target image through maximum entropy regularization to obtain a deblurred image.

[0077] In actual application, after obtaining the target image, the high-frequency details in the target image can be restored through blur direction estimation and maximum entropy regularization. That is, the blur direction of the target image can be estimated and adjusted to obtain a target blur direction, such as initializing the blur direction based on aircraft orientation estimation, optimizing the angle parameter through Radon transform spectrum analysis, and improving the estimation accuracy through confidence weighted fusion; and then based on the target blur direction, the target image is deblurred through maximum entropy regularization to obtain a deblurred image, such as constructing an energy function based on anisotropic total variation constraint and maximum entropy regularization, modeling a linear motion blur kernel along the blur direction, balancing edge sharpening and noise suppression, and realizing high-fidelity restoration of the degraded image.

[0078] In the example embodiment, in the process of blur direction estimation and adjustment of the target image, the aircraft heading angle estimation of the target image can be performed to obtain the initial blur direction and the confidence of the initial blur direction, such as the output angle estimation of the visible light aerial vehicle heading estimation method based on the cyclic angle coding as the initial blur direction and the confidence thereof; and then based on the initial blur direction, the angle search range is generated, such as setting the angle search range as and the like, represents the initial blur direction, represents the range influence parameter, such as and the like; the target image is processed to obtain a target spectrum; the direction energy distribution information of the target spectrum is determined according to the angle search range, and the optimized angle is generated based on the direction energy distribution information; the initial blur direction and the optimized angle are weighted and fused based on the confidence to obtain the target blur direction, such as through obtaining the target blur direction and the like, represents the target blur direction, represents the confidence, represents the optimized angle.

[0079] In a specific application scenario, the projection curve can be used to reflect the direction energy distribution information, that is, in the process of determining the direction energy distribution information of the target spectrum according to the angle search range and generating the optimized angle based on the direction energy distribution information, the Radon projection of the target spectrum at different angles can be performed in the angle search range, and the line integral of the target spectrum along the angle is performed to generate the projection curve for reflecting the direction energy distribution; the angle corresponding to the energy peak value in the projection curve is taken as the optimized angle.

[0080] It should be noted that the Radon transform is an integral transform method, and its core idea is to project a two-dimensional image along a straight line in different directions to generate a set of projection data. Mathematically, the definition of Radon transform is:

[0081] ;

[0082] wherein, represents the original image; represents the projection angle; represents the distance from the projection straight line to the origin; the physical meaning of the Radon transform is that the integral of the image along all parallel straight lines at the angle generates the projection value at the angle. In the frequency domain, the Radon transform is closely related to the Fourier slice theorem. The theorem points out that the image at a certain angle The one-dimensional Fourier transform of the Radon projection of the original image along the angle direction is equal to the slice of the two-dimensional Fourier transform of the original image along the angle direction. This characteristic enables the Radon transform to effectively capture the directional structural features in the image, and the direction extraction effect is as shown in Figure 2

[0083] In a specific application scenario, in the blur direction estimation, the Radon transform is mainly used to extract the main direction information of the motion blur from the frequency domain, and fine-tune the initial blur direction obtained by the aircraft heading estimation. That is, in the process of processing the target image to obtain the target spectrum diagram, the target image can be subjected to a fast Fourier transform to obtain an initial spectrum diagram; a logarithmic power spectrum of the initial spectrum diagram is generated; and a Gaussian filter is applied to the logarithmic power spectrum to obtain a target spectrum diagram. At this time, the generation process of the target blur direction is as shown in Figure 3 In this way, the Radon transform can accurately extract linear or directional features in the image through multi-angle projection. In the frequency domain, motion blur appears as stripes perpendicular to the blur direction. The Radon transform can efficiently locate the main direction of the blur by analyzing the energy distribution of these stripes. Compared with schemes that rely on a single data source, such as sensors or spatial edge detection, the present embodiment is less susceptible to noise interference. In other words, the present embodiment combines frequency domain analysis with spatial prior, i.e., aircraft heading estimation, to improve the robustness of direction estimation through multi-source data fusion. While suppressing noise interference, the present embodiment can accurately capture the main direction features of motion blur, providing key parameter support for subsequent deblurring and feature enhancement.

[0084] In an example embodiment, based on the target blur direction, the target image is deblurred by maximum entropy regularization to obtain a deblurred image, in which a linear motion blur kernel is generated along the target blur direction, where the blur length in the linear motion blur kernel can be estimated by the autocorrelation function method, which locates the first negative peak position to determine by analyzing the autocorrelation decay curve of the image along the blur direction; a smoothing constraint corresponding to the target blur direction is generated, the smoothing constraint is used for weak smoothing along the target blur direction and strong smoothing along the direction perpendicular to the target blur direction, and the degree of weak and strong smoothing can be flexibly determined according to the application scenario, such as weak smoothing according to a value less than or equal to a set intensity value, strong smoothing according to a value greater than the set intensity value, etc.; a deblurring energy function of the target image is generated based on the linear motion blur kernel, the smoothing constraint, and the maximum entropy regularization; and the deblurred image is determined according to the deblurring energy function.

[0085] ​In a specific application scenario, the maximum entropy regularization originates from the maximum entropy principle in information theory, and the core is to select the probability distribution with the maximum entropy as the optimal solution under the satisfaction of the known constraint condition. In the image restoration of the present application, the maximum entropy regularization forces the restoration result to tend to a natural smooth distribution by maximizing the information entropy of the image, avoiding artifacts such as ringing effect caused by local overfitting. Mathematically, the entropy is defined as wherein is the pixel intensity, the process of maximizing the entropy is to minimize the negative entropy term, thereby prompting the pixel intensity distribution to be more uniform and the information to be maximized. The total variation (TV) regularization realizes denoising and smoothing by minimizing the L1 norm of the image gradient. The traditional TV is isotropic, which applies the same penalty to the gradients of all directions, and is easy to cause excessive smoothing of the edge. The anisotropic total variation introduces a directional differentiation constraint, and weakens the gradient penalty along a specific direction, such as the blur direction, and strengthens the penalty in the vertical direction, thereby preserving the direction-sensitive structural features. The dynamic blur causes the gradient to diffuse along the motion direction, and the anisotropic TV preserves the edge information of the blur direction by directional differentiation constraint. Combined with the blur direction estimation result, the anisotropic TV can closely fit the motion degradation model, and improve the physical interpretability of the restoration process. Therefore, in the process of generating the deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization, the present application adopts the joint optimization of the maximum entropy regularization and the anisotropic total variation constraint to realize the balance between the degradation modeling and the detail recovery, that is, to generate the deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint and the maximum entropy regularization;

[0086] The deblurring energy function comprises:

[0087] ;

[0088] ;

[0089] ;

[0090] wherein, denotes the target image; denotes the deblurring image; denotes the minimum value; denotes the norm; denotes the linear motion blur kernel; denotes the tensor product; denotes the set parameter; denotes the smoothing constraint; denotes the set parameter; denotes the maximum entropy regularization; denotes the parameter for emphasizing the edge gradient; denotes the parameter for suppressing the noise and background interference; denotes taking an absolute value; denotes a gradient parallel to the target blur direction; denotes a gradient perpendicular to the target blur direction.

[0091] In this way, the maximum entropy regularization term forces the restored image to tend to be strongly smoothed in the perpendicular direction, avoiding local overfitting. In dynamic blur restoration, the ringing artifact caused by inaccurate blur kernel estimation is reduced by global smoothing constraint, thereby reducing the ringing effect. In addition, the maximum entropy maximizes the pixel intensity distribution to be closer to the statistical characteristics of natural scenes, thereby improving the visual rationality and separability of the restored image. Finally, combined with the directional constraint of anisotropic total variation, the maximum entropy regularization preserves key discriminative details while suppressing background interference.

[0092] Step S104: generating a to-be-identified image based on the deblurred image.

[0093] In actual applications, considering that the image information relied on in the classification and identification process of the aircraft may be different, after obtaining the deblurred image, a to-be-identified image can be generated based on the deblurred image, so as to subsequently apply the to-be-identified image to classify the aircraft.

[0094] In an example embodiment, in the process of generating the to-be-identified image based on the deblurred image, a target slice of the deblurred image can be determined, such as taking the wake area of the aircraft as the target slice; a blur kernel length is extracted from the linear motion blur kernel; an expansion parameter is generated based on the blur kernel length and an expansion coefficient, such as taking the product of the blur kernel length and the expansion coefficient as the expansion parameter; the target slice is expanded according to the expansion parameter to obtain an expanded slice, such as adding the expansion parameter to the target slice to obtain the expanded slice; the deblurred image is sliced according to the expanded slice to obtain a sliced image; and the to-be-identified image is generated according to the deblurred image and the sliced image. In this way, the expansion direction is along the main direction of the blur, ensuring that the linear structure of the slice is preserved to be continuous with the blur trail, and the slice can be decoupled from the background interference to become an auxiliary feature for classification rather than an interference source. For example, the wake diffusion patterns of aircrafts are different, and dynamic slice adjustment can capture such physical identifiers to improve the discriminability of fine-grained classification.

[0095] Step S105: identifying the to-be-identified image through a pre-trained ViT neural network model to obtain a classification result of the to-be-classified aircraft; wherein the position encoding of the ViT neural network model is generated based on the directional information of the image block; and the self-attention mechanism of the ViT neural network model is generated based on the similarity between the directional angles of the image blocks.

[0096] In practical applications, after obtaining the image to be identified, a pre-trained ViT neural network model can be used to identify the image and obtain the classification result of the aircraft to be classified. The overall framework of the ViT neural network model can be as follows: Figure 4 As shown.

[0097] It should be noted that the standard ViT positional encoding assigns fixed spatial positional information to each image patch through learnable vectors, but it lacks modeling of local directional characteristics. In scenarios such as motion blur and wake interference, features such as the linear structure of wakes and the directional continuity of blur trails have strong spatial directional correlations. Therefore, this application introduces the orientation angle information of local patches, embedding physical priors, such as the blur direction or even the wake diffusion direction, into the positional representation to enhance the model's ability to perceive orientation-sensitive structures. In other words, the positional encoding of the ViT neural network model in this application is generated based on the orientation information of image patches. Meanwhile, the standard self-attention mechanism achieves global feature interaction by calculating the similarity weights between queries and keys. However, in scenarios of motion blur and cloud interference, high-contrast interference areas may be misjudged as key features. To avoid this, this application introduces directional similarity constraints in its orientation-guided self-attention, forcing the model to focus on features consistent with the blur direction, such as wakes and fuselage outlines, and suppressing interference areas with inconsistent directions. In other words, the self-attention mechanism of the ViT neural network model in this application is generated based on the similarity between the orientation angles of image patches.

[0098] In an exemplary embodiment, fuzzy directional priors can be embedded into the global modeling of ViT through direction-sensitive positional encoding and direction-guided self-attention, thereby guiding the model to focus on wake regions with strong directional characteristics. First, ViT uses learnable positional encoding to assign a standard position vector to each image patch, then maps it to the same dimension as the standard position vector and fuses them to obtain the final positional encoding. That is, the generation formula of the positional encoding of the ViT neural network model includes:

[0099] ;

[0100] ;

[0101] in, Indicates the first The final position encoding of each image patch; Indicates the number of the image patch; Represented as the first Standard location vectors assigned to each image patch; Indicates the first The extraction direction is obtained by extracting the center position of each image patch; Indicates the first a vector obtained after encoding the extraction direction of the image block; denotes a set parameter; denotes a cosine operation; denotes a sine operation; denotes a set weight;

[0102] In this way, the linear structure of the object in the slice, the blurred trail extends along a certain direction, and the direction encoding makes the image blocks adjacent and consistent in direction have similar position representations, enhancing their semantic consistency, while the direction angle distribution of the background area, such as the cloud layer, is irregular, and the direction encoding can reduce its similarity with the target area, reduce false activation, thereby distinguishing the fuselage contour consistent in direction from the background interference with random wind direction, and improving the separability of the feature space.

[0103] In an example embodiment, the generation formula of the self-attention mechanism of the ViT neural network model can include:

[0104] ;

[0105] ;

[0106] wherein, denotes the self-attention mechanism value of the i-th image block and the j-th image block, so that tokens adjacent and consistent in direction obtain higher attention weights, and the model pays more attention to continuous structures along the blurred direction, such as aircraft contrails, etc.; denotes the number of the image block; denotes a normalized exponential function; denotes a query vector; denotes a key vector; denotes a scaling factor for stabilizing gradient propagation; is the dimension of ; denotes a direction penalty coefficient for balancing semantic similarity and direction consistency, and controls the strength of the direction prior; denotes a similarity score of the direction angle difference, when the two image blocks are consistent in direction, i.e. , the score tends to 1, and the greater the direction difference, the lower the score; denotes a sensitivity coefficient for controlling the penalty strength of the direction difference; denotes an exponential function with a natural constant as the base; denotes an extraction direction obtained after extracting the center position of the i-th image block; denotes taking the absolute value;

[0107] ​​​In this way, the ViT neural network model aggregates features along consistent directions, for example, image patches in the tail region enhance interaction through high directional scores, forming a coherent response pattern, while low scores in the background region suppress their participation in feature updates. In other words, by introducing a directional similarity constraint in the self-attention mechanism, the model is forced to focus on structures consistent with the ambiguous direction and tail diffusion direction. High-contrast regions in the background, such as cloud edges, are weakened in attention weights due to low directional scores. The directional similarity score explicitly embeds the motion degradation prior, giving the attention mechanism a physical meaning rather than being purely data-driven.

[0108] It should be noted that Vision Transformer (ViT) is a model that innovatively applies the Transformer architecture in natural language processing to computer vision tasks. Its core principle is to replace traditional convolution operations with a global self-attention mechanism to achieve global modeling of image features. The introduction of ViT marks the unification of visual and language model architectures, and subsequent work continues to promote its application in detection, segmentation, and other tasks through hierarchical design, hybrid architecture (such as CNN+ViT), and efficient attention mechanisms. The following analyzes its principles in five aspects:

[0109] 1) Image patching and embedding (Patch Embedding): ViT divides the input image into fixed-size non-overlapping blocks (such as 16x16 pixels), and each block is flattened and mapped to a low-dimensional vector (e.g., 768 dimensions) through a linear projection, forming a sequence of word embeddings similar to NLP. For example, a 224x224 RGB image is divided into 196 blocks, each block is flattened to 16x16x3=768, and then compressed to an embedding vector through a fully connected layer. This process converts the image into sequential data, laying the foundation for subsequent Transformer processing.

[0110] 2) Positional encoding (Positional Encoding): Since the Transformer itself does not have spatial perception capabilities, ViT introduces a learnable one-dimensional position encoding to encode the two-dimensional spatial relationship between image blocks into a vector and add it to the embedding vector. Positional encoding automatically learns the topological structure between blocks, such as the distance between adjacent blocks and the global layout, through training. Although images are inherently two-dimensional, experiments have shown that one-dimensional encoding is sufficient.

[0111] 3) Transformer encoder architecture: The core of ViT is stacked with multiple layers of Transformer encoder, each of which contains two key modules: Multi-head self-attention (MSA): The inter-block association weights are calculated by multiple sets of parallel attention heads (such as 12 heads), which can capture global dependencies. For example, some attention heads at the bottom can focus on most areas of the image, while other heads remain locally focused, achieving multi-scale feature fusion. Feed-forward network (FFN): composed of two fully connected layers and GELU activation function, which performs nonlinear transformation on the attention output. Layer normalization (LayerNorm) and residual connection are applied after each module to accelerate training convergence.

[0112] 4) Classification head and special token: ViT adds a learnable classification token ([class]token) before the input sequence, and the output vector after passing through all the encoding layers is used as the overall representation of the image. Finally, the MLP classification head is used to output the prediction result. This design is inspired by the [CLS] token in BERT, which aggregates global information for classification and avoids the information loss of traditional pooling operations.

[0113] 5) Training strategy and improvement direction: Supervised pre-training: ViT needs to be pre-trained on a large-scale dataset (such as JFT-300M) before migrating to downstream tasks. Studies have shown that when the pre-training data is sufficient, the accuracy of ViT on ImageNet can reach 88.55%, surpassing the CNN models of the same period. Self-supervised learning: Subsequent research introduces mask image modeling (such as MAE, BEiT), which improves the model's generalization ability by reconstructing the blocked blocks. For example, MAE randomly masks 75% of the blocks and uses the remaining blocks to reconstruct the pixels, achieving an accuracy of 83.6% on ImageNet. Efficiency optimization: To address the high computational complexity (O(N2)) of ViT, improved models such as Swin Transformer introduce local window attention, reducing computational complexity while maintaining multi-scale features.

[0114] It is also worth mentioning that the Transformer is a deep learning architecture based on self-attention mechanisms, which has achieved great success in natural language processing (NLP) and other fields. The Transformer has high parallel computing capability, as the attention mechanism calculation can be performed in parallel, making the Transformer model highly efficient and capable of parallel processing. It also has strong representation ability, as the Transformer model can effectively capture global information of input data, thus achieving significant performance improvement in natural language processing tasks such as language modeling and translation. The Transformer model has been widely applied to various tasks, including but not limited to: language translation, text generation, dialogue systems, language models, image processing, recommendation systems, etc. In summary, the Transformer is a powerful deep learning model that has greatly promoted the development of natural language processing and other fields. The core concepts of the Transformer include attention mechanisms, encoder-decoder architecture, and multi-head attention. Here is an introduction to the Transformer:

[0115] 1) Attention mechanism is an important concept in neural networks that allows the model to focus on certain parts of the input sequence while ignoring others. In the Transformer, self-attention mechanism, also known as internal attention, is used. Self-attention mechanism enables the model to automatically highlight important words or phrases to understand the meaning of a sentence.

[0116] 2) The Transformer consists of two parts: the encoder (Encoder) and the decoder (Decoder). The encoder is responsible for processing the input sequence, while the decoder generates the target sequence based on the output of the encoder. Each encoder and decoder is stacked with multiple identical layers (blocks), usually 6 layers.

[0117] 3) Multi-head attention is a key innovation in the Transformer, which allows the model to learn attention distributions in different representation subspaces. This means that the model can focus on different positions of the input sequence simultaneously, capturing richer contextual information.

[0118] The application provides an aerial vehicle classification method based on fuzzy direction prior. The target image of the vehicle to be classified is obtained. The fuzzy direction of the target image is estimated and adjusted to obtain the target fuzzy direction. Based on the target fuzzy direction, the target image is deblurred by maximum entropy regularization to obtain a deblurred image. The deblurred image is used to generate an image to be recognized. The ViT neural network model is pre-trained to recognize the image to be recognized, and the classification result of the vehicle to be classified is obtained. The position encoding of the ViT neural network model is generated based on the direction information of the image block. The self-attention mechanism of the ViT neural network model is generated based on the similarity between the direction angles of the image blocks. In the application, the fuzzy direction of the target image is estimated, adjusted and processed by maximum entropy regularization, which can balance edge sharpening and noise suppression, realize high-fidelity restoration of the degraded image, and restore the high-frequency details of the target image, so as to provide a clearer image for the ViT neural network model for classification, which can improve the classification accuracy from the image quality angle. The ViT neural network model integrates the direction similarity score into the self-attention mechanism, which can force the ViT neural network model to focus on the features consistent with the fuzzy direction and suppress the interference area inconsistent with the fuzzy direction, so that the aerial vehicle can be classified more accurately.

[0119] In order to facilitate understanding of the effect of the aerial vehicle classification method based on fuzzy direction prior of the application, it is assumed that the aerial vehicle is an airplane, and the effect of the application is reflected by experimental comparison. The experimental data set is composed of visible light aerial vehicle images, covering 64 fine-grained sub-classes of passenger aircraft. Among them, the civil aviation aircraft includes Boeing 737-800, Airbus A320 and other modifications. The experimental data set is divided into a training set (60%), a validation set (20%) and a test set (20%) in the ratio of 6:2:2 by using a random division strategy. The batch size is set to 16. The experiment is run under a Linux-based operating system, and the algorithm is realized by relying on the PyTorch deep learning framework.

[0120] To verify the effectiveness of the aircraft fine-grained classification method based on the fuzzy direction prior of the present application, a systematic comparison is made with various state-of-the-art classification frameworks in the field of fine-grained classification. Including NTS-Net, S3N, FDL, LIO, PMG, DP-Net, GaRD, SPS, P2P-Net, ViT, TransFG, DCAL and other methods. These comparison models not only cover methods based on region positioning and attention mechanism, methods based on double path and feature decoupling, but also include Transformer-based architecture. And the selected comparison algorithms have been verified on the fine-grained classification public dataset, as much as possible to ensure the persuasiveness of the comparison experiment. Table 1 shows the classification accuracy of different fine-grained classification methods on the visible light aerial vehicle classification dataset. Table 2 shows the performance comparison of different deblurring methods combined with the ViT architecture in the classification task.

[0121] Table 1 Performance comparison experiment result table

[0122]

[0123] Table 2 Comparison experiment result table of different deblurring methods

[0124]

[0125] From Table 1, the method proposed in the present application is significantly better than all comparison methods with an accuracy of 91.8%, which is 6.9 percentage points higher than the second place DCAL (84.9%). From Table 2, the method of the present application has an accuracy of 91.8%, an inference time of 36ms and an FLOPs of 81.4G, which is better than other schemes in terms of accuracy and efficiency balance.

[0126] Please refer to Figure 5 , Figure 5 A structure diagram of an aerial vehicle classification system based on a fuzzy direction prior provided by an embodiment of the present application.

[0127] An aerial vehicle classification system based on a fuzzy direction prior provided by an embodiment of the present application can include:

[0128] A target image acquisition module 101 is configured to acquire a target image of an aircraft to be classified;

[0129] A target blur direction generation module 102 is configured to estimate and adjust the blur direction of the target image to obtain a target blur direction;

[0130] A deblurring module 103 is configured to perform deblurring processing on the target image based on the target blur direction through maximum entropy regularization to obtain a deblurred image;

[0131] The to-be-identified image generation module 104 is configured to generate a to-be-identified image based on the deblurred image;

[0132] The identification module 105 is configured to identify the to-be-identified image by using the pre-trained ViT neural network model, and obtain a classification result of the to-be-classified aircraft.

[0133] The position encoding of the ViT neural network model is generated based on direction information of the image block; and the self-attention mechanism of the ViT neural network model is generated based on similarity between direction angles of the image blocks.

[0134] The aerial vehicle classification system based on the blurred direction prior provided in the embodiments of the present application can include:

[0135] The estimation unit is configured to estimate an aircraft heading angle of the target image, and obtain an initial blurred direction and a confidence degree of the initial blurred direction.

[0136] The range generation unit is configured to generate an angle search range based on the initial blurred direction.

[0137] The processing unit is configured to process the target image, and obtain a target spectrum.

[0138] The angle determination unit is configured to determine direction energy distribution information of the target spectrum according to the angle search range, and generate an optimized angle based on the direction energy distribution information.

[0139] The fusion unit is configured to perform weighted fusion on the initial blurred direction and the optimized angle based on the confidence degree, and obtain a target blurred direction.

[0140] The aerial vehicle classification system based on the blurred direction prior provided in the embodiments of the present application can include:

[0141] The processing unit can be configured to perform fast Fourier transform on the target image, and obtain an initial spectrum; generate a logarithmic power spectrum of the initial spectrum; and perform Gaussian filtering on the logarithmic power spectrum, and obtain the target spectrum.

[0142] The aerial vehicle classification system provided in the embodiments of the present application can include:

[0143] The blurred kernel generation unit is configured to generate a linear motion blurred kernel along the target blurred direction.

[0144] The constraint unit is configured to generate a smoothing constraint corresponding to the target blur direction, and the smoothing constraint is configured to perform weak smoothing along the target blur direction and strong smoothing along a direction perpendicular to the target blur direction.

[0145] The function generation unit is configured to generate a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint, and maximum entropy regularization.

[0146] The deblurring unit is configured to determine a deblurred image according to the deblurring energy function.

[0147] The function generation unit can be configured to:

[0148] generate a deblurring energy function of the target image based on the linear motion blur kernel, the smoothing constraint, and maximum entropy regularization.

[0149] The deblurring energy function comprises:

[0150]

[0151]

[0152]

[0153] wherein, represents the target image; represents the deblurred image; represents taking a minimum value; represents a norm; represents the linear motion blur kernel; represents a tensor product; represents a set parameter; represents the smoothing constraint; represents a set parameter; represents the maximum entropy regularization; represents a parameter for emphasizing an edge gradient; represents a parameter for suppressing noise and background interference; represents taking an absolute value; represents a gradient parallel to the target blur direction; represents a gradient perpendicular to the target blur direction.

[0154] The function generation unit can be configured to:

[0155] The slice determination unit is configured to determine a target slice of the deblurred image;

[0156] The extraction unit is configured to extract a blur kernel length from the linear motion blur kernel.​​​

[0157] an expansion parameter generation unit, configured to generate an expansion parameter based on the blur kernel length and the expansion coefficient;

[0158] a slice expansion unit, configured to expand the target slice according to the expansion parameter to obtain an expanded slice;

[0159] a slice unit, configured to slice the deblurred image according to the expanded slice to obtain a sliced image;

[0160] a to-be-recognized image generation unit, configured to generate a to-be-recognized image according to the deblurred image and the sliced image.

[0161] The position encoding generation formula of the ViT neural network model provided in the embodiment of the application includes:

[0162]

[0163]

[0164] The self-attention mechanism generation formula of the ViT neural network model includes:

[0165]

[0166]

[0167] wherein, denotes the final position encoding of the i-th image block; denotes the number of the image block; denotes the standard position vector assigned to the i-th image block; denotes the extraction direction obtained by extracting the center position of the i-th image block; denotes the vector obtained by encoding the extraction direction of the i-th image block; denotes a set parameter; denotes a cosine operation; denotes a sine operation; denotes a set weight; denotes the self-attention mechanism value of the i-th image block and the j-th image block; denotes the number of the image block; denotes a normalized exponential function; denotes a query vector; denotes a key vector; ​​​​​​​​​​denotes a scaling factor for stabilizing gradient propagation; denotes a dimension of the image block; denotes a dimension of the image block; denotes a direction penalty coefficient for balancing semantic similarity and direction consistency; denotes a similarity score of the difference of direction angles; denotes a sensitivity coefficient; denotes an exponential function with a natural constant as a base; denotes an extracted direction after extracting a center position of the i-th image block; denotes taking an absolute value.

[0168] The application further provides an electronic device and a computer readable storage medium, both of which have the corresponding effects of the aerial vehicle classification method based on the fuzzy direction prior provided in the embodiments of the application. Please refer to Figure 6 , Figure 6 which is a structural schematic diagram of an electronic device provided in the embodiments of the application.

[0169] The electronic device provided in the embodiments of the application comprises a memory 201 and a processor 202, the memory 201 stores a computer program, and the processor 202 implements the steps of the aerial vehicle classification method based on the fuzzy direction prior when executing the computer program.

[0170] Please refer to Figure 7 , the other electronic device provided in the embodiments of the application can further comprise: an input port 203 connected with the processor 202, used for transmitting an input command from the outside to the processor 202; a display unit 204 connected with the processor 202, used for displaying the processing result of the processor 202 to the outside; and a communication module 205 connected with the processor 202, used for realizing the communication between the electronic device and the outside. The display unit 204 can be a display panel, a laser scanning display, etc.; the communication mode adopted by the communication module 205 comprises but is not limited to a mobile high-definition link technology (MHL), a universal serial bus (USB), a high-definition multimedia interface (HDMI), wireless connection: wireless fidelity technology (WiFi), Bluetooth communication technology, low-power Bluetooth communication technology, and communication technology based on IEEE 802.11s.

[0171] ​The computer readable storage medium provided in the embodiment of the present application stores a computer program, and the computer program is executed by a processor to implement the steps of the method for classifying aerial vehicles based on fuzzy direction priori described in any of the above embodiments.

[0172] The computer readable storage medium involved in the present application includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable magnetic disk, a compact disc read-only memory (CD-ROM), or any other form of storage medium known in the technical field.

[0173] The system for classifying aerial vehicles based on fuzzy direction priori, the electronic device, and the related part of the computer readable storage medium provided in the embodiment of the present application are described in detail in the corresponding part of the method for classifying aerial vehicles based on fuzzy direction priori provided in the embodiment of the present application, which will not be described here. In addition, the part of the above technical solution provided in the embodiment of the present application which is consistent with the implementation principle of the corresponding technical solution in the prior art is not described in detail, so as not to be too verbose.

[0174] It should also be noted that, in this document, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0175] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for classifying airborne vehicles based on fuzzy directional priors, characterized in that, include: Acquire target images of the aircraft to be classified; The target image is subjected to blur direction estimation and adjustment to obtain the target blur direction; Based on the target blur direction, the target image is deblurred by maximum entropy regularization to obtain a deblurred image; A recognition image is generated based on the deblurred image; The image to be identified is identified by a pre-trained ViT neural network model, and the classification result of the aircraft to be classified is obtained. The position encoding of the ViT neural network model is generated based on the orientation information of image patches; the self-attention mechanism of the ViT neural network model is generated based on the similarity between the orientation angles of image patches. The step of deblurring the target image based on the target blur direction using maximum entropy regularization to obtain a deblurred image includes: Generate a linear motion blur kernel along the target blur direction; Generate a smoothing constraint corresponding to the target blur direction, wherein the smoothing constraint is used to perform weak smoothing along the target blur direction and strong smoothing along a direction perpendicular to the target blur direction; The deblurring energy function of the target image is generated based on the linear motion blur kernel, the smoothing constraint, and the maximum entropy regularization. The deblurred image is determined based on the deblurring energy function; The deblurring energy function includes: ； ； ； in, This refers to the target image; This refers to the deblurred image; This indicates taking the minimum value; Represents the norm; This represents the linear motion fuzzy kernel; Represents the tensor product; Indicates setting parameters; This represents the smoothing constraint; Indicates setting parameters; This represents the maximum entropy regularization; Parameters that emphasize edge gradients; Parameters representing the suppression of noise and background interference; Indicates taking the absolute value; This represents the gradient parallel to the blurring direction of the target; This represents the gradient perpendicular to the target blur direction.

2. The method according to claim 1, characterized in that, The step of estimating and adjusting the blur direction of the target image to obtain the target blur direction includes: The aircraft orientation angle is estimated from the target image to obtain the initial blur direction and the confidence level of the initial blur direction; Based on the initial fuzzy direction, an angle search range is generated; The target image is processed to obtain a target spectrogram; The directional energy distribution information of the target spectrogram is determined based on the angle search range, and an optimized angle is generated based on the directional energy distribution information; Based on the confidence level, the initial fuzzy direction and the optimized angle are weighted and fused to obtain the target fuzzy direction.

3. The method according to claim 2, characterized in that, The step of determining the directional energy distribution information of the target spectrogram based on the angle search range, and generating an optimized angle based on the directional energy distribution information, includes: Within the angle search range, Radon projections are performed on the target spectrogram at different angles, and line integrals are performed on the target spectrogram along the angle to generate projection curves that reflect the directional energy distribution. The angle corresponding to the energy peak in the projection curve is taken as the optimization angle.

4. The method according to claim 2, characterized in that, The process of processing the target image to obtain the target spectrogram includes: Perform a Fast Fourier Transform on the target image to obtain an initial spectrogram; Generate the logarithmic power spectrum of the initial spectrogram; Gaussian filtering is applied to the logarithmic power spectrum to obtain the target spectrum.

5. The method according to claim 2, characterized in that, The step of generating the image to be recognized based on the deblurred image includes: Determine the target slice of the deblurred image; Extract the fuzzy kernel length from the linear motion fuzzy kernel; Based on the fuzzy kernel length and the expansion coefficient, the expansion parameters are generated; The target slice is expanded according to the expansion parameters to obtain an expanded slice; The deblurred image is sliced ​​according to the extended slice to obtain a sliced ​​image; An image to be identified is generated based on the deblurred image and the sliced ​​image.

6. The method according to any one of claims 1 to 5, characterized in that, The formula for generating the positional encoding of the ViT neural network model includes: ； ； in, Indicates the first The final position encoding of each image patch; Indicates the number of the image patch; Represented as the first Standard location vectors assigned to each image patch; Indicates the first The extraction direction is obtained by extracting the center position of each image patch; Indicates the first The vector obtained by encoding the extraction direction of each image patch; Indicates setting parameters; Represents the cosine operation; Represents the sine operation; This indicates that a weight is set.

7. The method according to claim 6, characterized in that, The formula for generating the self-attention mechanism of the ViT neural network model includes: ； ； in, Indicates the first The image patch and the first The self-attention mechanism value of each image patch; Indicates the number of the image patch; This represents the normalized exponential function; Represents the query vector; Represents the key vector; This represents the scaling factor used for stabilizing gradient propagation; for The dimension; This represents the directional penalty coefficient used to balance semantic similarity and directional consistency. Similarity score representing the difference in orientation angle; Indicates the sensitivity coefficient; Represents an exponential function with the natural constant as its base; Indicates the first The extraction direction is obtained by extracting the center position of each image patch; This indicates taking the absolute value.

8. An airborne vehicle classification system based on fuzzy directional priors, characterized in that, include: The target image acquisition module is used to acquire target images of the aircraft to be classified. The target blur direction generation module is used to estimate and adjust the blur direction of the target image to obtain the target blur direction; The deblurring module is used to deblur the target image based on the target blur direction by using maximum entropy regularization to obtain a deblurred image. The image to be recognized generation module is used to generate an image to be recognized based on the deblurred image; The recognition module is used to recognize the image to be recognized using a pre-trained ViT neural network model to obtain the classification result of the aircraft to be classified. The position encoding of the ViT neural network model is generated based on the orientation information of image patches; the self-attention mechanism of the ViT neural network model is generated based on the similarity between the orientation angles of image patches. The deblurring module includes: A fuzzy kernel generation unit is used to generate a linear motion fuzzy kernel along the target fuzzy direction; A constraint unit is used to generate a smoothing constraint corresponding to the target fuzzing direction. The smoothing constraint is used to perform weak smoothing along the target fuzzing direction and strong smoothing along a direction perpendicular to the target fuzzing direction. The function generation unit is used to generate a deblurring energy function for the target image based on the linear motion blur kernel, the smoothing constraint, and the maximum entropy regularization. A deblurring unit is used to determine the deblurred image based on the deblurring energy function; The deblurring energy function includes: ； ； ； in, This refers to the target image; This refers to the deblurred image; This indicates taking the minimum value; Represents the norm; This represents the linear motion fuzzy kernel; Represents the tensor product; Indicates setting parameters; This represents the smoothing constraint; Indicates setting parameters; This represents the maximum entropy regularization; Parameters that emphasize edge gradients; Parameters representing the suppression of noise and background interference; Indicates taking the absolute value; This represents the gradient parallel to the blurring direction of the target; This represents the gradient perpendicular to the target blur direction.

Images