Image processing system and method, electronic equipment and computer storage medium

Abstract

The invention discloses an image processing system and method, electronic equipment and a computer storage medium, and the method comprises the steps: receiving a support target image and a query target image through a feature embedding network, and carrying out the feature extraction of the support target image and the query target image, so as to obtain an output feature; on the basis of the output features, obtaining feature representations of different weights of the support target image and the query target image at different positions of respective images; initial features of the metric classification network are obtained through the output features and the feature representations of different weights; and obtaining the similarity between the support target image and the query target image based on the initial features of the metric classification network, and classifying the query target image. According to the method, the WAAPNet model is designed, and the WAA module is introduced, so that more detailed and efficient features can be extracted while speckle noise of the SAR image is suppressed, and the complex features of the SAR image can be extracted more sufficiently and effectively by the network; the algorithm complexity of the model can be reduced, and the network operation rate is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image processing system, method, electronic equipment and computer storage medium. Background Art

[0002] At present, the complex imaging mechanism of SAR images determines that the sample annotation cost is too high in most SAR application scenarios, making it difficult to obtain a large number of trainable samples. Therefore, SAR image target recognition and classification tasks based on small sample learning have developed rapidly. Among them, the metric learning-based method measures the distance between two samples or multiple sample features through a given distance function and classifies them according to their size. It is one of the most commonly used and effective methods to solve the classification of small sample images. However, since the SAR image targets in the MSTAR dataset and the OpenSARShip dataset are very similar (the MSTAR dataset is all military vehicles such as tanks and artillery vehicles, and the OpenSARShip dataset is all ships), and the image background is full of speckle noise, SAR images usually have a small gap between classes, which makes the network model based on metric learning less effective in extracting image features, resulting in poor recognition accuracy of the model in small sample image classification tasks. Summary of the invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, the present application provides an image processing system, method, electronic device and computer storage medium to solve the problems that SAR image targets in the MSTAR dataset and the OpenSARShip dataset are similar, the inter-class gap caused by speckle noise is small, and the feature extraction capability of the traditional square convolution kernel is insufficient.

[0004] The first technical solution provided by the embodiment of the present application is: an image processing method, the method comprising the following steps:

[0005] Receiving a support target image and a query target image through a feature embedding network to extract features from the support target image and the query target image to obtain output features;

[0006] Based on the output features, obtaining feature representations of the support target image and the query target image at different positions of the respective images with different weights;

[0007] Obtaining initial features of a metric classification network through the output features and the feature representations with different weights;

[0008] The similarity between the support target image and the query target image is obtained based on the initial features of the metric classification network, and the query target image is classified.

[0009] Optionally, the output feature is an output feature of a wavelet asymmetric attention (WAA) module; the WAA module includes a wavelet transform module (WT), a spatial attention module (SA) and an asymmetric convolution (AC) module;

[0010] The step of receiving a support target image and a query target image through a feature embedding network to extract features from the support target image and the query target image to obtain output features includes:

[0011] A wavelet transform operation is introduced through the WT module to extract low-frequency features and high-frequency features in different directions of the support target image and the query target image for multi-scale noise reduction; the low-frequency features are used to obtain the shape and size of the target image; the high-frequency features in different directions are used to obtain image texture and edge information; the high-frequency features in different directions include obtaining the low-frequency features, high-frequency features in the horizontal direction, high-frequency features in the vertical direction, and high-frequency features in the diagonal direction; the high-frequency features include directional features and subtle features in the image;

[0012] The supporting target image and the query target image are decomposed into two asymmetric convolutional layers by the AC module to obtain a feature map of the AC module.

[0013] The wavelet transform operation is introduced through the WT module to extract low-frequency features of the support target image and the query target image and high-frequency features in different directions for multi-scale noise reduction, including:

[0014] The low-frequency features, the high-frequency features in the horizontal direction, the high-frequency features in the vertical direction, and the high-frequency features in the diagonal direction are spliced ​​to obtain primary features after noise reduction.

[0015] Optionally, decomposing the support target image and the query target image into two asymmetric convolutional layers through the AC module to obtain a feature map of the AC module includes:

[0016] Performing a convolution operation on the primary features through an asymmetric convolution (AC) module to obtain a plurality of asymmetric convolution layers;

[0017] Adding a plurality of the asymmetric convolutional layers and passing them through an activation function to obtain a spatial weight tensor;

[0018] The output features of the WAA module are obtained through the feature map and the spatial weight tensor.

[0019] Optionally, obtaining initial features of a metric classification network through the output features and the feature representations with different weights includes:

[0020] Multiplying the output feature map of the WAA module by the feature map of the AC module bit by bit to obtain the feature representation of the different weights based on the local part of the image as the initial feature of the metric classification network;

[0021] The prototype of each category of the support set is obtained by using the initial features of the metric classification network.

[0022] Optionally, the obtaining the similarity between the support target image and the query target image based on the initial features of the metric classification network and classifying the query target image includes:

[0023] Obtaining the distance between the feature representations of the query target image with different weights after passing through the feature embedding network and each of the prototypes; wherein the distance is obtained by a cosine distance function;

[0024] Determining the similarity between the support target image and the query target image based on the distance to generate the predicted label of the category corresponding to the prototype;

[0025] Predicting the probability that the query target image belongs to the category;

[0026] The classification loss of each query target image belonging to the category is obtained, so that the query target image is classified into the category corresponding to it.

[0027] The second technical solution provided by the embodiment of the present application is: to provide an image processing system, including: a feature embedding network, including multiple groups of convolution blocks and multiple groups of wavelet asymmetric attention (WAA) modules, each WAA module includes a wavelet transform module (WT), a spatial attention module (SA) and an asymmetric convolution (AC) module; wherein the WAA module introduces a wavelet transform operation to perform multi-scale denoising of a target image; the AC module includes multiple asymmetric convolution layers;

[0028] The metric classification network includes a prototype learning module and a class similarity module. The prototype learning module is used to average the feature map obtained by the feature embedding module; the class similarity module is used to obtain the similarity between the query target image and the support target image.

[0029] Optionally, the class similarity module obtains the similarity between the query target image and the support target image through a cosine metric function; the WT module performs multi-scale denoising on the target image by extracting low-frequency features and high-frequency features in different directions; wherein the low-frequency features are used to reflect the shape and size of the target image; the low-frequency features are used to obtain the shape and size of the target image; the high-frequency features in different directions are used to obtain image texture and edge information, and the high-frequency features in different directions include high-frequency features in horizontal, vertical and diagonal directions.

[0030] The third technical solution provided by the embodiment of the present application is: to provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned image processing methods when executing the computer program.

[0031] In order to solve the above technical problems, the fourth technical solution provided by the present invention is: providing a computer-readable storage medium storing a program file, and the program file can be executed to implement any of the above methods.

[0032] The beneficial effects of this application include:

[0033] The image processing method provided by the present application includes: receiving a support target image and a query target image through a feature embedding network to extract features from the support target image and the query target image to obtain output features; based on the output features, obtaining feature representations of different weights of the support target image and the query target image at different positions of each image; obtaining the initial features of the metric classification network through the output features and the feature representations of different weights; obtaining the similarity of the support target image and the query target image based on the initial features of the metric classification network, and classifying the query target image. The present application designs a WAAPNet model and introduces a WAA module, in which more detailed and efficient features can be extracted while suppressing the speckle noise of the SAR image, so that the network can extract complex features of the SAR image more fully and effectively; the asymmetric convolution structure decomposes the square convolution into horizontal convolution and vertical convolution, which can enhance the feature extraction capability of the convolution kernel center skeleton, thereby strengthening the features of the convolution kernel center and weakening the features of the edge, and can reduce the algorithm complexity of the model and improve the network operation rate. In addition, due to the unfriendliness of the Euclidean distance to high-dimensional data, the present application adopts a more robust cosine distance, which also improves the network classification accuracy to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 It is a schematic block diagram of the overall process of an image processing method provided by an embodiment of the present application;

[0036] Figure 2 It is a schematic diagram of the WAAPNet model architecture provided by an embodiment of the present application;

[0037] Figure 3 It is a schematic diagram of the architecture of a WAA module provided in an embodiment of the present application;

[0038] Figure 4 Schematic diagram of Euclidean distance and cosine distance provided in an embodiment of the present application;

[0039] Figure 5 3-way 5-shot test accuracy histogram provided in the embodiment of the present application; wherein (a) is the WAAPNet model; (b) is the WAAPNet-AC model;

[0040] Figure 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0041] Figure 7 It is a schematic block diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application.

[0042] Description of reference numerals:

[0043] 100. Image processing system; 10. Feature embedding network; 11. Convolution block; 12. WAA module; 120. WT module; 121. AC module; 1211. Asymmetric convolution layer; 122. SA module; 15. Target image; 151. Low-frequency features; 152. High-frequency features; 20. Metric classification network; 21. Prototype learning module; 22. Class similarity module; 31. Support set; 32. Query set. DETAILED DESCRIPTION

[0044] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant application, rather than to limit the application. It is also necessary to explain that, for ease of description, only the parts related to the application are shown in the accompanying drawings.

[0045] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0046] The inventors of the present application have found through research that the SAR image targets in the MSTAR dataset, a public dataset commonly used for small sample target recognition tasks of SAR (Synthetic Aperture Radar) images, are extremely similar, all of which are military vehicles such as tanks, artillery vehicles, and armored vehicles. In the OpenSARShip dataset, the targets are all ships, and there is severe speckle noise in the image backgrounds of both datasets, which leads to a small gap between different categories of SAR images. It is difficult for ordinary algorithm models to extract highly effective and fine-grained SAR image target features, resulting in low model recognition performance.

[0047] Due to the characteristics of SAR images that are all-day, all-weather, and highly penetrating, its target recognition task has high research and application value in the civilian and military fields. It aims to automatically locate and identify SAR targets from images in a short time. However, the SAR image targets in the common public dataset MSTAR dataset are very similar, all of which are military vehicles such as tanks, artillery vehicles, and armored vehicles. The targets in the OpenSARShip dataset are all ships. Due to the coherent imaging principle of SAR, there is serious speckle noise in the image, resulting in a small gap between different categories. Ordinary algorithm models are difficult to extract highly effective and fine-grained SAR image target features, resulting in low model recognition performance.

[0048] Compared with traditional SAR image target recognition, which has poor recognition results due to poor manual feature extraction performance and insufficient prior knowledge, deep learning has made great progress in the field of image recognition. It can realize automatic feature extraction through neural networks based on the diversity of image samples, and the autonomously extracted features can cover the image content to the maximum extent, with strong generalization ability. However, in actual scenarios, due to the complexity of SAR images, the cost of SAR image annotation is too high, resulting in the inability of recognition tasks to obtain sufficient training support from labeled samples. Deep learning requires a large number of trainable samples, otherwise there will be problems such as incomplete feature extraction and inability to fully train, and the classification performance will drop rapidly. Therefore, small sample learning ideas such as transfer learning, meta-learning, and metric learning are usually introduced into deep learning network models.

[0049] The complex imaging mechanism of SAR images determines that the sample annotation cost is too high in most SAR application scenarios, making it difficult to obtain a large number of trainable samples. Therefore, SAR image target recognition and classification tasks based on small sample learning have developed rapidly. Among them, the metric learning-based method measures the distance between two samples or multiple sample features through a given distance function and classifies them according to their size. It is one of the most commonly used and effective methods to solve the classification of small sample images. However, since the SAR image targets in the MSTAR dataset and the OpenSARShip dataset are very similar (the MSTAR dataset is all military vehicles such as tanks and artillery vehicles, and the OpenSARShip dataset is all ships), and the image background is full of speckle noise, SAR images usually have a small gap between classes, which makes the network model based on metric learning low in effectiveness when extracting image features, resulting in poor recognition accuracy of the model in small sample image classification tasks. Therefore, solving the challenging problem of SAR image target recognition requires the use of more complex and diverse methods to deal with the speckle noise caused by SAR coherent imaging and improve the accuracy and robustness of classification.

[0050] Aiming at the problems that the SAR images in the MSTAR dataset and the OpenSARShip dataset have similar targets, small inter-class gaps caused by speckle noise, and insufficient feature extraction capabilities of traditional square convolution kernels, this application proposes a new image processing system and method, a prototype network based on Wavelet Asymmetric Attention (WAAPNet), and verifies its effectiveness and superiority in SAR image target recognition tasks through a series of experiments on the MSTAR dataset and the OpenSARShip dataset. The Wavelet Asymmetric Attention (WAA) module of the model uses the multi-resolution analysis capability of the wavelet transform to perform multi-scale denoising on the SAR image while maintaining the important features of the SAR image, and extracts SAR target features from multiple dimensions, and combines the attention mechanism to weight them to form heterogeneous features. Since the targets of the SAR images in the dataset are mostly located in the center of the image, the attention mechanism combined with the asymmetric convolution structure can enhance the ability of the central skeleton to extract features, improve the ability to extract the detailed features of the SAR target, and thus improve the recognition accuracy of the model while reducing the complexity of the algorithm. In addition, the model uses cosine distance as the similarity measurement function, which can better handle high-dimensional data obtained by feature extraction than Euclidean distance.

[0051] Please refer to Figure 1 to Figure 3 as well as Figure 5 , Figure 1It is a schematic block diagram of the overall process of an image processing method provided by an embodiment of the present application; Figure 2 It is a schematic diagram of the WAAPNet model architecture provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the architecture of a WAA module provided in an embodiment of the present application; Figure 5 3-way 5-shot test accuracy histogram provided in the embodiment of the present application; wherein (a) is the WAAPNet model; and (b) is the WAAPNet-AC model.

[0052] The image processing method provided in this application is generally performed in the following steps:

[0053] First, both the support image and the query image are input into the CNN-based feature embedding network for feature extraction to obtain their corresponding feature representations based on local different weights of the image. Unlike the query image, each class in the support set needs to be further processed by the prototype learning module in the metric classification network to learn the prototype of the class. Then, the cosine distance function is used to measure the distance between the feature representation based on local different weights of the query image and the prototype of each class in the support set. Finally, the probability of the query image belonging to each class is predicted based on the softmax function.

[0054] In one embodiment, if Figure 1 As shown, the image processing method may include the following steps:

[0055] S1: receiving a support target image and a query target image through a feature embedding network to extract features of the support target image and the query target image to obtain output features.

[0056] During the meta-task training process of the prototype network, a large number of training classes and training samples from the relevant source domain (non-target category) are required to learn transferable knowledge and improve the generalization ability of unseen test data. However, compared with optical image datasets such as miniImageNet (containing 64 training classes and 600 samples per class), the number of training classes and training samples in the source domain of the SAR image dataset is much smaller, resulting in poor SAR image classification performance. Therefore, this application improves the feature extraction network of the prototype network, and solves the problem of small gaps between SAR image classes when there are very few labeled samples in the target domain and limited training samples in the source domain.

[0057] Specifically, as mentioned above, the WAAPNet model of the present application generally consists of a feature embedding network 10 and a metric classification network 20, and its overall architecture is as follows: Figure 2 For example, the support set and query set are denoted as S = {(x i ,y i ), i=1, ..., K s} and Q = {(x j ,y j ), j = 1, ..., K Q}. Among them, x i , x j ∈R D , is the D-dimensional feature vector of the sample, y i Represents x i The corresponding label, y j Represents x j The corresponding label.

[0058] The output features are the output features of the wavelet asymmetric attention (WAA) module. The feature embedding network 10 is used to extract features from image samples, and mainly includes four groups of convolution blocks 11 and four groups of WAA modules 12, and then the two are multiplied bit by bit.

[0059] Specifically, the structures of the four groups of convolutional blocks are as follows:

[0060] {3×3-conv (64 filters, padding=1), BatchNorm, ReLU, 2×2-Max pool}. Each WAA module 12 consists of a wavelet transform module (WT), a spatial attention (SA) module and an asymmetric convolution (AC) module, such as Figures 1-2 shown.

[0061] Wavelet transform is an excellent signal analysis tool. It not only inherits the localized analysis concept of short-time Fourier transform, but also successfully overcomes the limitations of its fixed window size and cannot be changed. Wavelet transform characterizes the signal through a finite length or fast decaying oscillating waveform called "mother wavelet". After scaling and translation operations, this waveform can automatically adjust the size of its "time-frequency" window according to the change of signal frequency, realize time subdivision in high-frequency areas and frequency subdivision in low-frequency areas, so as to more accurately capture the local characteristics of the signal. Wavelet transform can enhance the features of SAR images, highlight the details and edge information of SAR targets, and is conducive to extracting features with low noise influence.

[0062] In one embodiment, if Figure 3 As shown, step S1 of receiving a support target image and a query target image through a feature embedding network 10 to extract features from the support target image and the query target image to obtain output features may include:

[0063] S11: A wavelet transform operation is introduced through the WT module to extract low-frequency features 151 of the supporting target image and the query target image and high-frequency features 152 in different directions for multi-scale denoising.

[0064] Specifically, the low-frequency features 151 can reflect the basic shape and size of the target, capture the large-scale structure and global information of the image, and can better process speckle noise. However, the low-frequency features 151 are not sensitive to detailed textures and may lose the detailed features of the SAR image. Therefore, the WT module 120 extracts the high-frequency features 152 of the SAR image from the horizontal, vertical and diagonal directions respectively to obtain richer image texture and edge information, which is crucial for identifying the detailed structure and surface characteristics of the SAR image target. In the case of variable directionality and posture of SAR targets, for speckle noise and non-target elements in SAR images, combining high-frequency and low-frequency features 151 in different directions can fully understand the image content and increase the fault tolerance of the model.

[0065] S12: Decomposing the support target image and the query target image into two asymmetric convolutional layers 1211 through the AC module 121 to obtain a feature map of the AC module 121.

[0066] Specifically, the AC module 121 includes two asymmetric convolutional layers 1211, and its structure is as follows:

[0067] {1×3-conv(64 filters, padding=(0,1)), ReLU},

[0068] {3×1-conv(64 filters, padding=(1,0)), ReLU}.

[0069] The convolution kernel of a traditional CNN is generally square, and the fixed square convolution kernel is insufficient for target feature extraction of SAR images, has a large number of parameters and high computational complexity. The standard d×d convolution layer can be decomposed into two asymmetric convolution layers 1211 of d×1 and 1×d. This decomposition allows the network to learn horizontal and vertical features respectively, better capture the directional features of the target, help capture subtle features in SAR images, and reduce network parameters and the amount of required calculations. In the MSTAR dataset and the OpenSARShip dataset, the target size of SAR images is relatively small, the feature information is limited, and the edge information of the target is blurred. Therefore, the AC module 121 reduces the number of parameters and reduces the computational complexity by decomposing the standard convolution kernel, so that the network can learn the details of the SAR image more deeply.

[0070] The wavelet transform operation is introduced in the attention module to perform multi-scale denoising by extracting low-frequency and high-frequency features in different directions. Low-frequency features can reflect the basic shape and size of the target, capture the large-scale structure and global information of the image, and can better handle speckle noise. However, low-frequency features are insensitive to detailed textures and may lose detailed features of SAR images. Therefore, the WT module extracts high-frequency features of SAR images from the horizontal, vertical, and diagonal directions to obtain richer image texture and edge information, which is crucial for identifying the detailed structure and surface characteristics of SAR image targets. In the case of variable directionality and posture of SAR targets, for speckle noise and non-target elements in SAR images, combining high-frequency and low-frequency features in different directions can fully understand the image content and increase the fault tolerance of the model.

[0071] Further, step S11 of introducing a wavelet transform operation through the WT module 120 to extract low-frequency features 151 supporting the target image and the query target image and high-frequency features 152 in different directions for multi-scale denoising may include:

[0072] S111: concatenate the low-frequency feature 151, the horizontal high-frequency feature 152, the vertical high-frequency feature 152, and the diagonal high-frequency feature 152 to obtain a primary feature after noise reduction.

[0073] Specifically, the implementation process of the WAA module is as follows: First, the feature maps obtained by the AC convolution module are subjected to wavelet transform operations to obtain low-frequency features F L , high-frequency features in the horizontal direction Vertical high frequency characteristics And the high-frequency features in the diagonal direction The relevant calculation formula is as follows:

[0074]

[0075] Formula (1) is the wavelet transformation formula, where x(t) is a continuous signal, ψ is a wavelet basis function, γ represents a scale factor, and β represents a translation factor. In formula (2), f() represents a wavelet transformation operation.

[0076] Then these four features are concatenated to obtain the primary feature F after denoising. 1 :

[0077]

[0078] In formula (1), (,) is the concat operation.

[0079] In one embodiment, step S12 of decomposing the support target image and the query target image into two asymmetric convolutional layers 1211 through the AC module 121 to obtain a feature map of the AC module 121 may include:

[0080] S121: Perform convolution operation on the primary features through an asymmetric convolution (AC) module to obtain multiple asymmetric convolution layers 1211.

[0081] Specifically, the AC module 121 performs a convolution operation on the primary features to obtain two convolution branches (convolution layers).

[0082] S122: Add multiple asymmetric convolutional layers 1211 and pass them through an activation function to obtain a spatial weight tensor.

[0083] Specifically, the two convolution branches (convolution layers) are added together and passed through the Sigmoid activation function to obtain a H×W×1 spatial weight tensor S:

[0084] S=σ(f 1×3 (F 1 )+f 3×1 (F 1 )) (4)

[0085] Among them, f 1×3 is a 1×3 asymmetric convolution operation, f 3×1 is a 3×1 asymmetric convolution operation, and σ(·) represents the Sigmoid activation function.

[0086] S123: Obtain the output features of the WAA module 12 through the feature map and the spatial weight tensor.

[0087] Specifically, the spatial weight tensor S is multiplied by the feature map F to obtain the output feature map F′ of the WAA module:

[0088] F′=S×F (5)

[0089] S2: Based on the output features, feature representations with different weights for the support target image and the query target image at different positions of their respective images are obtained.

[0090] S3: Obtain the initial features of the metric classification network 20 through output features and feature representations with different weights.

[0091] Specifically, in steps S2 to S3, the feature embedding network 10 multiplies the output features of the convolution block 11 by the output features of the WAA module 12 bit by bit, and obtains a feature representation based on different local weights of the image as the initial feature of the metric classification network 20. Each weight corresponds to the relative importance of the corresponding position information on the feature map. Compared with the original embedding network in the prototype network, the feature embedding network 10 in the WAAPNet model introduces the WAA module 12, and introduces wavelet transform operations and asymmetric convolution structures in the spatial attention mechanism, which can effectively extract SAR image target features while suppressing speckle noise and reducing computational complexity. Among them, WAA uses wavelet transform to decompose SAR images, and obtains low-frequency features 151 and high-frequency features 152 in three directions respectively. The combination of these four types of features can greatly enhance the model's comprehensive understanding of the SAR image content and improve classification performance. The two asymmetric convolution layers 1211 of the AC module 121 perform feature extraction from the horizontal and vertical directions respectively, which can extract more refined and efficient features of the SAR image and improve the robustness of the network.

[0092] In one embodiment, step S3 of obtaining initial features of the metric classification network 20 by outputting features and feature representations with different weights may include:

[0093] S31: Multiply the output feature map of the WAA module 12 by the feature map of the AC module 121 bit by bit to obtain a feature representation with different weights based on the local part of the image as the initial feature of the metric classification network 20.

[0094] S32: Obtain the prototype of each category in the support set 31 by measuring the initial features of the classification network 20.

[0095] Specifically, the metric classification network 20 includes a prototype learning module 21 and a class similarity module 22. The feature representation based on different local weights of the image output by the feature embedding network 10 is averaged and evaluated by the prototype learning module 21 to obtain the prototype of each class in the support set 31. For example, the prototype representation C of each class in the support set 31 n :

[0096]

[0097] Among them, |S n | represents the number of samples in the sample set of category n in the support set, f φ (x i ) indicates support for sample x i After the feature embedding network f φ The output result is:

[0098] S4: Obtain the similarity between the supporting target image and the query target image based on the initial features of the metric classification network 20, and classify the query target image.

[0099] In one embodiment, the step S4 of acquiring the similarity between the support target image and the query target image based on the initial features of the measurement classification network 20 and classifying the query target image may include:

[0100] S41: Obtain the distance between the feature representations of the query target image with different weights after passing through the feature embedding network 10 and each prototype.

[0101] Specifically, the cosine distance function is used to calculate the local feature representation based on different weights after the query target image is mapped through the feature embedding network 10 and each prototype C n The distance between.

[0102] In metric learning, distance metrics and similarity metrics are generally used as metric functions to calculate the distance between sample features. Among them, the distance calculation formulas include Euclidean distance, Manhattan distance, Chebyshev distance, Minkowski distance, Hamming distance and Mahalanobis distance; the similarity (coefficient) algorithms include cosine similarity, Pearson correlation coefficient, Jaccard similarity coefficient and KL divergence. The two most commonly used and compared methods are Euclidean distance (euc) and cosine distance (cos). Figure 4 The Euclidean distance is the distance between two vectors, which can be obtained by the Pythagorean theorem:

[0103]

[0104] The cosine distance calculates the cosine of the angle between two vectors:

[0105]

[0106] In general, cosine distance is more suitable for high-dimensional data than Euclidean distance, because in high-dimensional space, Euclidean distance will have the so-called "curse of dimensionality" problem, that is, the Euclidean distance will become very large and lose its distinctiveness. In contrast, cosine distance measures the similarity of vectors by calculating the angle between them, which can better handle high-dimensional data. In addition, cosine distance is invariant to the scaling and rotation of the input data, that is, if the input data changes in scale or direction, the result of cosine distance will not be affected. This feature makes cosine distance more robust and reliable than Euclidean distance when processing images, text and other types of data. Therefore, when improving the prototype network, this application replaced the Euclidean distance measurement method in the native network with the cosine distance function.

[0107] S42: Determine the similarity between the supporting target image and the query target image based on the distance to generate a predicted label of the category corresponding to the prototype.

[0108] Specifically, the above distance is used to determine the similarity between the support target image and the query target image, and a predicted label is generated (the category corresponding to the prototype with the closest distance is found).

[0109] S43: Predict the probability that the query target image belongs to the category.

[0110] Specifically, Figure 1 As shown, based on the softmax function, the query target image x is predicted j The probability of belonging to class n is:

[0111]

[0112] In formula (5), cos(f φ (x j ), C n ) is the query target image feature f φ (x j ) to support sample prototype C n The cosine distance.

[0113] S44: Obtain the classification loss of each query target image to which the query target image belongs, so that the query target image is classified into the corresponding category.

[0114] Specifically, calculate the query target image x in a single task j The classification loss L belonging to class n c for:

[0115] L c = -log P(y j =n|x j ) (10)

[0116] By minimizing the query target image x j The classification loss makes f φ (x j ) is closer to the prototype of the true category n in the feature embedding space, so that the query target image x j can be correctly classified.

[0117] During the training phase, the Adaptive Moment Estimation (Adam) optimizer is used to search and update the network parameters to obtain the minimum loss function value.

[0118] Calculate the training set loss L in a single task c The process is shown in Table 1.

[0119] Table 1. Training set loss L c calculate

[0120]

[0121] The test data division is the same as that in the training process. The prototype network is a direct learning method, which is different from inductive learning methods such as meta-learning. Therefore, during testing, the WAAPNet model directly inputs all the support samples of the test set into the trained model, finds the prototype corresponding to each category, and then inputs the query sample into the model to obtain the query sample feature, calculates the similarity between it and the prototypes of different categories, and finally obtains the predicted probability of the category.

[0122] Please refer to Figure 4 , Figure 4 It is a schematic diagram of the Euclidean distance and cosine distance provided in the embodiment of the present application.

[0123] The image processing method disclosed in the present application includes: receiving a support target image and a query target image through a feature embedding network to extract features from the support target image and the query target image to obtain output features; based on the output features, obtaining feature representations of different weights of the support target image and the query target image at different positions of each image; obtaining the initial features of the metric classification network through the output features and the feature representations of different weights; obtaining the similarity of the support target image and the query target image based on the initial features of the metric classification network, and classifying the query target image. The present application designs a WAAPNet model and introduces a WAA module, in which more detailed and efficient features can be extracted while suppressing the speckle noise of the SAR image, so that the network can extract complex features of the SAR image more fully and effectively; the asymmetric convolution structure decomposes the square convolution into horizontal convolution and vertical convolution, which can enhance the feature extraction capability of the convolution kernel center skeleton, thereby strengthening the features of the convolution kernel center and weakening the features of the edge, and can reduce the algorithm complexity of the model and improve the network operation rate. In addition, due to the unfriendliness of the Euclidean distance to high-dimensional data, the present application adopts a more robust cosine distance, which also improves the network classification accuracy to a certain extent.

[0124] In order to solve the above problems, the present application also provides an image processing system.

[0125] like Figures 2-3As shown, the image processing system 100 may include a feature embedding network 10 and a metric classification network 20. The feature embedding network 10 may include multiple groups of convolution blocks 11 and multiple groups of wavelet asymmetric attention modules, namely WAA modules 12, each of which may include a wavelet transform module 120 (Wavelet Transform, WT), a space attention (SpaceAttention, SA) module (SA module 122) and an asymmetric convolution module (AC module 121). Among them, the WT module 120 introduces a wavelet transform operation to perform multi-scale denoising of the target image 15. The WT module 120 performs multi-scale denoising of the target image 15 by extracting low-frequency features 151 and high-frequency features 152 in different directions. Among them, the low-frequency features 151 are used to reflect the shape and size of the target image 15. The low-frequency features 151 are used to obtain the shape and size of the target image 15. The high-frequency features 152 in different directions are used to obtain image texture and edge information, and the high-frequency features 152 in different directions may include high-frequency features 152 in the horizontal direction, the vertical direction, and the diagonal direction. The AC module 121 may include a plurality of asymmetric convolutional layers 1211 .

[0126] The WAA module 12 can extract more detailed and efficient features while suppressing the speckle noise of the SAR image, making the network more effective in extracting complex features of the SAR image; the asymmetric convolution module (AC module 121) decomposes the square convolution into horizontal convolution and vertical convolution, which can enhance the feature extraction capability of the central skeleton of the convolution kernel, thereby strengthening the features of the center of the convolution kernel and weakening the features of the edge, and can reduce the algorithm complexity of the model and improve the network operation rate.

[0127] The metric classification network 20 may include a prototype learning module 21 and a class similarity module 22. The prototype learning module 21 is used to average the feature map obtained by the feature embedding module. The class similarity module 22 is used to obtain the similarity between the query target image and the support target image. The class similarity module 22 can further obtain the similarity between the query target image and the support target image through a cosine metric function.

[0128] For other specific contents about the feature embedding network 10 and the metric classification network 20 of the image processing system 100, please refer to the aforementioned contents, which will not be repeated here.

[0129] In order to make the technical methods and embodiments of the present application easier to understand, the advancement of the present application is described below using comparative examples and the experimental process and verification process of the present application.

[0130] 1. Experimental Setup

[0131] This application evaluates the effectiveness of the proposed method on the public MSTAR dataset and OpenSARShip dataset. In order to evaluate the performance of the proposed WAAPNet model, three-class target and five-class target classification experiments were conducted for the MSTAR dataset, and 3-way K-shot and 5-way K-shot were used as target tasks, where K was 1, 5, and 10. In the 3-way K-shot problem, the experiment will select three types of targets, BMP2, BTR70, and T72 with a pitch angle of 15° for small sample SAR image target classification tasks, and the remaining seven types of military targets with a pitch angle of 17° are used for model training. In the 5-way K-shot problem, the experiment will select five target types, 2S1, BRDM2, BMP2, BTR70, and T72 with a pitch angle of 15° for small sample SAR image target classification tasks, and the remaining five types of targets with a pitch angle of 17° are used for model training. The data in the training set and the test set are disjoint, so as to ensure the effectiveness of the trained network model for the new task set. For the OpenSARShip dataset, 3-way 1-shot and 3-way 5-shot classification experiments were conducted. The experiment selected three types of targets, Tug, Fishing, and Dredging, for the small sample SAR image target classification task, and three types of targets, Cargo, Tanker, and Other, for model training.

[0132] All experiments used the Adam optimizer with an initial learning rate of 0.001. The model was trained for a total of 5,000 iterations, with each iteration containing one task set. The batch size was 32, meaning that 576 images were sent to the network in one batch. To ensure the stability of the model parameters in the later training phase, the learning rate was exponentially decayed, i.e.:

[0133]

[0134] Among them, η 0 is the initial learning rate, g=0.9 is the hyperparameter, τ is the number of iterations, τ b =5000 is the number of training rounds. The data division during testing is the same as that during training, and the output category result is the predicted label of the test sample.

[0135] 2. Comparative Experiment

[0136] (1) Comparison with classic and advanced small sample learning methods

[0137] In order to verify the effectiveness and advancement of the proposed WAAPNet model, it is compared with existing classic and advanced small sample learning methods, such as matching network, prototype network, relation network, transduction propagation network, Meta-baseline, MSAR, MetaDeepBDC and CBLPN. Tables 2 and 3 show the average results of the relevant comparative experiments of WAAPNet and different methods on the MSTAR dataset and OpenSARShip dataset, respectively.

[0138] Table 2

[0139] Comparison of classification accuracy of different methods on MSTAR under 3-way K-shot and 5-way K-shot

[0140]

[0141] As can be seen from Table 2, in the classification tasks other than 5-way 1-shot, the proposed WAAPNet model has the highest classification accuracy for SAR images, which proves the superiority of the WAAPNet model in the small sample target classification task of SAR images. Under the 5-way 1-shot setting, the proposed WAAPNet model did not achieve the highest classification accuracy among all methods. This is because the prototype network itself cannot learn the prototype representation of the category by calculating the average value of the samples in the category when there is only a single training sample, so the classification performance is poor. The WAA module 12 in the WAAPNet model can perform multi-scale denoising on SAR images, and then extract more hierarchical and refined features, so that the model can better learn the prototype representation of different categories, thereby improving the recognition performance of the network.

[0142] Table 3

[0143] Comparison of classification accuracy of different methods in 3-way tasks on OpenSARShip

[0144]

[0145] As can be seen from Table 3, under the settings of 3-way 1-shot and 3-way 5-shot, the proposed WAAPNet model has the highest classification accuracy for SAR images, which fully demonstrates the superiority of the WAAPNet model for ship target recognition in the OpenSARShip dataset. Unlike the MSTAR dataset, the ship target sizes in the OpenSARShip dataset vary greatly. For example, the length of the ship ranges from a few to dozens of pixels, which places high demands on the multi-level and multi-scale feature extraction capabilities of the model. The WAA module 12 in the WAAPNet model uses a wavelet asymmetric attention mechanism to significantly reduce the influence of speckle noise in SAR images, extract deeper and finer features, and avoid generating biased feature distributions in the recognition process of new categories, thereby improving the classification accuracy of the model.

[0146] (2) WAA Module 12 Superiority Verification

[0147] In order to demonstrate the superiority of WAA module 12 in the WAAPNet model in the metric learning network, experiments were conducted by adding WAA module 12, spatial attention mechanism (SA), channel attention mechanism (CA), SENet (SE) and pyramid feature attention (PFA) to the prototype network. Tables 4 and 5 show the experimental comparison results of the above methods on the MSTAR dataset and OpenSARShip dataset respectively.

[0148] Table 4

[0149] Comparison of classification accuracy of different methods on MSTAR under 3-way K-shot and 5-way K-shot

[0150]

[0151] It can be clearly seen from Table 4 that under all experimental settings, after adding the WAA module 12 to the prototype network, the classification performance of the model is improved the most, which proves the superiority of the WAA module 12 over other attention mechanisms. The WAA module 12 proposed in this application first performs multi-scale denoising on the SAR image through a wavelet transform operation, and then extracts image features from the horizontal and vertical directions respectively through an asymmetric convolution structure, which complement each other, so that the model can extract more detailed and highly effective features, thereby improving the classification performance of the network.

[0152] Table 5

[0153] Comparison of classification accuracy of different methods in 3-way tasks on OpenSARShip

[0154]

[0155] The above conclusion can also be drawn from Table 5. Compared with the spatial attention mechanism, channel attention mechanism, SENet and pyramid feature attention, adding WAA module 12 to the feature extraction network of the prototype network can maximize the recognition performance of the model, indicating that WAA module 12 also has unique superior performance in the OpenSARShip dataset recognition task.

[0156] (3) Verification of the versatility of WAA module 12

[0157] In order to prove the versatility of the WAA module 12 in the proposed WAAPNet model in the metric learning network, experiments were conducted by adding the WAA module 12 to the matching network, prototype network, relational network, and transduction propagation network. Tables 6 and 7 are the experimental comparison results of the above methods based on the MSTAR dataset and the OpenSARShip dataset, respectively.

[0158] Table 6

[0159] Comparison of classification accuracy of different methods on MSTAR under 3-way K-shot and 5-way K-shot

[0160]

[0161] It can be clearly seen from Table 6 that under all experimental settings, after adding the WAA module 12 to the above network, the classification performance of the model is greatly improved, which proves that the WAA module 12 has good generalization. Models based on metric learning usually include an embedding module for feature extraction and a metric module for similarity calculation. The metric learning model has gradually matured since its development, and its focus has been on improvements in feature extraction in the embedding module and research on metrics based on neural networks. The WAA module 12 in the WAAPNet proposed in this application, due to the nature of its attention mechanism, can be inserted into the embedding module of the metric learning network in a model-independent manner. While denoising the SAR image, it can extract more in-depth and effective features of the SAR image, so that the model can better learn the feature representation of the target, thereby improving the recognition accuracy of the network.

[0162] Table 7

[0163] Comparison of classification accuracy of different methods in 3-way tasks on OpenSARShip

[0164]

[0165] It can be clearly seen from Table 7 that adding WAA module 12 to the feature extraction network of the metric method can improve the recognition performance of the model to a large extent, indicating that WAA module 12 also has good generalization in the recognition task of the OpenSARShip dataset.

[0166] 3. Ablation Experiment

[0167] In order to verify the effectiveness of the WT module 120, AC module 121, SA module 1222 and cosine distance function in the proposed WAAPNet model, a series of ablation experiments were performed on the 3-way K-shot small sample target recognition task based on the MSTAR dataset.

[0168] (1) Verify the effectiveness of the WT module in WAAPNet in improving network classification accuracy

[0169] The benchmark model in the experiment adopts the WAAPNet model mentioned in this section. Then, based on WAAPNet, a 3×3 square convolution is used to replace the asymmetric convolution module (denoted as WAAPNet-AC), and the WT module 120 is eliminated (denoted as WAAPNet-WT). Then, based on WAAPNet-AC, the WT module 120 is eliminated (denoted as WAAPNet-AC-WT). The above groups of experiments are compared and analyzed, and the comparison results are shown in Table 8.

[0170] Table 8 Comparison of ablation experiments under 3-way K-shot

[0171]

[0172] As can be seen from Table 8, under all 3-way K-shot settings, the proposed WAAPNet model has the best classification performance among all the ablation experiment comparison methods, which shows the advanced nature of the proposed method. For 3-way 1-shot, 3-way 5-shot and 3-way 10-shot, the recognition accuracy of WAAPNet-AC decreased by 2.32%, 1.57% and 3.41% respectively compared with WAAPNet, which shows the effectiveness of the AC module. This is because the AC module strengthens feature extraction from the horizontal and vertical directions respectively, enhances the ability of the central skeleton part of the convolution kernel, and can extract deeper features of SAR images, thereby improving the classification accuracy of the model. Compared with the proposed WAAPNet, WAAPNet-WT in the ablation experiment retains the attention mechanism and asymmetric convolution structure, but its classification accuracy in the three groups of experiments has decreased by 3% to 6%, which proves the importance of wavelet transform operation. Its multi-resolution analysis capability can perform multi-scale denoising on SAR images while maintaining important image features. Since the AC module and WT module are added to the attention mechanism, eliminating the attention mechanism alone will make the AC module and WT module ineffective. Therefore, the ablation experiment of the SA module is based on WAAPNet-AC. In the 3-way1-shot, 3-way 5-shot and 3-way 10-shot recognition tasks, the recognition accuracy of WAAPNet-AC-WT decreased by 5.57%, 3.14% and 7.30% respectively compared with WAAPNet-AC, and by 3.13%, 1.43% and 4.47% respectively compared with WAAPNet-WT. However, WAAPNet-WT decreased by as much as 3% to 6% compared with WAAPNet. It can be seen that although the attention mechanism can suppress useless or inefficient information and find the region of interest in the process of image feature extraction, the presence of speckle noise in SAR images will reduce the extraction performance of the attention mechanism, while the wavelet transform can decompose the details of the SAR image into frequency subbands of different scales, and the threshold processing strategy can be designed to suppress the noise in a targeted manner and maintain the important features of the image.

[0173] In addition, in the 3-way 1-shot and 3-way 10-shot cases, WAAPNet has a relatively obvious improvement compared to WAAPNet-AC-WT. The reason for this is that when there is only one supporting sample, the process of finding the category prototype in the prototype learning module in the metric classification network of the model is meaningless, so the improvement effect is obvious after adding the attention module; on the other hand, when the number of supporting samples is large, the learning effect of the attention mechanism is better.

[0174] (2) Verify the effectiveness of the AC module in WAAPNet in reducing model complexity

[0175] For the WAAPNet and WAAPNet-AC models, batch-size=32, a 3-way5-shot experiment was conducted on the MSTAR dataset to calculate the number of parameters of the feature embedding network of the WAAPNet and WAAPNet-AC models and the running time during the test. The results are shown in Table 9.

[0176] Table 9 Comparison of parameters and running time in a single test task in the 3-way 5-shot case

[0177] Comparison of the number of parameters and running time in a single test task in the 3-way 5-shot case

[0178]

[0179] As can be seen from Table 9, the number of parameters is reduced by about 19% when using the asymmetric convolution module. This is because the AC module 121 consists of a 1×3 and a 3×1 asymmetric convolution, which can be decomposed from a 3×3 square convolution. Decomposing the convolution can reduce model parameters, making the network more lightweight while reducing the risk of overfitting. The AC module 121 can learn horizontal and vertical features separately, and then combine these features for deeper learning, which helps to capture subtle features in SAR images and improve the SAR image target recognition performance. In addition, when using the AC module 121, the running time of a single test task is also reduced. This is because the reduction in the number of model parameters reduces the model complexity and runs at a higher rate.

[0180] (3) Verify the effectiveness of AC module 121 in WAAPNet in improving network robustness

[0181] For the WAAPNet and WAAPNet-AC models, 1000 independent test experiments were constructed respectively. The test accuracy histograms of the two models under the 3-way5-shot setting are shown in the following figure. Figure 4As shown. It can be clearly seen from the figure that compared with WAAPNet-AC, the accuracy distribution of WAAPNet is more concentrated, indicating that its standard deviation is smaller and the model is less affected by the data. Since the effective targets in the SAR images in the experimental data are small, the use of standard square convolutions is easily affected by the rotation of the target and the different aspect ratios, and is prone to problems such as feature information loss. The AC module 121 in the proposed WAAPNet fuses two asymmetric convolutions on the basis of the standard square convolution, and extracts features from the SAR image in the horizontal and vertical directions respectively, which can better capture the texture, shape and other features of the target, thereby improving the feature representation and further improving the performance of the model. After eliminating AC module 121 on the basis of WAAPNet, the variance of its accuracy distribution becomes larger. It can be seen that adding AC module 121 to the model can effectively improve the robustness of the model.

[0182] (4) Verify the effectiveness of cosine distance in WAAPNet

[0183] like Figure 4 As shown in Table 10, relevant ablation comparison experiments were carried out on the prototype network and WAAPNet model using Euclidean distance and cosine distance, and the results are shown in Table 10.

[0184] Table 10. Comparison of ablation experiments under 3-way K-shot conditions

[0185]

[0186] As can be seen from Table 10, under all 3-way K-shot settings, using cosine distance as the network metric function, the recognition accuracy is higher than that using Euclidean distance, which shows the effectiveness of cosine distance for the prototype network. Under 1-shot conditions, using Euclidean distance in the WAAPNet model, the accuracy dropped by 1.71% compared to using cosine distance, which is slightly larger than the 0.86% and 0.83% drops under 5-shot and 10-shot conditions. This is because in the case of a single training sample, the prototype network has poor learning performance for the category prototype, and adding the WAA module 12 to the feature embedding network 10 is not as good as the other two cases. Therefore, the effect of using the cosine distance function is more prominent.

[0187] See also Figure 6 , Figure 6 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0188] The electronic device 200 may specifically include a processor 210 and a memory 220 . The memory 220 is coupled to the processor 210 .

[0189] The processor 210 is used to control the operation of the electronic device 200. The processor 210 may also be referred to as a CPU (Central Processing Unit). The processor 210 may be an integrated circuit chip having signal processing capabilities. The processor 210 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 210 may also be any conventional processor, etc.

[0190] The memory 220 is used to store computer programs, and may be a RAM, a ROM, or other types of storage devices. Specifically, the memory 220 may include one or more computer-readable storage media, which may be non-transitory. The memory may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory is used to store at least one program code.

[0191] The processor 210 is used to execute the computer program stored in the memory 220 to implement the image processing method described in the embodiment of the image processing method of the present application.

[0192] In some embodiments, the electronic device 200 may further include: a peripheral device interface 230 and at least one peripheral device. The processor 210, the memory 220 and the peripheral device interface 230 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 230 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 240, a display screen 250, an audio circuit 260 and a power supply 270.

[0193] The peripheral device interface 230 may be used to connect at least one peripheral device related to I / O (Input / output) to the processor 210 and the memory 220. In some embodiments, the processor 210, the memory 220, and the peripheral device interface 230 are integrated on the same chip or circuit board; in some other implementations, any one or two of the processor 210, the memory 220, and the peripheral device interface 230 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0194] The radio frequency circuit 240 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 240 communicates with the communication network and other communication devices through electromagnetic signals, and the radio frequency circuit 240 is the communication circuit of the electronic device 200. The radio frequency circuit 240 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 240 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 240 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 240 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0195] The display screen 250 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 250 is a touch display screen, the display screen 250 also has the ability to collect touch signals on the surface or above the surface of the display screen 250. The touch signal can be input to the processor 210 as a control signal for processing. At this time, the display screen 250 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 250 can be one, set on the front panel of the electronic device 200; in other embodiments, the display screen 250 can be at least two, respectively set on different surfaces of the electronic device 200 or in a folding design; in other embodiments, the display screen 250 can be a flexible display screen, set on the curved surface or folding surface of the electronic device 200. Even, the display screen 250 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 250 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0196] The audio circuit 260 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 210 for processing, or input them into the radio frequency circuit 240 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the electronic device 200. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 210 or the radio frequency circuit 240 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 260 may also include a headphone jack.

[0197] The power supply 270 is used to power various components in the electronic device 200. The power supply 270 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 270 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged through a wired line, and a wireless rechargeable battery is a battery that is charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0198] For a detailed description of the functions and execution processes of each functional module or component in the embodiment of the electronic device 200 of the present application, reference may be made to the description in the above-mentioned embodiment of the image processing method of the present application, which will not be repeated here.

[0199] In the several embodiments provided in the present application, it should be understood that the disclosed electronic device 200 and image processing method can be implemented in other ways. For example, the various embodiments of the electronic device 200 described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0201] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0202] See also Figure 7 , Figure 7 It is a schematic block diagram of the structure of a computer-readable storage medium provided in an embodiment of the present application.

[0203] See also Figure 7 , if the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium 300. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions / computer programs to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of each embodiment of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical disks, and electronic devices such as computers, mobile phones, laptops, tablet computers, cameras, etc. having the above-mentioned storage media.

[0204] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned application concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. An image processing method, characterized in that: The method comprises the following steps: Receiving a support target image and a query target image through a feature embedding network to extract features from the support target image and the query target image to obtain output features; Based on the output features, obtaining feature representations of the support target image and the query target image at different positions of the respective images with different weights; Obtaining initial features of a metric classification network through the output features and the feature representations with different weights; The similarity between the support target image and the query target image is obtained based on the initial features of the metric classification network, and the query target image is classified.

2. The method according to claim 1, characterized in that The output features are output features of a wavelet asymmetric attention (WAA) module; the WAA module includes a wavelet transform module (WT), a spatial attention module (SA) and an asymmetric convolution (AC) module; The step of receiving a support target image and a query target image through a feature embedding network to extract features from the support target image and the query target image to obtain output features includes: A wavelet transform operation is introduced through the WT module to extract low-frequency features and high-frequency features in different directions of the support target image and the query target image for multi-scale noise reduction; the low-frequency features are used to obtain the shape and size of the target image; the high-frequency features in different directions are used to obtain image texture and edge information; the high-frequency features in different directions include obtaining the low-frequency features, high-frequency features in the horizontal direction, high-frequency features in the vertical direction, and high-frequency features in the diagonal direction; the high-frequency features include directional features and subtle features in the image; The supporting target image and the query target image are decomposed into two asymmetric convolutional layers by the AC module to obtain a feature map of the AC module.

3. The method according to claim 2, characterized in that The wavelet transform operation is introduced through the WT module to extract low-frequency features of the support target image and the query target image and high-frequency features in different directions for multi-scale noise reduction, including: The low-frequency features, the high-frequency features in the horizontal direction, the high-frequency features in the vertical direction, and the high-frequency features in the diagonal direction are spliced ​​to obtain primary features after noise reduction.

4. The method according to claim 3, characterized in that Decomposing the support target image and the query target image into two asymmetric convolutional layers through the AC module to obtain a feature map of the AC module includes: Convolutionally performing a convolution operation on the primary features through an asymmetric convolution (AC) module to obtain a plurality of asymmetric convolutional layers; Adding a plurality of the asymmetric convolutional layers and passing them through an activation function to obtain a spatial weight tensor; The output features of the WAA module are obtained through the feature map and the spatial weight tensor.

5. The method according to claim 4, characterized in that The obtaining of initial features of the metric classification network through the output features and the feature representations with different weights includes: Multiplying the output feature map of the WAA module by the feature map of the AC module bit by bit to obtain the feature representation of the different weights based on the local part of the image as the initial feature of the metric classification network; The prototype of each category of the support set is obtained by using the initial features of the metric classification network.

6. The method according to claim 5, characterized in that The obtaining the similarity between the support target image and the query target image based on the initial features of the metric classification network and classifying the query target image includes: Obtaining the distance between the feature representations of the query target image with different weights after passing through the feature embedding network and each of the prototypes; wherein the distance is obtained by a cosine distance function; Determining the similarity between the support target image and the query target image based on the distance to generate the predicted label of the category corresponding to the prototype; Predicting the probability that the query target image belongs to the category; The classification loss of each query target image belonging to the category is obtained, so that the query target image is classified into the category corresponding to it.

7. An image processing system, characterized in that: include: A feature embedding network, comprising a plurality of groups of convolution blocks and a plurality of groups of wavelet asymmetric attention (WAA) modules, each WAA module comprising a wavelet transform module (WT), a spatial attention module (SA) and an asymmetric convolution (AC) module; wherein the WT module introduces a wavelet transform operation to perform multi-scale denoising of a target image; and the AC module comprises a plurality of asymmetric convolution layers; The metric classification network includes a prototype learning module and a class similarity module. The prototype learning module is used to average the feature map obtained by the feature embedding module; the class similarity module is used to obtain the similarity between the query target image and the support target image.

8. The image processing system according to claim 7, characterized in that: The class similarity module obtains the similarity between the query target image and the support target image through a cosine metric function; The WT module performs multi-scale denoising on the target image by extracting low-frequency features and high-frequency features in different directions; wherein the low-frequency features are used to reflect the shape and size of the target image; the low-frequency features are used to obtain the shape and size of the target image; the high-frequency features in different directions are used to obtain image texture and edge information, and the high-frequency features in different directions include high-frequency features in horizontal, vertical and diagonal directions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image processing method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium, characterized in that: A program file is stored, and the program file can be executed to implement the method according to any one of claims 1 to 6.

Images