Small sample water surface target detection method based on wavelet parallel deep meta-learning

Abstract

The application discloses a target detection method based on wavelet parallel deep network meta-learning, which comprises the following steps: step 1, constructing a meta-learning framework combined with a neural network; step 2, dividing an image training set into two parts; step 3, training the meta-learning framework through pre-processing of the training set; step 4, training the meta-learning framework by taking images in a common training set as a query set on the basis of the meta-learning framework trained in step 3, and completing the training of the meta-learning framework; and step 5, putting real-time images into the trained meta-learning framework to detect targets in the images. The application combines the advantages of the neural network and the meta-learning, so that the training speed is faster and the training accuracy is higher; meanwhile, the idea of multi-resolution analysis is introduced, the image features are strengthened under multiple resolutions, and the detection effect on water surface targets is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a small sample water surface target detection method based on wavelet parallel deep meta learning. BACKGROUND

[0002] The wavelet neural network is a feedforward neural network model organically combining wavelet transform and neural network. The wavelet neural network shows excellent zooming capability and time-frequency local characteristics for non-stationary signals, and helps to improve the prediction accuracy. The wavelet neural network combines the advantages of wavelet analysis theory and artificial neural network, and thus has strong advantages in image approximation and processing of nonlinear problems.

[0003] Meta learning mainly solves the problem of learning to learn, that is, using past knowledge and experience to guide the learning of new tasks, so that the network has the ability to learn to learn. Combining meta learning for target detection usually has the problems of insufficient model generalization ability and slow network convergence speed, and the problems are more prominent in the case of small samples.

[0004] Target detection is the basis of computer vision tasks such as image segmentation and target tracking. There are two kinds of mainstream target detection algorithms at present: two-stage target detection algorithms R-CNN, Faster R-CNN and the like based on region proposal, and single-stage target detection algorithms such as YOLO series, RetinaNet, EfficientDet and the like based on regression analysis. At present, the detection effect of the target detection algorithm based on the deep learning model depends largely on the quality and quantity of the data set, so as to cause the problems of poor model generalization ability and slow network convergence speed. SUMMARY

[0005] In view of the deficiencies in the prior art, the present application provides a small sample water surface target detection method based on wavelet parallel deep meta learning, to solve the technical problems of low detection efficiency, low detection precision and poor generalization ability in the prior art.

[0006] The present application provides a small sample water surface target detection method based on wavelet parallel deep meta learning, comprising the following steps:

[0007] Step 1: constructing a meta learning framework combined with a neural network, specifically: setting the meta learning training task target as predicting the labeling information of other images according to the labeling information in the given image; taking two neural networks as a network encoder and a node evaluator of the meta learning framework respectively;

[0008] Step 2: divide the image training set into two parts, preprocess one part of the image training set to form a preprocessed training set; the other part of the image training set is not preprocessed to form a normal training set, wherein the preprocessing process is:

[0009] Step 2: divide the image training set into two parts, preprocess one part of the image training set to form a preprocessed training set; the other part of the image training set is not preprocessed to form a normal training set, wherein the preprocessing process is:

[0010] Step 3: train the meta-learning framework through the preprocessed training set, specifically: take the original image as the input image; take three sub-images of the original image as the support set; take another sub-image of the original image as the query set, and train the meta-learning framework;

[0011] Step 4: based on the meta-learning framework trained in step 3, train the meta-learning framework with the images in the normal training set as the query set, complete the training of the meta-learning framework;

[0012] Step 5: put the real-time image into the trained meta-learning framework to detect the target in the image.

[0013] Further, in step 1, two identical neural networks are selected as the network encoder and node evaluator of the meta-learning framework, or two different neural networks are selected as the network encoder and node evaluator of the meta-learning framework.

[0014] Further, in step 1, a multi-level wavelet pooling structure is used as the pooling layer between adjacent convolution layers in the neural network; a wavelet function is used as the activation function of the full connection layer, and a mother wavelet function is used as the transfer function of the hidden layer node.

[0015] Further, the multi-level wavelet pooling structure includes two parts: image multi-resolution analysis and sub-band recombination.

[0016] Further, the multi-level wavelet pooling structure starts from the first convolution layer, uses two different wavelet basis functions to perform multi-resolution analysis on the input image, generates four wavelet transform sub-bands LL, LH, HL and HH as output, and connects any combination of the four sub-bands to the next convolution layer to form four parallel channels, each channel has a pooling function, and the optimal channel is selected through the evaluation result of the node evaluator.

[0017] Further, in step 2, the specific process of multi-resolution analysis on the input image is: first, perform row decomposition on the input image to obtain low-frequency components of the image in horizontal and vertical directions, low-frequency components of the image in horizontal direction and high-frequency components in vertical direction, high-frequency components of the image in horizontal direction and low-frequency components in vertical direction, and high-frequency components of the image in horizontal and vertical directions; then, reconstruct the four groups of components respectively to obtain four sub-images with different resolutions from the input image.

[0018] Further, in step 3, the meta-learning framework is trained, and the meta-learning training task of each round is as follows:

[0019] S m ={(v1,y1),(v2,y2),...,(v N×K ,y N×K )}

[0020]

[0021] T m ={S m ,Q m}

[0022] Wherein, T m represents the meta-learning training task, S m represents the support set, Q m represents the query set, N represents the number of categories of the data set, K represents the number of nodes of each class contained in the support set S m , M represents the number of nodes of each class contained in the query set T m , (v i , y i ) represents the node of each class, which is given in the form of coordinates.

[0023] The beneficial effects of the application are:

[0024] 1. A small sample water surface target detection method based on wavelet parallel deep meta-learning, which combines the advantages of neural networks and meta-learning, makes the training speed faster and the accuracy higher, and at the same time introduces the idea of multi-resolution analysis, strengthens the image features at multiple resolutions, and further improves the detection effect of water surface targets.

[0025] 2. A small sample water surface target detection method based on wavelet parallel deep meta-learning, which constructs a new meta-learning training framework to effectively improve the training speed of deep learning neural network and further improve the efficiency.

[0026] 3. A small sample water surface target detection method based on wavelet parallel deep meta-learning, which constructs an evaluation method combining attention mechanism and Softmax normalization to evaluate the training effect of meta-learning, so that the network converges faster and is more likely to reach the optimal parameters.

[0027] 4. A small sample water surface target detection method based on wavelet parallel deep meta learning, a multi-level wavelet pooling structure is constructed in the neural network structure, the pooling layer in the traditional neural network is optimized, and a parallel pooling channel is designed, which not only retains the low frequency information, but also retains the high frequency information, prevents the loss of information of the input image, prevents the image size from increasing all the time, and further improves the speed and accuracy of target detection.

[0028] 5. A small sample water surface target detection method based on wavelet parallel deep meta learning, which fully utilizes the output results of wavelet parallel image preprocessing, divides them into a training set and a support set, introduces the idea of meta learning in training to construct a learning task, predicts the vertex label in a given graph, and selects the image with the lowest loss for training, so that the model can more accurately use prior information and improve the generalization ability of the model. BRIEF DESCRIPTION OF DRAWINGS

[0029] The features and advantages of the present application will be more clearly understood through reference to the following drawings, which are schematic and should not be construed as limiting the application, in which:

[0030] Figure 1 is a flowchart of a specific embodiment of the present application;

[0031] Figure 2 is a schematic diagram of a meta learning framework of a specific embodiment of the present application combined with a neural network;

[0032] Figure 3 is a schematic diagram of wavelet parallel image preprocessing constructed by a specific embodiment of the present application;

[0033] Figure 4 is a schematic diagram of a multi-level parallel wavelet pooling structure constructed by a specific embodiment of the present application;

[0034] Figure 5 is a result diagram after multi-resolution analysis of an image by a specific embodiment of the present application;

[0035] Figure 6 is a result diagram of a neural network combined with meta learning for target framing by a specific embodiment of the present application;

[0036] Figure 7 is a comparison result of the accuracy of the network and other networks realized by a specific embodiment of the present application;

[0037] Figure 8 is a comparison result of the loss value of the network and other networks realized by a specific embodiment of the present application;

[0038] Figure 9 is a result diagram of a network for detecting an image realized by a specific embodiment of the present application;

[0039] Figure 10 is a result diagram when the existing Swin-Transform network detects a picture. DETAILED DESCRIPTION

[0040] To make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0041] The present application will be described in further detail below with reference to specific embodiments. Those skilled in the art should understand that these embodiments are only used to illustrate the present application and not to limit the scope of the present application. Various modifications of the equivalent forms of the present application are all within the scope of the appended claims of the present application.

[0042] The present application provides a small sample water surface target detection method based on wavelet parallel deep meta-learning, as shown in Figure 1 , comprising the following steps:

[0043] Step 1: Construct a meta-learning framework combined with a neural network, specifically: set the meta-learning training task goal as predicting the labeling information in other pictures according to the labeling information in a given picture; take two neural networks as the network encoder and node evaluator of the meta-learning framework respectively;

[0044] The meta-learning framework combined with a neural network and its training process are as shown in Figure 2 , wherein g θ is the network encoder, and f θ is the node evaluator.

[0045] The meta-learning training task of each round is as follows:

[0046] S m ={(v1, y1), (v2, y2),..., (v N×K , y N×K )}

[0047]

[0048] T m ={S m , Q m}

[0049] , wherein T m represents the meta-learning training task, S m represents the support set, and Q mN represents the number of classes of the data set, K represents the support set S m The number of nodes contained in each class, M represents the query set T m The number of nodes contained in each class, (v i , y i ) represents the nodes of each class, given in coordinate form.

[0050] The specific implementation of the node evaluator is as follows:

[0051] First, stack multiple layers of CNN to obtain the dependency between samples:

[0052] T 1 = CNN 1 (A, X),

[0053] …

[0054] Z = CNN L (A, T L-1 )

[0055] A, X are the feature points of the support set and the query set input into the network, T 1 represents the output nodes of the first layer network, Z is the node representation learned from the network encoder, and the calculation method of the class prototype is:

[0056]

[0057] Where Sc represents the labeled example set, P ROTO is a prototype calculation function, and z i represents each node.

[0058] A scoring aggregation layer is constructed using a shared attention mechanism to calculate the importance score of each node, and then the scores are normalized using the Softmax function. Finally, the loss between the prediction and the true value of each query set is calculated, which can be defined as the average negative log likelihood probability of matching the correct label.

[0059] The defined scoring aggregation layer is as follows:

[0060]

[0061] Where is the importance score of the Lth layer node v i , and is the attention weight between nodes v i and v j .

[0062] The formula for calculating its score through the shared attention mechanism is:

[0063]

[0064] where || is a concatenation operator, a T is the transpose of a weight vector.

[0065] Step 2: Divide the image training set into two parts, pre-process one part of the image training set to form a pre-processed training set; the other part of the image training set is not pre-processed to form a normal training set.

[0066] wherein the pre-processing process is:

[0067] Performing multi-resolution analysis on the input image to obtain four sub-images with different resolutions from the input image;

[0068] The basic principle is:

[0069] Discrete wavelet transform operation is performed on the image using a wavelet function, that is, the image is first decomposed in rows to obtain the low-frequency component L and the high-frequency component H in the horizontal direction of the original image, and then each column of the transformed data is decomposed in columns to obtain the low-frequency component LL1, the low-frequency in the horizontal direction and the high-frequency in the vertical direction LH1, the high-frequency in the horizontal direction and the low-frequency in the vertical direction HL1, and the high-frequency components HH1 in the horizontal and vertical directions of the original image. Retain all subbands, and reconstruct the four groups of components respectively, that is, four sub-images with different resolutions from the input image can be obtained, which all retain the outline of the original image while strengthening the details and features, and provide a method for increasing samples in the case of insufficient samples.

[0070] The wavelet basis function here is Morlet wavelet.

[0071] Step 3: Train the meta-learning framework through the pre-processed training set, specifically: take the original image as the input image; take three sub-images of the original image as the support set; take the other sub-image of the original image as the query set, and train the meta-learning framework;

[0072] The schematic diagram of the multi-resolution analysis process of the image is shown in Figure 3 . Taking one resolution of one image as an example, the pre-processing result of the image is visualized, and the schematic diagram is shown in Figure 5 .

[0073] The initial importance score needs to be calculated at the beginning of the training, and a non-linear feedforward layer is used to compress the node features. The initial score of the node v i is:

[0074]

[0075] wherein is a learnable weight vector, b s is a bias corresponding to each node, which can be defined by itself, and is set to 0.2 in the embodiment.

[0076] After the initial importance score is obtained, the network starts training, and the label information node of the learned image is used as the query node v * of the unlearned image to define a predictor, and the distance between the query node v * and each learned image label information node is used to calculate a probability for each class c of the data set, and the specific formula is as follows:

[0077]

[0078] The label with the maximum probability is assigned as the correct label, and is compared with the support set to calculate the classification loss of each meta-training task, and the loss is defined as the average negative log likelihood probability of the assigned correct class label, and the specific formula is as follows:

[0079]

[0080] Since the multi-level parallel pooling structure is adopted, that is, the classification loss is calculated for each channel, the loss values of each channel are compared, and the network training parameters are updated according to the channel with the minimum loss value.

[0081] The schematic diagram of the multi-level wavelet pooling structure is shown in Figure 4 The principle is that after each convolution, the pooling layer is used to solve the problem of image size increasing after convolution, and to reduce overfitting.

[0082] In the embodiment, for a given image P with a size of (n, n, m), first, two different wavelet bases are selected according to the characteristics of the water surface target detection data set; then the input image features are extracted and converted into matrix form, and DWT discrete wavelet transform is performed on the two matrices.

[0083] The matrix form of the image features extracted by the two different wavelet bases is as follows:

[0084]

[0085]

[0086] The above H and G are matrices with a size of ,

[0087] And the corresponding DWT operation is:

[0088]

[0089]

[0090] where P is the matrix form corresponding to the original image. After the DWT operation, LL, HL, LH and HH at different resolutions are obtained, which are combined in pairs using a parallel manner to obtain new image features, and the two results are connected and transmitted to the next layer of convolutional neural network, and the working process is as shown in Figure 4 The neural network in all parallel channels repeats the above pooling process.

[0091] Step 4: On the basis of the meta-learning framework trained in step 3, the images in the ordinary training set are used as the query set to train the meta-learning framework, and the training of the meta-learning framework is completed;

[0092] In this step, the images in the ordinary training set are not preprocessed, but are directly trained in the meta-learning framework, and node evaluation and parameter updating are also performed, and finally the optimal model is obtained;

[0093] Step 5: Put the real-time image into the trained meta-learning framework to detect the target in the image.

[0094] The visualization result of the detection process is as shown in Figure 6

[0095] Figure 7 and Figure 8 The comparison results of the wavelet parallel deep meta-learning network, Swin Transformer and Fast R-CNN network under the same data set are given, wherein Figure 7 The accuracy of the three networks is compared, Figure 8 The loss values of the three networks are compared. It can be seen that the wavelet parallel deep meta-learning network according to the present application is superior to the other two classical networks in terms of accuracy and loss value.

[0096] In this embodiment, when the naval ship is detected, it can be seen that the wavelet parallel deep meta-learning network according to the present application can accurately detect the target and output the accuracy, as shown in Figure 9 Figure 10 The result of detecting the same target by using Swin-transformer can be seen, and the accuracy of the network according to the present application is obviously higher than the prediction effect of Swin-transformer.

[0097] ​​The application introduces the multi-resolution analysis thought, the meta learning thought and the wavelet pooling thought for the target detection problem. For different data sets, the wavelet base function and the backbone network can be replaced to adapt to different situations, so as to achieve the best detection effect. Finally, experiments are conducted on a small-scale embodiment constructed on a ship classification data set. The experiments show that the network structure can still quickly converge when facing new target categories during testing, and has high accuracy and strong generalization ability in the case of small data set and poor data set quality.

[0098] Although the embodiments of the present application are described in conjunction with the drawings, various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application, and such modifications and changes fall within the scope defined by the appended claims.

Claims

1. A small sample water surface target detection method based on wavelet parallel deep meta-learning, characterized in that, The method comprises the following steps: Step 1: constructing a meta-learning framework combined with a neural network, specifically: setting the meta-learning training task target as predicting the label information of other graphs according to the label information of a given graph; taking two neural networks as the network encoder and node evaluator of the meta-learning framework respectively; and using a multi-level wavelet pooling structure as the pooling layer between adjacent convolution layers in the neural network, wherein the multi-level wavelet pooling structure comprises image multi-resolution analysis and sub-band reorganization two parts; Step 2: dividing the image training set into two parts, pre-processing one part of the image training set to form a pre-processed training set; and not pre-processing the other part of the image training set to form a normal training set, wherein the pre-processing process is as follows: performing multi-resolution analysis on the input image to obtain four sub-images with different resolutions from the input image; Step 3: training the meta-learning framework through the pre-processed training set, specifically: taking the original image as the input image; taking three sub-images of the original image as the support set; taking the other sub-image of the original image as the query set, and training the meta-learning framework; Step 4: training the meta-learning framework through the images in the normal training set as the query set based on the meta-learning framework trained in step 3, to complete the training of the meta-learning framework; Step 5: putting the real-time image into the trained meta-learning framework to detect the target in the image.

2. The small sample water surface target detection method based on wavelet parallel deep meta-learning according to claim 1, wherein, In step 1, two identical neural networks are selected as the network encoder and node evaluator of the meta-learning framework, or two different neural networks are selected as the network encoder and node evaluator of the meta-learning framework.

3. The small sample water surface target detection method based on wavelet parallel deep meta-learning according to claim 1 or 2, characterized in that, In step 1, a wavelet function is used as the activation function of the full connection layer, and a mother wavelet function is used as the transfer function of the hidden layer node.

4. The small sample water surface target detection method based on wavelet parallel deep meta-learning according to claim 1, wherein, The multi-level wavelet pooling structure starts from the first convolution layer, uses two different wavelet basis functions to perform multi-resolution analysis on the input image, generates four wavelet transform sub-bands LL, LH, HL and HH as output, connects any combination of the four sub-bands to the next convolution layer to form four parallel channels, each channel has a pooling function, and the optimal channel is selected through the evaluation result of the node evaluator.

5. The small sample water surface target detection method based on wavelet parallel deep meta-learning according to claim 1, wherein, In step 2, the specific process of performing multi-resolution analysis on the input image is as follows: first, perform row decomposition on the input image to obtain the low-frequency components of the image in the horizontal and vertical directions, the low-frequency components of the image in the horizontal direction and the high-frequency components of the image in the vertical direction, the high-frequency components of the image in the horizontal direction and the low-frequency components of the image in the vertical direction, and the high-frequency components of the image in the horizontal and vertical directions; and then reconstruct the four groups of components to obtain four sub-images with different resolutions from the input image.

6. The small sample water surface target detection method based on wavelet parallel deep meta-learning according to claim 1, wherein, In step 3, the meta-learning framework is trained, and the meta-learning training task of each round is as follows: where T m represents the meta-learning training task, S m represents the support set, Q m represents the query set, N represents the number of classes of the dataset, K represents the number of nodes of each class contained in the support set S m , M represents the number of nodes of each class contained in the query set T m , (v i , y i ) represents the nodes of each class, given in coordinate form.

Images