A small sample target detection and recognition method, device and computing equipment

Abstract

The application provides a small sample target detection and recognition method and device and computing equipment, the method comprising: acquiring a small sample target data set in a target domain; training an initial detection network model and an initial classification network model pre-trained by a source domain data set using the small sample target data set to obtain a target detection network model and a target classification network model; inputting a to-be-detected and recognized image of the target domain into the target detection network model to obtain a target image and a regression result; inputting the target image and semantic information, attribute information and image information related to the target image into the target classification network model to obtain a classification result; and obtaining a recognition result of the to-be-detected and recognized image according to the regression result and the classification result. The small sample target detection and recognition method provided by the scheme can realize accurate recognition of small sample data and improve the recognition accuracy based on small sample data.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method, apparatus and computing device for small sample target detection and recognition. Background Technology

[0002] Object detection and recognition remains a popular research area in computer vision. Its wide range of engineering applications has led to rapid development and innovation in academic research. Intelligent recognition systems typically rely on deep learning-based object recognition techniques, but several challenges remain: 1) Object recognition requires a large amount of training data and diverse training data types to meet accuracy requirements. However, in most practical applications, harsh environments and hardware limitations make it difficult to obtain large amounts of high-quality data. 2) Traditional methods process only single image data, neglecting additional information for guidance. This leads to overfitting in limited-sample object recognition tasks, resulting in low detection accuracy. Furthermore, under limited-sample conditions, overfitting occurs with increasing training iterations. Reducing network parameters can mitigate overfitting to some extent, but shallower network structures also affect the final accuracy. These problems have made it difficult for the performance of the target recognition model to meet the technical requirements in practical applications. Therefore, how to build a target recognition model using a small amount of sample data has become a bottleneck problem for academia, industry and even defense in recent years. Summary of the Invention

[0003] This invention provides a method, apparatus, and computing device for small sample target detection and recognition, which can achieve accurate recognition of small sample data and improve recognition accuracy.

[0004] In a first aspect, the present invention provides a method for small-sample target detection and recognition, comprising:

[0005] Obtain a small sample target dataset within the target domain;

[0006] The initial detection network model and the initial classification network model, which were pre-trained from the source domain dataset, are used to train the small sample target dataset to obtain the target detection network model and the target classification network model.

[0007] The target image to be detected and recognized is input into the target detection network model to obtain the target image and regression result; the semantic information, attribute information and image information related to the target image are input into the target classification network model to obtain the classification result.

[0008] Based on the regression results and the classification results, the recognition result of the image to be detected is obtained.

[0009] Optionally, the source domain dataset includes a public dataset; wherein the initial detection network model is obtained by pre-training the pre-built detection network model using the public dataset.

[0010] The detection network model includes a detection encoder, a gradient decoupling layer, a region of interest extraction network, a region of interest alignment structure, and a regression module. The input of the detection network model is an image from the public dataset, and the output is a regression result, which contains the location information of the target in the image.

[0011] Optionally, the initial detection network model and the target detection network model are constrained by a first loss function during training;

[0012] The first loss function is determined by the following formula:

[0013]

[0014]

[0015]

[0016] Where, p i The prediction category score used to characterize the i-th prediction box; The classification score used to characterize the i-th predicted bounding box; t i Regression parameters used to characterize the i-th prediction box; Used to characterize the true regression parameters corresponding to the i-th prediction box; L cls Used to characterize the classification loss function; L reg Used to characterize the regression loss function; N cls The number of predicted boxes used to characterize the output; N reg The number of positive samples is used to characterize the number of samples; λ is a hyperparameter.

[0017] Optionally, the source domain dataset includes a historical image dataset; wherein, the pre-built classification network model is pre-trained using the historical image dataset to obtain the initial classification network model;

[0018] The classification network model includes a semantic encoder, an attribute encoder, an attribute predictor, an image encoder, a multi-source class center feature fusion module, and an image attribute feature fusion module. The input of the classification network model is an image in the historical image dataset and the corresponding semantic information, attribute information, and image information of that image. The output is the recognition result of that image.

[0019] Optionally, the step of inputting the target image to be detected into the target detection network model to obtain the target image and regression results includes:

[0020] Based on the regression results, the image to be detected and identified is cropped to obtain the target image; wherein, the regression results include the location information of the target in the image to be detected and identified.

[0021] Optionally, the step of inputting the semantic information, attribute information, and image information related to the target image into the target classification network model to obtain the classification result includes:

[0022] The target image is input into the target classification network model to predict the attribute features of the target image and obtain the attribute prediction vector; in parallel, the target image is transformed into a feature vector to obtain a first image feature vector.

[0023] The attribute prediction vector and the first image feature vector are fused to obtain the image attribute feature vector;

[0024] The target image is parsed to obtain the semantic information, attribute information, and image information related to the target image;

[0025] The semantic information, the attribute information, and the image information are input into the target classification network model to obtain the semantic feature vector, the attribute feature vector, and the second image feature vector, respectively.

[0026] The obtained semantic feature vector, the attribute feature vector, and the second image feature vector are fused to obtain a fused class center feature vector;

[0027] The similarity between the image attribute feature vector and the fused class center feature vector is calculated to obtain the classification result.

[0028] Optionally, obtaining the recognition result of the image to be detected based on the regression result and the classification result includes:

[0029] The regression results and the classification results are subjected to non-maximum suppression processing to obtain the recognition results; wherein, the recognition results include the attribute information corresponding to the target in the image to be detected.

[0030] Secondly, the present invention also provides a small sample target detection and recognition device, comprising:

[0031] The acquisition module is used to acquire a small sample target dataset within the target domain;

[0032] The training module is used to train the initial detection network model and the initial classification network model pre-trained from the source domain dataset using the small sample target dataset, so as to obtain the target detection network model and the target classification network model.

[0033] The detection and recognition module is used to input the target image to be detected and recognized in the target domain into the target detection network model to obtain the target image and regression results; input the semantic information, attribute information and image information related to the target image into the target classification network model to obtain the classification results; and obtain the recognition result of the target image based on the regression results and the classification results.

[0034] Thirdly, the present invention also provides a computing device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the small sample target detection and recognition method described in any of the above claims.

[0035] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the small sample target detection and recognition method described in any of the preceding claims.

[0036] This invention provides a method, apparatus, and computing device for small-sample object detection and recognition. The method first pre-trains an initial detection network model and an initial classification network model using a source domain dataset. Then, it trains these models using a small-sample target dataset within the target domain to obtain an object detection network model and a target classification network model. The object detection network model is then used to obtain the target image and regression results for the image to be detected and recognized. The target classification network model is then used to obtain the classification results of the target image. Finally, the recognition result of the image to be detected and recognized is obtained based on the regression and classification results. Thus, given a small-sample target dataset, by using an initial model pre-trained on the source domain dataset and then fine-tuning the initial model in the target domain to obtain the target model, the semantic, attribute, and image information contained in the sample data within the target domain can be fully extracted to obtain accurate category feature representations, achieving accurate recognition of small-sample data. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of a small sample target detection and recognition method provided in an embodiment of the present invention;

[0039] Figure 2 This is a structural diagram of a detection network model provided in an embodiment of the present invention;

[0040] Figure 3 This is a structural diagram of a classification network model provided in an embodiment of the present invention;

[0041] Figure 4 This is a network model structure diagram of an attribute predictor provided in an embodiment of the present invention;

[0042] Figure 5 This is a hardware architecture diagram of a computing device provided in an embodiment of the present invention;

[0043] Figure 6 This is a structural diagram of a small sample target detection and recognition device provided in an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0045] Due to the limited sample size, it is impossible to establish an effective mapping relationship between sample data and categories. Traditional target recognition methods cannot effectively integrate the semantic information of the target into the recognition process, nor can they construct intelligent target recognition models that incorporate additional auxiliary information. This results in low recognition performance in finite sample problems, failing to meet the technical needs of intelligent target recognition in the new era. To address these issues, this invention introduces external prior information to assist the model in establishing better general feature extraction and class center establishment, improving the generalization ability and feature extraction capability of the network model, thereby increasing the accuracy of target detection and recognition in small samples.

[0046] The following describes the specific implementation of the concept in this application.

[0047] Please refer to Figure 1 This invention provides a method for small-sample target detection and recognition, the method comprising:

[0048] Step 100: Obtain a small sample target dataset within the target domain;

[0049] Step 102: Train the initial detection network model and the initial classification network model obtained by pre-training the source domain dataset using the small sample target dataset to obtain the target detection network model and the target classification network model;

[0050] Step 104: Input the target image to be detected into the target detection network model to obtain the target image and regression results; input the semantic information, attribute information and image information related to the target image into the target classification network model to obtain the classification results;

[0051] Step 106: Based on the regression results and classification results, obtain the recognition results of the image to be detected.

[0052] In this embodiment of the invention, an initial detection network model and an initial classification network model are first pre-trained using a source domain dataset. Then, a small sample target dataset within the target domain is used to train these initial detection and classification network models, resulting in a target detection network model and a target classification network model. The target detection network model is then used to obtain the target image and regression results for the image to be detected and recognized. The target classification network model is then used to obtain the classification results for the target image. Finally, the recognition result for the image to be detected and recognized is obtained based on the regression and classification results. Thus, given a small sample target dataset, by using an initial model pre-trained on the source domain dataset and then fine-tuning the initial model in the target domain to obtain the target model, the semantic, attribute, and image information contained in the sample data within the target domain can be fully extracted, resulting in accurate category feature representation and achieving accurate recognition of small sample data.

[0053] The following description Figure 1 The execution method for each step is shown.

[0054] First, for step 100, the small sample target dataset includes a visible light dataset and an infrared dataset. Specifically, the final visible light target detection network model and visible light target classification network model are trained on the visible light dataset; the final infrared target detection network model and infrared target classification network model are trained on the infrared dataset. More specifically, for example, if the target classification network model contains five categories, then 8 samples are selected from each category in the visible light dataset and the infrared dataset, i.e., 40 samples are selected from each dataset. These 40 samples are labeled with attributes (e.g., 1 represents that the image contains the attribute, and 0 represents that the image does not contain the attribute, forming attribute file information; the category information of each image, i.e., the English name or Chinese name, is used as the semantic information corresponding to the image), and used as the training data set for fine-tuning the target classification network model and the target detection network model; the remaining visible light and infrared data are manually labeled and used as the test dataset in the target domain for performance testing of the final target model.

[0055] Then, in step 102, the source domain dataset includes a public dataset and a historical image dataset; wherein, the public dataset is used to pre-train the pre-built detection network model to obtain an initial detection network model; the historical image dataset is used to pre-train the pre-built classification network model to obtain an initial classification network model;

[0056] like Figure 2 As shown, the detection network model includes a detection encoder, a gradient decoupling layer, a region of interest extraction network, a region of interest alignment structure, and a regression module. The input of the detection network model is an image from a public dataset, and the output is a regression result containing the location information of the target in the image.

[0057] like Figure 3 and Figure 4 As shown, the classification network model includes a semantic encoder, an attribute encoder, an attribute predictor, an image encoder, a multi-source class center feature fusion module, and an image attribute feature fusion module. The input of the classification network model is an image in the historical image dataset and the corresponding semantic information, attribute information, and image information of that image. The output is the recognition result of that image.

[0058] It should be noted that the public dataset can be the object detection image data in the PASCAL VOC dataset, used for end-to-end pre-training of the detection network model to obtain the initial detection network model. The historical image dataset can be images of known targets and their classifications collected from the network or existing databases; each image is manually annotated with fine-grained image attributes to form corresponding attribute information (in this invention, this refers to a series of characteristic combinations abstracted from the object category and image context information in the image; for example, attribute information for the category of tank includes gun barrel, tracks, etc.; attribute information for the category of aircraft includes engine, wings, tail, landing gear, etc.). At the same time, semantic information is formed by using the Chinese names (e.g., barracks, tank, aircraft) or English words (e.g., barracks, tank, plane) corresponding to the actual categories of known targets in each image.

[0059] In one specific implementation, for example, the detection encoder in the detection network adopts the ResNet50 network model; the gradient decoupling layer consists of a fully connected layer; the region of interest extraction network mainly consists of a 3×3 convolutional layer and two parallel 1×1 convolutional layers; the region of interest alignment structure mainly uses the bilinear interpolation algorithm to unify the feature maps of the region of interest to the same size; and the regression module mainly consists of multiple fully connected layers.

[0060] In one specific implementation, for example: 1) A semantic encoder is constructed using a Word2vec word vector model and two linear layers. The Word2vec word vector model loads a pre-trained Word2vec model, which is pre-trained using the GloVe vocabulary. The historical image dataset is sourced from Wikipedia and the Gigaword dataset. After vocabulary pre-training, the model can calculate the similarity between each seed word and each word contained in the pre-trained model, thereby converting semantic information into semantic features. Finally, the semantic features are converted into semantic feature vectors through two linear layers. 2) The attribute encoder consists of five linear layers. The first four linear layers are followed by ReLU activation functions. Through the calculation of these linear layers and activation functions, the input attribute labels or attribute prediction vectors are mapped to the feature layer, and the output is an attribute feature vector. 3) The image encoder is constructed using a deep convolutional neural network ResNet50 pre-trained on the ImageNet dataset to convert the input image into a feature vector of the corresponding category, i.e., an image feature vector. 4) The multi-source class center feature fusion module consists of two fully connected layers and a ReLU activation function. The ReLU activation function is located after the first fully connected layer. It inputs the semantic feature vector, attribute feature vector, and image feature vector corresponding to the support set images (i.e., the historical image dataset) into the multi-source class center feature fusion module to obtain fused class center feature vectors for each class. 5) The attribute predictor consists of a baseline network BN-Inception and an attribute localization module. The former comprises multiple Inception blocks with different feature levels, enabling the extraction of feature information at different levels. The latter consists of a channel attention network and a spatial transformation network. After the predicted features at different levels pass through the attribute localization module, corresponding attribute prediction values ​​are obtained. Finally, the maximum value of the attribute prediction values ​​from the three levels is selected to obtain the attribute prediction vector for the sample. 6) The image attribute feature fusion module is used to weightedly fuse the image feature vector and the attribute prediction vector to obtain the image attribute fusion feature.

[0061] In a specific implementation, following the previous example, in step 102, when training the initial detection network model and the initial classification network model pre-trained from the source domain dataset using the small sample target dataset, the network model parameters of the target domain are initialized using the pre-trained network parameter weights. Except for setting the network parameters of stage 5 of the image encoder ResNet50 in the initial classification network model and the network parameters of the region of interest extraction network and regression module in the initial detection network model to be trainable, all other network parameters are frozen.

[0062] In a preferred embodiment, a first loss function is used to constrain the training of the initial detection network model and the object detection network model;

[0063] The first loss function is determined by the following formula:

[0064]

[0065]

[0066]

[0067] Where, p i The prediction category score used to characterize the i-th prediction box; The classification score used to characterize the i-th predicted bounding box; t i Regression parameters used to characterize the i-th prediction box; Used to characterize the true regression parameters corresponding to the i-th prediction box; L cls Used to characterize the classification loss function; L reg Used to characterize the regression loss function; N cls The number of predicted boxes used to characterize the output; N reg The number of positive samples is used to characterize the number of samples; λ is a hyperparameter.

[0068] It should be noted that for each image, L predicted bounding boxes are generated, where L is the total number of predicted bounding boxes, and i represents the i-th predicted bounding box. L(p i ,t i Let ) represent the first loss function, which is used when the i-th predicted box is a positive sample. on the contrary This characterizes the calculation of regression loss when dealing with positive samples. R stands for Smoothness. L1 (x) function, Where x represents the regression loss. The value of, i.e.

[0069] In a preferred embodiment, a second loss function is used to constrain the training of the initial classification network model and the target classification network model;

[0070] The second loss function is determined by the following formula:

[0071]

[0072] Among them, l n U is the second loss function; c The formula used to represent cosine similarity; C is the number of training classes; GT is used to represent the true labels; F q Used to characterize the true label information of the image to be detected; P c Predicted label information used to characterize the image to be detected; U c (F q,P c |F q =GT) represents the cosine similarity between the image to be detected and the query set of its true label class.

[0073] More specifically, in this invention, when pre-training the classification network model to obtain the initial classification network model, the entire pre-training process employs an episode-based learning approach. This approach differs from conventional supervised classification training; instead of directly inputting the entire training set into the network model at once, each training round begins by randomly dividing the support set and query set from the historical image dataset. This training method simulates the network input under actual testing conditions, making the network model more suitable for small-sample learning tasks. Simultaneously, the classification network model is constrained using the following second loss function to obtain the initial classification network model.

[0074] In step 102, the initial detection network model and the initial classification network model are retrained using a small sample target dataset from the target domain. Most of the network parameters in the deep network model are kept unchanged to prevent overfitting, so that the deep network model can maintain good target detection and recognition performance in the target domain.

[0075] In step 104, the target image to be detected is input into the target detection network model to obtain the target image and regression results, including:

[0076] Based on the regression results, the image to be detected and identified is cropped to obtain the target image; the regression results include the location information of the target in the image to be detected and identified.

[0077] In this invention, the image to be detected is processed by a monitoring encoder, a gradient decoupling layer, a region of interest extraction network, a region of interest alignment structure, and a regression module. The resulting regression result contains the location information of potential target objects in the image, thus confirming that the potential target can be detected. Based on the location information in the regression result, image information at corresponding locations is cropped from the image to be detected to remove other images containing non-target information, thus obtaining the target image.

[0078] In step 104, the semantic information, attribute information, and image information related to the target image are input into the target classification network model to obtain the classification result, including:

[0079] The target image is input into the target classification network model to predict the attribute features of the target image and obtain the attribute prediction vector; in parallel, the target image is transformed into a feature vector to obtain the first image feature vector.

[0080] The attribute prediction vector and the first image feature vector are fused to obtain the image attribute feature vector;

[0081] The target image is parsed to obtain semantic information, attribute information, and image information related to the target image;

[0082] Semantic information, attribute information, and image information are input into the target classification network model to obtain semantic feature vector, attribute feature vector, and second image feature vector, respectively.

[0083] The semantic feature vector, attribute feature vector, and second image feature vector are fused to obtain the fused class center feature vector.

[0084] The similarity between the image attribute feature vector and the fused class center feature vector is calculated to obtain the classification result.

[0085] In a preferred embodiment, the target image is parsed to obtain semantic information, attribute information, and image information related to the target image, including:

[0086] The target image is analyzed to determine the types of targets in the target image;

[0087] Determine the semantic information, attribute information, historical target image information, and image information corresponding to historical target components for this target type.

[0088] Specifically, such as Figure 4 As shown, the classification network module inputs the cropped target image into the attribute predictor and image encoder. The attribute predictor outputs an attribute prediction vector, and the image encoder outputs a first image feature vector. These are then fused by the image attribute feature fusion module to output an image attribute feature vector. Simultaneously, based on the parsing results of the target image, the semantic, attribute, and image information related to the target image included in the historical image dataset are processed by the semantic encoder, attribute encoder, and image encoder, respectively, to output corresponding feature vector information. This information is then fused into a class center feature vector by the multi-source class center feature fusion module. Finally, the similarity between the fused class center feature vector and the image attribute feature vector is calculated to output the classification result. This classification result includes the specific attribute and semantic information of the identified target.

[0089] Finally, regarding step 106, based on the regression and classification results, the recognition result of the image to be detected is obtained, including:

[0090] Non-maximum suppression is applied to the regression and classification results to obtain the recognition results; the recognition results include the attribute information corresponding to the target in the image to be detected.

[0091] In this invention, in order to accurately identify the target in the image to be detected, non-maximum suppression processing is further applied to the regression results and classification results obtained by the two network models respectively, so as to obtain the optimal solution by filtering out local maxima and improving the recognition accuracy.

[0092] Specifically, following the previous example, the small sample target dataset within the target domain includes visible light and infrared datasets. The performance of the final deep network model is tested by calculating and measuring accuracy. The experimental results are as follows: Using the visible light dataset in the target domain to train the target detection network model and the target classification network model, and inputting the image to be detected based on visible light into the deep network model composed of the target detection network model and the target classification network model, the accuracy of the recognition result is 74.8%. However, using the classification network model and the target detection network model, which do not include semantic encoders, attribute encoders, or multi-source class center feature fusion modules in existing technologies, as the deep network model, the accuracy of the recognition result obtained is 66.9%. 2) Using an infrared dataset in the target domain, target detection and classification network models were trained. The images to be detected based on infrared data were then input into a deep network model composed of these two models, achieving an accuracy of 77.9%. In contrast, using classification and target detection network models (which lack semantic encoders, attribute encoders, and multi-source class center feature fusion modules) as deep network models yielded an accuracy of 70.5%. These accuracy results demonstrate that the network model incorporating external prior information (i.e., semantic and attribute information) achieved significantly better performance in both visible light and infrared datasets. This verifies the effectiveness of introducing semantic and attribute information, demonstrating that this approach effectively optimizes the model, enhancing its classification capabilities and recognition accuracy.

[0093] The small-sample target detection and recognition method provided by this invention utilizes external prior knowledge, including semantic and attribute information, to enrich the expression of image feature information, improve the accuracy of target object category representation, and enhance intelligent recognition performance in small-sample data environments. Moreover, this method is simple and easy to implement; it only requires fine-tuning the deep network model composed of the initial detection network model and the initial classification network model using limited target domain labeled data, thus overcoming the problem of insufficient training samples in the target domain.

[0094] like Figure 5 , Figure 6 As shown, this embodiment of the invention provides a small-sample target detection and recognition device. The device embodiment can be implemented through software, hardware, or a combination of both. From a hardware perspective, as... Figure 5The diagram shown is a hardware architecture diagram of a computing device housing a small-sample target detection and recognition device provided in an embodiment of the present invention. (Except for...) Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, the computing device in the embodiment may also include other hardware, such as a forwarding chip responsible for processing packets. Taking software implementation as an example, such as... Figure 6 As shown, a device in a logical sense is formed by the CPU of its computing device reading the corresponding computer program from non-volatile memory into memory and running it. This embodiment provides a small-sample target detection and recognition device, including:

[0095] Module 600 is used to acquire small sample target datasets within the target domain.

[0096] Training module 602 is used to train the initial detection network model and the initial classification network model pre-trained from the source domain dataset using a small sample target dataset, so as to obtain the target detection network model and the target classification network model.

[0097] The detection and recognition module 604 is used to input the target image to be detected and recognized into the target detection network model to obtain the target image and regression results; input the target image and related semantic information, attribute information and image information into the target classification network model to obtain the classification results; and obtain the recognition result of the target image based on the regression results and classification results.

[0098] In some specific implementations, the acquisition module 600 can be used to perform the above step 100, the training module 602 can be used to perform the above step 102, and the detection and recognition module 604 can be used to perform the above steps 104 and 106.

[0099] In some specific implementations, the source domain dataset includes a public dataset; wherein, the public dataset is used to pre-train the pre-built detection network model to obtain the initial detection network model;

[0100] The detection network model includes a detection encoder, a gradient decoupling layer, a region of interest extraction network, a region of interest alignment structure, and a regression module. The input of the detection network model is an image from a public dataset, and the output is a regression result containing the location information of the target in the image.

[0101] In some specific implementations, the training of the initial detection network model and the object detection network model is constrained by a first loss function;

[0102] The first loss function is determined by the following formula:

[0103]

[0104]

[0105]

[0106] Where, p i The prediction category score used to characterize the i-th prediction box; The classification score used to characterize the i-th predicted bounding box; t i Regression parameters used to characterize the i-th prediction box; Used to characterize the true regression parameters corresponding to the i-th prediction box; L cls Used to characterize the classification loss function; L reg Used to characterize the regression loss function; N cls The number of predicted boxes used to characterize the output; N reg The number of positive samples is used to characterize the number of samples; λ is a hyperparameter.

[0107] In some specific implementations, the source domain dataset includes a historical image dataset; wherein, the historical image dataset is used to pre-train a pre-built classification network model to obtain an initial classification network model;

[0108] The classification network model includes a semantic encoder, an attribute encoder, an attribute predictor, an image encoder, a multi-source class center feature fusion module, and an image attribute feature fusion module. The input of the classification network model is an image in the historical image dataset and the corresponding semantic information, attribute information, and image information. The output is the recognition result of the image.

[0109] In some specific implementations, a second loss function is used to constrain the training of the initial classification network model and the target classification network model;

[0110] The second loss function is determined by the following formula:

[0111]

[0112] Among them, l n U is the second loss function; c The formula used to represent cosine similarity; C is the number of training classes; GT is used to represent the true labels; F q Used to characterize the true label information of the image to be detected; P c Predicted label information used to characterize the image to be detected; U c (F q ,P c |F q =GT) represents the cosine similarity between the image to be detected and the query set of its true label class.

[0113] In some specific implementations, the detection and identification module 604 is also used to perform the following operations:

[0114] Based on the regression results, the image to be detected and identified is cropped to obtain the target image; the regression results include the location information of the target in the image to be detected and identified.

[0115] In some specific implementations, the detection and identification module 604 is also used to perform the following operations:

[0116] The target image is input into the target classification network model to predict the attribute features of the target image and obtain the attribute prediction vector; in parallel, the target image is transformed into a feature vector to obtain the first image feature vector.

[0117] The attribute prediction vector and the first image feature vector are fused to obtain the image attribute feature vector;

[0118] The target image is parsed to obtain semantic information, attribute information, and image information related to the target image;

[0119] Semantic information, attribute information, and image information are input into the target classification network model to obtain semantic feature vector, attribute feature vector, and second image feature vector, respectively.

[0120] The semantic feature vector, attribute feature vector, and second image feature vector are fused to obtain the fused class center feature vector.

[0121] The similarity between the image attribute feature vector and the fused class center feature vector is calculated to obtain the classification result.

[0122] In some specific implementations, the detection and identification module 604 is also used to perform the following operations:

[0123] Non-maximum suppression is applied to the regression and classification results to obtain the recognition results; the recognition results include the attribute information corresponding to the target in the image to be detected.

[0124] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on a small sample target detection and recognition device. In other embodiments of the present invention, a small sample target detection and recognition device may include more or fewer components than illustrated, or combine some components, or split some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0125] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description of the method embodiment of the present invention, and will not be repeated here.

[0126] This invention also provides a computing device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements a small sample target detection and recognition method according to any embodiment of this invention.

[0127] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform a small-sample target detection and recognition method according to any embodiment of this invention.

[0128] Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer (or CPU or MPU) of the system or apparatus may read and execute the program code stored in the storage medium.

[0129] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0130] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0131] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0132] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion module connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion module execute some and all of the actual operations, thereby realizing the function of any of the above embodiments.

[0133] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0134] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for small-sample target detection and recognition, characterized in that, include: Obtain a small sample target dataset within the target domain; The initial detection network model and the initial classification network model, which were pre-trained from the source domain dataset, are used to train the small sample target dataset to obtain the target detection network model and the target classification network model. The target image to be detected and recognized in the target domain is input into the target detection network model to obtain the target image and regression results; The semantic information, attribute information, and image information related to the target image are input into the target classification network model to obtain the classification result; Based on the regression results and the classification results, the recognition result of the image to be detected is obtained; The semantic information, attribute information, and image information related to the target image are input into the target classification network model to obtain the classification result, including: The target image is input into the target classification network model to predict the attribute features of the target image and obtain the attribute prediction vector; in parallel, the target image is transformed into a feature vector to obtain a first image feature vector. The attribute prediction vector and the first image feature vector are fused to obtain the image attribute feature vector; The target image is parsed to obtain the semantic information, attribute information, and image information related to the target image; The semantic information, the attribute information, and the image information are input into the target classification network model to obtain the semantic feature vector, the attribute feature vector, and the second image feature vector, respectively. The obtained semantic feature vector, the attribute feature vector, and the second image feature vector are fused to obtain a fused class center feature vector; The similarity between the image attribute feature vector and the fused class center feature vector is calculated to obtain the classification result.

2. The method according to claim 1, characterized in that, The source domain dataset includes a public dataset; wherein, the pre-built detection network model is pre-trained using the public dataset to obtain the initial detection network model; The detection network model includes a detection encoder, a gradient decoupling layer, a region of interest extraction network, a region of interest alignment structure, and a regression module. The input of the detection network model is an image from the public dataset, and the output is a regression result, which contains the location information of the target in the image.

3. The method according to claim 1, characterized in that, The initial detection network model and the target detection network model are constrained by a first loss function during training; The first loss function is determined by the following formula: in, p i The prediction category score used to characterize the i-th prediction box; The classification score used to characterize the i-th predicted bounding box; t i Regression parameters used to characterize the i-th prediction box; Used to characterize the true regression parameters corresponding to the i-th prediction box; L cls Used to characterize the classification loss function; L reg Used to characterize the regression loss function; N cls Used to characterize the number of predicted boxes in the output; N reg The number of positive samples is used to characterize the number of samples; λ is a hyperparameter.

4. The method according to claim 1, characterized in that, The source domain dataset includes a historical image dataset; wherein, the pre-built classification network model is pre-trained using the historical image dataset to obtain the initial classification network model; The classification network model includes a semantic encoder, an attribute encoder, an attribute predictor, an image encoder, a multi-source class center feature fusion module, and an image attribute feature fusion module. The input of the classification network model is an image in the historical image dataset and the corresponding semantic information, attribute information, and image information of that image. The output is the recognition result of that image.

5. The method according to claim 1, characterized in that, The step of inputting the target image to be detected into the target detection network model to obtain the target image and regression results includes: Based on the regression results, the image to be detected and identified is cropped to obtain the target image; wherein, the regression results include the location information of the target in the image to be detected and identified.

6. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the recognition result of the image to be detected based on the regression result and the classification result includes: The regression results and the classification results are subjected to non-maximum suppression processing to obtain the recognition results; wherein, the recognition results include the attribute information corresponding to the target in the image to be detected.

7. A small sample target detection and recognition device, characterized in that, For implementing the method as described in any one of claims 1 to 6, comprising: The acquisition module is used to acquire a small sample target dataset within the target domain; The training module is used to train the initial detection network model and the initial classification network model pre-trained from the source domain dataset using the small sample target dataset, so as to obtain the target detection network model and the target classification network model. The detection and recognition module is used to input the target image to be detected and recognized in the target domain into the target detection network model to obtain the target image and regression results; input the semantic information, attribute information and image information related to the target image into the target classification network model to obtain the classification results; and obtain the recognition result of the target image based on the regression results and the classification results.

8. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-6.

Images