Small sample visual identification method and device, equipment and storage medium

Abstract

The invention relates to a small sample visual identification method and device, equipment and a storage medium. According to the method, the obtained training sample is input into the initial recognition model, the feature extraction is performed on the training sample by adopting the diversity convolution layer, multi-level semantic features can be extracted, and the local feature representation in each branch is enhanced; a dense classifier is adopted to classify the extracted features, and the category of the image can be judged based on the global features and the local features; and the feature similarity between the extracted features is calculated, so that the nuisance between the categories can be more accurately captured, and the classification accuracy of the model in the small sample visual recognition scene is further improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision and image processing, and in particular to a small sample visual recognition method, apparatus, device and storage medium. Background Art

[0002] In order to mimic the human ability to recognize unseen categories in practical applications, few-shot visual recognition has triggered a research boom and attracted widespread attention in recent years.

[0003] Few-shot visual recognition aims to recognize new categories from scarce instances given a priori base classes. This is a very challenging problem because there are only limited instances of new categories in the few-shot setting, and the traditional way to deal with this challenge is to encode a global representation of the image and achieve feature alignment. Most global representation-based methods focus on training the learner by leveraging a meta-learning mechanism and then aligning the vector features encoded by the learner, such as constructing the center or relationship of the global representation. In order to obtain a more powerful learner, the meta-learning mechanism is abandoned, and only fully supervised pre-training on the base class can generate a baseline model with global representation and achieve significant progress. However, the global representation-based method obtains features through global average pooling before the classification layer, which ignores the semantic information of the features, limits the performance of the model, and reduces the classification accuracy of the model. Summary of the invention

[0004] Based on this, it is necessary to provide a small sample visual recognition method, device, equipment and storage medium to address the above technical problems.

[0005] In a first aspect, the present application provides a small sample visual recognition method. The method comprises:

[0006] Acquire a training sample, wherein the training sample includes a plurality of images;

[0007] Inputting the training samples into the initial recognition model, and extracting features from the training samples using a diversity convolutional layer;

[0008] A dense classifier is used to classify the extracted features to obtain a first classification result;

[0009] Calculating feature similarity between the extracted features, and determining a second classification result according to the similarity score;

[0010] According to the first classification result and the second classification result, under the constraint of the loss function, iteratively training the initial recognition model until a preset condition is met to obtain a trained recognition model;

[0011] The trained recognition model is used to identify the image to be identified and obtain the image category.

[0012] In one embodiment, the extracting features of the training samples using a diversity convolutional layer includes:

[0013] Construct a diverse convolutional feature extractor to encode different levels of features for each image and obtain tensor features, which are expressed as:

[0014]

[0015] in, Represents the tensor features of the jth branch in the i-th image in the training sample;

[0016] S represents the set of images in the training sample;

[0017] F Θ Representation diversity convolutional feature extractor;

[0018] C m , W m , H m Represents the mth tensor feature respectively channels, weights and heights;

[0019] Represents the i-th image in the training sample;

[0020] B represents the number of images; J represents the number of branches in an image.

[0021] In one embodiment, the step of using a dense classifier to classify the extracted features to obtain a first classification result includes:

[0022] Convert tensor features into vector form;

[0023] Construct weight parameters and use dense classifier to classify to get category probability, which is expressed as:

[0024]

[0025] Among them, w q and b q Respectively The corresponding weight parameters and bias of the categories; t and T represent the temperature factor and transposition operation respectively, and the temperature factor is used to balance the distribution of features;

[0026] P stands for category;

[0027] The probability of each category constitutes the first classification result.

[0028] In one embodiment, the training sample includes a training set, and the training set includes a query set and a support set; then the calculating of feature similarity between the extracted features and determining the second classification result according to the similarity score includes:

[0029] For each tensor feature in the query set of the training set, align it with all spatial positions of the tensor features of the corresponding branch in the support set of the training set, and the expression is:

[0030]

[0031] Among them, D represents The kth 1 spatial features to the jth branch The cosine distance scores of all spatial features in; Represents the query features in the query set;

[0032] represents the support features in the support set;

[0033] The cosine distance scores of each tensor feature in the query set constitute the second classification result.

[0034] In one embodiment, the iterative training of the initial recognition model according to the first classification result and the second classification result under the constraint of a loss function includes:

[0035] In each iteration process, a first loss function is used to calculate a first loss value of the first classification result, and a second loss function is used to calculate a second loss value of the second classification result;

[0036] Calculating a weighted sum of the first loss value and the second loss value to obtain a weighted result;

[0037] Based on the weighted result, the parameters of the initial recognition model are adjusted.

[0038] In one embodiment, the training sample includes a test set, and the test set includes a query set and a support set; the method further includes:

[0039] After obtaining the trained recognition model, the test set is input into the trained recognition model, the score of the feature similarity between the query set of the test set and the support set of the test set is calculated, and based on the score, it is determined whether the accuracy of the trained recognition model meets the training requirements.

[0040] In one embodiment, the method of determining whether the preset condition is met includes:

[0041] If the number of optimizations of the initial recognition model reaches a preset number, or the output accuracy of the initial recognition model meets the preset accuracy, it is considered that the preset condition is met.

[0042] In a second aspect, the present application also provides a small sample visual recognition device. The device comprises:

[0043] An acquisition module, used for acquiring training samples, wherein the training samples include a plurality of images;

[0044] A training module is used to input the training samples into the initial recognition model, use a diversity convolution layer to extract features from the training samples; use a dense classifier to classify the extracted features to obtain a first classification result; calculate the feature similarity between the extracted features, and determine a second classification result according to the similarity score; according to the first classification result and the second classification result, under the constraint of a loss function, iteratively train the initial recognition model until a preset condition is met to obtain a trained recognition model;

[0045] The recognition module is used to use the trained recognition model to recognize the image to be recognized and obtain the image category.

[0046] In a third aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method steps described in any one of the first aspects are implemented.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method steps described in any one of the first aspects are implemented.

[0048] The above-mentioned small sample visual recognition method, device, equipment and storage medium have at least the following advantages:

[0049] This application inputs the acquired training samples into the initial recognition model, and uses a diverse convolutional layer to extract features from the training samples, which can extract multi-level semantic features and enhance the local feature representation in each branch; a dense classifier is used to classify the extracted features, and the image category can be determined based on global and local features; the feature similarity between the extracted features is calculated, which can more accurately capture the subtle differences between categories, further improving the classification accuracy of the model in small sample visual recognition scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings constituting a part of this application are used to provide a further understanding of the present invention. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A diagram showing an application environment of a small sample visual recognition method in an embodiment;

[0053] Figure 2 A schematic diagram of a flow chart of a small sample visual recognition method in an embodiment;

[0054] Figure 3 A schematic diagram of a process for obtaining a first classification result in one embodiment;

[0055] Figure 4 A schematic diagram of a flow chart of a step of determining a second classification result in one embodiment;

[0056] Figure 5 A schematic diagram of a process of iteratively training an initial recognition model in one embodiment;

[0057] Figure 6 is a structural block diagram of a small sample visual recognition device in an embodiment;

[0058] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0060] While some exemplary embodiments of the present invention have been described for the purpose of illustration, it should be understood that the present invention may be implemented in other ways not specifically shown in the drawings.

[0061] The small sample visual recognition method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the terminal 102 communicates with the server 104 through a network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0062] Among them, the terminal 102 can send training samples to the server 104 so that the server 104 processes the training samples. For example, the server 104 inputs the acquired training samples into the initial recognition model, and uses the diversity convolution layer to extract features from the training samples; uses a dense classifier to classify the extracted features to obtain a first classification result; calculates the feature similarity between the extracted features, and determines the second classification result according to the similarity score; according to the first classification result and the second classification result, under the constraint of the loss function, iteratively trains the initial recognition model until the preset conditions are met to obtain a trained recognition model. The terminal 102 also sends the image to be recognized to the server 104, so that the server 104 uses the trained recognition model to recognize it and obtain the image category. In this way, the server 104 feeds back the image category to the terminal 102.

[0063] The above-mentioned small-sample visual recognition method inputs the acquired training samples into the initial recognition model, and uses a diverse convolutional layer to extract features from the training samples, thereby extracting multi-level semantic features and enhancing the local feature representation in each branch; a dense classifier is used to classify the extracted features, and the image category can be determined based on global and local features; the feature similarity between the extracted features is calculated, which can more accurately capture the subtle differences between categories, further improving the classification accuracy of the model in the small-sample visual recognition scenario.

[0064] The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0065] In an exemplary embodiment, the present application provides a small sample visual recognition method. Figure 1 The server 104 in FIG. 1 is used as an example for explanation.

[0066] See also Figure 2 , Figure 2 The following is a flow chart of a small sample visual recognition method of this embodiment, which specifically includes the following steps:

[0067] Step S202: obtaining a training sample, where the training sample includes a plurality of images.

[0068] Specifically, the training sample includes a training set, and the training set includes a query set and a support set. Generally speaking, the query set is a set of images that need to be classified, and the support set is a set of images containing a certain category, which serves as a reference to assist classification. The purpose of the small sample visual recognition of the embodiment of the present application is to identify the categories of each category that only exist in the target domain that is invisible in a limited number of samples by learning a given base class source domain. Among them, the base class source domain includes an image set and labels corresponding to each image, and the target domain also includes an image set and labels corresponding to each image, and the categories of the source domain and the target domain do not overlap.

[0069] Step S204: input the training samples into the initial recognition model, and use a diversity convolutional layer to extract features from the training samples.

[0070] Specifically, the traditional method mainly relies on the standard single receptive field convolution layer to extract features. This structure has a single receptive field and is difficult to capture effective alignment features in different samples. Therefore, inspired by visual recognition work, the embodiment of the present application proposes a diversity convolution layer structure, which introduces a diversity design in the convolution layer structure to enhance the feature alignment capability in small sample visual recognition, and combines different receptive fields with multi-branch tensor feature representations to encode richer spatial information. Through this structure, multi-level semantic features can be extracted, effectively characterizing the features of the image and improving the alignment capability of different samples.

[0071] Step S206: classify the extracted features using a dense classifier to obtain a first classification result.

[0072] Specifically, a dense classifier has the ability to densely process or predict each pixel or region in an image, and is capable of fine-grained, pixel-level classification or prediction of images. A dense classifier combines high-level, mid-level, and low-level features, and shares weight parameters at each spatial position, enabling the model to use all spatial positions of multi-level features for classification and prediction, effectively improving the model's discriminative power and generalization ability.

[0073] Step S208, calculating the feature similarity between the extracted features, and determining the second classification result according to the similarity score.

[0074] Specifically, the embodiment of the present application uses a multi-branch semantic alignment method to align the multi-layer local features of the query image and the support image, and calculates the similarity score between the features to determine the classification category of the query image. Through the construction of the support set and the query set, each spatial position of the query feature is aligned with the support feature, and the distance score between the two is calculated. Finally, the semantic alignment of the multi-branch is balanced and the final classification result is determined.

[0075] Step S210, according to the first classification result and the second classification result, under the constraint of the loss function, iteratively train the initial recognition model until the preset conditions are met to obtain a trained recognition model.

[0076] Specifically, the first classification result and the second classification result are the prediction results of the model. The loss function is used to calculate the difference between the prediction result of the model and the true value. According to the difference, the hyperparameters of the model are gradually adjusted to finally obtain the optimal prediction result.

[0077] Optionally, the determination method of satisfying the preset condition includes:

[0078] If the number of optimizations of the initial recognition model reaches a preset number, or the output accuracy of the initial recognition model meets the preset accuracy, it is considered that the preset conditions are met.

[0079] Step S212: using the trained recognition model to recognize the image to be recognized, and obtaining the image category.

[0080] Specifically, after the image to be recognized is input into the trained recognition model, a diversity convolutional layer is used to perform multi-branch feature extraction on the image, the feature similarity between the extracted features is calculated, and the image category is obtained based on the similarity score.

[0081] The above-mentioned small-sample visual recognition method inputs the acquired training samples into the initial recognition model, and uses a diverse convolutional layer to extract features from the training samples, thereby extracting multi-level semantic features and enhancing the local feature representation in each branch; a dense classifier is used to classify the extracted features, and the image category can be determined based on global and local features; the feature similarity between the extracted features is calculated, which can more accurately capture the subtle differences between categories, further improving the classification accuracy of the model in the small-sample visual recognition scenario.

[0082] Optionally, a diversity convolutional layer is used to extract features from the training samples, including:

[0083] Construct a diverse convolutional feature extractor to encode different levels of features for each image and obtain tensor features, which are expressed as:

[0084]

[0085] in, Represents the tensor features of the jth branch in the i-th image in the training sample;

[0086] S represents the set of images in the training sample;

[0087] F ΘRepresentation diversity convolutional feature extractor;

[0088] C m , W m , H m Represents the mth tensor feature respectively channels, weights and heights;

[0089] Represents the i-th image in the training sample;

[0090] B represents the number of images; J represents the number of branches in an image.

[0091] Specifically, the constructed diverse convolutional feature extractor F Θ :X→R C×W×H , F Θ Each layer consists of a 1×1 convolution, a 1×1-3×3 convolution, and a 1×1 convolution AVG, where AVG represents an average pooling operation (AveragePooling, AVG).

[0092] In the above-mentioned small sample visual recognition method, the role of the diverse convolutional feature extractor is to capture spatial information at different levels and receptive fields, enrich the feature representation, and improve the image representation ability and the ability to capture detail information.

[0093] See also Figure 3 Optionally, a dense classifier is used to classify the extracted features to obtain a first classification result, including:

[0094] Step 302: convert the tensor feature into a vector form.

[0095] Step 304: construct weight parameters and use a dense classifier to perform classification to obtain category probabilities.

[0096] Step 306: The probabilities of each category constitute the first classification result.

[0097] Specifically, the embodiment of the present application adopts a diversity convolution feature extractor to extract multi-branch features Then the spatial information in the feature is represented as a vector Where i represents image, j represents branch, k represents space, K = W j ×H j The input image is thus embedded into a three-dimensional tensor structure, and then the spatial information is used for dense classification. This scheme avoids the use of global average pooling for dimensionality reduction, so as to retain more spatial semantic information.

[0098] Further, construct the weight parameter W that needs to be learned p =[w 1 ,w2 ,…,w p ]∈C j ×P,W p represents the weight parameter belonging to class P. For each spatial position of the feature, they share the weight parameter. Therefore, the Softmax function of the spatial feature can be expressed as:

[0099]

[0100] Among them, w q and b q Respectively The corresponding weight parameters and bias of the categories; t and T represent the temperature factor and transposition operation respectively, and the temperature factor is used to balance the distribution of features.

[0101] The result of the above expression is the class probability obtained by transforming the output of the dense classifier.

[0102] In the above small sample visual recognition method, the dense classifier can classify and predict the extracted multi-branch features at all spatial positions, effectively improving the discrimination and generalization ability of the model.

[0103] See also Figure 4 Optionally, the training sample includes a training set, and the training set includes a query set and a support set. Then, the feature similarity between the extracted features is calculated, and the second classification result is determined according to the similarity score, including:

[0104] Step 402, for each tensor feature in the query set of the training set, align it with all spatial positions of the tensor features of the corresponding branch in the support set of the training set.

[0105] Step 404: The cosine distance score of each tensor feature in the query set constitutes a second classification result.

[0106] Specifically, the second classification result is obtained by aligning the multi-layer local features of the query image in the query set and the support image in the support set, and calculating the similarity scores between the features.

[0107] For example, a query set S1 and a support set S2 are constructed in the training set. Given a query image X in the query set S1, s1 , and its corresponding multi-branch features can be expressed as:

[0108] Z j,s1 =F Θ (X s1 ),j=1,…,J.

[0109] Similarly, for the input support image in the support set S2, s , and its corresponding multi-branch features can be expressed as:

[0110] Z j,s2 =F Θ (X s2 ),j=1,…,J

[0111] For each query feature in the branch Each spatial position of the feature is related to the corresponding branch feature in the support image. All spatial positions of are aligned as follows:

[0112]

[0113] Among them, D represents The kth 1 spatial features to the jth branch The cosine distance scores of all spatial features in; Represents the query features in the query set;

[0114] Represents the support features in the support set.

[0115] The above-mentioned small sample visual recognition method can more accurately capture the subtle differences between categories by calculating the feature similarity between the extracted features, further improving the classification accuracy of the model in the small sample visual recognition scenario.

[0116] See also Figure 5 Optionally, according to the first classification result and the second classification result, under the constraint of the loss function, iteratively training the initial recognition model includes:

[0117] Step 502: In each iteration process, a first loss function is used to calculate a first loss value of a first classification result, and a second loss function is used to calculate a second loss value of a second classification result.

[0118] Step 504, calculate the weighted sum of the first loss value and the second loss value to obtain a weighted result.

[0119] Step 506: Adjust the parameters of the initial recognition model based on the weighted result.

[0120] Specifically, the first loss function is used to calculate the loss value of the dense classifier, and the second loss function is used to calculate the multi-branch semantic alignment loss value.

[0121] Exemplarily, the first loss function of the embodiment of the present application adopts the minimization of the cross entropy loss function, and the dense classifier loss constructed based on the multi-level spatial feature optimization network model is as follows:

[0122]

[0123] Dense classifier loss L dense Utilizing all spatial locations of multi-layer features aims to enable spatial information to achieve correct predictions anywhere. By making this prediction, more useful discriminative features can be explored and the robustness and generalization ability of the model can be improved.

[0124] Furthermore, the score L is calculated based on the spatial features of the query feature and the supporting feature. j , we can obtain the multi-branch semantic alignment loss, which is expressed as:

[0125]

[0126] Among them, L j Represents a hyperparameter, which aims to optimize the model performance by balancing the semantic alignment of different branches. By constructing multi-branch semantic alignment, the contribution of different branch features is balanced to ensure that the key features of all spatial positions are effectively measured.

[0127] Furthermore, the above-mentioned dense classifier loss and multi-branch semantic alignment loss are weightedly summed, and the weighted sum result is the overall loss value of the model. Based on the overall loss value, the parameters of the initial recognition model are gradually adjusted to finally obtain a trained recognition model.

[0128] In the above small sample visual recognition method, the dense classifier loss is used to measure whether each level of features contributes sufficient distinguishing ability to the classification. The multi-branch semantic alignment loss is used to ensure that the model effectively aligns the features of the query image and the support image. Using the above method to train the initial recognition model, the extracted multi-branch features can be classified and predicted at all spatial locations, effectively improving the model's discrimination and generalization capabilities; at the same time, by calculating the feature similarity between the extracted features, the subtle differences between categories can be more accurately captured, further improving the classification accuracy of the model in the small sample visual recognition scenario.

[0129] Optionally, the training sample includes a test set, and the test set includes a query set and a support set. Then the small sample visual recognition method of the embodiment of the present application further includes:

[0130] After obtaining the trained recognition model, the test set is input into the trained recognition model, the score of the feature similarity between the query set of the test set and the support set of the test set is calculated, and the accuracy of the trained recognition model is determined based on the score to determine whether it meets the training requirements. If not, it can be trained on the basis of the last training to obtain the optimal recognition result, thereby further improving the recognition accuracy of the model. Among them, the step of calculating the similarity score is the same as the step in the above step 402, and will not be repeated here.

[0131] The above-mentioned small sample visual recognition method uses a variety of convolutional layers to extract features from training samples, which can capture spatial information at different levels and receptive fields, enrich the representation of features, and improve the image representation ability and the ability to capture detail information.

[0132] Furthermore, a dense classifier is used to classify the extracted multi-branch features, which can be used to make classification predictions at all spatial locations based on global and local features, effectively improving the model's discriminative power and generalization capabilities. Calculating the feature similarity between the extracted features can more accurately capture the subtle differences between categories.

[0133] Furthermore, the dense classifier loss can be used to measure whether the features at each level contribute sufficient distinguishing ability to the classification; the multi-branch semantic alignment loss is used to ensure that the model effectively aligns the features of the query image and the support image.

[0134] The above method is used to train the initial recognition model, which improves the classification accuracy of the model in small sample visual recognition scenarios.

[0135] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0136] Based on the same inventive concept, the embodiment of the present application also provides a small sample visual recognition device for implementing the small sample visual recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more small sample visual recognition device embodiments provided below can refer to the limitations of the small sample visual recognition method above, and will not be repeated here.

[0137] See also Figure 6 In an exemplary embodiment, the present application provides a small sample visual recognition device, including: an acquisition module, a training module and a recognition module, wherein:

[0138] The acquisition module is used to acquire training samples, which include multiple images.

[0139] The training module is used to input the training samples into the initial recognition model, and use the diversity convolution layer to extract features of the training samples; use the dense classifier to classify the extracted features to obtain the first classification result; calculate the feature similarity between the extracted features, and determine the second classification result according to the similarity score; according to the first classification result and the second classification result, under the constraint of the loss function, iteratively train the initial recognition model until the preset conditions are met to obtain a trained recognition model.

[0140] The recognition module is used to use the trained recognition model to recognize the image to be recognized and obtain the image category.

[0141] Optionally, the training module uses a diverse convolutional layer to extract features from training samples, including: constructing a diverse convolutional feature extractor, encoding features of different levels for each image, and obtaining tensor features.

[0142] Optionally, the training module uses a dense classifier to classify the extracted features to obtain a first classification result, including: converting tensor features into vector form; constructing weight parameters and using a dense classifier to classify to obtain category probabilities; and each category probability constitutes the first classification result.

[0143] Optionally, the training module calculates feature similarity between the extracted features and determines a second classification result based on the similarity score, including: for each tensor feature in the query set of the training set, aligning it with all spatial positions of the tensor features of the corresponding branch in the support set of the training set; the cosine distance score of each tensor feature in the query set constitutes the second classification result.

[0144] Optionally, the training module iteratively trains the initial recognition model according to the first classification result and the second classification result under the constraint of the loss function, including: in each iteration process, using the first loss function to calculate the first loss value of the first classification result, and using the second loss function to calculate the second loss value of the second classification result; calculating the weighted sum of the first loss value and the second loss value to obtain a weighted result; based on the weighted result, adjusting the parameters of the initial recognition model.

[0145] Optionally, the training module is also used to: after obtaining the trained recognition model, input the test set into the trained recognition model, calculate the score of the feature similarity between the query set of the test set and the support set of the test set, and determine whether the accuracy of the trained recognition model meets the training requirements based on the score.

[0146] The above-mentioned small sample visual recognition device uses a variety of convolutional layers to extract features from training samples, which can capture spatial information at different levels and receptive fields, enrich the feature representation, and improve the image representation ability and the ability to capture detail information.

[0147] Furthermore, a dense classifier is used to classify the extracted multi-branch features, which can be used to make classification predictions at all spatial locations based on global and local features, effectively improving the model's discriminative power and generalization capabilities. Calculating the feature similarity between the extracted features can more accurately capture the subtle differences between categories.

[0148] Furthermore, the dense classifier loss can be used to measure whether the features at each level contribute sufficient distinguishing ability to the classification; the multi-branch semantic alignment loss is used to ensure that the model effectively aligns the features of the query image and the support image.

[0149] The above method is used to train the initial recognition model, which improves the classification accuracy of the model in small sample visual recognition scenarios.

[0150] Each module in the above-mentioned small sample visual recognition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0151] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a small sample visual recognition method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0152] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned small sample visual recognition method when executing the computer program.

[0154] In an exemplary embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned small sample visual recognition method are implemented.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0156] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0157] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A small sample visual recognition method, characterized in that: The method comprises: Acquire a training sample, wherein the training sample includes a plurality of images; Inputting the training samples into the initial recognition model, and extracting features from the training samples using a diversity convolutional layer; A dense classifier is used to classify the extracted features to obtain a first classification result; Calculating feature similarity between the extracted features, and determining a second classification result according to the similarity score; According to the first classification result and the second classification result, under the constraint of the loss function, iteratively training the initial recognition model until a preset condition is met to obtain a trained recognition model; The trained recognition model is used to identify the image to be identified and obtain the image category.

2. The method according to claim 1, characterized in that The adopting a diversity convolutional layer to extract features from the training samples includes: Construct a diverse convolutional feature extractor to encode different levels of features for each image and obtain tensor features, which are expressed as: in, Represents the tensor features of the jth branch in the i-th image in the training sample; S represents the set of images in the training sample; F Θ Representation diversity convolutional feature extractor; C m , W m , H m Represents the mth tensor feature respectively channels, weights and heights; Represents the i-th image in the training sample; B represents the number of images; J represents the number of branches in an image.

3. The method according to claim 2, characterized in that The method of using a dense classifier to classify the extracted features to obtain a first classification result includes: Convert tensor features into vector form; Construct weight parameters and use dense classifier to classify to get category probability, which is expressed as: Among them, w q and b q Respectively The corresponding weight parameters and bias of the categories; t and T represent the temperature factor and transposition operation respectively, and the temperature factor is used to balance the distribution of features; P stands for category; The probability of each category constitutes the first classification result.

4. The method according to claim 2, characterized in that: The training sample includes a training set, and the training set includes a query set and a support set; the feature similarity between the extracted features is calculated, and the second classification result is determined according to the similarity score, including: For each tensor feature in the query set of the training set, align it with all spatial positions of the tensor features of the corresponding branch in the support set of the training set, and the expression is: Among them, D represents The k1th spatial feature in the jth branch The cosine distance scores of all spatial features in; Represents the query features in the query set; represents the support features in the support set; The cosine distance scores of each tensor feature in the query set constitute the second classification result.

5. The method according to claim 1, characterized in that: The iterative training of the initial recognition model according to the first classification result and the second classification result under the constraint of a loss function includes: In each iteration process, a first loss function is used to calculate a first loss value of the first classification result, and a second loss function is used to calculate a second loss value of the second classification result; Calculating a weighted sum of the first loss value and the second loss value to obtain a weighted result; Based on the weighted result, the parameters of the initial recognition model are adjusted.

6. The method according to any one of claims 1 to 5, characterized in that The training sample includes a test set, and the test set includes a query set and a support set; the method further includes: After obtaining the trained recognition model, the test set is input into the trained recognition model, the score of the feature similarity between the query set of the test set and the support set of the test set is calculated, and based on the score, it is determined whether the accuracy of the trained recognition model meets the training requirements.

7. The method according to any one of claims 1 to 5, characterized in that The determination method of satisfying the preset condition includes: If the number of optimizations of the initial recognition model reaches a preset number, or the output accuracy of the initial recognition model meets the preset accuracy, it is considered that the preset condition is met.

8. A small sample visual recognition device, characterized in that: The device comprises: An acquisition module, used for acquiring training samples, wherein the training samples include a plurality of images; A training module is used to input the training samples into the initial recognition model, use a diversity convolution layer to extract features from the training samples; use a dense classifier to classify the extracted features to obtain a first classification result; calculate the feature similarity between the extracted features, and determine a second classification result according to the similarity score; according to the first classification result and the second classification result, under the constraint of a loss function, iteratively train the initial recognition model until a preset condition is met to obtain a trained recognition model; The recognition module is used to use the trained recognition model to recognize the image to be recognized and obtain the image category.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images