Small sample learning processing method, device, equipment and storage medium

Abstract

The present disclosure relates to the technical field of machine learning, and in particular to a small sample learning processing method, device, equipment and storage medium. The method comprises: extracting local inquiry features of an inquiry sample; based on the local inquiry features, obtaining global inquiry features of the inquiry sample; based on the local inquiry features and the global inquiry features, obtaining inquiry interaction features of the inquiry sample, wherein the inquiry interaction features are formed by the interaction of the local inquiry features and the global inquiry features; determining the category to which the inquiry sample belongs through the inquiry interaction features and a supporting sample set, wherein the supporting sample set includes at least one sample category, and each of the sample categories includes at least one supporting sample. The present disclosure is used to solve the defect of low learning efficiency caused by the small sample learning method in the prior art being based only on local features, and to achieve the use of the interaction between local features and global features to guide semantic alignment.

Description

Technical Field

[0001] The present disclosure relates to the technical field of machine learning, and in particular to a small sample learning processing method, device, equipment and storage medium. Background Art

[0002] At present, machine learning is an important research field in artificial intelligence. When processing local areas with aligned semantics, small sample learning in machine learning can reduce the dependence of deep learning on large-scale labeled training data sets and can achieve tasks quickly and at low cost. However, small sample learning mainly processes the local features of the query sample, and the processing results can only reflect the information of the local area of ​​the query sample, but ignore the global features, resulting in excessively high semantic weights that are irrelevant to the global features of the query sample, which in turn reduces the efficiency of small sample learning and inaccurate learning processing results. Summary of the invention

[0003] The present disclosure provides a small sample learning processing method, device, equipment and storage medium to solve the defect of low learning efficiency caused by small sample learning methods in the prior art based only on local features, and to achieve the use of the interaction between local features and global features to guide semantic alignment.

[0004] The present disclosure provides a small sample learning processing method, comprising: extracting a local query feature of a query sample; obtaining a global query feature of the query sample based on the local query feature; obtaining a query interaction feature of the query sample based on the local query feature and the global query feature, wherein the query interaction feature is formed by the interaction of the local query feature and the global query feature; determining the category to which the query sample belongs through the query interaction feature and a supporting sample set, wherein the supporting sample set includes at least one sample category, and each of the sample categories includes at least one supporting sample.

[0005] According to a small sample learning processing method provided by the present disclosure, the extracting of the inquiry local features of the inquiry sample includes: extracting the inquiry local feature matrix of the inquiry sample; the obtaining of the inquiry global features of the inquiry sample based on the inquiry local features includes: obtaining at least one row vector contained in the inquiry local feature matrix; multiplying each of the row vectors by a preset first learnable parameter to obtain at least one weighted row vector; obtaining a first average value of the at least one weighted row vector; and multiplying the first average value by a preset second learnable parameter to obtain the inquiry global feature matrix of the inquiry sample.

[0006] According to a small sample learning processing method provided by the present disclosure, the inquiry interaction feature of the inquiry sample is obtained based on the inquiry local feature and the inquiry global feature, including: superimposing the inquiry local feature matrix and the inquiry global feature matrix to obtain the inquiry interaction feature matrix of the inquiry sample.

[0007] According to a small sample learning processing method provided by the present disclosure, the category to which the query sample belongs is determined through the query interaction features and the supporting sample set, including: obtaining the category interaction features corresponding to each of the sample categories in the supporting sample set; calculating the similarity between the query sample and each of the sample categories based on the query interaction features and the category interaction features; and determining the category to which the query sample belongs based on at least one of the similarities.

[0008] According to a small sample learning processing method provided by the present disclosure, the processing process for obtaining each of the category interaction features is as follows: extract the supporting local features of each of the supporting samples contained in the sample category; calculate the second average value of at least one of the supporting local features corresponding to the sample category; based on the supporting local features of each of the supporting samples in the sample category, respectively obtain the supporting global features of each of the supporting samples contained in the sample type; calculate the third average value of at least one of the supporting global features corresponding to the sample category; superimpose the second average value and the third average value to obtain the category interaction feature corresponding to the sample category.

[0009] According to a small sample learning processing method provided by the present disclosure, the query interaction feature is a query interaction feature matrix, and the category interaction feature is a category interaction feature matrix; the similarity between the query sample and each of the sample categories is calculated based on the query interaction feature and the category interaction feature, including: calculating the query interaction feature matrix and the cross covariance matrix of the category interaction feature matrix corresponding to each of the sample categories; and based on each of the cross covariance matrices, calculating the similarity between the query sample and each of the sample categories.

[0010] According to a small sample learning processing method provided by the present disclosure, the query sample includes any one of a picture sample or a video sample.

[0011] The present disclosure also provides a small sample learning processing device, including: an extraction module, used to extract the inquiry local features of the inquiry sample; a first acquisition module, used to acquire the inquiry global features of the inquiry sample based on the local features; a second acquisition module, used to acquire the inquiry interaction features of the inquiry sample based on the inquiry local features and the inquiry global features, wherein the inquiry interaction features are formed by the interaction of the inquiry local features and the inquiry global features; a category determination module, used to determine the category to which the inquiry sample belongs through the inquiry interaction features and a supporting sample set, wherein the supporting sample set includes at least one sample category, and each of the sample categories includes at least one supporting sample.

[0012] The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any one of the small sample learning processing methods described above are implemented.

[0013] The present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the small sample learning processing methods described above.

[0014] The small sample learning processing method, device, equipment and storage medium provided by the present disclosure, when processing the inquiry sample, in addition to extracting the inquiry local features of the inquiry sample, also obtains the inquiry global features of the inquiry sample through the inquiry local features. Then the inquiry interaction features formed by the interaction of the inquiry local features and the inquiry global features are obtained. Finally, the category to which the inquiry sample belongs is obtained based on the inquiry interaction features and the supporting sample set. The small sample learning processing process provided above does not process the inquiry local features of the inquiry sample separately, but obtains the inquiry interaction features containing the inquiry local features and the inquiry global features, and processes the inquiry interaction features. In this way, the interaction between the inquiry global features and the inquiry local features is used to semantically align the local areas of the inquiry samples, thereby improving the processing efficiency of the small sample learning process and making the inquiry sample category obtained by the small sample learning process more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present disclosure or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 It is one of the flowcharts of the small sample learning processing method provided by the present disclosure;

[0017] Figure 2 This is the second flow chart of the small sample learning processing method provided by the present disclosure;

[0018] Figure 3 is a structural schematic diagram of a small sample learning processing device provided by the present disclosure;

[0019] Figure 4 It is a schematic diagram of the structure of an electronic device provided by the present disclosure. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present disclosure.

[0021] The small sample learning processing method provided by the present disclosure is used to learn and classify inquiry samples. The method can be implemented by a software algorithm, specifically, by relying on a constructed small sample learning model. For example, a small sample learning model is constructed based on a convolutional neural network model. After the model is trained, it can determine the category to which the inquiry sample belongs based on local features and global features. The physical execution subject of the method can be any of a variety of processing devices such as computers, processors, servers, etc. Figure 1-Figure 2 The small sample learning processing method provided by the embodiment of the present disclosure is described.

[0022] In one embodiment, if Figure 1 As shown in the figure, the process steps for implementing the small sample learning processing method are as follows:

[0023] Step 101: extract local features of the query sample.

[0024] In this embodiment, Few-shot learning (FSL) is a method of deep learning that enables the learning system to learn and summarize through a small number of training samples. A small sample learning model is constructed based on the small sample learning principle, and the model can implement the small sample learning processing method provided by the present disclosure. Among them, the small sample learning model can be trained by selecting the original intelligent model according to actual conditions and needs, for example, the small sample learning model is formed by training the original convolutional neural network model.

[0025] When training the original intelligent model, a training sample set is required, which includes at least one training sample, and each training sample has been pre-labeled. The category to which each training sample in the training sample set belongs is included in the pre-set M sample categories, where M is an integer greater than 1. However, the number of samples in the training sample set is very small compared to a large amount of machine learning sample data. For example, the number of samples in the training sample set does not exceed 100.

[0026] When the original intelligent model is trained using the above training sample set, the training sample set needs to be re-divided into a support sample set and a query set. S represents the support sample set, (x i ,y i ) represents a training sample, i is an integer greater than or equal to 1, then the divided support sample set S consists of MN samples, M represents the M sample categories included in the support sample set, N represents that each sample category includes N support samples, N is an integer greater than or equal to 1, then the support samples contained in the support sample set S are as shown in formula (1):

[0027]

[0028] Q represents the query sample set, (x i ,y i ) represents a training sample, i is an integer greater than or equal to 1, then the divided query sample set Q consists of MB samples, M represents the M sample categories included in the query sample set, B represents that each sample category includes B query samples, B is an integer greater than or equal to 1, then the support samples contained in the query sample set Q are as shown in formula (2):

[0029]

[0030] The above N and B are both very small values. For example, the values ​​of N and B do not exceed 10.

[0031] In this embodiment, the original intelligent model constructed by the present method is trained using a support sample set and an inquiry sample set, and the learnable parameters in the original intelligent model are adjusted until the result output by the original intelligent model meets the preset condition threshold, and the original intelligent model is determined to be a small sample learning model. After the small sample learning model is trained, it is put into normal use. During the use phase, the inquiry sample input during the use phase is processed in the form of a small sample learning model to predict the category to which the inquiry sample belongs. In other words, the method provided by the present disclosure is used to construct a small sample learning model and realize the processing process of the inquiry sample within the small sample learning model. The processing process of the inquiry sample using this method can be implemented in both the model training phase and the model use phase.

[0032] Specifically, Represents the query sample, then in the model training phase, is a query sample in the query sample set Q; in the model usage phase, Any query sample whose category needs to be determined.

[0033] In this embodiment, after the query sample is input into the small sample learning model, the query local features of the query sample are extracted, and the query local features refer to the local features of the query sample to complete the processing of small sample learning.

[0034] Specifically, a local feature extractor is pre-constructed in the small sample learning model, and the local feature extractor may be a sub-convolutional neural network. When the query sample is input into the small sample learning model, the local feature extractor extracts the query local feature from the query sample.

[0035] In one embodiment, the inquiry sample includes any one of a picture sample or a video sample. Preferably, the picture sample is a two-dimensional picture sample; the video sample is a video sample that needs to be classified into video actions. Of course, the inquiry sample can also be set to any one of the samples in other data formats according to actual conditions and needs. When the inquiry sample is a picture sample, the training samples in the training sample set mentioned above are also pictures; when the inquiry sample is a video sample, the training samples in the training sample set mentioned above are also videos. It is sufficient to ensure that the data formats of the training samples and inquiry samples used in model training are consistent.

[0036] Step 102: obtaining a query global feature of the query sample based on the query local feature.

[0037] In this embodiment, after extracting the local inquiry feature of the inquiry sample, the global inquiry feature of the inquiry sample may be obtained according to the local inquiry feature, wherein the global inquiry feature refers to the global feature of the inquiry sample.

[0038] In one embodiment, both the local inquiry features and the global inquiry features of the inquiry sample can be expressed in the form of a matrix. Specifically, the local inquiry features of the inquiry sample are extracted, and the implementation process is as follows: extract the local inquiry feature matrix of the inquiry sample. Based on the local inquiry features, the global inquiry features of the inquiry sample are obtained, and the implementation process is as follows: obtain at least one row vector contained in the local inquiry feature matrix; multiply each row vector by a preset first learnable parameter to obtain at least one weighted row vector; obtain a first average value of at least one weighted row vector; multiply the first average value by a preset second learnable parameter to obtain the global inquiry feature matrix of the inquiry sample.

[0039] In this embodiment, the query local features and the query global features are expressed in the form of a matrix, which makes the calculation process easier to implement, simplifies the implementation process of the method, and indirectly improves the small sample learning processing efficiency implemented by the method.

[0040] Specifically, the query local feature matrix of the query sample is obtained by a preset local feature extractor, and the query local feature matrix is ​​represented as F L ∈R Z×C , F L ∈R Z×C The meaning is to query the local feature matrix F L is a real number (R) matrix with Z features and C feature dimensions, that is, the local feature matrix F L ∈R Z×C is a Z×C matrix.

[0041] In this embodiment, a global feature extractor is pre-constructed in the small sample learning model. The global feature extractor obtains the query global feature according to the principle shown in the following formula (3):

[0042]

[0043] Among them, F L (:,i)∈R 1×C is the local feature F L A characteristic component of L (:,i) is the query local feature matrix F L A row vector containing, is the first learnable parameter of the global feature extractor, is the second learnable parameter of the global feature extractor. L (:,i)W1 represents the weighted row vector, represents the first mean value, F G Represents the query global feature matrix of the query sample.

[0044] The row vector is a 1×C matrix, and the first learnable parameter is The matrix of , so the weighted row vector is The second learnable single number is The final query global feature matrix F G ∈R 1×C is a 1×C matrix.

[0045] In this embodiment, during the model training phase, it is necessary to continuously adjust the first learnable parameter and the second learnable parameter through training of the training sample set until the result output by the model meets the preset conditional threshold.

[0046] In this embodiment, by querying the local feature matrix, the specific method of querying the global feature matrix is ​​calculated, which realizes the extraction of the global features of the query sample, provides a basis for the interaction between global features and local features, and the matrix calculation process is simple and easier to implement through code, further improving the processing speed and efficiency of small sample learning.

[0047] Step 103: acquiring an inquiry interaction feature of the inquiry sample based on the inquiry local feature and the inquiry global feature, wherein the inquiry interaction feature is formed by the interaction of the inquiry local feature and the inquiry global feature.

[0048] In this embodiment, after obtaining the local inquiry feature and the global inquiry feature, the local inquiry feature and the global inquiry feature are interacted to form the inquiry interaction feature of the final inquiry sample. The inquiry interaction feature contains both the local feature information of the inquiry sample and the global feature information of the inquiry sample.

[0049] In one embodiment, based on the expression of the inquiry local features and the inquiry global features in the form of a matrix in the above embodiment, the inquiry interaction features of the inquiry sample are obtained based on the inquiry local features and the inquiry global features. The specific implementation process is as follows: the inquiry local feature matrix and the inquiry global feature matrix are superimposed to obtain the inquiry interaction feature matrix of the inquiry sample.

[0050] In this embodiment, since the global feature matrix F is queried G ∈R 1×C is a 1×C matrix, and the local feature matrix F L ∈R Z×C is a Z×C matrix. Then, through the broadcast operation of tensor, the query local feature matrix and the query global feature matrix are superimposed, and the query interaction feature matrix is ​​finally obtained as F G+L =F G +F L Among them, F G+L ∈R C×Z is a Z×C matrix.

[0051] Step 104 , determining the category to which the query sample belongs by querying the interactive features and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each sample category includes at least one supporting sample.

[0052] In this embodiment, after obtaining the query interaction feature, the query interaction feature is used to determine the category to which the query sample belongs, which is a specific one of at least one sample category included in the support sample set, that is, the category to which the query sample belongs is one of the sample categories included in the support sample set. After determining the category to which the query sample belongs, the query sample is processed by the small sample learning method.

[0053] In one embodiment, the category to which the query sample belongs is determined by the query interaction feature of the query sample and the category interaction feature corresponding to each sample category in the supporting sample set. Specifically, the category to which the query sample belongs is determined by the query interaction feature and the supporting sample set, and the implementation process is as follows: obtaining the category interaction feature corresponding to each sample category in the supporting sample set; calculating the similarity between the query sample and each sample category respectively according to the query interaction feature and the category interaction feature; and determining the category to which the query sample belongs according to at least one similarity.

[0054] In this embodiment, the supporting sample set includes at least one sample category, and each sample category includes at least one supporting sample. For each sample category, the corresponding category interaction feature can be obtained through the supporting samples included in the sample category. Then, based on the query interaction feature of the query sample and the category interaction feature corresponding to each sample category, the similarity between the query sample and each sample category is calculated respectively. Finally, based on the calculated similarities, the category to which the query sample belongs is determined.

[0055] In one embodiment, the corresponding category interaction features are obtained through the supporting samples included in the sample category. Specifically, the processing process for obtaining each category interaction feature is as follows: extract the supporting local features of each supporting sample included in the sample category; calculate the second average value of at least one supporting local feature corresponding to the sample category; based on the supporting local features of each supporting sample in the sample category, respectively obtain the supporting global features of each supporting sample included in the sample type; calculate the third average value of at least one supporting global feature corresponding to the sample category; superimpose the second average value and the third average value to obtain the category interaction feature corresponding to the sample category.

[0056] In this embodiment, for each supporting sample included in a sample category, the supporting local features are extracted respectively, and then the second average of the supporting local features of each supporting sample is calculated. For each supporting sample, the supporting global features of the supporting sample are obtained through the supporting local features of the supporting sample; and the third average of the supporting global features of each supporting sample is calculated. After superimposing the second average and the third average, the category interaction feature corresponding to the sample category can be obtained.

[0057] In one embodiment, when extracting supporting local features of supporting samples, the local feature extractor provided in the above embodiment can be used to implement the extraction. When the supporting samples are input into the small sample learning model, the local feature extractor extracts supporting local features from the supporting samples.

[0058] In one embodiment, the data format of each supporting sample in the supporting sample set is the same, and the data format of the supporting sample is consistent with the data format of the query sample.

[0059] In one embodiment, the supporting local features and supporting global features of the supporting sample are expressed in the form of a matrix. Specifically, the supporting local features of the supporting sample are extracted, and the implementation process is as follows: extract the supporting local feature matrix of the supporting sample. Based on the supporting local features, the supporting global features of the supporting sample are obtained, and the implementation process is as follows: obtain at least one row vector contained in the supporting local feature matrix; multiply each row vector by a preset first learnable parameter to obtain at least one weighted row vector; obtain a first average value of at least one weighted row vector; multiply the first average value by a preset second learnable parameter to obtain the supporting global feature matrix of the supporting sample.

[0060] In this embodiment, when the supporting local features and supporting global features of the supporting samples are expressed in the form of matrices, the specific processing procedure for obtaining each category interaction feature is as follows: extracting the supporting local feature matrix of each supporting sample contained in the sample category; calculating the second average value of at least one supporting local feature matrix corresponding to the sample category; based on the supporting local feature matrix of each supporting sample in the sample category, respectively obtaining the supporting global feature matrix of each supporting sample contained in the sample type; calculating the third average value of at least one supporting global feature matrix corresponding to the sample category; superimposing the second average value and the third average value to obtain the category interaction feature matrix corresponding to the sample category.

[0061] Specifically, for the mth sample category in the support sample set, m is an integer greater than or equal to 1, Represents the category interaction feature matrix corresponding to the sample category. Then get the category interaction feature matrix The principle is as follows:

[0062]

[0063] in, represents the second average value of the m-th sample category, representing the local features of the m-th sample category. The principle of acquisition is as shown in formula (5):

[0064]

[0065] represents the support local feature data matrix of the i-th support sample in the m-th sample category, and the m-th sample category includes N support samples.

[0066] represents the third average value of the m-th sample category, representing the global feature of the m-th sample category. The principle of acquisition is as shown in formula (6):

[0067]

[0068] represents the support global feature matrix of the i-th support sample in the m-th sample category, and the m-th sample category includes N support samples.

[0069] Category interaction feature matrix of the mth sample category By the second average and the third mean Obtained by superposition. Represents the category interaction feature matrix Equivalent to the average value of the support interaction feature matrix of N support samples in the mth sample category, represents the support interaction feature matrix of the i-th support sample in the m-th sample category. The support local feature matrix of the i-th support sample in the m-th sample category can be obtained and the support global feature matrix of the i-th support sample in the m-th sample category Obtained by superposition.

[0070] In one embodiment, when the query interaction feature is a query interaction feature matrix and the category interaction feature is a category interaction feature matrix, the similarity between the query sample and each sample category is calculated respectively based on the query interaction feature and the category interaction feature. The specific implementation process is as follows: the query interaction feature matrix and the cross-covariance matrix of the category interaction feature matrix corresponding to each sample category are calculated respectively; based on each cross-covariance matrix, the similarity between the query sample and each sample category is calculated respectively.

[0071] In this embodiment, when calculating the similarity between the query sample and the mth sample category in the support sample set, a similarity calculation method based on the Hilbert-Schmidt Independence Criterion (HSIC) of a linear kernel function is adopted to use a cross-covariance estimation method.

[0072] Specifically, the principle of calculating the similarity between the query sample and the mth sample category in the support sample set is as shown in formula (7):

[0073]

[0074] in, represents the similarity between the query sample and the mth sample category in the support sample set, C is the feature dimension of the query sample, represents the cross covariance matrix between the category interaction feature matrix of the mth sample category and the query interaction feature matrix of the query sample, Indicates that the cross-covariance matrix is ​​subjected to Frobenius processing.

[0075] The specific calculation principle of the cross-covariance matrix is ​​as shown in formula (8):

[0076]

[0077] in, represents the category interaction feature matrix of the mth sample category, represents the query interaction feature matrix of the query sample, and T represents the transpose calculation.

[0078] In this embodiment, after calculating the similarity between the query sample and each sample category in the support sample set, the category to which the query sample belongs is determined according to each similarity. The implementation process of determining the category according to the similarity can be set according to the actual situation and needs. For example, the similarities are sorted from large to small, and the sample category corresponding to the maximum similarity is used as the category to which the query sample belongs. Of course, other parameters or processing methods can also be introduced to process the similarity, and the scope of protection of this application is not limited to the specific implementation method of determining the category to which the query sample belongs by similarity.

[0079] In a specific embodiment, the method is applied to an image classification scenario, the support samples in the support sample set S, and the query samples to be classified The data formats are all pictures. Figure 2 As shown in the figure, a global-local information concurrent learning module is set up, which is used to obtain local features, global features and interactive features. After input into the global-local information concurrent learning module, the global-local information concurrent learning module extracts the query sample The query local features are obtained, and the query samples are obtained according to the query local features. Finally, according to the query local features and query global features, the query sample is obtained. The query interaction feature (interaction feature is also called global-local interaction information feature) of the support sample set S containing at least one sample category is input into the global-local information concurrent learning module one by one. The global-local information concurrent learning module extracts the support local feature of each support sample, and obtains the support global feature of each support sample based on the support local feature of each support sample. Finally, the category interaction feature (interaction feature is also called global-local interaction information feature) corresponding to each sample category in the support sample set S is obtained based on the support local feature and the support global feature of each support sample.

[0080] Get Inquiry Sample After the query interaction features and the category interaction features corresponding to each sample category in the support sample set S, the query sample is calculated by cross covariance estimation The similarity between each sample category and the support sample set.

[0081] Using this similarity, on the one hand, in the model training phase of the small sample learning model, the loss function can be constructed using the similarity to complete the model training; on the other hand, in the use phase of the small sample learning model, the similarity can be used to complete the model's query sample The prediction of the category.

[0082] The small sample learning processing method provided by the present disclosure, when processing the inquiry sample, in addition to extracting the inquiry local features of the inquiry sample, also obtains the inquiry global features of the inquiry sample through the inquiry local features. Then the inquiry interaction features formed by the interaction of the inquiry local features and the inquiry global features are obtained. Finally, the category to which the inquiry sample belongs is obtained based on the inquiry interaction features and the supporting sample set. The small sample learning processing process provided above does not process the inquiry local features of the inquiry sample separately, but obtains the inquiry interaction features containing the inquiry local features and the inquiry global features, and processes the inquiry interaction features. In this way, the interaction between the inquiry global features and the inquiry local features is used to semantically align the local areas of the inquiry samples, thereby improving the processing efficiency of the small sample learning process and making the inquiry sample category obtained by the small sample learning process more accurate.

[0083] In addition, the small sample learning algorithm based on global-local information interaction uses the interaction between global features and local features to guide semantic alignment, which can overcome the existing technology's neglect of global information, significantly reduce the weight of semantics irrelevant to global information, and reduce the variability of learning features in each class in the feature space, thereby reducing the time cost of learning, further improving the learning efficiency of the small sample learning method, and improving the performance of the trained small sample learning model.

[0084] The following describes a small sample learning processing device provided by an embodiment of the present disclosure. The small sample learning processing device described below and the small sample learning processing method described above can be referred to each other, and the repetitions are not repeated. Figure 3 As shown, the small sample learning processing device includes:

[0085] An extraction module 301 is used to extract local features of the inquiry sample;

[0086] A first acquisition module 302 is used to acquire a query global feature of the query sample based on the local feature;

[0087] A second acquisition module 303 is used to acquire an inquiry interaction feature of the inquiry sample based on the inquiry local feature and the inquiry global feature, wherein the inquiry interaction feature is formed by the interaction of the inquiry local feature and the inquiry global feature;

[0088] The category determination module 304 is used to determine the category to which the query sample belongs through the query interaction feature and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each sample category includes at least one supporting sample.

[0089] In one embodiment, the extraction module 301 is specifically used to extract the query local feature matrix of the query sample;

[0090] The first acquisition module 302 is specifically used to obtain at least one row vector contained in the inquiry local feature matrix; multiply each row vector by a preset first learnable parameter to obtain at least one weighted row vector; obtain a first average value of at least one weighted row vector; multiply the first average value by a preset second learnable parameter to obtain an inquiry global feature matrix of the inquiry sample.

[0091] In one embodiment, the second acquisition module 303 is specifically configured to superimpose the inquiry local feature matrix and the inquiry global feature matrix to obtain an inquiry interaction feature matrix of the inquiry sample.

[0092] In one embodiment, the category determination module 304 is specifically used to obtain the category interaction features corresponding to each sample category in the support sample set; calculate the similarity between the query sample and each sample category based on the query interaction features and the category interaction features; and determine the category to which the query sample belongs based on at least one similarity.

[0093] In one embodiment, the category determination module 304 is specifically used to obtain each category interaction feature, and the processing process for obtaining each category interaction feature is as follows: extract the supporting local features of each supporting sample contained in the sample category; calculate the second average value of at least one supporting local feature corresponding to the sample category; based on the supporting local features of each supporting sample in the sample category, respectively obtain the supporting global features of each supporting sample contained in the sample type; calculate the third average value of at least one supporting global feature corresponding to the sample category; superimpose the second average value and the third average value to obtain the category interaction feature corresponding to the sample category.

[0094] In one embodiment, the category determination module 304 is specifically used to respectively calculate the query interaction feature matrix and the cross covariance matrix of the category interaction feature matrix corresponding to each sample category; based on each cross covariance matrix, the similarity between the query sample and each sample category is respectively calculated, wherein the query interaction feature is the query interaction feature matrix, and the category interaction feature is the category interaction feature matrix.

[0095] In one embodiment, the query sample includes any one of a picture sample or a video sample.

[0096] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402 and the memory 403 communicate with each other through the communication bus 404. The processor 401 may call the logic instructions in the memory 403 to execute the small sample learning processing method, which includes: extracting the inquiry local features of the inquiry sample; obtaining the inquiry global features of the inquiry sample based on the inquiry local features; obtaining the inquiry interaction features of the inquiry sample based on the inquiry local features and the inquiry global features, wherein the inquiry interaction features are formed by the interaction of the inquiry local features and the inquiry global features; determining the category to which the inquiry sample belongs through the inquiry interaction features and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each sample category includes at least one supporting sample.

[0097] In addition, the logic instructions in the above-mentioned memory 403 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the embodiment of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0098] On the other hand, the present disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the small sample learning processing method provided by the above methods, and the method includes: extracting local inquiry features of the inquiry sample; based on the local inquiry features, obtaining global inquiry features of the inquiry sample; based on the local inquiry features and the global inquiry features, obtaining inquiry interaction features of the inquiry sample, wherein the inquiry interaction features are formed by the interaction of the local inquiry features and the global inquiry features; determining the category to which the inquiry sample belongs through the inquiry interaction features and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each sample category includes at least one supporting sample.

[0099] On the other hand, the present disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned small sample learning processing methods, the methods comprising: extracting local inquiry features of the inquiry sample; based on the local inquiry features, obtaining global inquiry features of the inquiry sample; based on the local inquiry features and the global inquiry features, obtaining inquiry interaction features of the inquiry sample, wherein the inquiry interaction features are formed by the interaction of the local inquiry features and the global inquiry features; determining the category to which the inquiry sample belongs through the inquiry interaction features and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each sample category includes at least one supporting sample.

[0100] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0101] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0102] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A small sample learning processing method, characterized in that: include: Extracting the inquiry local features of the inquiry sample; wherein the inquiry local features refer to the local features of the inquiry sample; the inquiry local features are extracted based on a local feature extractor pre-built in a small sample learning model; Based on the local inquiry feature, the inquiry global feature of the inquiry sample is obtained; wherein the inquiry global feature refers to the global feature of the inquiry sample; the obtaining of the inquiry global feature of the inquiry sample based on the local inquiry feature includes: obtaining at least one row vector contained in the inquiry local feature matrix; multiplying each of the row vectors by a preset first learnable parameter to obtain at least one weighted row vector; obtaining a first average value of the at least one weighted row vector; multiplying the first average value by a preset second learnable parameter to obtain the inquiry global feature matrix of the inquiry sample; Based on the local inquiry feature and the global inquiry feature, obtaining an inquiry interaction feature of the inquiry sample, wherein the inquiry interaction feature is formed by the interaction of the local inquiry feature and the global inquiry feature; the local inquiry feature and the global inquiry feature are both in the form of a matrix; The category to which the query sample belongs is determined by using the query interaction feature and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each of the sample categories includes at least one supporting sample.

2. The small sample learning processing method according to claim 1, characterized in that: The extracting of the local features of the query sample includes: A query local feature matrix of the query sample is extracted.

3. The small sample learning processing method according to claim 2, characterized in that: The acquiring the inquiry interaction feature of the inquiry sample based on the inquiry local feature and the inquiry global feature includes: The query local feature matrix and the query global feature matrix are superimposed to obtain a query interaction feature matrix of the query sample.

4. The small sample learning processing method according to claim 1, characterized in that: The determining the category to which the query sample belongs through the query interaction feature and the supporting sample set includes: Obtaining the category interaction features corresponding to each sample category in the supporting sample set; Calculating the similarity between the query sample and each of the sample categories respectively according to the query interaction feature and the category interaction feature; The category to which the query sample belongs is determined according to at least one of the similarities.

5. The small sample learning processing method according to claim 4, characterized in that: The process of obtaining each of the category interaction features is as follows: Extracting supporting local features of each supporting sample included in the sample category; Calculating a second average value of at least one of the supporting local features corresponding to the sample category; Based on the supporting local features of each supporting sample in the sample category, respectively obtaining supporting global features of each supporting sample included in the sample type; Calculating a third average value of at least one of the supporting global features corresponding to the sample category; The second average value and the third average value are superimposed to obtain the category interaction feature corresponding to the sample category.

6. The small sample learning processing method according to claim 4, characterized in that: The inquiry interaction feature is an inquiry interaction feature matrix, and the category interaction feature is a category interaction feature matrix; The calculating the similarity between the query sample and each of the sample categories respectively according to the query interaction feature and the category interaction feature includes: Respectively calculating the query interaction feature matrix and the cross covariance matrix of the category interaction feature matrix corresponding to each of the sample categories; Based on each of the cross-covariance matrices, the similarity between the query sample and each of the sample categories is calculated respectively.

7. The small sample learning processing method according to claim 1, characterized in that: The query sample includes any one of a picture sample or a video sample.

8. A small sample learning processing device, characterized in that: include: An extraction module, used for extracting the inquiry local features of the inquiry sample; wherein the inquiry local features refer to the local features of the inquiry sample; the inquiry local features are extracted based on a local feature extractor pre-built in a small sample learning model; A first acquisition module is used to acquire the inquiry global feature of the inquiry sample based on the local feature; wherein the inquiry global feature refers to the global feature of the inquiry sample; the acquisition of the inquiry global feature of the inquiry sample based on the inquiry local feature includes: acquiring at least one row vector contained in the inquiry local feature matrix; multiplying each of the row vectors by a preset first learnable parameter to obtain at least one weighted row vector; acquiring a first average value of the at least one weighted row vector; multiplying the first average value by a preset second learnable parameter to obtain the inquiry global feature matrix of the inquiry sample; A second acquisition module is used to acquire the inquiry interaction feature of the inquiry sample based on the inquiry local feature and the inquiry global feature, wherein the inquiry interaction feature is formed by the interaction of the inquiry local feature and the inquiry global feature; the inquiry local feature and the inquiry global feature are both in the form of a matrix; The category determination module is used to determine the category to which the query sample belongs based on the query interaction feature and the supporting sample set, wherein the supporting sample set includes at least one sample category, and each of the sample categories includes at least one supporting sample.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the small sample learning processing method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the small sample learning processing method as described in any one of claims 1 to 7 are implemented.

Images