Few-shot model learning method and device, equipment and storage medium

Abstract

Embodiments of the present application disclose a few-sample model learning method and device, equipment and a storage medium. The method comprises: obtaining a few-sample database, a natural database, a verification model and a few-sample model trained based on the few-sample database; performing data reasoning on natural data in the natural database based on the verification model to obtain a verification reasoning result; in a case where the class of the natural data represented by the verification reasoning result is consistent with a target class of few-sample data of the few-sample database, adding the natural data to the few-sample database to obtain an updated few-sample database; and updating the few-sample model according to the updated few-sample database to obtain an updated few-sample model.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of deep learning technology, and in particular to a few-shot model learning method, apparatus, device, and storage medium. Background Technology

[0002] Due to the rapid development of neural networks, significant breakthroughs have been achieved in many tasks based on deep neural networks, including image classification, object detection, natural language processing, and speech recognition. However, current neural network models are highly dependent on the amount of training data. If there is only a small amount of training data, the model often overfits, resulting in poor performance in actual use. Summary of the Invention

[0003] Based on the problems existing in related technologies, this application provides a few-shot model learning method, apparatus, device and storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] This application provides a few-shot model learning method, the method comprising:

[0006] Acquire a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database;

[0007] Based on the verification model, data reasoning is performed on the natural data in the natural database to obtain verification reasoning results;

[0008] If the category of the natural data represented by the verification inference result is consistent with the target category of the few-sample database, the natural data is added to the few-sample database to obtain an updated few-sample database;

[0009] The few-shot model is updated based on the updated few-shot database to obtain the updated few-shot model.

[0010] This application provides a few-shot model learning device, the device comprising:

[0011] The acquisition module is used to acquire a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database.

[0012] The data reasoning module is used to perform data reasoning on the natural data in the natural database based on the verification model, and obtain the verification reasoning result;

[0013] An add module is used to add the natural data to the few-sample database when the category of the natural data represented by the verification inference result is consistent with the target category of the few-sample database, thereby obtaining an updated few-sample database;

[0014] The update module is used to update the few-sample model according to the updated few-sample database to obtain the updated few-sample model.

[0015] This application provides a few-shot model learning device, including a processor and a memory. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the aforementioned few-shot model learning method.

[0016] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned few-shot model learning method.

[0017] This application provides a computer program product, which includes executable instructions stored in a computer-readable storage medium. When the processor of a few-shot model learning device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the aforementioned few-shot model learning method is implemented.

[0018] The few-shot model learning method, apparatus, device, and storage medium provided in this application perform data inference on natural data using a pre-trained validation model, and add natural data whose categories match the target category of the few-shot database to the few-shot database, thus expanding and updating the data in the few-shot database, and thereby training a new few-shot model. Firstly, it enables the mining of data suitable for few-shot model learning from natural data through a validation model, not only solving the problem of insufficient few-shot model data, but also continuously expanding the data pool of the few-shot database, thereby continuously expanding the cognitive boundary of the few-shot model and improving the accuracy of the few-shot model in test sets or actual use. Secondly, the few-shot model learning method provided in this application is applicable to any few-shot model task, providing a universal few-shot model learning process.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0021] Figure 1This is a schematic diagram illustrating an application scenario of the few-shot model learning method provided in the embodiments of this application;

[0022] Figure 2 This is a schematic diagram illustrating the implementation process of a few-shot model learning method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram illustrating the implementation process of a few-shot model learning method provided in an embodiment of this application;

[0024] Figure 4 This is a schematic diagram illustrating the implementation process of a few-shot model learning method provided in an embodiment of this application;

[0025] Figure 5 This is a schematic diagram illustrating the implementation process of a few-shot model learning method provided in an embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the framework structure corresponding to a few-shot model learning method provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram illustrating the updating of a small sample database provided in an embodiment of this application;

[0028] Figure 8 This is a schematic diagram of a few-shot model learning device provided in an embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the hardware entity of a few-shot model learning device provided in an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0032] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0034] Currently, few-shot learning is one of the important emerging research areas in deep learning. However, the research focus of current few-shot learning technologies is on the model itself, rather than the data. The limited amount of data is a significant and unavoidable problem that few-shot learning faces, yet current developments in few-shot learning have precisely overlooked this aspect.

[0035] Based on the problems existing in related technologies, this application embodiment performs data inference on natural data using a pre-trained validation model, and adds natural data whose categories are consistent with the target category of the few-shot database to the few-shot database, thus expanding and updating the data in the few-shot database, and thereby training a new few-shot model. Firstly, it realizes the mining of data suitable for few-shot model learning from natural data through the validation model, which not only solves the problem of insufficient few-shot model data, but also continuously expands the data pool of the few-shot database, thereby continuously expanding the cognitive boundary of the few-shot model. Secondly, the few-shot model learning method provided in this application is applicable to any few-shot model task, providing a general few-shot model learning process.

[0036] The few-shot model learning method provided in this application can be executed by electronic devices such as few-shot model learning devices. These electronic devices can be various types of terminals, including laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or they can be implemented as servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0037] The following will describe an exemplary application of the few-shot model learning device as a server. The technical solutions in the embodiments of this application will be clearly and completely described in conjunction with the accompanying drawings.

[0038] Figure 1This is a schematic diagram illustrating an application scenario of the few-shot model learning method provided in this application embodiment. The few-shot model learning system 10 provided in this application embodiment includes a terminal 100, a network 200, and a server 300. The network 200 can be a wide area network (WAN), a local area network (LAN), or a combination of both. The server 300 and the terminal 100 can be physically separate or integrated. During few-shot model learning, the server 300 acquires a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database from the network 200. Based on the validation model, it performs data inference on the natural data in the natural database to obtain validation inference results. If the category of the natural data represented by the validation inference results matches the target category of the few-shot data in the few-shot database, the natural data is added to the few-shot database to obtain an updated few-shot database. Based on the updated few-shot database, the few-shot model is updated to obtain an updated few-shot model, and the updated few-shot model is sent to the terminal 100 via the network 200.

[0039] Figure 2 This is a schematic diagram illustrating the implementation process of a few-shot model learning method provided in an embodiment of this application, as shown below. Figure 2 As shown, the method is implemented through steps S201 to S204:

[0040] Step S201: Obtain a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database.

[0041] In this embodiment, the data in the few-shot database is labeled data and may include multiple few-shot data sets. These few-shot data sets are used for training and optimizing the few-shot model. Here, few-shot data refers to data corresponding to categories with a small amount of data. For example, if a category has only 1 to 10 data sets, then the data corresponding to that category is considered few-shot data. Few-shot data can be text, image, or speech data, etc. The type of few-shot data is related to the type of few-shot model. If the few-shot model is an image classification model, then the few-shot data is image data; when the few-shot model is an object detection model or an object segmentation model, the few-shot data can also be image data; when the few-shot model is a text classification model, sentiment analysis, or language modeling, the few-shot data is text data; when the few-shot model is a speech recognition model, the few-shot data is speech data.

[0042] In some embodiments, few-shot models learn using a small amount of data, enabling the model to quickly learn new tasks and adapt to new environments. For example, in the field of medical diagnostics, they can identify rare diseases and other abnormalities with limited data, which can help personalize treatment and predict patient outcomes.

[0043] In some embodiments, the natural database may include multiple natural data sources, which can be from a wide range of sources, such as online data, public datasets, or private industrial datasets. Natural data is used to filter data that meets certain criteria and add it to the small sample database to expand the small sample data.

[0044] In some embodiments, the validation model is used to perform inference on natural data and obtain the inference results. Here, the validation model can be used for inference such as classification, object detection, or image segmentation of natural data. The validation model can consist of a pre-trained model and an output layer. The pre-trained model is a model that was not trained using data from a small sample database. The validation model can be a common supervised pre-trained model, such as an ImageNet pre-trained model, a COCO pre-trained model, or a pre-trained model for text tasks, etc. The pre-trained model can also be an unsupervised trained model. Taking image classification models as an example, the validation model can be a neural network model trained by unsupervised methods such as Moco, SimCLR, Byol, Dino, or MAE. The output layer can be a structure with classification, object detection, or image segmentation functions after training, used to output the inference results of the validation model.

[0045] In this embodiment, the few-shot model is trained based on few-shot data. The few-shot model can be any type of few-shot model architecture, such as a few-shot classification model, a few-shot object detection model, a few-shot image segmentation model, or a few-shot natural language processing model. Different types of few-shot models correspond to different types of few-shot databases.

[0046] Step S202: Based on the verification model, perform data reasoning on the natural data in the natural database to obtain the verification reasoning result.

[0047] In some embodiments, the validation model consists of a pre-trained model and an output layer. The pre-trained model provides a semantically rich backbone network for extracting semantic information from natural data (e.g., natural images). The output layer is used to adjust the output format and can be a learnable fully connected layer or a learnable output layer, such as a network structure based on the k-Nearest Neighbor (KNN) algorithm and support vector machines.

[0048] In some embodiments, the natural database may contain multiple natural data. Data reasoning on the natural data may refer to classifying the natural data to obtain a verification reasoning result containing the categories of the natural data. Data reasoning on the natural data may also refer to object detection on the natural data to obtain a verification reasoning result containing the target objects in the natural data.

[0049] Here, the type of data inference is the same as the type of few-shot model. When the few-shot model is a classification model, data inference involves classifying natural data and verifying that the inference result is the category of each natural data point. When the few-shot model is an image segmentation model, data inference involves segmenting natural data into images and verifying that the inference result is the segmented image. The following embodiments all use the few-shot model as a classification model and data inference as the classification of natural data as an example to describe in detail the few-shot model learning method provided in this application.

[0050] Step S203: If the category of the natural data represented by the verification inference result is consistent with the target category of the few-sample data in the few-sample database, the natural data is added to the few-sample database to obtain an updated few-sample database.

[0051] In some embodiments, the few-shot database may include few-shot data for multiple target categories, such as flying objects, animals, or other data categories with limited data volume. Here, data in the natural data that belongs to the same category as the target category can be identified as natural data that meets the screening criteria, meaning that this natural data can be used for training the few-shot model.

[0052] In some embodiments, natural data in the natural database with the same category as the target category can be used for training the few-shot model. Adding natural data to the few-shot database can increase the few-shot data in the few-shot database, expand the few-shot data pool, and obtain an updated few-shot database.

[0053] Step S204: Update the few-sample model according to the updated few-sample database to obtain the updated few-sample model.

[0054] In some embodiments, updating a few-shot model means retraining the few-shot model by updating the few-shot database, thereby training a new few-shot model, continuously expanding the cognitive boundaries of the model, and avoiding poor performance of the few-shot model in use.

[0055] In some embodiments, updating a few-shot model may refer to updating the model parameters of the few-shot model.

[0056] The application embodiment can also update the natural database using network data or other publicly available data to obtain an updated natural database. The updated natural database can then be used to identify natural data whose categories match the target category of the few-shot database. Based on the natural data whose categories match the target category of the few-shot database again, the few-shot database is expanded again, and the few-shot model is trained again using the expanded few-shot database to obtain a new few-shot model. In this way, not only is the data expansion of the few-shot database realized, but also the cyclical updating of the few-shot model is realized, which improves the accuracy of the few-shot model in the test set or in actual use.

[0057] First, this application addresses the problem of prioritizing models over data in few-shot learning research. It focuses on the data level by providing a validation model (i.e., a parallel framework for few-shot models) to mine scarce samples in natural data for few-shot model learning. Second, to address the lack of data for few-shot models, this application proposes continuously expanding the data pool by mining natural data whose categories match the target category of the few-shot database, thereby continuously expanding the cognitive boundaries of the model. Finally, current few-shot research frameworks are mostly domain-specific, such as focusing only on images or text, while the few-shot model learning method proposed in this application is applicable to any few-shot task, establishing a universal few-shot learning process.

[0058] In some embodiments, the validation model includes at least a pre-trained model and an output layer. The output layer can be a learnable fully connected layer or a non-learnable output layer. When the output layer of the validation model is a learnable fully connected layer, i.e., the validation model consists of a pre-trained model and a first output layer, such as... Figure 3 As shown, step S203 can be achieved through steps S301 to S302:

[0059] Step S301: Based on the pre-trained model, extract feature information from the natural data to obtain the first feature data.

[0060] In some embodiments, the pre-trained model is not trained on a few-sample database during model training. The pre-trained model can be a common supervised pre-trained model or an unsupervised training model, used to extract feature information from natural data (e.g., natural images). Here, feature extraction refers to extracting semantic information for classification from natural data. Semantic information can be low-level semantic information such as contours, edges, colors, textures, and shapes, or high-level semantic information such as eyes, noses, blue skies, and seawater. Specifically, the pre-trained model can extract feature information from the feature information of each natural data point in the natural database to obtain first feature data containing the semantic information of each natural data point.

[0061] Step S302: Based on the first output layer, classify the first feature data to obtain the first classification result of the natural data; wherein, the first output layer is an output layer with classification function obtained by classification learning based on the few sample data in the few sample database.

[0062] In some embodiments, the first output layer refers to an output layer with classification capabilities obtained by performing classification learning on a few-shot dataset from a few-shot database. Here, the first output layer is trained on few-shot data to achieve the same classification ability as the few-shot model.

[0063] Based on the semantic information of natural data, the first feature data is classified through the first output layer with classification function. This determines the category of each natural data and forms the first classification result of the natural data. The category of each natural data constitutes the verification reasoning result after the verification model performs data reasoning on the natural data.

[0064] In some embodiments, before determining whether the category of the natural data in the verification inference result is consistent with the target category of the few-sample data in the few-sample database, the classification result of the few-sample model on the natural data can also be determined. If the category of some natural data in the verification inference result is the same as the category classified by the few-sample model, and the category is the same as the target category of the few-sample database, then some natural data in the verification inference result can be added to the few-sample database. Based on the foregoing embodiments, after step S202, step S303 may also be included:

[0065] Step S303: Based on the few-sample model, classify the natural data to obtain the second classification result of the natural data.

[0066] In this embodiment of the application, the few-shot model is a classification model trained on a few-shot database. The few-shot model is used to classify natural data to obtain natural data with different categories, i.e., the second classification result.

[0067] In this embodiment, after both the validation model and the few-shot model classify the natural data to obtain validation inference results and classification results, the consistency between the output categories of the few-shot model and the validation model can be used to determine whether the natural data corresponds to the target category, i.e., the data required in the few-shot database. Based on the foregoing embodiment, step S203 can be implemented through steps S304 and S305:

[0068] Step S304: Compare the second classification result and the verification reasoning result, and determine the natural data that have the same category in the first classification result and the second classification result and belong to the target category as the updated data.

[0069] In some embodiments, after obtaining the second classification result and the validation inference result, the second classification result and the validation inference result are compared. If there is a portion of the natural data in the validation inference result whose category is the same as the category corresponding to that natural data in the second classification result, and that same category is the target category of the few-shot database, then that portion of the natural data is determined as updated data. For example, in the validation inference result, some data is classified as flying objects, the target category in the few-shot database includes flying objects, and the category of this portion of natural data after classification by the few-shot model is also flying objects, then this portion of natural data can be used for training the few-shot model; therefore, this portion of data is determined as updated data.

[0070] Step S305: Add the updated data to the few-sample database to obtain the updated few-sample database.

[0071] In some embodiments, adding updated data to the minimum sample database can increase the number of sample data in the minimum sample database, expand the minimum sample data pool, and obtain an updated minimum sample database.

[0072] This application embodiment classifies and mines natural data by comparing classification results and verification inference results, and determines natural data in the natural database that can be used for training of few-shot models, thereby realizing the data expansion of the few-shot database and continuously expanding the cognitive boundaries of the model.

[0073] In some embodiments, when the output layer of the validation model is a non-learnable output layer, i.e., the validation model consists of a pre-trained model and a second output layer, such as... Figure 4 As shown, step S202 can be achieved through steps S401 and S402:

[0074] Step S401: Based on the pre-trained model, extract feature information from the natural data to obtain second feature data.

[0075] Here, the pre-trained model is the same as the pre-trained model in the previous embodiment, both of which are used to extract feature information from the semantic information of natural data to obtain the second feature data.

[0076] Step S402: Based on the second output layer, perform feature extraction on the second feature data to obtain the verification inference result that includes at least the natural feature vector.

[0077] In some embodiments, the second output layer can be a network structure based on the k-Nearest Neighbor (KNN) algorithm and support vector machine. The second output layer can be used to extract features from the second feature data to obtain the natural feature vector of each natural data. The natural feature vector of each natural data constitutes the verification inference result after the verification model performs inference on the natural data.

[0078] In some embodiments, the verification model can also perform data inference on the few-sample data in the few-sample database to obtain the few-sample feature vector for each few-sample data. The feature distance between each natural feature vector and each few-sample feature vector can be calculated based on the KNN algorithm. Natural data with a feature distance less than a threshold are identified as data suitable for training the few-sample model. Based on the foregoing embodiments, steps S403 and S404 may be included after step S202:

[0079] Step S403: Based on the pre-trained model, extract feature information from the few sample data in the few sample database to obtain the third feature data.

[0080] In this embodiment, the pre-trained model is the same as the pre-trained model in the previous embodiment. The semantic information of each few sample data in the few sample database can be obtained through this pre-trained model, and the third feature data can be obtained.

[0081] Step S404: Based on the second output layer, perform feature extraction on the third feature data to obtain the few-sample inference result that includes at least the few-sample feature vector.

[0082] In some embodiments, the second output layer can also be used to extract features from the third feature data to obtain the few sample feature vector of each few sample data. The few sample feature vector of each few sample data constitutes the few sample inference result of the validation model after inferring the few sample data.

[0083] Based on the foregoing embodiments, step S203 can also be implemented via steps S405 to S407:

[0084] Step S405: Based on the verification inference result, the few-sample inference result, and the second output layer, determine the feature distance between the few-sample feature vector and the natural feature vector.

[0085] In this embodiment, after determining the verification inference result containing natural feature vectors and the few-sample inference result containing few-sample feature vectors, the feature distance between each natural feature vector and each few-sample feature vector can be determined based on the second output layer of the verification model. Here, the feature distance can be the Euclidean distance or cosine distance between each natural feature vector and each few-sample feature vector, or other feature distances that can characterize the similarity between feature vectors.

[0086] Step S406: Determine the natural data corresponding to the natural feature vectors whose feature distance is less than the distance threshold as the updated data.

[0087] In some embodiments, when Euclidean distance is used to calculate the feature distance between each natural feature vector and each few-sample feature vector, the feature distance value between each natural feature vector and each few-sample feature vector can be obtained. The natural data corresponding to the natural feature vector whose feature distance with the few-sample feature vector is less than the distance threshold can be determined as the updated data that can be used for few-sample model training. That is, the category of the natural data is consistent with the category of the data in the few-sample database.

[0088] Step S407: Add the updated data to the few-sample database to obtain the updated few-sample database.

[0089] In some embodiments, adding updated data to the minimum sample database can increase the number of sample data in the minimum sample database, expand the minimum sample data pool, and obtain an updated minimum sample database.

[0090] This application embodiment determines the feature distance between the few-shot feature vector in the few-shot inference result and the natural feature vector in the verification inference result, and determines the natural data in the natural database that can be used for few-shot model training, thereby realizing the data expansion of the few-shot database and continuously expanding the cognitive boundary of the model.

[0091] In some embodiments, natural data can be iteratively updated to meet the needs of more models. After updating the natural data, more data for few-shot models can be mined from the natural data, thereby updating the model parameters of the few-shot model and obtaining a new few-shot model. Based on the foregoing embodiments, as follows... Figure 5As shown, after step S204, the few-shot model learning method provided in this application embodiment may further include steps S501 to S504:

[0092] Step S501: Update the natural data in the natural database to obtain an updated natural database.

[0093] In some embodiments, the data update of the Xiran database can be achieved by updating the data source of the natural data, adding new datasets to the natural database, or acquiring network data and adding it to the natural database, thereby obtaining an updated natural database.

[0094] Step S502: Based on the verification model, perform data reasoning on the natural data in the updated natural database to obtain the updated verification reasoning result.

[0095] In this embodiment of the application, after the natural database is updated, the natural data in the updated natural database can be used to perform data reasoning through the verification model to obtain the updated verification reasoning result.

[0096] Step S503: Add the natural data whose category in the updated verification inference result is consistent with the target category of the few-sample database to the updated few-sample database to obtain the first updated few-sample database.

[0097] Step S504: Based on the first updated few-sample database, update the updated few-sample model to obtain the first updated few-sample model.

[0098] In this embodiment of the application, by updating the updated verification inference results, natural data with the same category as the target category in the updated verification inference results can be added to the updated few-sample database to obtain the first updated few-sample database. Based on the first updated few-sample database, the model parameters of the updated few-sample model are adjusted again to obtain the first updated few-sample model, thereby realizing the update of the few-sample database and the few-sample model again.

[0099] This application embodiment continuously updates the natural database and filters the natural data through a validation model to obtain data that can be used for training a few-shot model. This results in a continuous increase in the data in the few-shot database, allowing the few-shot model to be trained based on more training data, thereby improving the accuracy of the updated few-shot model in the test set and in actual use.

[0100] This application provides another application of a few-shot model learning method in a real-world scenario.

[0101] Figure 6 This is a schematic diagram of the framework structure corresponding to a few-shot model learning method provided in an embodiment of this application, as shown below. Figure 6 As shown, the framework structure corresponding to the few-shot model learning method in this application embodiment includes at least a few-shot learning model 601 (i.e., a few-shot model), a validation model 602 (i.e., a pre-trained model), an output layer 603 (i.e., a first output layer or a second output layer), a result comparison module 604, a few-shot database 605, and a natural database 606.

[0102] In some embodiments, the input data can be divided into a few-shot database 605 and a natural database 606. The few-shot database 605 contains only data for some scarce categories, meaning that a scarce category has very few samples, such as only 1-10 data units for a single category. The data in the few-shot database is labeled data used to train the few-shot model. The data units in the few-shot database can be text, images, and audio clips, etc. The natural data in the natural database 606 comes from a wide range of sources, including web data, public datasets, and private industrial datasets, etc.

[0103] The few-shot data in the few-shot database 605 is used to optimize the few-shot learning model, while the natural data is used for inference of the few-shot model 601, the validation model 602, and the output layer 603. The inference results of the two models are sent to the result comparison module 604 to filter the natural data. Natural data that meets the filtering conditions will be added to the few-shot database.

[0104] In some embodiments, the few-shot model receives few-shot data and is trained based on the few-shot data to obtain a new few-shot model. The new few-shot model and the validation model receive natural data from the natural database and output inference results respectively. The result comparison module, based on the output results of the few-shot model and the validation model, determines the natural data in the natural data that meets the screening criteria (i.e., the category is consistent with the target category of the few-shot database), and adds the natural data that meets the screening criteria to the few-shot database to obtain an updated few-shot database. The data in the updated few-shot database is then put into the new few-shot model, which is further trained to obtain an updated few-shot model. Following the previous logic, the updated few-shot model and the validation model receive another batch of updated natural data, and the result comparison module obtains another batch of natural data that meets the screening criteria. The natural data that meets the screening criteria and the updated few-shot database are merged again, and the aforementioned steps are repeated to achieve cyclical updates of the few-shot database and the few-shot model.

[0105] Here, natural data can be updated in batches, such as updating the source of natural data or adding new datasets.

[0106] In some embodiments, the few-shot model supports any type of few-shot model architecture, such as few-shot classification models, few-shot object detection models, few-shot image segmentation models, few-shot natural language processing models, etc. The validation model is responsible for inference on natural data, obtaining the inference results for subsequent result verification. The validation model must be a common model or a model for which the corresponding scarce samples have not been seen before. For example, it can be a common supervised pre-trained model, including ImageNet pre-trained models, COCO pre-trained models, or pre-trained models for text tasks, etc. In addition, the validation model can also be an unsupervised trained model. Taking image tasks as an example, this includes neural network models trained using unsupervised methods such as Moco, SimCLR, Byol, Dino, or MAE. These models do not require data labels, which aligns with the original intention of few-shot learning.

[0107] The validation model 602 provides a semantically rich backbone network for extracting semantic information from natural images. The output layer 603 is used to adjust the output format. The output layer 603 can be a learnable fully connected layer or a learnable output layer, such as KNN or support vector machine.

[0108] In this embodiment of the application, the result comparison module 604 is used to determine whether the current natural data is scarce category data that meets the screening conditions. The judgment criteria include two: (1) whether the output categories of the few-sample model and the validation model for the natural data are consistent; and (2) the category judgment results of the validation model for the few-sample data and the natural data.

[0109] For the first judgment criterion, the output layer is trained using a few-shot database. At this point, the output layer after validating the model has the ability to distinguish categories. Then, the output results of the few-shot model and the output results of the validation model can be compared. If the two category predictions are consistent and the category belongs to a scarce sample in the few-shot database (i.e., the target category), then the corresponding natural data is added to the few-shot database by comparing the results.

[0110] For the second judgment criterion, the output layer is a class discrimination network structure that does not require training, such as KNN or support vector machine. The output layer outputs the feature vectors of the few sample data and the feature vectors of the natural data. Based on the feature distance between the feature vectors of the few sample data and the feature vectors of the natural data, it is determined whether the natural data meets the screening conditions. When the feature distance is less than the feature threshold, the natural data is considered to be in the same category as the few sample data. Then, through the result comparison model, the corresponding natural data is added to the few sample database.

[0111] In some embodiments, the two judgment criteria mentioned above can be in the form of OR (one of them must be satisfied) or in the form of AND (both of them must be satisfied).

[0112] In this embodiment, the cyclic update includes data updates and model updates. The cyclic update, based on the output of the result comparison module, selects whether to add or not add current natural data to the few-sample database. The updated few-sample database can then be used to train a new few-sample model, thus obtaining more added data. During the cyclic update process, the natural data can also be iteratively updated to meet the needs of more samples. Figure 7 This is a schematic diagram illustrating the updating of a small sample database provided in an embodiment of this application, such as... Figure 7 As shown, through cyclical updates, the few-sample database expands from d0 to d1, and then to d2. The new samples added during this process can also continuously expand the cognitive boundaries of the few-sample model.

[0113] First, this application addresses the problem of prioritizing models over data in few-shot learning research. It focuses on the data level by providing a validation model (i.e., a parallel framework for few-shot models) to mine scarce samples in natural data for few-shot model learning. Second, to address the lack of data for few-shot models, this application proposes continuously expanding the data pool by mining natural data whose categories match the target category of the few-shot database, thereby continuously expanding the cognitive boundaries of the model. Finally, current few-shot research frameworks are mostly domain-specific, such as focusing only on images or text. The few-shot model learning method proposed in this application is applicable to any few-shot task, establishing a universal few-shot learning process that can be applied to various learning tasks, including images, text, and speech, such as in smart cameras integrating few-shot algorithms.

[0114] Based on the above embodiments, this application provides a few-shot model learning device. Figure 8 This is a schematic diagram of a few-shot model learning device provided in an embodiment of this application, as shown below. Figure 8 As shown, the device 80 includes an acquisition module 801, a data inference module 802, an addition module 803, and an update module 804.

[0115] The acquisition module 801 is used to acquire a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database.

[0116] The data reasoning module 802 is used to perform data reasoning on the natural data in the natural database based on the verification model, and obtain the verification reasoning result.

[0117] The addition module 803 is used to add the natural data to the few-sample database when the category of the natural data represented by the verification inference result is consistent with the target category of the few-sample data in the few-sample database, thereby obtaining an updated few-sample database.

[0118] The update module 804 is used to update the few-sample model according to the updated few-sample database to obtain the updated few-sample model.

[0119] In some embodiments, the verification inference result includes a first classification result; the verification model includes at least a pre-trained model and a first output layer; the data inference module 802 is further configured to extract feature information from the natural data based on the pre-trained model to obtain first feature data; and to perform classification processing on the first feature data based on the first output layer to obtain a first classification result of the natural data; wherein, the first output layer is an output layer with classification function obtained by classification learning based on few sample data in the few sample database.

[0120] In some embodiments, the apparatus further includes a classification processing module for classifying the natural data based on the few-sample model to obtain a classification result; correspondingly, the adding module 803 is further configured to compare the second classification result and the verification inference result, determine the natural data in the first classification result and the second classification result that have the same category and belong to the target category as updated data; and add the updated data to the few-sample database to obtain an updated few-sample database.

[0121] In some embodiments, the verification model includes at least a pre-trained model and a second output layer; the data inference module 802 is further configured to extract feature information from the natural data based on the pre-trained model to obtain second feature data; and to extract features from the second feature data based on the second output layer to obtain the verification inference result including at least the natural feature vector.

[0122] In some embodiments, the apparatus further includes a feature information extraction module, configured to extract feature information from the few-sample data in the few-sample database based on the pre-trained model to obtain third feature data; a feature extraction module, configured to extract features from the third feature data based on the second output layer to obtain the few-sample inference result including at least the few-sample feature vector; correspondingly, the adding module 803 is further configured to determine the feature distance between the few-sample feature vector and the natural feature vector based on the verification inference result, the few-sample inference result and the second output layer; determine the natural data corresponding to the natural feature vector whose feature distance is less than a distance threshold as updated data; and add the updated data to the few-sample database to obtain an updated few-sample database.

[0123] In some embodiments, the apparatus further includes: a first update module, configured to update the natural data in the natural database to obtain an updated natural database; a first data inference, based on the verification model, to perform data inference on the natural data in the updated natural database to obtain an updated verification inference result; a first addition module, configured to add natural data in the updated verification inference result whose category is consistent with the target category of the few-sample database to the updated few-sample database to obtain a first updated few-sample database; and a second update module, configured to update the updated few-sample model based on the first updated few-sample database to obtain a first updated few-sample model.

[0124] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0125] It should be noted that, in the embodiments of this application, if the above-described few-shot model learning method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a few-shot model learning device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0126] This application provides a few-shot model learning device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the aforementioned few-shot model learning method.

[0127] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned few-shot model learning method. The computer-readable storage medium can be transient or non-transient.

[0128] This application provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program. When the computer program is read and executed by a computer, it implements some or all of the steps in the above-described method. This computer program product can be implemented specifically through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied as a computer storage medium; in another optional embodiment, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0129] It should be noted that, Figure 9 This is a schematic diagram of the hardware entity of a few-shot model learning device provided in an embodiment of this application, such as... Figure 9 As shown, the hardware entity of the few-shot model learning device 90 includes: a processor 901, a communication interface 902, and a memory 903, wherein:

[0130] The processor 901 typically controls the overall operation of the few-shot model learning device 90.

[0131] Communication interface 902 enables the few-shot model learning device to communicate with other terminals or servers over a network.

[0132] The memory 903 is configured to store instructions and applications executable by the processor 901, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 901 and the few-shot model learning device 90. It can be implemented using flash memory or random access memory (RAM). Data transfer between the processor 901, the communication interface 902, and the memory 903 can be performed via bus 904.

[0133] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0134] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0135] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0137] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0138] In addition, each functional unit in the embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0139] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0140] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a few-shot model learning device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0141] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A few-shot model learning method, characterized in that, The method includes: Acquire a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database; the data category of the few-shot database is one of the following: images, text, or speech; Based on the verification model, data reasoning is performed on the natural data in the natural database to obtain verification reasoning results; the verification reasoning results include a first classification result; the verification model includes at least a pre-trained model and a first output layer, the pre-trained model is used to extract features from the natural data, and the first output layer is used to classify the data obtained from feature extraction to obtain a first classification result of the natural data; If the category of the natural data represented by the verification inference result is consistent with the target category of the few-sample data in the few-sample database, the natural data is added to the few-sample database to obtain an updated few-sample database. Based on the updated few-shot database, the few-shot model is updated to obtain the updated few-shot model; The method further includes: Based on the few-shot model, the natural data is classified to obtain a second classification result for the natural data; Correspondingly, if the category of the natural data represented by the verification inference result is consistent with the target category of the few-sample database, the natural data is added to the few-sample database, including: The second classification result and the verification reasoning result are compared, and the natural data that have the same category in the first classification result and the second classification result and belong to the target category are determined as the updated data; The updated data is added to the few-sample database to obtain the updated few-sample database.

2. The method according to claim 1, characterized in that, Based on the verification model, data reasoning is performed on the natural data in the natural database to obtain verification reasoning results, including: Based on the pre-trained model, feature information is extracted from the natural data to obtain the first feature data; Based on the first output layer, the first feature data is classified to obtain the first classification result of the natural data; wherein, the first output layer is an output layer with classification function obtained by classification learning based on the few sample data in the few sample database.

3. The method according to claim 1 or 2, characterized in that, After obtaining the updated few-shot model, the method further includes: The natural data in the natural database is updated to obtain an updated natural database; Based on the verification model, data reasoning is performed on the natural data in the updated natural database to obtain the updated verification reasoning result; Natural data whose category in the updated verification inference result matches the target category in the few-sample database are added to the updated few-sample database to obtain the first updated few-sample database; Based on the first updated few-sample database, the updated few-sample model is updated to obtain the first updated few-sample model.

4. A few-shot model learning device, characterized in that, The device includes: The acquisition module is used to acquire a few-shot database, a natural database, a validation model, and a few-shot model trained based on the few-shot database; the data category of the few-shot database is one of the following: images, text, or speech; The data inference module is used to perform data inference on natural data in the natural database based on the verification model to obtain a verification inference result; the verification inference result includes a first classification result; the verification model includes at least a pre-trained model and a first output layer, the pre-trained model is used to extract features from the natural data, and the first output layer is used to classify the data obtained from the feature extraction to obtain a first classification result of the natural data; An addition module is used to add natural data whose category in the verification inference result is consistent with the target category of the few-sample database to the few-sample database, thereby obtaining an updated few-sample database; The update module is used to update the few-sample model according to the updated few-sample database to obtain the updated few-sample model; The device further includes: The classification module is used to classify the natural data based on the few-shot model to obtain a second classification result of the natural data; Correspondingly, the addition module is also used to compare the second classification result and the verification inference result, and to determine the natural data that have the same category in the first classification result and the second classification result and belong to the target category as updated data; and to add the updated data to the few-sample database to obtain the updated few-sample database.

5. A few-shot model learning device, characterized in that, It includes a memory and a processor, the memory storing a computer program that can run on the processor, the processor executing the program to implement the few-shot model learning method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, It stores executable instructions that cause the processor to execute the executable instructions to implement the few-shot model learning method according to any one of claims 1 to 3.

Images