Unified paradigm, task construction, model optimization and error control methods for few-shot learning

Abstract

The present invention proposes a unified paradigm, task construction, model optimization and error control method for small sample learning. The method establishes a small sample learning theory for limited supervised information, that is, universal prior information is obtained by learning cross-category data, and it is transferred to the small sample target task to achieve recognition and prediction. The method establishes a unified framework for small sample learning from data division to task construction to optimization strategy, and realizes the complete expression of the whole process of small sample learning.

Description

Technical Field

[0001] The present invention belongs to the fields of machine learning, deep learning, and computer vision technology. The technical fields that can be directly applied include structural health monitoring and damage identification, intelligent operation and maintenance of infrastructure, intelligent bridge inspection, and intelligent structural disaster prevention, especially the unified paradigm, task construction, model optimization, and error control methods related to small sample learning. Background Art

[0002] In recent years, deep learning and computer vision technologies have developed rapidly, especially for supervised learning image understanding problems. At present, researchers at home and abroad have collected relevant data sets for different tasks in different fields, and proposed a series of network models such as image classification, object detection and semantic segmentation based on machine learning, deep learning and computer vision recognition tasks, realizing multi-level feature extraction and scene understanding of input images. These emerging network models mainly improve the following two aspects:

[0003] (1) Establish new network architectures, such as ResNet / U-net / DenseNet / Deeplab, etc., by increasing the connection between adjacent or different levels of feature maps and feature fusion / aggregation operations or operations, directly or indirectly increasing the depth and width of the network model, thereby improving the model's feature extraction capabilities;

[0004] (2) Design special functional modules, such as channel / spatial self-attention mechanism and Transformer series, etc., to improve the model's ability to perceive certain important features by adding new functional modules or replacing the original modules, thereby improving model performance.

[0005] The above model based on deep learning mechanism can extract multi-level feature information of images through deep network, which guarantees the efficiency and accuracy of image recognition to a certain extent, but it has the following limitations:

[0006] (1) The above methods are highly dependent on supervision when solving specific problems. They require the collection of a large amount of training data and their labels for specific fields and tasks in advance. In addition, their generalization ability and recognition accuracy are highly correlated with the number of training samples, category completeness, and label accuracy.

[0007] (2) The above methods are all performed on specific datasets, so the trained models usually have better expression ability for the dataset, but weaker generalization ability for new categories of images, and are not universal and generalizable.

[0008] (3) In actual scenarios, data sets often have the characteristics of incomplete information, insufficient samples, unbalanced data, and inaccurate labels, which to a certain extent lead to incomplete model expression capabilities. The generalization ability and accuracy of the model are difficult to guarantee on data sets with a limited number of samples. Summary of the invention

[0009] The purpose of the present invention is to solve the problems in the prior art and propose a unified paradigm, task construction, model optimization and error control method for small sample learning.

[0010] The present invention is implemented by the following technical scheme. The present invention proposes a unified paradigm, task construction, model optimization and error control method for small sample learning. The method comprises the following steps:

[0011] Step 1: Data division and task construction for small sample learning;

[0012] The training process of small sample learning is task-based. The strategy of sampling with replacement is used to construct training tasks and test tasks in the training data set and query data set. Each task consists of two mutually exclusive sets, the support set and the query set.

[0013] Step 2: Design an optimization strategy for small sample learning;

[0014] The typical optimization strategies used by small sample learning models when learning prior knowledge are divided into two modes, namely single-stage optimization learning and double-stage optimization learning, and are defined respectively.

[0015] Step 3: Establish a classification paradigm and mathematical framework for small sample learning;

[0016] According to the differences in the intrinsic learning mechanisms and implementation principles of small sample learning methods for limited supervised information, based on different processes of model optimization, a mathematical framework for small sample learning is established, and four types of small sample learning paradigms are formed and defined, namely, the small sample learning paradigm based on metric learning, the small sample learning paradigm based on optimization algorithms, the small sample learning paradigm based on transfer learning, and the small sample learning paradigm based on generative models.

[0017] Step 4: Establish an error control method for small sample learning;

[0018] According to the four types of optimization paradigms of small sample learning and based on the data-driven machine learning error analysis theory, error control of three typical errors in machine learning in small sample scenarios, namely optimization error, generalization error and approximation error, is achieved.

[0019] Furthermore, the step 1 specifically includes the following steps:

[0020] Step 1: Design a model training and testing process based on supervised learning rules;

[0021] Let a deep network model be f, whose optimizable parameter is θ, in a labeled training set D train The model is trained on the dataset D and the trained model is used to test an unlabeled test set D test Make predictions; the optimization goal of the supervised learning model is to train The model parameters θ are optimized and the optimized parameters containing prior knowledge are used For the test set D test The data in is predicted, and the process of training optimization and test prediction is expressed as:

[0022]

[0023] In the formula, Train and Test represent the training optimization and test prediction processes respectively, I i and Y i They represent the i-th input sample and its corresponding label in the training set, L represents the loss function selected for model optimization according to the target task, and I j and They represent the jth sample to be predicted in the test set and its prediction result respectively;

[0024] Step 1 and 2: Design the data partitioning method and training and testing tasks for small sample learning under limited supervision conditions;

[0025] For small sample learning scenarios, the training set D train The labeled samples in the test set D test The unlabeled samples in are composed of data of multiple categories, and the categories included are represented as C train and C test The basic learning unit of small-sample learning is the N-way-K-shot task, which means that each learning task includes N categories and K samples per category. train and the test set D test In the training task and test task, the training task T is formed by sampling with replacement. train and test task T test They are composed of a support set S and a query set Q, where S is set to N-way-K-shot and Q is set to N-way-1-shot. The construction process of training and testing tasks for small sample learning is expressed as:

[0026]

[0027] Step 13: Design a prediction task for small-sample learning on the test query set under limited supervision conditions;

[0028] The test process of designing small sample learning is to use the test support set S test A small number of samples in the training task T are used as guidance information, and train The prior parameters obtained in Implement the test query set Q test Accurate prediction of samples in :

[0029]

[0030] Furthermore, step 2 specifically includes the following steps:

[0031] Step 21: Design a single-stage optimization learning strategy;

[0032] The single-stage optimization learning strategy of small samples directly uses the data in the training task for gradient descent, and its model parameter update process is expressed as:

[0033]

[0034] In the formula, Represents the use of a specific training task T train The trained model parameters obtained by optimizing the samples in the training set;

[0035] Step 22: Design a two-stage optimization learning strategy;

[0036] The two-stage optimization learning strategy for small samples first uses the support set data of a specific task to optimize the model parameters as the first stage, and then uses it as the initial parameters for subsequent query set optimization as the second stage. The optimization process is expressed as:

[0037]

[0038] In the formula, StageI and StageII represent the first and second stages of the two-stage optimization process of small sample learning, respectively. For a specific training task T train Using its support set data S train Optimize the trained model parameters; Indicates the use of the corresponding query set data Q train Optimize the fine-tuned model parameters;

[0039] Step 2 and 3: Select an appropriate single-stage or two-stage optimization learning strategy based on the amount of existing data, the nature of the target task, and the computing resource requirements, and convert the model parameters obtained through single-stage or two-stage optimization into As the prior parameter in formula (3) Used for subsequent small sample prediction tasks.

[0040] Furthermore, the step three specifically includes the following steps:

[0041] Step 31: Define four classification paradigms for small sample learning;

[0042] Step 32: Establish a mathematical framework for the small sample learning paradigm based on metric learning;

[0043] Step 33: Establish a mathematical framework for small sample learning paradigm based on optimization algorithm;

[0044] Step 3 and 4: Establish a mathematical framework for small sample learning paradigm based on transfer learning;

[0045] Step 35: Establish a mathematical framework for the small sample learning paradigm based on generative models.

[0046] Furthermore, the small sample learning paradigm based on metric learning adopts a two-stage strategy for model optimization, using the support loss L based on similarity measurement. S Optimize feature space representation for query set sample prediction and query loss L Q The mathematical framework of the small sample learning paradigm based on metric learning is:

[0047]

[0048] In the formula, · represents the similarity metric function of the feature space, Y represents the one-hot encoded label value, λ represents the hyperparameter to ensure that the loss is positive, α and β represent the learning rates of the support set and the query set, and t represents the number of iterations;

[0049] The small sample learning paradigm based on optimization algorithm adopts a two-stage optimization paradigm to achieve rapid adaptation to multiple tasks. Its mathematical framework is:

[0050]

[0051] Furthermore, the small sample learning paradigm based on transfer learning adopts a single-stage optimization strategy to directly learn the supervision information in the training task. Its mathematical framework is:

[0052]

[0053] Furthermore, the mathematical framework of the small sample learning paradigm based on the generative model is:

[0054]

[0055] In the formula, G represents the pre-trained generative model, which is used to generate more support set samples and their labels.

[0056] Furthermore, the step 4 specifically includes the following steps:

[0057] Step 41: Establish a data-driven machine learning error analysis process;

[0058] Step 42: Establish an error decomposition method for machine learning;

[0059] Step 43: Establish a small sample learning error control method under limited supervision.

[0060] The present invention proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the unified paradigm, task construction, model optimization and error control method of small sample learning are implemented.

[0061] The present invention proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the unified paradigm, task construction, model optimization and error control method of small sample learning.

[0062] The unified paradigm, task construction, model optimization and error control method for small sample learning proposed in the present invention have the following advantages:

[0063] 1. A unified framework for small sample learning from data partitioning to task construction and then to optimization strategy is established, achieving a complete expression of the entire small sample learning process;

[0064] 2. Based on different utilization modes of limited supervision information and intrinsic learning mechanisms, four types of learning paradigms for small sample learning are established, achieving a high degree of induction of different small sample learning methods;

[0065] 3. Incorporate the small sample learning process into a unified mathematical framework, and realize the mathematical expression of four types of small sample learning paradigms;

[0066] 4. The error control theory of small sample learning under limited supervision was established. Based on the machine learning error decomposition, the control principles of the four types of small sample learning paradigms on optimization error, generalization error and approximation error were explained. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0068] Figure 1It is an overall flow chart of the unified paradigm, task construction, model optimization and error control methods for small sample learning.

[0069] Figure 2 It is a schematic diagram of the four-category paradigm of small sample learning based on optimization process classification.

[0070] Figure 3 Schematic diagram of the small sample learning paradigm based on metric learning.

[0071] Figure 4 Schematic diagram of the small sample learning paradigm based on optimization algorithms.

[0072] Figure 5 This is a schematic diagram of the small sample learning paradigm based on transfer learning.

[0073] Figure 6 Schematic diagram of the small sample learning paradigm based on generative models.

[0074] Figure 7 It is a schematic diagram of the approximation error, generalization error, and optimization error of a general machine learning model driven by data.

[0075] Figure 8 It is a schematic diagram of the control principle of small sample learning on approximation error, generalization error and optimization error under limited supervision conditions. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] This paper proposes a unified paradigm, task construction, model optimization and error control method for small sample learning, and establishes a small sample learning theory for limited supervised information, that is, obtaining universal prior information through learning cross-category data, and transferring it to small sample target tasks to achieve recognition and prediction. The basic idea is explained as follows:

[0078] (1) Randomly select samples of different categories from the training set to form a support set and a query set, which constitute different training tasks; for each task, build a deep network model (which can be an image classification, object detection, or semantic segmentation model depending on the recognition target), use the data and labels of the support set for supervised learning, and then adjust the parameters of the trained model using the data and labels of the query set;

[0079] (2) According to the form of training tasks, sampling is performed in the test set to form a test task including a support set and a query set, where the support set is labeled samples and the query set is unlabeled samples to be predicted; the model obtained after learning on all training tasks can be transferred to the test task to achieve prediction of the query set of the test task.

[0080] Specifically, the present invention proposes a unified paradigm, task construction, model optimization and error control method for small sample learning, and the method includes the following steps:

[0081] Step 1: Data division and task construction for small sample learning;

[0082] The training process of small sample learning is task-based. The strategy of sampling with replacement is used to construct training tasks and test tasks in the training data set and query data set. Each task consists of two mutually exclusive sets, the support set and the query set.

[0083] Step 2: Design an optimization strategy for small sample learning;

[0084] The typical optimization strategies used by small sample learning models when learning prior knowledge are divided into two modes, namely single-stage optimization learning and double-stage optimization learning, and are defined respectively.

[0085] Step 3: Establish a classification paradigm and mathematical framework for small sample learning;

[0086] According to the differences in the intrinsic learning mechanisms and implementation principles of small sample learning methods for limited supervised information, based on different processes of model optimization, a mathematical framework for small sample learning is established, and four types of small sample learning paradigms are formed and defined, namely, the small sample learning paradigm based on metric learning, the small sample learning paradigm based on optimization algorithms, the small sample learning paradigm based on transfer learning, and the small sample learning paradigm based on generative models.

[0087] Step 4: Establish an error control method for small sample learning;

[0088] According to the four types of optimization paradigms of small sample learning and based on the data-driven machine learning error analysis theory, error control of three typical errors in machine learning in small sample scenarios, namely optimization error, generalization error and approximation error, is achieved.

[0089] The overall flow chart of the unified paradigm, task construction, model optimization and error control method of small sample learning proposed in this invention is as follows: Figure 1 shown.

[0090] The step 1 performs data division and task construction for small sample learning, specifically including the following steps:

[0091] Step 1: Design a model training and testing process based on supervised learning rules;

[0092] Let a deep network model be f, whose optimizable parameter is θ, in a labeled training set D train The model is trained on the dataset D and the trained model is used to test an unlabeled test set D test Make predictions; the optimization goal of the supervised learning model is to train The model parameters θ are optimized and the optimized parameters containing prior knowledge are used For the test set D test The data in is predicted, and the process of training optimization and test prediction is expressed as:

[0093]

[0094] In the formula, Train and Test represent the training optimization and test prediction processes respectively, I i and Y i They represent the i-th input sample and its corresponding label in the training set, L represents the loss function selected for model optimization according to the target task, and I j and They represent the jth sample to be predicted in the test set and its prediction result respectively;

[0095] Step 1 and 2: Design the data partitioning method and training and testing tasks for small sample learning under limited supervision conditions;

[0096] For small sample learning scenarios, the training set D train The labeled samples in the test set D test The unlabeled samples in are composed of data of multiple categories, and the categories included are represented as C train and C test The basic learning unit of small-sample learning is the N-way-K-shot task, which means that each learning task includes N categories and K samples per category. train and the test set D test In the training task and test task, the training task T is formed by sampling with replacement. train and test task T test Both are composed of a support set S and a query set Q, where S is set to N-way-K-shot and Q is set to N-way-1-shot; T train and T test The sample categories contained in S can be different, and the number of samples in S is much smaller than that in the training set D train The number of samples, the training task and test task construction process of small sample learning is expressed as:

[0097]

[0098] Step 13: Design a prediction task for small-sample learning on the test query set under limited supervision conditions;

[0099] The goal of small sample learning is to use the training set D train Train a general model that contains prior knowledge of multiple categories, which is test There are multiple sample categories C involved in test With the ability to adapt quickly, the test process of designing small sample learning is to use the test support set S test A small number of samples in the training task T are used as guidance information, and train The prior parameters obtained in Implement the test query set Q test Accurate prediction of samples in :

[0100]

[0101] Step 2: Design an optimization strategy for small sample learning, which includes the following steps:

[0102] Step 21: Design a single-stage optimization learning strategy;

[0103] Similar to the optimization mechanism of traditional supervised learning, the single-stage optimization learning strategy of small samples directly uses the data in the training task for gradient descent, and its model parameter update process is expressed as:

[0104]

[0105] In the formula, Represents the use of a specific training task T train The trained model parameters obtained by optimizing the samples in the training set;

[0106] Step 22: Design a two-stage optimization learning strategy;

[0107] The two-stage optimization learning strategy for small samples first uses the support set data of a specific task to optimize the model parameters as the first stage, and then uses it as the initial parameters for subsequent query set optimization as the second stage. The optimization process is expressed as:

[0108]

[0109] In the formula, StageI and StageII represent the first and second stages of the two-stage optimization process of small sample learning, respectively. For a specific training task T train Using its support set data S train Optimize the trained model parameters; Indicates the use of the corresponding query set data Q train Optimize the fine-tuned model parameters;

[0110] Step 2 and 3: Select an appropriate single-stage or two-stage optimization learning strategy based on the amount of existing data, the nature of the target task, and the computing resource requirements, and convert the model parameters obtained through single-stage or two-stage optimization into As the prior parameter in formula (3) Used for subsequent small sample prediction tasks.

[0111] The step 3 establishes a classification paradigm and mathematical framework for small sample learning, which specifically includes the following steps:

[0112] Step 31: Define four classification paradigms for small sample learning;

[0113] Few-shot learning learns a robust and generalized model representation from a variety of training tasks, and by capturing the basic features of multiple categories, allows for reasonable model generalization of unknown categories with limited supervised samples. According to the intrinsic learning mechanism of supervised information and the extraction mode of prior knowledge, the few-shot learning methods are divided into four categories: the few-shot learning paradigm based on metric learning, the few-shot learning paradigm based on optimization algorithms, the few-shot learning paradigm based on transfer learning, and the few-shot learning paradigm based on generative models. The schematic diagram is shown in the figure. Figure 2 shown.

[0114] Step 32: Establish a mathematical framework for the small sample learning paradigm based on metric learning;

[0115] The few-shot learning paradigm based on metric learning captures the underlying data structure embedded in the feature space, projects the deep features into the feature space, and applies appropriate similarity metric functions in the metric space to predict unlabeled samples, such as Figure 3 As shown, the red mark represents that the optimization object of small sample learning under this paradigm is the consistency of the metric space.

[0116] The small sample learning paradigm based on metric learning adopts a two-stage strategy for model optimization, using the support loss L based on similarity measurement. S Optimize feature space representation for query set sample prediction and query loss L Q The mathematical framework of the small sample learning paradigm based on metric learning is:

[0117]

[0118] where · represents the similarity metric function of the feature space, Y represents the one-hot encoded label value, λ represents the hyperparameter to ensure that the loss is positive, α and β represent the learning rates of the support set and query set, and t represents the number of iterations.

[0119] Step 33: Establish a mathematical framework for small sample learning paradigm based on optimization algorithm;

[0120] The small sample learning paradigm based on optimization algorithms makes better use of limited supervision information by designing new optimization algorithms, and uses the prior knowledge of pre-trained models to predict unknown samples in target data, such as Figure 4 As shown, the red mark represents that the optimization object of small sample learning under this paradigm is the accuracy of the designed optimization algorithm.

[0121] The small sample learning paradigm based on optimization algorithm adopts a two-stage optimization paradigm to achieve rapid adaptation to multiple tasks. Its mathematical framework is:

[0122]

[0123] Step 3 and 4: Establish a mathematical framework for small sample learning paradigm based on transfer learning;

[0124] The small sample learning paradigm based on transfer learning focuses on the learning process of prior knowledge in the pre-trained model and transfers the integrated multi-task information to the target task to achieve predictive reasoning, such as Figure 5 As shown, the red mark represents the transferability of the target model for small sample learning under this paradigm.

[0125] The small sample learning paradigm based on transfer learning adopts a single-stage optimization strategy to directly learn the supervision information in the training task. Its mathematical framework is:

[0126]

[0127] Step 35: Establish a mathematical framework for the small sample learning paradigm based on generative models.

[0128] The small sample learning paradigm based on generative models simulates the probability distribution of actual data through the prior knowledge in the generative model, and samples and synthesizes new samples in the simulated distribution, thereby enriching the data space and improving learning ability. Figure 6 As shown, the red mark represents that the optimization object of small sample learning under this paradigm is the data generator aimed at generating more realistic samples.

[0129] The mathematical framework of the few-shot learning paradigm based on generative models is:

[0130]

[0131] In the formula, G represents the pre-trained generative model, which is used to generate more support set samples and their labels.

[0132] The fourth step is to establish an error control method for small sample learning, which specifically includes the following steps:

[0133] Step 41: Establish a data-driven machine learning error analysis process;

[0134] By modeling and analyzing the machine learning errors driven by data, machine learning errors are divided into three typical errors: approximation error, generalization error, and optimization error. The final mathematical error of the machine learning network model inference refers to the difference between the objective function f and the numerical solution that can be obtained. The distance between them, the error analysis diagram of the data-driven machine learning model is as follows Figure 7 shown.

[0135] (1) First, the specific optimization algorithm used in the model learning process is used for gradient descent, and the final numerical solution is Difficult to find the global optimal solution on a known dataset Correspondingly, this leads to optimization errors in machine learning.

[0136] (2) Secondly, in actual scenarios, the complete data distribution is usually unknown, and the model training can only be based on a limited sample data set sampled from the approximate distribution of the data. Therefore, the expected error of the network model for the complete data domain cannot be calculated and optimized, and the global optimal solution of the hypothesis function φ for the complete data domain cannot be obtained. This causes generalization error in machine learning.

[0137] (3) Again, in the actual operation of the machine learning model, a specific model structure needs to be used to represent the hypothesis function φ, which inevitably leads to the approximation error in machine learning between the hypothesis function φ and the expected target function f.

[0138] Step 42: Establish an error decomposition method for machine learning;

[0139] For the complete data domain And its data distribution The objective function f on the machine learning model is to use a data set containing finite samples. To learn the hypothesis function φ, so that it keeps approaching the target function f. Therefore, in order to enable the machine learning model to infer and predict unknown data, it should be given a known data set. Minimize the empirical error Based on the above error analysis, the assumed network model φ is The numerical solution obtained on The relationship between the objective function f(x) and the complete data domain The inference error can be broken down into:

[0140]

[0141] Where Approximationerror represents the approximation error, Optimizationerror represents the optimization error, and Generalizationerror represents the generalization error.

[0142] Step 43: Establish a small sample learning error control method under limited supervision.

[0143] The four types of paradigms of small-sample learning focus on dealing with the above-mentioned approximation error, generalization error, and optimization error, respectively, corresponding to corresponding solutions from different error control perspectives. The error control methods are specifically described as follows. Figure 8 Schematic diagram of the control principle of approximation error, generalization error and optimization error in small sample learning under limited supervision conditions.

[0144] (1) The small sample learning paradigm based on metric learning reduces the approximation error.

[0145] The small-shot learning paradigm based on metric learning transforms the direct reasoning process of the hypothesis function φ(x):x→f(x) into two mapping reasoning processes φ(x):x→C(x)→f(x) that introduce the embedded feature space, where C represents the implicit expression of the latent feature space. The small-shot learning paradigm based on metric learning reduces the approximation error between the hypothesis function φ and the target function f by strengthening the supervised learning of the embedded features.

[0146] (2) The small sample learning paradigm based on the optimization algorithm reduces the optimization error.

[0147] The small sample learning paradigm based on the optimization algorithm adopts a special learning machine optimization strategy, which directly transforms the optimization target of the traditional machine learning model parameters into the optimization of the set learning machine model, making the fine-tuning parameters of the network model on specific tasks Closer to the global optimal solution This reduces the optimization error of the model.

[0148] (3) The small sample learning paradigm based on transfer learning reduces generalization error.

[0149] The small sample learning paradigm based on transfer learning utilizes the prior knowledge extracted from large-scale datasets and quickly transfers it to specific targets through parameter fine-tuning, i.e. This reduces the generalization error of the model.

[0150] (4) The small sample learning paradigm based on generative model reduces generalization error.

[0151] Different from the above methods, the small sample learning paradigm based on generative model focuses on data distribution and fits the complete data domain through pre-trained generative model. distribution, and sample more data in the fitting distribution to increase the supervision information of the training process, so that the solution value of the hypothesis function Close to the global optimal solution This reduces the generalization error.

[0152] The present invention proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the unified paradigm, task construction, model optimization and error control method of small sample learning are implemented.

[0153] The present invention proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the unified paradigm, task construction, model optimization and error control method of small sample learning.

[0154] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0155] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0156] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0157] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0158] The unified paradigm, task construction, model optimization and error control method of small sample learning proposed in the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A unified paradigm, task construction, model optimization, and error control method for small sample learning, characterized by: The method comprises the following steps: Step 1: Image data division and task construction for small sample learning; The training process of small sample learning is task-based. The strategy of sampling with replacement is used to construct training tasks and test tasks in the training image dataset and query image dataset. Each task consists of two mutually exclusive sets, the support set and the query set. Step 2: Design an optimization strategy for small sample learning; The typical optimization strategies used by small sample learning models when learning prior knowledge are divided into two modes, namely single-stage optimization learning and double-stage optimization learning, and are defined respectively. Step 3: Establish a classification paradigm and mathematical framework for small sample learning; According to the differences in the intrinsic learning mechanisms and implementation principles of small sample learning methods for limited supervised information, based on different processes of model optimization, a mathematical framework for small sample learning is established, and four types of small sample learning paradigms are formed and defined, namely, the small sample learning paradigm based on metric learning, the small sample learning paradigm based on optimization algorithms, the small sample learning paradigm based on transfer learning, and the small sample learning paradigm based on generative models. Step 4: Establish an error control method for small sample learning; According to the four types of optimization paradigms of small sample learning and based on the data-driven machine learning error analysis theory, error control of three typical errors in machine learning in small sample scenarios, namely optimization error, generalization error and approximation error, is achieved.

2. The method according to claim 1, characterized in that The step 1 specifically includes the following steps: Step 1: Design a model training and testing process based on supervised learning rules; Let a deep network model be f, the deep network model is an image classification model, and its optimizable model parameter is θ. In a labeled training set D train The model is trained on the dataset D and the trained model is used to test an unlabeled test set D test Make predictions; the optimization goal of the supervised learning model is to train The model parameters θ are optimized and the optimized prior parameters containing prior knowledge are used For the test set D test The data in is predicted, and the process of training optimization and test prediction is expressed as: In the formula, Train and Test represent the training optimization and test prediction processes respectively, I i and Y i They represent the i-th input sample and its corresponding label in the training set, L represents the loss function selected for model optimization according to the target task, and I j and They represent the jth sample to be predicted in the test set and its prediction result respectively; Step 1 and 2: Design the data partitioning method, training tasks, and testing tasks for small sample learning under limited supervision conditions; For small sample learning scenarios, the training set D train The labeled samples in the test set D test The unlabeled samples in are composed of data of multiple categories, and the categories included are represented as C train and C test The basic learning unit of small-sample learning is the N-way-K-shot task, which means that each learning task includes N categories and K samples per category. train and the test set D test In the training task and test task, the training task T is formed by sampling with replacement. train and test task T test They are composed of a support set S and a query set Q, where S is set to N-way-K-shot and Q is set to N-way-1-shot. The construction process of training and testing tasks for small sample learning is expressed as: Step 13: Design a prediction task for small-sample learning on the test query set under limited supervision conditions; The test process of designing small sample learning is to use the test support set S test A small number of samples in the training task T are used as guidance information, and train The prior parameters obtained in Implement the test query set Q test Accurate prediction of samples in :

3. The method according to claim 2, characterized in that Step 2 specifically includes the following steps: Step 21: Design a single-stage optimization learning strategy; The single-stage optimization learning strategy of small samples directly uses the data in the training task for gradient descent, and its model parameter update process is expressed as: In the formula, Represents the use of a specific training task T train The trained model parameters obtained by optimizing the samples in the training set; Step 22: Design a two-stage optimization learning strategy; The two-stage optimization learning strategy for small samples first uses the support set data of a specific task to optimize the model parameters as the first stage, and then uses it as the initial parameters for subsequent query set optimization as the second stage. The optimization process is expressed as: In the formula, Stage I and Stage II represent the first and second stages of the two-stage optimization process of small sample learning, respectively. For a specific training task T train Using its support set data S train Optimize the trained model parameters; Indicates the use of the corresponding query set data Q train Optimize the fine-tuned model parameters; Step 2 and 3: Select an appropriate single-stage or two-stage optimization learning strategy based on the amount of existing image data, the nature of the target task, and the computing resource requirements, and convert the model parameters obtained through single-stage or two-stage optimization into As the prior parameter in formula (3) Used for subsequent small sample prediction tasks.

4. The method according to claim 3, characterized in that The step three specifically includes the following steps: Step 31: Define four classification paradigms for small sample learning; Step 32: Establish a mathematical framework for the small sample learning paradigm based on metric learning; Step 33: Establish a mathematical framework for small sample learning paradigm based on optimization algorithm; Step 3 and 4: Establish a mathematical framework for small sample learning paradigm based on transfer learning; Step 35: Establish a mathematical framework for the small sample learning paradigm based on generative models.

5. The method according to claim 4, characterized in that The small sample learning paradigm based on metric learning adopts a two-stage strategy for model optimization, using the support loss L based on similarity measurement. S Optimize feature space representation for query set sample prediction and query loss L Q The mathematical framework of the small sample learning paradigm based on metric learning is: In the formula, ||·|| represents the similarity measurement function of the feature space, Y j , Y k Respectively represent the corresponding labels of the j-th and k-th input samples, λ represents the hyperparameter that ensures the loss is positive, α and β represent the learning rates of the support set and query set, and t represents the number of iterations; The small sample learning paradigm based on optimization algorithm adopts a two-stage optimization paradigm to achieve rapid adaptation to multiple tasks. Its mathematical framework is:

6. The method according to claim 5, characterized in that The small sample learning paradigm based on transfer learning adopts a single-stage optimization strategy to directly learn the supervision information in the training task. Its mathematical framework is:

7. The method according to claim 6, characterized in that The mathematical framework of the few-shot learning paradigm based on generative models is: In the formula, G represents the pre-trained generative model, which is used to generate more support set samples and their labels.

8. The method according to claim 7, characterized in that The step 4 specifically includes the following steps: Step 41: Establish a data-driven machine learning error analysis process; Step 42: Establish an error decomposition method for machine learning; Step 43: Establish a small sample learning error control method under limited supervision.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images