Method and apparatus for training a model for image classification

Abstract

A method and apparatus for training a model for image classification are provided. The method for training the model for image classification includes: receiving a first set of image samples representing a base class for a base task; training a model based on the first set of image samples to obtain base classification weights for the base class of the base task from the trained model; sequentially receiving a plurality of new tasks; and upon receiving any one of the plurality of new tasks: receiving a second set of image samples representing a new class for the new task; training a weight generator based on the base classification weights, one or more other new classification weights for new classes of one or more previously received new tasks among the plurality of new tasks, and one or more from the first and second set of image samples to obtain new classification weights for the new class of the new task; and updating the model using the new classification weights for the new class of the new task.

Description

[0001] This application is based on and claims priority to U.S. Provisional Patent Application No. 63 / 071,067, filed with the United States Patent and Trademark Office (USPTO) on August 27, 2020, and U.S. Non-Provisional Patent Application No. 17 / 156,126, filed with the United States Patent and Trademark Office (USPTO) on January 22, 2021, the contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure generally relates to machine learning methods, and more specifically, to methods and apparatus for training models for image classification. Background Technology

[0003] In the field of machine learning, accumulating enough data to improve model accuracy can be challenging. In scenarios with limited data, few-shot learning algorithms have been employed to discover patterns in the data and make inferences. This technique is commonly used in computer vision, particularly for classifying or categorizing photographs.

[0004] In few-shot learning tasks (where N is the number of classes and K is the number of samples (or images) in each class), a small training set D is provided. The size of the training set is |D| = N·K.

[0005] Transferable knowledge for improved few-shot learning can be learned using a basic training set D0. The basic training set D0 contains a large number of labeled samples from a large number of classes. However, the classes in the basic training set D0 are different from the classes in the training set D. Therefore, traditional few-shot learning utilizes a small amount of training data or samples without using the basic classes to train the model.

[0006] An episode represents a training and test pair for a few-shot learning task. Figure 1 This diagram illustrates an episodic few-shot learning method. The first training task 102, the second training task 104, and the first test task 106 each include corresponding support sets 108, 110, and 112 with three classes (N=3) and two samples (images) per class (K=2). The first training task 102, the second training task 104, and the first test task 106 also include corresponding query sets 114, 116, and 118 with three samples (images). The classes in each of the first training task, the second training task, and the first test task are different.

[0007] Both metric-based and gradient-based training algorithms are developed on top of fragmented learning frameworks. For example, self-supervised loss can be added to the feature extractor training process to achieve robust semantic feature learning and improve few-shot classification. Furthermore, Wasserstein-based methods can be added to better align the distribution of features with the distribution of the classes considered. However, as mentioned above, traditional few-shot learning does not consider the base classes used in training.

[0008] Few-shot learning that does not forget the base class has been developed to classify a new class when only a small number of labeled samples are provided for the new class, while also preserving the ability of the feature embedding network to classify the base class on which it is trained. For example, the feature embedding network and the classification weights for the base class are pre-trained via regular supervised learning and are then fixed.

[0009] Figure 2 This diagram illustrates few-shot learning focused on generating classification weights for new classes without forgetting the base classes. Sample or test images 202 are provided to feature extractor 204, which outputs the features of the samples to classifier 206. Classifier 206 obtains base classification weights (or, classification weights for the base classes) 208 from training data 210 of the base classes. Few-shot classification weight generator 212 generates new classification weights (or, classification weights for the new classes) 214 for the limited training data 216 of the new classes and provides the new classification weights 214 to classifier 206.

[0010] More specifically, for the few-shot classification weight generator 212, the weight imprinting method computes prototypes of the new class based on a pre-trained feature embedding network and uses them as classification weights for the new class. Furthermore, by using the new class prototypes and the classification weights 208 for the base class as input to the weight generator, an attention-based mechanism is employed in the generation of the new classification weights 214 to learn the generation of the classification weights 214 for the new class, leveraging the relationship between the base class and the new class.

[0011] Based on the base classification weight 208 and the new classification weight 214, the classifier outputs the probabilities of the base class and the new class for sample 202.

[0012] Furthermore, new classification weights can be trained using the cross-entropy loss of some labeled samples from the new class through gradient-based optimization until they converge. Since the loss for training the new classification weights is computed only using samples from the new class, a forgetting problem with respect to the base class can occur. To prevent this, an attention-based regularization method is applied. The regularization loss is provided by an attention attractor network. The attention attractor network generates attractor vectors using the base class weights, and the regularization loss is calculated based on the Mahalanobis distance between the new classification weights and the attractor vectors. Summary of the Invention

[0013] According to one embodiment, a method for continuous few-shot learning is provided. A model for a base task is generated using base classification weights for a base class of the base task. A series of new tasks are received sequentially. Upon receiving each new task in the series, the model is updated by a weight generator using new classification weights for the new class of the corresponding new task. The new classification weights are generated based on one or more of the following: base classification weights and one or more other new classification weights for the new classes of the one or more other new tasks in the series that were previously received. Furthermore, for each new task, a first sample set of the corresponding new task is classified into a new class using the updated model.

[0014] According to one embodiment, a UE is provided, the UE including a processor and a non-transitory computer-readable storage medium storing instructions. When executed, the instructions cause the processor to: generate a model for a base task using base classification weights of a base class for the base task, and sequentially receive a series of new tasks. The instructions further cause the processor to: update the model using new classification weights of a new class for the corresponding new task upon receiving each new task in the series. The new classification weights are generated by a weight generator based on one or more of the following: base classification weights and one or more other new classification weights for new classes of the one or more other new tasks received previously in the series. The instructions further cause the processor to: classify a first sample set of the corresponding new task into a new class using the updated model upon receiving each new task. Attached Figure Description

[0015] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become clearer from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 This is a diagram illustrating fragmented few-shot learning;

[0017] Figure 2 This is a diagram illustrating few-shot learning without forgetting the base class;

[0018] Figure 3 This is a diagram illustrating an example of continuous few-shot learning in three stages according to an embodiment;

[0019] Figure 4 This is a flowchart illustrating a method for continuous few-shot learning according to an embodiment; and

[0020] Figure 5 This is a block diagram of an electronic device in a network environment according to an embodiment. Detailed Implementation

[0021] In the following description, embodiments of the present disclosure are described in detail with reference to the accompanying drawings. It should be noted that the same elements will be designated by the same reference numerals, although they are shown in different drawings. In the following description, specific details such as detailed configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of the present disclosure. Therefore, it will be apparent to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Furthermore, for clarity and brevity, descriptions of well-known functions and constructions have been omitted. The terminology described below is defined in consideration of the functions in this disclosure and may vary depending on the user, the user's intent, or habit. Therefore, the definitions of the terms should be determined based on the content throughout this specification.

[0022] This disclosure can have various modifications and embodiments, which are described in detail below with reference to the accompanying drawings. However, it should be understood that this disclosure is not limited to the embodiments, but includes all modifications, equivalents, and substitutions within the scope of this disclosure.

[0023] Although terms including ordinal numbers such as first, second, etc., may be used to describe various elements, structural elements are not limited by such terms. The term is used only to distinguish one element from another. For example, a first structural element may be referred to as a second structural element without departing from the scope of this disclosure. Similarly, a second structural element may also be referred to as a first structural element. As used herein, the term "and / or" includes any and all combinations of one or more related items.

[0024] The terminology used herein is merely for describing various embodiments of this disclosure and is not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In this disclosure, it should be understood that the terms “comprising” or “having” indicate the presence of features, quantities, steps, operations, structural elements, components, or combinations thereof, and do not preclude the possibility of the presence of one or more other features, quantities, steps, operations, structural elements, components, or combinations thereof, or the addition of one or more other features, quantities, steps, operations, structural elements, components, or combinations thereof.

[0025] Unless otherwise defined, all terms used herein shall have the same meaning as understood by one of skill in the art to which this disclosure pertains. Unless clearly defined in this disclosure, terms (such as those defined in a general dictionary) shall be interpreted as having the same meaning as in the context of the relevant technical field and shall not be interpreted as having an ideal or overly formal meaning.

[0026] The electronic device according to one embodiment can be one of various types of electronic devices. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance. According to one disclosed embodiment, the electronic device is not limited to those described above.

[0027] The terminology used in this disclosure is not intended to limit the disclosure, but rather to include various modifications, equivalents, or substitutions of corresponding embodiments. Regarding the description of the drawings, similar reference numerals may be used to refer to similar or related elements. Unless the relevant context clearly indicates otherwise, the singular form of the noun corresponding to an item may include one or more things. As used herein, each of the phrases such as “A or B,” “at least one of A and B,” “A, B, or C,” and “at least one of A, B, and C” may include all possible combinations of the items listed together in the corresponding one of the phrases. As used herein, terms such as “first,” “second,” “first,” and “second” may be used to distinguish a corresponding component from another component, but are not intended to limit the components in other respects (e.g., importance or order). The intention is that if an element (e.g., a first element) is referred to as being "combined" with, "combined to" another element (e.g., a second element), "connected" to, or "attached to" another element (e.g., a second element), whether or not the terms "operably" or "communically" are used, it indicates that the element can be combined with the other element directly (e.g., wired), wirelessly, or via a third element.

[0028] As used herein, the term "module" may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with other terms such as "logic," "logic block," "component," and "circuit." A module may be a single integrated component or its smallest unit or component adapted to perform one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0029] The disclosed embodiments provide a framework for continuous few-shot learning. Such a framework can be used in many applications to progressively learn new tasks (e.g., new classes, new objects, etc.) without forgetting previously learned old tasks.

[0030] The model for the base task (with a base class) is pre-trained. For example, the model can be trained based on a first set of image samples from the base class of the base task. Then, the model for the new task (with a new class) is updated. For the new task, as in traditional few-shot learning, it is assumed that only a few samples (e.g., 1 or 5 samples) are provided for each new class. The base task T0 is provided given a base training set D0. Given a training set D... i Given i = 1, 2, ..., a series of N-way K-shot few-shot learning tasks T are learned sequentially. i .

[0031] The number of few-shot learning tasks is uncertain. The training set D i From class C i The set of C such that C i ∩C j =φ,

[0032] For each task T i Using only the current training set D i (For example, task T) i The model is updated using a second set of image samples for the new class. Data from past tasks cannot be revisited. However, in tests, data from all previous classes (i.e., C0∪C1∪…∪C) are updated. i Evaluate the trained model.

[0033] Because the model remembers all the classes it has learned as training data gradually arrives, continuous few-shot learning has more practical applications than traditional few-shot learning.

[0034] The following provides a more detailed description of the continuous few-shot learning steps described above.

[0035] First, feature extractor F θand a set of classification weight vectors It is pre-trained. For example, assuming linear classification after feature extraction, the inferred (or test) output y for input x is given by the following equation (1):

[0036]

[0037] In learning task T i Then, the inference is performed as described in equation (2) below:

[0038]

[0039] Given W0, W1, ..., W i-1 In this case, all classes learned are of interest, and it needs to be tailored to task T. i New class C i A set of classification weight vectors

[0040] In order to obtain the new class C i A set of classification weight vectors W i The weight generator utilizes a small training set D i and classification weights for previously learned classes (i.e., W0, W1, ..., W...). i-1 Specifically, where g φ This represents a weight generator, and uses a few-sample training set D. i The feature extractor output, W, is explained in equation (3) below. i :

[0041] W i =g φ (F θ (D i ), W0, W1, ..., W i-1 (3)

[0042] Therefore, in continuous few-shot learning, as in few-shot learning without forgetting the base class, as a series of few-shot tasks are gradually learned, the weight generator uses all the previously learned classification weights as its input (instead of just using the base class weights) to produce classification weights for the new class.

[0043] As the number of few-shot tasks learned increases, the number of classification weights provided to the aforementioned weight generator also increases. Therefore, two methods for training the weight generator are presented.

[0044] In the first approach, the weight generator is optimized for a random number of new base classes. First, a "fake" few-shot learning task is constructed from D0. Specifically, new base classes are randomly selected. The quantity (i.e., ). This can be the minimum number of classes. A "new" base class is randomly selected from the base class C0. And the new category of "fake" (Right now, ).against K samples are randomly selected from each class, and the K samples are placed into K samples are fed into the weight generator. For samples from... and Some samples are randomly selected from each class and placed into... and in, It is a new training set corresponding to D0. This is a new training set corresponding to D1. Randomly selected samples are used to calculate the cross-entropy.

[0045] Optimization is performed according to the following equation (4):

[0046]

[0047] in, It corresponds to W0 The classification weight vector, CrossEntropy can be used to calculate the cross-entropy function, softmax can be used to calculate the maximum flexibility, and E can be used to calculate the expected value.

[0048] In another embodiment, multiple weight generators are trained for a random number of "new" base classes. Assuming the number of base classes falls within a fixed, finite range (e.g., 50 to 100), a separate weight generator is trained for each random number of base classes. The weights of the multiple weight generators are averaged to obtain a fused weight generator.

[0049] When the number of few-shot learning tasks added after the base model is arbitrary and not predetermined, it becomes difficult to optimize the weight generator for an arbitrary number of few-shot learning tasks. In a second approach for training the weight generator, the number of few-shot learning tasks is limited (e.g., three), and the weight generator is trained to minimize the classification error on a fixed number of few-shot learning tasks.

[0050] Figure 3This is a diagram illustrating an example of continuous few-shot learning in three stages according to an embodiment. In the first stage, the model is trained against a base class. Using base class weights (or, classification weights against the base class) 302 and a first set of new class samples 304, a weight generator 306 generates a first set of new classification weights 308 for a first few-shot task. In the second stage, the base class classification weights 302 and the generated first set of classification weights 308 are combined with a second set of new class samples 310 and used by the weight generator 306 to generate a second set of classification weights 312 for a second few-shot learning task. In the third stage, the base class classification weights 302, the generated first set of classification weights 308, and the generated second set of classification weights 312 are combined with a third set of new class samples 314 and used by the weight generator 306 to generate a third set of classification weights 316 for a third few-shot learning task.

[0051] This continuous few-shot learning is performed a fixed number of times. In each stage, the loss used to train the weight generator is defined as the cross-entropy loss between the base class and the newly learned class. The average classification loss for all stages is determined, and the weight generator is optimized to minimize this average classification loss.

[0052] Specifically, regarding the optimization of the weight generator for a fixed number of few-shot learning tasks, we first construct k "fake" few-shot learning tasks from D0. Specifically, as illustrated in equation (5) below, a "new" base class is randomly selected from the base class C0. And group k "fake" new class

[0053]

[0054] For 1≤i≤k, for K samples are randomly selected from each class, and the K samples are placed into K randomly selected samples are fed into the weight generator. For 0 ≤ i ≤ k, some samples for each class are generated from... They were randomly selected and placed into These randomly selected samples are used to calculate the cross-entropy.

[0055] Optimization is performed according to the following equation (6):

[0056]

[0057] Among them, W i =g φ (S i W0, W1, ..., W i-1 )and It corresponds to W0 The classification weight vector.

[0058] The first architecture of the weight generator includes a dual-attention weight generator. Set to D i The data of class c in [the context]. When... Then, the classification weights w for class c generated by the dual attention weight generator are described in equation (7) below. c :

[0059]

[0060] Among them, W Q W K and W V It is a linear transformation weight of the query, keyword, and value in the dual attention module, and mean can be represented as a function to calculate the average value.

[0061] The second architecture of the weight generator includes a self-attention weight generator, where, for c1, c N ∈C i , The classification weights generated by the self-attention weight generator are illustrated in equation (8) below:

[0062] W all ←W V W all Att(W Q W all W K W alI ) T (8)

[0063] The difference between a self-attention weight generator and a dual-attention weight generator is that the base class weights are updated in a self-attention weight generator.

[0064] The proposed weight generator modification can be applied to attention-attracting subnetworks, which have outputs for regularizing weights (instead of generating weights).

[0065] The bidirectional attention weight generator and the self-attention weight generator can be modified into a multi-head bidirectional attention weight generator and a multi-head self-attention weight generator, respectively.

[0066] Now refer to Figure 4The flowchart illustrates a method for continuous few-shot learning according to an embodiment. At 402, a model for a basic task is pre-trained using basic classification weights for the basic class of the basic task. The model includes a feature extractor. At 404, a new basic class and a pseudo-new class are selected from the basic classes. At 406, the average cross-entropy loss is determined using randomly selected samples from each class to be used to optimize the weight generator. At 408, the weight generator is optimized using a random number of pseudo-new tasks with the new basic class and the pseudo-new class, or using a fixed number of pseudo-new tasks with the pseudo-new class.

[0067] In step 410, a new task is received within a series of new tasks. In step 412, features are extracted from a sample set in the second image sample set of the new task. The sample set is pre-classified into a new class that is different from the base class.

[0068] In step 414, new classification weights are generated for the new class. In one example, the weight generator can be trained based on one or more of the following to generate new classification weights for the new class: base classification weights, one or more other new classification weights, a first image sample set for the base task, and a second image sample set for the new task. For example, the new classification weights can be generated by the weight generator using features extracted from the image sample set used to train the weight generator, the base classification weights, and one or more other new classification weights. The one or more other new classification weights are used for the new class of one or more other new tasks in the previously received series. The number of one or more other new tasks can be less than or equal to three.

[0069] As another example, new classification weights can be generated using a weight generator with classification weights for classes selected from one or more other new tasks in a previously received series, along with extracted features. For each new task, a random number of classes can be selected for the classification weights used to generate the new classification weights.

[0070] In 416, the model is updated using new classification weights for the new class of the new task.

[0071] At 418, the updated model is used to classify the unclassified sample set of the new task (e.g., the sample set in the second image sample set of the new task) into a new class. This allows for more efficient and accurate image classification. At 420, it is determined whether to receive another new task in a series of new tasks. If an additional new task is expected, the method returns to 410. If no additional new task is expected, the method terminates at 422.

[0072] Figure 5 This is a block diagram of an electronic device (or user equipment) in a network environment according to one embodiment. (Refer to...) Figure 5In network environment 500, electronic device 501 can communicate with electronic device 502 via a first network 598 (e.g., a short-range wireless communication network), or with electronic device 504 or server 508 via a second network 599 (e.g., a long-range wireless communication network). Electronic device 501 can communicate with electronic device 504 via server 508. Electronic device 501 may include processor 520, memory 530, input device 550, sound output device 555, display device 560, audio module 570, sensor module 576, interface 577, haptic module 579, camera module 580, power management module 588, battery 589, communication module 590, connection terminal 578, subscriber identification module (SIM) 596, or antenna module 597. In one embodiment, at least one of the components (e.g., display device 560 or camera module 580) may be omitted from electronic device 501, or one or more other components may be added to electronic device 501. Some of the components may be implemented as a single integrated circuit (IC). For example, sensor module 576 (e.g., fingerprint sensor, iris sensor, or illuminance sensor) may be embedded in display device 560 (e.g., display).

[0073] Processor 520 may execute, for example, software (e.g., program 540) to control at least one other component (e.g., hardware or software component) of electronic device 501 connected to processor 520, and may perform various data processing or computations (e.g., the methods described above for continuous few-shot learning). As at least part of the data processing or computation, processor 520 may load commands or data received from other components (e.g., sensor module 576 or communication module 590) into volatile memory 532, process the commands or data stored in volatile memory 532, and store the resulting data in non-volatile memory 534. Processor 520 may include a main processor 521 (e.g., central processing unit (CPU) or application processor (AP)) and an auxiliary processor 523 (e.g., graphics processing unit (GPU), image signal processor (ISP), sensor hub processor, or communication processor (CP)) that may operate independently of or in conjunction with the main processor 521. Alternatively or concurrently, the auxiliary processor 523 may be adapted to consume less power than the main processor 521, or to perform specific functions. The auxiliary processor 523 may be implemented separately from or as part of the main processor 521.

[0074] The auxiliary processor 523 may replace the main processor 521 when the main processor 521 is inactive (e.g., in sleep) or, when the main processor 521 is active (e.g., executing an application), work with the main processor 521 to control at least some of the functions or states associated with at least one component of the electronic device 501 (e.g., display device 560, sensor module 576, or communication module 590). The auxiliary processor 523 (e.g., an image signal processor or a communication processor) may be implemented as part of another component (e.g., camera module 580 or communication module 590) that is functionally associated with the auxiliary processor 523.

[0075] The memory 530 may store various data used by at least one component of the electronic device 501 (e.g., processor 520 or sensor module 576). The various data may include, for example, software (e.g., program 540) and input or output data for commands associated therewith. The memory 530 may include volatile memory 532 or non-volatile memory 534.

[0076] The program 540 may be stored as software in the memory 530 and may include, for example, an operating system (OS) 542, middleware 544, or application 546.

[0077] Input device 550 can receive commands or data from outside electronic device 501 (e.g., a user) that will be used by other components of electronic device 501 (e.g., processor 520). Input device 550 may include, for example, a microphone, mouse, or keyboard.

[0078] The sound output device 555 can output sound signals to the outside of the electronic device 501. The sound output device 555 may include, for example, a speaker or a receiver. The speaker can be used for general purposes (such as playing multimedia or recording), and the receiver can be used to receive incoming calls. The receiver can be implemented separately from the speaker or as part of the speaker.

[0079] Display device 560 can visually provide information to the outside of electronic device 501 (e.g., to a user). Display device 560 may include, for example, a display, a holographic device, or a projector, and control circuitry for controlling a respective one of the display, holographic device, and projector. Display device 560 may include touch circuitry adapted to detect touch, or sensor circuitry adapted to measure the intensity of the force caused by touch (e.g., a pressure sensor).

[0080] The audio module 570 can convert sound into electrical signals and vice versa. The audio module 570 can acquire sound via the input device 550, or output sound via the sound output device 555 or headphones of the external electronic device 502 that is directly (e.g., wired) or wirelessly connected to the electronic device 501.

[0081] Sensor module 576 can detect the operating state of electronic device 501 (e.g., power or temperature) or the environmental state outside electronic device 501 (e.g., user state), and then generate an electrical signal or data value corresponding to the detected state. Sensor module 576 may include, for example, a gesture sensor, gyroscope sensor, atmospheric pressure sensor, magnetic sensor, accelerometer, grip sensor, proximity sensor, color sensor, infrared (IR) sensor, biometric sensor, temperature sensor, humidity sensor, or illuminance sensor.

[0082] Interface 577 may support one or more specified protocols for direct (e.g., wired) or wireless connection of electronic device 501 to external electronic device 502. Interface 577 may include, for example, a High Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB) interface, a Secure Digital Card (SD) interface, or an audio interface.

[0083] Connection terminal 578 may include a connector through which electronic device 501 can be physically connected to external electronic device 502. Connection terminal 578 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0084] The haptic module 579 can convert electrical signals into mechanical stimulation (e.g., vibration or motion) or electrical stimulation that can be recognized by a user via touch or kinesthesia. The haptic module 579 may include, for example, a motor, a piezoelectric element, or an electrical stimulator.

[0085] Camera module 580 can capture still or moving images. Camera module 580 may include one or more lenses, an image sensor, an image signal processor, or a flash.

[0086] The power management module 588 manages the power supplied to the electronic device 501. The power management module 588 may be implemented as at least part of, for example, a power management integrated circuit (PMIC).

[0087] Battery 589 can power at least one component of electronic device 501. Battery 589 may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0088] Communication module 590 can support the establishment of a direct (e.g., wired) or wireless communication channel between electronic device 501 and external electronic devices (e.g., electronic device 502, electronic device 504, or server 508), and perform communication via the established communication channel. Communication module 590 may include one or more communication processors that can operate independently of processor 520 (e.g., AP), and support direct (e.g., wired) or wireless communication. Communication module 590 may include wireless communication module 592 (e.g., cellular communication module, short-range wireless communication module, or Global Navigation Satellite System (GNSS) communication module) or wired communication module 594 (e.g., local area network (LAN) communication module or power line communication (PLC) module). A corresponding one of these communication modules can communicate via a first network 598 (e.g., such as Bluetooth). TM The communication module 592 communicates with external electronic devices via a short-range communication network (such as a Wi-Fi Direct or Infrared Data Association (IrDA) standard network) or a second network 599 (such as a long-range communication network like a cellular network, the Internet, or a computer network (e.g., a LAN or a wide area network (WAN)) . These various types of communication modules can be implemented as a single component (e.g., a single IC) or as multiple components that are separate from each other (e.g., multiple ICs). The wireless communication module 592 can use user information (e.g., International Mobile Subscriber Identity (IMSI)) stored in the user identification module 596 to identify and authenticate electronic device 501 in a communication network (such as a first network 598 or a second network 599).

[0089] Antenna module 597 can transmit or receive signals or power to or from the outside of electronic device 501 (e.g., external electronic device). Antenna module 597 may include one or more antennas, and at least one antenna suitable for a communication scheme used in a communication network (such as a first network 598 or a second network 599) may be selected, for example, by communication module 590 (e.g., wireless communication module 592). Signals or power can then be transmitted or received between communication module 590 and external electronic device via the selected at least one antenna.

[0090] At least some of the aforementioned components can be interconnected and transmit signals (e.g., commands or data) between them via peripheral communication schemes (e.g., bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industrial processor interface (MIPI)).

[0091] Commands or data can be sent or received between electronic device 501 and external electronic device 504 via server 508 connected to the second network 599. Each of electronic devices 502 and 504 can be a device of the same or different type as electronic device 501. All or some of the operations to be performed at electronic device 501 can be performed at one or more of the external electronic devices 502, 504, or server 508. For example, if electronic device 501 is required to perform a function or service automatically or in response to a request from a user or another device, electronic device 501 may perform the function or service in lieu of performance, or may request one or more external electronic devices to perform at least a portion of the function or service in addition to performing the function or service. One or more external electronic devices receiving the request may perform at least a portion of the requested function or service, or additional functions or services related to the request, and transmit the result of the performance to electronic device 501. Electronic device 501 may provide the result, with or without further processing, as at least part of a response to the request. For this purpose, cloud computing, distributed computing, or client-server computing technologies may be used, for example.

[0092] One embodiment may be implemented as software (e.g., program 540) including one or more instructions stored in a storage medium (e.g., internal memory 536 or external memory 538) readable by a machine (e.g., electronic device 501). For example, a processor of electronic device 501 may invoke at least one of the one or more instructions stored in the storage medium and execute at least one instruction under the control of the processor with or without one or more other components. Thus, the machine may be operable to perform at least one function according to the invoked at least one instruction. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. The term "non-transitory" indicates that the storage medium is a tangible device and does not include signals (e.g., electromagnetic waves), but the term does not distinguish between a location where data is stored semi-permanently in the storage medium and a location where data is temporarily stored in the storage medium.

[0093] According to one embodiment, the disclosed method may include being in and providing a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., an optical disc read-only memory (CD-ROM)) or via an app store (e.g., the Play Store). TMThe computer program product may be distributed online (e.g., downloaded or uploaded) or directly between two user devices (e.g., smartphones). If distributed online, at least a portion of the computer program product may be temporarily generated or at least temporarily stored in a machine-readable storage medium (such as the memory of a manufacturer's server, an app store's server, or a relay server).

[0094] According to one embodiment, each of the above components (e.g., a module or program) may include a single entity or multiple entities. One or more of the above components may be omitted, or one or more other components may be added. Optionally or additionally, multiple components (e.g., modules or programs) may be integrated into a single component. In this case, the integrated component may still perform one or more functions of each of the multiple components in the same or similar manner as one or more functions of each of the multiple components were performed by the corresponding components of the multiple components prior to integration. Operations performed by modules, programs, or other components may be performed sequentially, in parallel, repeatedly, or heuristically, or one or more operations may be performed in a different order or omitted, or one or more other operations may be added.

[0095] Although specific embodiments of this disclosure have been described in detail, this disclosure can be modified in various forms without departing from its scope. Therefore, the scope of this disclosure should not be determined solely based on the described embodiments, but rather on the appended claims and their equivalents.

Claims

1. A training method for a model used for image classification, comprising: Receive the first image sample set of the base class of the basic task; The model is trained based on the first image sample set to obtain the basic classification weights of the basic class for the basic task from the trained model; Receive multiple new tasks sequentially; as well as Upon receiving any one of the plurality of new tasks: Receive the second image sample set of the new class for any new task. A weight generator is trained based on base classification weights, one or more other new classification weights for new classes of one or more previously received other new tasks among the plurality of new tasks, and one or more from a first image sample set and a second image sample set, to obtain new classification weights for new classes of any one of the new tasks using the base classification weights and the one or more other new classification weights as input to the weight generator. The model is updated using new classification weights for the new class of any new task.

2. The training method according to claim 1 further includes: The weight generator is trained using either a random number of base classes selected from the base classes and fake new tasks of fake new classes selected from the base classes, or a fixed number of fake new tasks of fake new classes selected from the base classes.

3. The training method according to claim 2, wherein, The steps for training the weight generator include: determining the average cross-entropy loss using samples randomly selected from the image sample set used to train the weight generator class.

4. The training method according to claim 1, wherein, The updated model is used to classify the first sample set in the second image sample set of any new task into the new class.

5. The training method according to claim 1, wherein, The steps for training a model based on a first image sample set include: extracting features from the first image sample set and training the model based on the extracted features.

6. The training method according to claim 1, wherein, The steps for training a weight generator include: Extract features from the second sample set in the second image sample set of any new task; and New classification weights are generated by using the extracted features, the basic classification weights, and one or more other new classification weights through a weight generator.

7. The training method according to claim 6, wherein, The number of the one or more other new tasks is less than or equal to three.

8. The training method according to claim 1, wherein, The steps for training a weight generator include: Extract features from the second sample set in the second image sample set of any new task; and New classification weights are generated by using a weight generator with classification weights for classes selected from the base class and new classes for one or more other new tasks, and extracted features.

9. The training method according to claim 8, wherein, For each new task, a random number of classes are selected for the classification weights used to generate the new classification weights.

10. The training method according to any one of claims 1 to 9, wherein, The weight generator is either a bidirectional attention weight generator or a self-attention weight generator.

11. A user device for training a model for image classification, comprising: processor; as well as A non-transitory computer-readable storage medium for storing instructions that, when executed, cause the processor to: Receive the first image sample set of the base class of the basic task; The model is trained based on the first image sample set to obtain the basic classification weights of the basic class for the basic task from the trained model; Receive multiple new tasks sequentially; as well as Upon receiving any one of the plurality of new tasks: Receive the second image sample set of the new class for any new task. A weight generator is trained based on base classification weights, one or more other new classification weights for new classes of one or more previously received other new tasks among the plurality of new tasks, and one or more from a first image sample set and a second image sample set, to obtain new classification weights for new classes of any one of the new tasks using the base classification weights and the one or more other new classification weights as input to the weight generator. The model is updated using new classification weights for the new class of any new task.

12. The user equipment according to claim 11, wherein, The processor is also configured to train the weight generator using either a random number of base classes and fake new classes selected from the base classes, or a fixed number of fake new classes selected from the base classes.

13. The user equipment according to claim 12, wherein, When training the weight generator, the processor is also configured to determine the average cross-entropy loss using samples randomly selected from the image sample set of the class used to train the weight generator.

14. The user equipment according to claim 11, wherein, The updated model is used to classify the first sample set in the second image sample set of any new task into the new class.

15. The user equipment according to claim 11, wherein, The steps for training a model based on a first image sample set include: extracting features from the first image sample set and training the model based on the extracted features.

16. The user equipment according to claim 11, wherein, During the training of the weight generator, the processor is also configured as follows: Extract features from the second sample set in the second image sample set of any new task; as well as New classification weights are generated by using the extracted features, the basic classification weights, and one or more other new classification weights through a weight generator.

17. The user equipment according to claim 16, wherein, The number of the one or more other new tasks is less than or equal to three.

18. The user equipment according to claim 11, wherein, During the training of the weight generator, the processor is also configured as follows: Extract features from the second sample set in the second image sample set of any new task; as well as New classification weights are generated by using a weight generator with classification weights for classes selected from the base class and new classes for one or more other new tasks, and extracted features.

19. The user equipment according to claim 18, wherein, For each new task, a random number of classes are selected for the classification weights used to generate the new classification weights.

20. The user equipment according to any one of claims 11 to 19, wherein, The weight generator is either a bidirectional attention weight generator or a self-attention weight generator.

Images