A method of continuously learning a classifier for classifying images of a client using a continuous learning server, and a continuous learning server using the same

Abstract

A method is disclosed for continuously learning a classifier for classifying client images using a continuous learning server, comprising: the continuous learning server (a) inputting each first-difficulty image from a first classifier of the client into an adversarial autoencoder, such that the adversarial autoencoder outputs multiple latent vectors from the multiple first-difficulty images through an encoder, outputs multiple reconstructed images from the multiple latent vectors through a decoder, outputs multiple attribute information and multiple second classification information through a discriminator and a second classifier to determine multiple second-difficulty images to be stored in a first learning dataset, and generating multiple augmented images to be stored in a second learning dataset by adjusting the multiple latent vectors corresponding to each reconstructed image determined not to be a second-difficulty image; (b) continuously learning a third classifier corresponding to the first classifier; and (c) transmitting multiple updated parameters to the client.

Description

Technical Field

[0001] This invention relates to a method for continuously learning a classifier for classifying client images using a continuous learning server, and a continuous learning server using the same. More specifically, it relates to a method and a continuous learning server using the same, which generates multiple learning data by applying reconstruction and augmentation processing to hard images determined to be difficult for the client's classifier to classify. The continuous learning server then generates parameters for continuous learning from the learning data and transmits them to the client, thereby updating the client's classifier. Background Technology

[0002] Image classification is used by humans or machines to identify and analyze objects in various fields such as security systems, robots, automobiles, healthcare, and social media.

[0003] In such image classification, the performance and learning methods required by the image classifier can vary depending on the type of input image, the purpose of image classification, and the characteristics of the environment in which image classification is used.

[0004] Therefore, in image classification, depending on the type of available data and the purpose of image classification, at least one of the following learning methods has been adopted and used as the learning method for learning image classifier algorithms: supervised learning, unsupervised learning, reinforcement learning, and continuous learning.

[0005] In particular, in existing image classification methods, as a method for learning image classifiers using neural networks (NNs), a new model is learned through conventional continuous learning methods. If the new model is better than the previous model, the new model is selected; if the previous model is better, the previous model is selected.

[0006] However, in existing image classification methods, when various variables such as weather and noise arise during the image classification process, creating difficult-to-classify images, humans or machines may struggle to identify at least some of these images. Accurately identifying these difficult images requires a very large amount of training data, and the network capacity increases with the amount of new training data it can accommodate, thus necessitating more computing power and higher learning costs.

[0007] In addition, as network capacity increases, a large amount of computing power is required. To accommodate this computing power, image classifiers may become physically larger and heavier. In this case, there is a problem that it is difficult to install such large and heavy image classifiers in mobile devices or small terminals.

[0008] Furthermore, existing image classification methods may suffer from deep-seated problems such as catastrophic forgetting (CF), where information about previously learned tasks is lost during the learning of new tasks.

[0009] Therefore, an improved solution is needed to address the aforementioned problem. Summary of the Invention

[0010] Technical issues

[0011] The purpose of this invention is to solve all the above-mentioned problems.

[0012] Furthermore, the present invention aims to train the classifier of the client's classifier using learning data generated by the Adversarial Autoencoder (AAE) of the continuous learning server, so as to update the client's classifier using only updated parameter information.

[0013] Furthermore, another objective of this invention is to learn the classifier more efficiently with less computation than existing methods by using continuous learning techniques that update only a portion of the layers or slightly extend or increase the layers when updating the classifier on the client using parameters of the classifier learned in the continuous learning server.

[0014] Technical solution

[0015] In order to achieve the above-mentioned objectives of the present invention and to realize the characteristic effects of the present invention described later, the characteristic structure of the present invention is as follows.

[0016] According to one aspect of the present invention, a method for continuously learning a classifier for classifying client images using a continuous learning server includes: (a) when the client receives multiple first classification information corresponding to each acquired image from the output of a first classifier located on the client, and multiple first hard images are determined to be unclassifiable by the first classifier based on the multiple first classification information corresponding to each acquired image, the continuous learning server performs or supports performing the following processing: (i) inputting each first hard image into an adversarial autoencoder (AAE), causing the adversarial autoencoder (i-1) to encode each first hard image through an encoder included in the adversarial autoencoder to output each latent vector. (i-2) Each latent vector is decoded by the decoder included in the adversarial autoencoder to output each reconstructed image corresponding to each of the first difficult images; (i-3) Multiple attribute information regarding whether each reconstructed image is true or false is output by the discriminator included in the adversarial autoencoder; (i-4) Multiple second classification information regarding each latent vector is output by the second classifier included in the adversarial autoencoder; and (ii) The multiple reconstructed images are determined to be second difficult images that the adversarial autoencoder cannot distinguish by referring to the attribute information and the second classification information; (ii-1) Multiple first reconstructed images that are judged to be second difficult images are stored in the first learning dataset; (ii-2) Multiple latent vectors corresponding to each second reconstructed image that is judged not to be a second difficult image are randomly adjusted by the decoder to generate multiple augmented images. (a) and store it in the second learning dataset; (b) the continuous learning server performs or supports the following process: continuously learns a third classifier located on the continuous learning server and corresponding to the first classifier using the first learning dataset and the second learning dataset; and (c) the continuous learning server performs or supports the following process: transmits at least one updated parameter of the learned third classifier to the client so that the client updates the first classifier using the updated plurality of said parameters.

[0017] As an example, prior to (c), the continuous learning server performs or supports performing the following process: continuously learning the adversarial autoencoder using (i) first existing labeled training data including multiple first existing labeled images and (ii) first newly labeled training data obtained by labeling each of the first reconstructed images in the first training dataset.

[0018] As an example, the continuous learning server performs or supports the following processing: performing continuous learning for the adversarial autoencoder, wherein the autoencoder constituting the adversarial autoencoder and the discriminator are learned alternately, the autoencoder including the encoder and the decoder, and the continuous learning server performs or supports the following processing: (i) for each of the autoencoder and the discriminator, using an existing autoencoder model previously trained with the first existing labeled learning data or an existing discriminator model previously trained with the first existing labeled learning data to obtain a first base loss as the average loss for the first existing labeled learning data; (ii) for each iteration of the continuous learning, after sampling a first minibatch by selecting portions of data from the first existing labeled learning data and the first new labeled learning data at specific ratios respectively; and (iii) inputting the first minibatch into the existing autoencoder model or the existing discriminator model so that the existing autoencoder model or the existing discriminator model references the ground data corresponding to the first existing labeled learning data. (iv) Generate the first existing loss corresponding to the first existing labeled learning data, and generate the first new loss corresponding to the first new labeled learning data with reference to the real data corresponding to the first new labeled learning data. (iv) (iv-1) For the first new loss, learn it through back-propagation in each iteration of the continuous learning. (iv-2) For the first existing loss, learn it through back-propagation only for the part of the iteration where the first existing loss is greater than the first basic loss.

[0019] As an example, in step (a), the continuous learning server performs or supports performing the following processing: (i) when the discriminator determines that multiple augmented images are true, and the second_1 classification information generated for the multiple augmented images is the same as the second_2 classification information generated for the multiple second reconstructed images corresponding to the multiple augmented images, the multiple augmented images are stored in the second learning dataset; and (ii) when the discriminator determines that multiple augmented images are false, or when the second_1 classification information generated for the multiple augmented images is different from the second_2 classification information generated for the multiple second reconstructed images corresponding to the multiple augmented images, the multiple augmented images are not stored in the second learning dataset.

[0020] As an example, in step (a), the continuous learning server performs or supports performing the following processing: (i) when the discriminator determines that multiple reconstructed images are false, or when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is less than a first threshold preset by the second classifier, the reconstructed image is determined to be the second difficult image; and (ii) when the discriminator determines that multiple reconstructed images are true, and when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is greater than a second threshold preset by the second classifier, the reconstructed image is determined not to be the second difficult image.

[0021] As an example, in step (a), the continuous learning server performs or supports performing the following processing: (i) when the deviation of the probability distribution of each category of the first classification information generated by the first classifier for multiple acquired images is less than a first threshold preset by the first classifier, the acquired image is determined to be the first difficult image; and (ii) when the deviation of the probability distribution of each category of the first classification information generated by the first classifier for multiple acquired images is greater than a second threshold preset by the first classifier, the acquired image is determined not to be the first difficult image.

[0022] As an example, the continuous learning server performs or supports the following process: enabling the encoder to generate multiple latent vectors representing multiple feature values ​​corresponding to each of the first difficult images by downsampling through applying at least one convolution operation and at least one pooling operation to each of the first difficult images.

[0023] As an example, the continuous learning server performs or supports the following process: enabling the decoder to generate multiple reconstructed images corresponding to each of the first-difficulty images by upsampling each of the latent vectors through at least one deconvolution operation and at least one un-pooling operation.

[0024] As an example, in step (b), the continuous learning server performs or supports the following processing: in the method of continuous learning of the third classifier, (i) a second base loss is obtained as the average loss for the second existing labeled training data using an existing third classifier model trained with second existing labeled training data including multiple second existing labeled images; (ii) for each iteration of the continuous learning, a second minibatch is sampled by selecting portions of data from the second existing labeled training data and the second newly labeled training data at specific ratios, wherein the second newly labeled training data is generated by labeling multiple first reconstructed images included in the first training dataset and multiple augmented images included in the second training dataset; and (iii) the second minibatch is input into the existing third classifier, causing the existing third classifier to generate the second existing loss corresponding to the second existing labeled training data by referring to the ground truth corresponding to the second existing labeled training data. (iv) (iv-1) For the second new loss, it is learned through back-propagation in each iteration of the continuous learning process. (iv-2) For the second existing loss, it is learned through back-propagation only for the part of the iterations where the second existing loss is greater than the second basic loss.

[0025] As an example, the continuous learning server performs or supports the following process: transmitting only update information about at least one updated parameter, an updated neural network layer, and an updated category to the first classifier of the client, so that the client updates the first classifier using only the received update information about the updated parameters, the updated neural network layers, and the updated categories.

[0026] As an example, the continuous learning server performs or supports performing the following processes: causing the client to update the first classifier by (i) selectively updating at least a portion of the neural network layers constituting the first classifier with reference to the update information about the updated plurality of parameters; (ii) adding at least one new neural network layer with reference to the update information about the updated neural network layers; and (iii) updating the first classifier by at least one of the following processes: adding at least one category with reference to the update information about the updated category.

[0027] According to another aspect of the present invention, a continuous learning server is provided for continuously learning a classifier for classifying client images using a continuous learning server, comprising: at least one memory storing a plurality of instructions; and at least one processor for executing the plurality of instructions, the processor performing or supporting the following processing: (I) upon receiving from the client a plurality of first classification information corresponding to each acquired image output by a first classifier located on the client, and a plurality of first hard images determined by the first classifier to be unclassifiable based on the plurality of first classification information corresponding to each acquired image, (i) inputting each hard image to an adversarial autoencoder (AAE), causing the adversarial autoencoder (i-1) to encode each hard image through an encoder included in the adversarial autoencoder to output each latent vector. (i-2) Each latent vector is decoded by the decoder included in the adversarial autoencoder to output each reconstructed image corresponding to each of the first difficult images; (i-3) The discriminator included in the adversarial autoencoder outputs multiple attribute information about whether each reconstructed image is true or false. (i-4) Outputting multiple second classification information about each latent vector through the second classifier included in the adversarial autoencoder; and (ii) determining whether multiple reconstructed images are second-difficult images that the adversarial autoencoder cannot distinguish by referring to the attribute information and the second classification information; (ii-1) storing multiple first reconstructed images that are judged to be second-difficult images in the multiple reconstructed images in the first learning dataset; (ii-2) randomly adjusting multiple latent vectors corresponding to each second reconstructed image that is judged not to be a second-difficult image in the multiple reconstructed images by the decoder to generate multiple augmented images and storing them in the second learning dataset; (ii) continuously learning a third classifier located in the processor and corresponding to the first classifier using the first learning dataset and the second learning dataset; and (iii) transmitting at least one updated parameter of the learned third classifier to the client so that the client updates the first classifier using the updated multiple parameters.

[0028] As an example, prior to the (III) process, the processor performs or supports the following process: continuously learning the adversarial autoencoder using (i) first existing labeled training data including multiple first existing labeled images and (ii) first newly labeled training data obtained by labeling each of the first reconstructed images in the first training dataset.

[0029] As an example, the processor performs or supports performing the following processing: performing the continuous learning for the adversarial autoencoder, wherein the autoencoder constituting the adversarial autoencoder and the discriminator are learned alternately, the autoencoder including the encoder and the decoder, and the processor performs or supports performing the following processing: (i) for each of the autoencoder and the discriminator, using an existing autoencoder model previously trained with the first existing labeled learning data or an existing discriminator model previously trained with the first existing labeled learning data to obtain a first base loss as the average loss for the first existing labeled learning data; (ii) for each iteration of the continuous learning, after sampling a first minibatch by selecting portions of data from the first existing labeled learning data and the first new labeled learning data at specific ratios respectively; and (iii) inputting the first minibatch into the existing autoencoder model or the existing discriminator model so that the existing autoencoder model or the existing discriminator model references the ground data corresponding to the first existing labeled learning data. (iv) Generate the first existing loss corresponding to the first existing labeled learning data, and generate the first new loss corresponding to the first new labeled learning data with reference to the real data corresponding to the first new labeled learning data. (iv) (iv-1) For the first new loss, it is learned through back-propagation in each iteration of the continuous learning. (iv-2) For the first existing loss, it is learned through back-propagation only for the part of the iterations where the first existing loss is greater than the first basic loss.

[0030] As an example, in step (I), the processor performs or supports performing the following processing: (i) when the discriminator determines that the plurality of augmented images are true, and the second_1 classification information generated for the plurality of augmented images is the same as the second_2 classification information generated for the plurality of second reconstructed images corresponding to the plurality of augmented images, the plurality of augmented images are stored in the second learning dataset; and (ii) when the discriminator determines that the plurality of augmented images are false, or when the second_1 classification information generated for the plurality of augmented images is different from the second_2 classification information generated for the plurality of second reconstructed images corresponding to the plurality of augmented images, the plurality of augmented images are not stored in the second learning dataset.

[0031] As an example, in step (I), the processor performs or supports performing the following processing: (i) when the discriminator determines that multiple reconstructed images are false, or when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is less than a first threshold preset by the second classifier, the reconstructed image is determined to be the second difficult image; and (ii) when the discriminator determines that multiple reconstructed images are true, and when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is greater than a second threshold preset by the second classifier, the reconstructed image is determined to be not the second difficult image.

[0032] As an example, in step (I), the processor performs or supports performing the following processing: (i) determining that the acquired image is the first difficult image when the deviation of the probability distribution of each category of the first classification information generated by the first classifier for the multiple acquired images is less than a first threshold preset by the first classifier; and (ii) determining that the acquired image is not the first difficult image when the deviation of the probability distribution of each category of the first classification information generated by the first classifier for the multiple acquired images is greater than a second threshold preset by the first classifier.

[0033] As an example, the processor performs or supports performing the following process: causing the encoder to generate multiple latent vectors representing multiple feature values ​​corresponding to each of the first difficult images by applying at least one convolution operation and at least one pooling operation to each of the first difficult images.

[0034] As an example, the processor performs or supports performing the following process: causing the decoder to generate multiple reconstructed images corresponding to each of the first hard images by upsampling through applying at least one deconvolution operation and at least one un-pooling operation to each of the latent vectors.

[0035] As an example, in step (II), the processor performs or supports performing the following processing: in the method of continuous learning of the third classifier, (i) using an existing third classifier model trained with second existing labeled training data including multiple second existing labeled images to obtain a second base loss as the average loss for the second existing labeled training data; (ii) for each iteration of the continuous learning, sampling a second minibatch by selecting portions of data from the second existing labeled training data and the second newly labeled training data at a specific ratio, wherein the second newly labeled training data is generated by labeling multiple first reconstructed images included in the first training dataset and multiple augmented images included in the second training dataset; and (iii) inputting the second minibatch into the existing third classifier, causing the existing third classifier to generate a second existing loss corresponding to the second existing labeled training data with reference to the ground truth corresponding to the second existing labeled training data. (iv) (iv-1) For the second new loss, it is learned through back-propagation in each iteration of the continuous learning process. (iv-2) For the second existing loss, it is learned through back-propagation only for the part of the iterations where the second existing loss is greater than the second basic loss.

[0036] As an example, the processor performs or supports performing the following process: transmitting only update information about at least one updated parameter, an updated neural network layer, and an updated category to the first classifier of the client, so that the client updates the first classifier using only the received update information about the updated multiple parameters, the updated multiple neural network layers, and the updated multiple categories.

[0037] As an example, the processor performs or supports performing the following processes: causing the client to update the first classifier by at least one of the following processes: (i) selectively updating at least a portion of the neural network layers constituting the first classifier with reference to the update information regarding the updated plurality of parameters; (ii) adding at least one new neural network layer with reference to the update information regarding the updated neural network layers; and (iii) adding at least one category with reference to the update information regarding the updated categories.

[0038] In addition, the present invention also provides a computer-readable recording medium for recording a computer program for performing the method of the present invention.

[0039] Beneficial effects

[0040] The advantage of this invention is that it uses learning data generated by the Adversarial Autoencoder (AAE) of the continuous learning server to train the classifier of the client's classifier, so that the client's classifier can be updated only with updated parameter information.

[0041] Furthermore, another advantage of the present invention is that when updating the classifier on the client using the parameters of the classifier learned in the continuous learning server, the continuous learning technique of updating only a portion of the layers or slightly expanding or increasing the layers can efficiently learn the classifier with less computation than existing methods. Attached Figure Description

[0042] The following drawings, which are used to describe embodiments of the present invention, are only a part of the embodiments of the present invention, and those skilled in the art to which the present invention pertains (hereinafter referred to as "skilled persons") can obtain other drawings based on these drawings without any creative work.

[0043] Figure 1This is a schematic diagram of a continuous learning server for updating the classifier of a client classifying images, according to an embodiment of the present invention.

[0044] Figure 2 This is a schematic diagram of a method for updating a client's classifier using a continuous learning server according to an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram of a continuous learning method for an adversarial autoencoder according to an embodiment of the present invention.

[0046] Figure 4 This is a schematic diagram illustrating a method for continuously learning the classifier of a server corresponding to the classifier of a client according to an embodiment of the present invention.

[0047] Figure 5 This is a schematic diagram of a method for updating a client's classifier using update information transmitted from a classifier of a continuous learning server, according to an embodiment of the present invention. Detailed Implementation

[0048] The following detailed description of the invention is illustrated in the accompanying drawings, which show specific embodiments in which the invention can be practiced to illustrate the objectives, technical solutions, and advantages of the invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Furthermore, the invention includes all possible combinations of the embodiments shown in this specification. It should be understood that the various embodiments of the invention, although different, are not necessarily mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in one embodiment by other embodiments without departing from the spirit and scope of the invention. Furthermore, it should be understood that the position or configuration of the components in each disclosed embodiment may be varied without departing from the spirit and scope of the invention. Therefore, the detailed description that follows is not intended to limit the invention; the scope of the invention should be defined by all scopes equivalent to the scope of its claims and the appended claims, provided that appropriate description is possible. Similar reference numerals in the drawings indicate the same or similar functions in several respects.

[0049] The various images involved in this invention may include images related to paved or unpaved roads. In this case, it can be assumed that objects that may appear in a road environment (e.g., cars, people, animals, plants, objects, buildings, aircraft such as airplanes or drones, and other obstacles) can be present, but are not necessarily limited to this. The various images involved in this invention may also be images unrelated to roads (e.g., images related to unpaved roads, alleys, open spaces, seas, lakes, rivers, mountains, forests, deserts, skies, and interiors). In this case, it can be assumed that objects that may appear in unpaved roads, alleys, open spaces, seas, lakes, rivers, mountains, forests, deserts, skies, and interiors (e.g., cars, people, animals, plants, objects, buildings, aircraft such as airplanes or drones, and other obstacles) can be present, but are not necessarily limited to this. The titles and abstracts of the disclosure provided herein are for convenience only and should not be construed as limiting the scope or meaning of the embodiments.

[0050] Furthermore, in the description and claims of this invention, the term "comprising" and its variations are not intended to exclude other technical features, additions, components, or steps. Other objects, advantages, and features of this invention will be apparent to those skilled in the art, in part from this specification and in part from practice of the invention. The following illustrations and figures are provided as examples and are not intended to limit the scope of the invention.

[0051] The titles and abstracts of the disclosure provided herein are for convenience only and should not be construed as limiting the scope or meaning of the embodiments. Preferred embodiments of the invention will now be described in detail with reference to the accompanying drawings to enable those skilled in the art to readily implement the invention.

[0052] Figure 1 This is a schematic diagram of a continuous learning server 2000 for updating the classifier of a client classifying images, according to an embodiment of the present invention.

[0053] See Figure 1 The continuous learning server 2000 may include: a memory 2001 storing a plurality of instructions for updating a classifier that classifies images of a client; and a processor 2002 that, in response to the plurality of instructions stored in the memory 2001, performs operations to update the classifier that classifies images of a client.

[0054] Specifically, the Continuous Learning Server 2000 typically uses a combination of computing devices (e.g., computer processors, memory, storage devices, input and output devices, and other components that may include conventional computing devices; electronic communication devices such as routers, switches, etc.; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that enable the computing devices to operate in a particular manner) to achieve the required system performance.

[0055] Additionally, the processor of a computing device may include hardware configurations such as a microprocessor unit (MPU) or central processing unit (CPU), cache memory, and a data bus. Furthermore, the computing device may include a software configuration such as an operating system and applications for executing specific purposes.

[0056] However, it is not excluded that the computing device may include an integrated processor in the form of an integrated medium, processor, and memory for the purpose of implementing the present invention.

[0057] On the other hand, as an embodiment of the present invention, the continuous learning server 2000 continuously learns from multiple classifiers located on multiple clients based on multiple hard images transmitted from each client, and then can transmit update information to each client to update the multiple classifiers located on each client. Each acquired image determined as difficult to recognize by each classifier of each client can be defined as a hard image.

[0058] On the other hand, as another embodiment of the present invention, multiple classifiers located on multiple clients may correspond to a single classifier located on the continuous learning server 2000. In this case, the single corresponding classifier located on the continuous learning server 2000 is trained using learning data generated from multiple difficult images transmitted from multiple classifiers on multiple clients. The single corresponding classifier may simultaneously transmit update information to multiple classifiers located on multiple clients, but the present invention is not limited thereto.

[0059] The following will combine Figures 2 to 5 In describing this invention, for ease of explanation, the invention is described with respect to a single classifier in a single client and a single classifier in a corresponding continuous learning server 2000. However, according to another embodiment of the invention, the classifier in the client or the classifier in the continuous learning server 2000 may be configured as multiple classifiers.

[0060] The following will refer to Figures 2 to 5This describes a method for updating a first classifier 1100 that classifies images of a client 1000 using a continuous learning server 2000 configured as described above, according to an embodiment of the present invention.

[0061] In the rest of the description of the invention, for ease of explanation, "acquiring an image" may be labeled as "image" or "multiple images".

[0062] First, see Figure 2 The client 1000 can operate based on the first classification information in the first classifier 1100 for multiple images, and determine whether the multiple images are multiple first hard images based on the first classification information of the multiple images. At this time, the client 1000 can acquire video images. In this case, the client 1000 can acquire multiple images corresponding to each frame of the video images.

[0063] At this time, the first classification information can be the class-specific probabilities set by the first classifier 1100 for classifying multiple images. The client 1000 can determine whether multiple acquired images are first-difficulty images that cannot be classified by the first classifier 1100 by referring to the first classification information.

[0064] As an example, when the probabilities of each category in the first classification information are similar and the deviation of the probability distribution of each category is lower than the preset first threshold, the client 1000 can classify multiple images into multiple first-difficulty images, because the multiple images are multiple images that the first classifier 1100 has not learned or multiple images that have been learned but not recognized, which are data that the first classifier 1100 has difficulty classifying.

[0065] On the other hand, when the probability of a category in the first classification information is higher than a preset value and the deviation of the probability distribution of each category is higher than the preset first-second threshold, since the client 1000 can identify multiple such images, it can be classified as a non-first-class image. The preset first-first threshold and the preset first-second threshold can have the same value, but are not limited to this.

[0066] Then, the client 1000 can transmit multiple images identified as the first difficult image to the continuous learning server 2000. At this time, the client 1000 can transmit multiple first difficult images to the continuous learning server 2000 in real time, or, while storing multiple first difficult images in the internal storage device, when predetermined conditions are met, such as predetermined time conditions, predetermined image volume conditions, etc., the stored multiple first difficult images can be transmitted to the continuous learning server 2000.

[0067] On the other hand, the client 1000 may include: a memory storing a plurality of instructions for classifying a plurality of images to transmit a plurality of first-difficulty images to the continuous learning server 2000 and update the first classifier 1100; and a processor 2002 that, in response to the plurality of instructions stored in the memory, performs operations to classify a plurality of images, transmit a plurality of first-difficulty images to the continuous learning server 2000 and update the first classifier 1100.

[0068] Specifically, the client 1000 can typically achieve the required system performance using a combination of computing devices (e.g., computer processors, memory, storage devices, input and output devices, and other components that may include conventional computing devices; electronic communication devices such as routers, switches, etc.; electronic information storage systems such as network attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., instructions that enable the computing devices to operate in a particular manner).

[0069] Additionally, the processor of a computing device may include hardware configurations such as a microprocessor unit (MPU) or central processing unit (CPU), cache memory, and a data bus. Furthermore, the computing device may include a software configuration such as an operating system and applications for executing specific purposes.

[0070] However, it is not excluded that the computing device may include an integrated processor in the form of an integrated medium, processor, and memory for the purpose of implementing the present invention.

[0071] Additionally, the client 1000 can be an autonomous vehicle that can detect objects, lanes, etc., from driving images obtained during vehicle operation using a first classifier 1100. However, the client 1000 is not limited to autonomous vehicles and may also include computing devices that identify objects by classifying multiple images and perform operations based on the identified object information, such as smartphones, manufacturing robots, etc.

[0072] Next, when multiple first-difficulty images are transmitted from the client 1000, the continuous learning server 2000 can input each first-difficulty image into the adversarial autoencoder 2100 so that the adversarial autoencoder 2100 performs additional classification operations on the multiple first-difficulty images.

[0073] At this point, the adversarial autoencoder 2100 encodes each first-difficulty image by the encoder included in the adversarial autoencoder 2100 to output multiple latent vectors, and each latent vector can be decoded by the decoder included in the adversarial autoencoder 2100 to output multiple reconstructed images corresponding to each first-difficulty image.

[0074] As an example, the encoder 2110 of the continuous learning server 2000 can generate multiple latent vectors representing multiple feature values ​​corresponding to each first-difficulty image by applying at least one convolution operation and at least one pooling operation to down-sampling each first-difficulty image, but the present invention is not limited thereto.

[0075] In addition, the continuous learning server 2000 decoder 2120 can generate multiple reconstructed images corresponding to each first-difficulty image by applying at least one deconvolution operation and at least one un-pooling operation to each input latent vector for up-sampling, but the present invention is not limited thereto.

[0076] When multiple latent vectors and multiple reconstructed images are obtained, the continuous learning server 2000 can output multiple attribute information about whether each reconstructed image is true or false through the discriminator 2130 included in the adversarial autoencoder 2100, and output multiple second classification information about each latent vector through the second classifier 2140 included in the adversarial autoencoder.

[0077] Then, the continuous learning server 2000 can refer to the attribute information of the discriminator 2130 output from the adversarial autoencoder 2100 and the second classification information to determine whether multiple first-difficulty images are second-difficulty images that are difficult to classify even when using the adversarial autoencoder 2100.

[0078] As an example, multiple first reconstructed images that are determined to be false by discriminator 2130 or whose probability distributions for each category of the second classification information are determined by second classifier 2140 have a deviation less than a second threshold preset by second classifier 2140 can be stored as multiple second-difficult images in the first learning dataset. Furthermore, multiple reconstructed images that are difficult to classify even in adversarial autoencoder 2100 can be referred to as multiple first reconstructed images and stored as multiple second-difficult images in the first learning dataset. Additionally, the multiple first-difficult images are selected from multiple images acquired by client 1000 with reference to first classification information, while the multiple second-difficult images are selected from multiple reconstructed images generated from the multiple first-difficult images with reference to attribute information and second classification information.

[0079] On the other hand, multiple second reconstructed images that are determined to be true by discriminator 2130 and whose probability distributions of the categories of the second classification information determined by second classifier 2140 have a deviation greater than the second threshold preset by the second classifier can be excluded from being stored as second reconstructed images. Among them, the multiple reconstructed images that can be classified in the adversarial autoencoder 2100 can be referred to as multiple second reconstructed images.

[0080] Therefore, at this point, the output of encoder 2110 corresponding to each second reconstructed image, i.e., the latent vector, can be randomly adjusted so that decoder 2120 generates multiple augmented images corresponding to multiple randomly adjusted latent vectors. The preset second_1 threshold and the preset second_2 threshold can have the same value, but are not limited to this.

[0081] When randomly adjusting latent vectors for data augmentation, the degree of adjustment can be controlled so that the adjusted latent vectors, generated to the extent that the second classification information generated based on the augmented image is completely different from the existing latent vectors generated based on the second reconstructed image on which the augmented image is based, do not deviate from the existing latent vectors. For example, when adjusting each element of the latent vector, the degree of adjustment can be controlled by adjusting the latent vector itself, so that the difference between each element and the original value is within a preset deviation range, but the invention is not limited to this.

[0082] Therefore, when multiple augmented images are determined to be true by the discriminator 2130 and the second_1 classification information generated for the multiple augmented images is the same as the second_2 classification information generated for the second reconstructed image corresponding to the multiple augmented images, the continuous learning server 2000 can store the multiple augmented images in the second learning dataset.

[0083] On the other hand, when multiple augmented images are determined to be false by the discriminator 2130 or when the second_1 classification information generated for multiple augmented images is different from the second_2 classification information generated for the second reconstructed image corresponding to multiple augmented images, the multiple augmented images may not be stored in the second learning dataset.

[0084] Next, the continuous learning server 2000 can manually label each of the first reconstructed images included in the first learning dataset as multiple second-difficulty images determined by the multiple outputs of the adversarial autoencoder (AAE) 2100, to generate first newly labeled training data. Therefore, the continuous learning server 2000 can use the generated first newly labeled training data and the first existing labeled training data storing multiple first existing labeled images to continuously learn the adversarial autoencoder 2100.

[0085] Specifically, the continuous learning server 2000 can perform continuous learning against the adversarial autoencoder 2100, but it learns the autoencoder and discriminator 2130 that constitute the adversarial autoencoder 2100 in an overlapping manner. The autoencoder includes an encoder 2110 and a decoder 2120, and the adversarial autoencoder 2100 includes an autoencoder and a discriminator 2130.

[0086] Figure 3 This is a schematic diagram of a continuous learning method for an adversarial autoencoder according to an embodiment of the present invention.

[0087] See Figure 3 For each of the 2000 pairs of autoencoders and discriminators in the continuous learning server, the first base loss is obtained as the average loss for the first existing labeled learning data, using either the existing autoencoder model or the existing discriminator model previously trained with the first existing labeled learning data. For each iteration of continuous learning, the first minibatch is sampled from the first existing labeled learning data and the first new labeled learning data at specific ratios.

[0088] Next, the continuous learning server 2000 can input the first mini-batch into the existing autoencoder model or the existing discriminator model, so that the existing autoencoder model or the existing discriminator model can generate the first existing loss corresponding to the first existing labeled learning data by referring to the ground truth corresponding to the first existing labeled learning data, and generate the first new loss corresponding to the first new labeled learning data by referring to the ground truth corresponding to the first new labeled learning data.

[0089] Then, the continuous learning server 2000 (i) learns the first new loss through back-propagation in each iteration of the continuous learning, and (ii) learns the first existing loss only through back-propagation for the partial iterations in which the first existing loss is greater than the first basic loss, in order to complete the learning of the autoencoder or discriminator 2130.

[0090] That is, after the tagger analyzes and classifies multiple second-difficulty images identified in the adversarial autoencoder 2100, the performance of the adversarial autoencoder 2100 can be improved by selectively training only some neural network layers that constitute the adversarial autoencoder 2100 to update only the parameters of the selected neural network layers, or by adding some new neural network layers, or by adding classes, etc.

[0091] Next, see you again. Figure 2 The continuous learning server 2000 can use the first and second learning datasets classified by the adversarial autoencoder 2100 to continuously learn the third classifier 2200 located on the continuous learning server 2000. At this time, the third classifier 2200 can correspond to the first classifier 1100 located on the client 1000.

[0092] The multiple augmented images stored as the second learning dataset by the adversarial autoencoder 2100 can be multiple results of augmentation of multiple images that are difficult to classify by the first classifier 1100. The multiple second-difficult images stored as the first learning dataset can be the results of additional classification by multiple labelers of multiple images that are judged to be difficult to classify even using the adversarial autoencoder 2100. Therefore, the first classifier 1100 of the client 1000 can be updated by continuously learning the third classifier 2200 using multiple augmented images and multiple second-difficult images.

[0093] For reference, in this invention, since the adversarial autoencoder 2100 is located in the continuous learning server 2000 and is not limited by size, additional classification operations can be performed on multiple first-difficulty images that are classified as difficult to classify in the first classifier, which has a higher performance than the first classifier 1100. Similarly, the third classifier 2200 can also have the same or higher performance as the first classifier 1100, and depending on the physical constraints such as the size and weight of the client 1000, only at least a portion of the updated information of parameters, categories, and layers updated as a result of continuous learning can be transmitted to the first classifier 1100.

[0094] Figure 4 This is a schematic diagram illustrating a method for continuously learning the classifier of a server corresponding to the classifier of a client according to an embodiment of the present invention.

[0095] See Figure 4 In the continuous learning server 2000's method of continuously learning the third classifier 2200, a second base loss can be obtained as the average loss for the second existing labeled training data using an existing third classifier model previously trained with the second existing labeled training data. For each iteration of continuous learning, a second minibatch is sampled by selecting portions of data from the second existing labeled training data and the second newly labeled training data at specific ratios. The second existing labeled training data may include multiple second existing labeled images, while the second newly labeled training data can be generated by labeling multiple first reconstructed images included in the first learning dataset and multiple augmented images included in the second learning dataset.

[0096] Next, the continuous learning server 2000 can input the second mini-batch into the existing third classifier model, so that the existing third classifier model can generate the second existing loss corresponding to the second existing labeled learning data by referring to the ground truth corresponding to the second existing labeled learning data, and generate the second new loss corresponding to the second new labeled learning data by referring to the ground truth corresponding to the second new labeled learning data.

[0097] Then, the continuous learning server 2000 (i) learns the second new loss through back-propagation in each iteration of the continuous learning, and (ii) learns the second existing loss only through back-propagation for the partial iterations in which the second existing loss is greater than the second basic loss, to complete the continuous learning of the third classifier 2200.

[0098] That is, in this continuous learning method, the second set of newly labeled learning data is always reflected in the continuous learning of the third classifier 2200, while only a portion of the second set of existing labeled learning data is reflected in the continuous learning of the third classifier 2200 when performance is poor. This allows for the training of a new third classifier model that achieves almost the same performance as the existing third classifier model on the second set of existing labeled learning data, and better performance than the existing third classifier model on the second set of newly labeled learning data. Specifically, since the second set of existing labeled learning data can help improve the performance of the new third classifier model, it can be reflected in the continuous learning of the third classifier 2200.

[0099] Next, see you again. Figure 2 The continuous learning server 2000 uses methods such as OTA (over-the-air) to transmit at least one parameter updated in the continuously learned third classifier 2200 to the client 1000, so that the client 1000 can use the updated multiple parameters transmitted from the continuous learning server 2000 to update the client 1000's first classifier 1100.

[0100] That is, see Figure 5 Since the continuous learning server 2000 only needs to transmit at least one update information for updated parameters, updated neural network layers, and updated categories to the client 1000 through continuous learning in the third classifier 2200, more efficient data transmission can be achieved. The client 1000 can also improve efficiency by using only the received information for updated parameters, updated neural network layers, and updated categories.

[0101] The continuous learning server 2000 enables the client 1000 to update the first classifier 1100 by selectively updating at least a portion of the neural network layers constituting the first classifier 1100 with reference to update information about the updated multiple parameters; adding at least one new neural network layer with reference to update information about the updated neural network layers; and adding at least one category with reference to update information about the updated categories.

[0102] Thus, this invention proposes a structure and method for an adversarial autoencoder (AAE) that performs classification, reconstruction, and enhancement processing on multiple first-order images. In this regard, an overall framework is proposed that can train multiple classifiers and efficiently transmit only updated data.

[0103] The embodiments described above according to the present invention can be implemented in the form of program instructions executable by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include individual or combined program instructions, data files, data structures, etc. The program instructions recorded in the computer-readable recording medium may be specifically designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floppy disks; and hardware devices specifically configured for storing and executing program instructions, such as ROMs, RAMs, flash memory, etc. Examples of program instructions include not only machine language code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter, etc. The hardware device may be configured to operate as at least one software module to perform the processing according to the present invention, and vice versa.

[0104] In the foregoing, the invention has been described with reference to specific matters such as particular components, as well as limited embodiments and accompanying drawings. However, this is only to help to understand the invention more fully, and the invention is not limited to the embodiments described above. Various modifications and variations can be designed by those skilled in the art based on these descriptions.

[0105] Therefore, the spirit of the present invention should not be limited to the above embodiments. Except for the appended claims, all modifications that are equivalent or analogous to these claims should be included within the spirit and scope of the present invention.

Claims

1. A method for continuously learning a classifier for images from a classification client using a continuous learning server, wherein, include: (a) When the client receives multiple first classification information corresponding to each acquired image from the first classifier located on the client, and the multiple first classification information corresponding to each acquired image is determined to be multiple first-difficulty images that the first classifier cannot classify, the continuous learning server performs or supports performing the following processing: (i) inputting each first-difficulty image into an adversarial autoencoder, so that the adversarial autoencoder (i-1) encodes each first-difficulty image through the encoder included in the adversarial autoencoder to output each latent vector, (i-2) decoding each latent vector through the decoder included in the adversarial autoencoder to output each reconstructed image corresponding to each first-difficulty image, (i-3) outputting multiple attribute information about whether each reconstructed image is true or false through the discriminator included in the adversarial autoencoder; (i-4) outputting multiple second classification information about each latent vector through the second classifier included in the adversarial autoencoder; (ii) Determine whether the multiple reconstructed images are second-difficult images that the adversarial autoencoder struggles to discriminate by referring to the attribute information and the second classification information; (ii-1) Store the multiple first reconstructed images that are judged to be the second-difficult images in the first learning dataset; (ii-2) Randomly adjust the multiple latent vectors corresponding to each second reconstructed image that is judged not to be the second-difficult image by the decoder to generate multiple enhanced images and store them in the second learning dataset. (b) The continuous learning server performs or supports the following process: continuously learning a third classifier located on the continuous learning server and corresponding to the first classifier using the first learning dataset and the second learning dataset; as well as (c) The continuous learning server performs or supports performing the following process: transmitting at least one updated parameter of the learned third classifier to the client so that the client updates the first classifier using the updated plurality of said parameters.

2. The method according to claim 1, characterized in that: Before (c), The continuous learning server performs or supports the following processing: continuously learning the adversarial autoencoder using (i) first existing labeled learning data including multiple first existing labeled images and (ii) first new labeled learning data obtained by labeling each first reconstructed image in the first learning dataset.

3. The method according to claim 2, characterized in that: The continuous learning server performs or supports performing the following process: performing continuous learning against the adversarial autoencoder, wherein the autoencoder constituting the adversarial autoencoder and the discriminator are learned alternately, the autoencoder including the encoder and the decoder. The continuous learning server performs or supports performing the following processes: (i) for each of the autoencoder and the discriminator, using either an existing autoencoder model previously trained with the first existing labeled learning data or an existing discriminator model previously trained with the first existing labeled learning data to obtain a first basic loss as the average loss for the first existing labeled learning data; (ii) for each iteration of the continuous learning, after sampling a first mini-batch by selecting portions of data from the first existing labeled learning data and the first new labeled learning data at specific ratios, respectively; and (iii) inputting the first mini-batch into the existing autoencoder. The model or the existing discriminator model is configured such that the existing autoencoder model or the existing discriminator model generates a first existing loss corresponding to the first existing labeled learning data with reference to the real data corresponding to the first existing labeled learning data, and generates a first new loss corresponding to the first new labeled learning data with reference to the real data corresponding to the first new labeled learning data, (iv) (iv-1) For the first new loss, it is learned through backpropagation in each iteration of the continuous learning, (iv-2) For the first existing loss, it is learned through backpropagation only for the part of the iterations where the first existing loss is greater than the first basic loss.

4. The method according to claim 1, characterized in that: In step (a), The continuous learning server performs or supports the following processing: (i) when the discriminator determines that multiple enhanced images are true, and the second_1 classification information generated for the multiple enhanced images is the same as the second_2 classification information generated for the multiple second reconstructed images corresponding to the multiple enhanced images, the multiple enhanced images are stored in the second learning dataset; (ii) If the discriminator determines that multiple augmented images are false, or if the second classification information generated for multiple augmented images is different from the second classification information generated for multiple second reconstructed images corresponding to multiple augmented images, the multiple augmented images shall not be stored in the second learning dataset.

5. The method according to claim 1, characterized in that: In step (a), The continuous learning server performs or supports the following processing: (i) when the discriminator determines that multiple reconstructed images are false, or when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is less than a first threshold preset by the second classifier, the reconstructed image is determined to be the second difficult image; (ii) when the discriminator determines that multiple reconstructed images are true, and when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is greater than a second threshold preset by the second classifier, the reconstructed image is determined to be not the second difficult image.

6. The method according to claim 1, characterized in that: In step (a), The continuous learning server performs or supports the following processing: (i) when the deviation of the probability distribution of each category of the first classification information generated by the first classifier for multiple acquired images is less than a first threshold preset by the first classifier, the acquired image is determined to be the first difficult image; (ii) If the deviation of the probability distribution of each category of the first classification information generated by the first classifier for multiple acquired images is greater than a second threshold preset by the first classifier, the acquired image is determined to be not the first difficult image.

7. The method according to claim 1, characterized in that: The continuous learning server performs or supports the following process: the encoder generates multiple latent vectors representing multiple feature values ​​corresponding to each of the first difficult images by downsampling the image by applying at least one convolution operation and at least one pooling operation to each of the first difficult images.

8. The method according to claim 1, characterized in that: The continuous learning server performs or supports the following process: the decoder generates multiple reconstructed images corresponding to each of the first difficult images by upsampling each of the latent vectors by applying at least one deconvolution operation and at least one unpooling operation.

9. The method according to claim 1, characterized in that: In step (b), The continuous learning server performs or supports the following processing: in the method of continuous learning of the third classifier, (i) a second base loss is obtained as the average loss for the second existing labeled learning data using an existing third classifier model trained with second existing labeled learning data including multiple second existing labeled images; (ii) for each iteration of the continuous learning, a second mini-batch is sampled by selecting portions of data from the second existing labeled learning data and the second new labeled learning data at specific ratios, wherein the second new labeled learning data is obtained by processing multiple first reconstructed images included in the first learning dataset and multiple augmented images included in the second learning dataset. The image is labeled and generated, and then (iii) the second mini-batch is input into the existing third classifier model, so that the existing third classifier model generates a second existing loss corresponding to the second existing labeled learning data with reference to the real data corresponding to the second existing labeled learning data, and generates a second new loss corresponding to the second new labeled learning data with reference to the real data corresponding to the second new labeled learning data. (iv) (iv-1) For the second new loss, it is learned through backpropagation in each iteration of the continuous learning. (iv-2) For the second existing loss, it is learned through backpropagation only for the part of the second existing loss that is greater than the second basic loss.

10. The method according to claim 9, characterized in that: The continuous learning server performs or supports performing the following process: transmitting only update information about at least one updated parameter, updated neural network layer, and updated category to the first classifier of the client, so that the client updates the first classifier using only the received update information about the updated parameters, updated neural network layers, and updated categories.

11. The method according to claim 10, characterized in that: The continuous learning server performs or supports performing the following processes: causing the client to (i) selectively update at least a portion of the neural network layers constituting the first classifier with reference to the update information regarding the updated plurality of parameters; and (ii) add at least one new neural network layer with reference to the update information regarding the updated neural network layers. (iii) Updating the first classifier by adding at least one of the processes of at least one category with reference to the update information regarding the updated category.

12. A continuous learning server for continuously learning a classifier for classifying images of a classification client using a continuous learning server, wherein, include: At least one memory that stores multiple instructions; and At least one processor for executing the plurality of said instructions, The processor executes or supports the following processing: (I) When receiving multiple first classification information corresponding to each acquired image from the first classifier located on the client, and multiple first-difficulty images that are determined to be unclassifiable by the first classifier based on the multiple first classification information corresponding to each acquired image, (i) each first-difficulty image is input into an adversarial autoencoder, so that the adversarial autoencoder (i-1) encodes each first-difficulty image through the encoder included in the adversarial autoencoder to output each latent vector, (i-2) decodes each latent vector through the decoder included in the adversarial autoencoder to output each reconstructed image corresponding to each first-difficulty image, (i-3) outputs multiple attribute information about whether each reconstructed image is true or false through the discriminator included in the adversarial autoencoder; (i-4) outputs multiple second classification information about each latent vector through the second classifier included in the adversarial autoencoder. (ii) Determine whether the multiple reconstructed images are second-difficult images that the adversarial autoencoder struggles to discriminate by referring to the attribute information and the second classification information; (ii-1) Store the multiple first reconstructed images that are judged to be the second-difficult images in the first learning dataset; (ii-2) Randomly adjust the multiple latent vectors corresponding to each second reconstructed image that is judged not to be the second-difficult image by the decoder to generate multiple enhanced images and store them in the second learning dataset; (ii) Continuously learn the third classifier located in the processor and corresponding to the first classifier using the first learning dataset and the second learning dataset. (III) Transmit at least one updated parameter of the learned third classifier to the client so that the client updates the first classifier using the updated plurality of the parameters.

13. The continuous learning server according to claim 12, characterized in that: Prior to the (III) process, The processor performs or supports the following processing: continuously learning the adversarial autoencoder using (i) first existing labeled learning data including multiple first existing labeled images and (ii) first new labeled learning data obtained by labeling each of the first reconstructed images included in the first learning dataset.

14. The continuous learning server according to claim 13, characterized in that: The processor performs or supports performing the following process: performing continuous learning for the adversarial autoencoder, wherein the learning is performed alternately on the autoencoder constituting the adversarial autoencoder and the discriminator, the autoencoder including the encoder and the decoder. The processor performs or supports performing the following processing: for each of the autoencoder and the discriminator, (i) using an existing autoencoder model previously trained with the first existing labeled learning data or an existing discriminator model previously trained with the first existing labeled learning data to obtain a first basic loss as the average loss for the first existing labeled learning data; (ii) for each iteration of the continuous learning, after sampling a first mini-batch by selecting portions of data from the first existing labeled learning data and the first new labeled learning data at specific ratios respectively; and (iii) inputting the first mini-batch into the existing autoencoder model or... The existing discriminator model is configured such that the existing autoencoder model or the existing discriminator model generates a first existing loss corresponding to the first existing labeled learning data with reference to the real data corresponding to the first existing labeled learning data, and generates a first new loss corresponding to the first new labeled learning data with reference to the real data corresponding to the first new labeled learning data. (iv) (iv-1) For the first new loss, it is learned through backpropagation in each iteration of the continuous learning. (iv-2) For the first existing loss, it is learned through backpropagation only for the part of the iterations where the first existing loss is greater than the first basic loss.

15. The continuous learning server according to claim 12, characterized in that: In the process described in (I), The processor performs or supports performing the following process: (i) when the discriminator determines that multiple enhanced images are true, and the second_1 classification information generated for the multiple enhanced images is the same as the second_2 classification information generated for the multiple second reconstructed images corresponding to the multiple enhanced images, the multiple enhanced images are stored in the second learning dataset; (ii) If the discriminator determines that multiple augmented images are false, or if the second classification information generated for multiple augmented images is different from the second classification information generated for multiple second reconstructed images corresponding to multiple augmented images, the multiple augmented images shall not be stored in the second learning dataset.

16. The continuous learning server according to claim 12, characterized in that: In the process described in (I), The processor performs or supports performing the following processing: (i) when the discriminator determines that multiple reconstructed images are false, or when the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is less than a first threshold preset by the second classifier, the reconstructed image is determined to be the second difficult image; (ii) When the discriminator determines that multiple reconstructed images are true, and it is determined that the deviation of the probability distribution of each category of the second classification information generated by the second classifier for multiple reconstructed images is greater than a second threshold preset by the second classifier, the reconstructed image is determined to be not the second difficult image.

17. The continuous learning server according to claim 12, characterized in that: In the process described in (I), The processor performs or supports performing the following processing: (i) when the deviation of the probability distribution of each category of the first classification information generated by the first classifier for multiple acquired images is less than a first threshold preset by the first classifier, the acquired image is determined to be the first difficult image; (ii) If the deviation of the probability distribution of each category of the first classification information generated by the first classifier for multiple acquired images is greater than a second threshold preset by the first classifier, the acquired image is determined to be not the first difficult image.

18. The continuous learning server according to claim 12, characterized in that: The processor performs or supports performing the following process: causing the encoder to generate multiple latent vectors representing multiple feature values ​​corresponding to each of the first difficult images by downsampling the image by applying at least one convolution operation and at least one pooling operation to each of the first difficult images.

19. The continuous learning server according to claim 12, characterized in that: The processor performs or supports performing the following process: causing the decoder to generate multiple reconstructed images corresponding to each of the first hard images by upsampling each of the latent vectors by applying at least one deconvolution operation and at least one unpooling operation.

20. The continuous learning server according to claim 12, characterized in that: In the process described in (II), The processor performs or supports the following processing: in a method for continuous learning of a third classifier, (i) a second base loss is obtained as the average loss for the second existing labeled learning data using an existing third classifier model trained with second existing labeled learning data including multiple second existing labeled images; (ii) for each iteration of the continuous learning, a second mini-batch is sampled by selecting portions of data from the second existing labeled learning data and the second new labeled learning data at specific ratios, wherein the second new labeled learning data is obtained by processing multiple first reconstructed images included in the first learning dataset and multiple enhanced images included in the second learning dataset. The row labels are generated, and then (iii) the second mini-batch is input into the existing third classifier model, so that the existing third classifier model generates the second existing loss corresponding to the second existing label learning data with reference to the real data corresponding to the second existing label learning data, and generates the second new loss corresponding to the second new label learning data with reference to the real data corresponding to the second new label learning data. (iv) (iv-1) For the second new loss, it is learned through backpropagation in each iteration of the continuous learning. (iv-2) For the second existing loss, it is learned through backpropagation only for the part of the iteration where the second existing loss is greater than the second basic loss.

21. The continuous learning server according to claim 20, characterized in that: The processor performs or supports performing the following process: transmitting only update information about at least one updated parameter, updated neural network layer, and updated category to the first classifier of the client, so that the client updates the first classifier using only the received update information about the updated multiple parameters, updated multiple neural network layers, and updated multiple multiple categories.

22. The continuous learning server according to claim 21, characterized in that: The processor performs or supports performing the following processes: causing the client to (i) selectively update at least a portion of the neural network layers constituting the first classifier with reference to the update information regarding the updated plurality of parameters; and (ii) add at least one new neural network layer with reference to the update information regarding the updated neural network layers. (iii) Updating the first classifier by adding at least one of the processes of at least one category with reference to the update information regarding the updated category.

Images