Learning system

Abstract

To provide a learning system which can prevent increase in a cost due to image transmission or reduce a leakage risk of images.SOLUTION: A learning system 1 includes a plurality of client terminals 11 and a server 12. Each of the client terminals 11 is configured to learn a first discriminator and transmits the first discriminator to a server 12, the first discriminator being configured to identify a distribution of a first feature quantity extracted by a feature extractor from a client image and a distribution of a second feature quantity extracted from a first proxy image obtained by performing a predetermined process on the client image. The server 12 is configured to learn the feature extractor on the basis of the first discriminator and learn the first discriminator by machine learning using third feature quantity so that a distribution of a third feature quantity extracted by the feature extractor from a server image may be approximate to the distribution of the first feature quantity.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This disclosure relates to a learning system. [Background technology]

[0002] Patent Document 1 discloses a technique for synchronizing the learning model on each client terminal with the master model on the integration server before each of the multiple client terminals begins learning. Each client terminal performs machine learning on the learning model using data stored in the medical institution and sends the learning results to the integration server. The integration server divides the multiple client terminals into multiple client clusters and integrates the learning results for each client cluster to create a candidate master model. The integration server evaluates the inference accuracy of each candidate master model and extracts the client terminals that are causing the accuracy to deteriorate. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] International Publication No. 2021 / 059607 [Overview of the project] [Problems that the invention aims to solve]

[0004] Patent Document 1 requires the cost of transmitting and storing data stored in medical institutions on client terminals. Furthermore, there is the problem of the risk of image leakage.

[0005] This disclosure was made in light of these issues, and aims to realize a learning system that can reduce the increased costs associated with image transmission and the risk of image leakage. [Means for solving the problem]

[0006] A learning system according to one aspect of the present disclosure is a learning system comprising a plurality of client terminals and a server, wherein each client terminal learns a first classifier that distinguishes between the distribution of first features extracted from a client image by a feature extractor and the distribution of second features extracted from a first proxy image obtained by performing a predetermined process on the client image, transmits the first classifier to the server, and the server learns the feature extractor based on the first classifier such that the distribution of third features extracted from a server image by the feature extractor approaches the distribution of the first features, and learns a first classifier by machine learning using the third features. [Effects of the Invention]

[0007] According to this disclosure, it is possible to realize a learning system that can reduce the increased costs associated with image transmission and the risk of image leakage. [Brief explanation of the drawing]

[0008] [Figure 1] This diagram illustrates a learning system that performs domain adaptation. [Figure 2] This diagram illustrates the configuration of the learning system according to Embodiment 1. [Figure 3] This figure illustrates the learning method for the feature extractor according to Embodiment 1. [Modes for carrying out the invention]

[0009] The following describes specific embodiments applying this disclosure with reference to the drawings. However, this disclosure is not limited to the following embodiments. Also, for clarity, the following description and drawings have been simplified as appropriate.

[0010] Background leading to this disclosure First, a general federated learning method will be described. In federated learning, a server learns a machine learning model (e.g., a model for object recognition) using images collected from the web or images generated from 3D models (also referred to as simulators), and distributes the initial values of the weight coefficients of the machine learning model to a plurality of client terminals. The learning images (referred to as server images) are also called source images.

[0011] Each client terminal collects client images (also referred to as target images) indoors or the like, updates the weight coefficients of the machine learning model with the client images, and transmits the weight coefficients of the machine learning model to the server. Next, the server integrates (updates) the weight coefficients received from the plurality of client terminals, and distributes the updated weight coefficients of the machine learning model to the plurality of client terminals.

[0012] In general federated learning, there is a problem that the domains of server images and client images are different, and the accuracy of the machine learning model decreases. Therefore, domain adaptation needs to be performed. However, transmitting client images to the server has problems from the perspective of privacy, so it is necessary to transmit server images to the client terminals.

[0013] FIG. 1 is a diagram for explaining a learning system 2 that performs domain adaptation. The client terminal 11 extracts features from the client image x n n and the server image x s using a feature extractor F g and learns an identifier D t n for identifying the client image x n and transmits the identifier D n to the server 12. n is any integer from 1 to N, and N represents the number of client terminals 11. The server 12 learns the feature extractor F s so that the distribution of the features of the server image x n (n = 1 to N) approaches the distribution of the features of the client image x g and at the same time a classifier C g(Also called a classifier, corresponding to the above machine learning model) is trained, and a feature extractor F is created. g and classifier C g Send this to client terminal 11.

[0014] In learning system 2, server image x s The distribution of features in client image x t n The feature extractor F is designed to approximate the distribution of features. g Domain adaptation is performed by learning from the server image x. s The data is sent to the client terminal 11, and the identifier D n It is necessary to learn the server image x. s It was necessary to send this to all client terminals 11, but there were problems in terms of licensing and data management. The inventor of the present invention came up with the learning system according to Embodiment 1 based on the above circumstances.

[0015] Embodiment 1 The learning system according to Embodiment 1 will be described below with reference to the drawings. Figure 2 is a diagram illustrating the configuration of the learning system 1 according to Embodiment 1. The learning system 1 uses a server image x s Domain adaptation is performed using a proxy image as an alternative.

[0016] Client terminal 11 displays client image x t n Proxy image T after performing the prescribed processing ps (x t n Generates the first proxy image. ps This represents a predetermined process. The content of the predetermined process is arbitrary and can include, for example, a process to generate a random noise image, a process to add adversarial noise, a process to convert an RGB image to a BGR image, a process called dropout, a process to fractalize, a process to convert to a grayscale image, a rotation process, a process to add Gaussian noise, a process to divide into patches and swap them, or a process to overlay with other images (called mixup).

[0017] Client terminal 11 is a feature extractor F g by client image x t n Extract features (referred to as the first feature) from the image, and create a proxy image T. ps (x t n ) extracts a feature (called the second feature). n is an integer from 1 to N, where N represents the number of client terminals 11. Client image x t n The processing related to this is represented by a solid line, and the proxy image T ps (x t n The processing related to ) is shown by a dotted line. The client terminal 11 is a classifier D that identifies the distribution of the first feature and the distribution of the second feature according to equation (1). n k (Also called the first classifier) ​​is trained, and classifier D n k Send to server 12. Identifier D n k The number of these may be multiple. Also, for use in the knowledge distillation described below, discriminator D n k This may be sent to other client terminals 11.

number

[0018] Furthermore, the client terminal 11 is connected to classifier C, which will be described later. g Through knowledge distillation using (also called the first classifier), classifier C can be created through unsupervised learning. n The client terminal 11 learns the classifier C (also called the second classifier). Specifically, the client terminal 11 learns the classifier C according to equation (2). n Learn D s This is the discriminator D, which will be described later. s This represents the (also called the second classifier). p is a hyperparameter. KL represents the Kullback-Leibler divergence. V is the feature extractor F. g This represents the extracted features. m represents an integer other than n from 1 to N. K is the classifier D. mk This represents the number of [number]. As mentioned above, classifier D m k The number of items can be multiple.

number

[0019] Classifier C n This is the discriminator D s and discriminator D m k This allows us to consider the similarity of the feature distributions while also considering other classifiers C. g and C m Classifier C is trained to produce classification results that are as close as possible to the original classification. n This is a classifier optimized for each client terminal 11, but classifier C n Without learning, classifier C as shown in Figure 1 (related technology) g Classifier C may be used. n This is sent to other client terminals 11.

[0020] Server 12 is server image x s Feature extractor F g The server extracts a feature (referred to as the third feature) using the classifier D. Server 12 adjusts the distribution of the third feature so that it approaches the distribution of the first feature. n k Based on this, feature extractor F g Classifier C is created by learning and using machine learning with a third feature. g Server 12 learns the feature extractor F. g and classifier C g Send this to client terminal 11.

[0021] Specifically, the server 12 uses a feature extractor F according to equation (3). g and classifier C g Learn L ce This represents the loss function using cross-entropy, and y s is server image x s This represents the label assigned to λ. sis a hyperparameter. The first term in Equation (3) represents ordinary supervised learning, and the second term represents the alignment of the feature distribution (space). [Number]

[0022] FIG. 3 is a diagram for explaining a method by which the server 12 learns the feature extractor F g The distribution C1 surrounded by the solid line represents the distribution of the feature amounts (first feature amounts) extracted from the client image x t n . The distribution C2 surrounded by the dotted line represents the distribution of the feature amounts (second feature amounts) extracted from the proxy image T ps (x t n ). The symbol L represents a classification line for classifying the distribution C1 and the distribution C2, and corresponds to the discriminator D n k . The distribution S surrounded by the dashed-dotted line represents the distribution of the feature amounts (third feature amounts) extracted from the server image x s . The feature extractor F g learns so that the distribution S approaches the distribution C1. In FIG. 1 showing the related art, a classification line (discriminator) for classifying the distribution C1 and the distribution S is learned, and the feature extractor F g is learned so that the distribution S approaches the distribution C1. However, in Embodiment 1, a classification line for classifying the distribution C1 and the distribution C2 is learned, and the feature extractor F g is learned so that the distribution S approaches the distribution C1. Therefore, in Embodiment 1, it is not necessary to transmit the server image x s to the client terminal 11.

[0023] Returning to FIG. 2, the server 12 also generates a proxy image T s (referred to as a second proxy image) obtained by performing the above-described predetermined process on the server image x ps (x s ). The processing related to the server image x s is represented by a solid line, and the processing related to the proxy image T ps (x s ) is represented by a dotted line. The server 12 uses the proxy image Tps (x s ) extracts a feature (referred to as the fourth feature). Server 12 uses a classifier D to distinguish between the distribution of the third feature and the distribution of the fourth feature according to equation (4). s Learns and sends to client terminal 11. Classifier D s This is used in the distillation of knowledge described above.

number

[0024] By repeating the process represented by equation (1), the process represented by equation (2), the process represented by equation (3), and equation (4), classifier C is generated. n Classifier C g It can improve performance.

[0025] In conventional technology, the client terminal 11 receives the server image x s It was necessary to receive and perform domain adaptation. On the other hand, in Embodiment 1, the server image x s Instead of proxy image T ps (x t n ) is used, so server image x s There is no need to send the server image x to the client terminal 11. s While preventing leakage, classifier C based on domain differences g This reduces the performance degradation of the server image x. s Reduce the cost of sending and storing server images x s This reduces the risk of users leaking information.

[0026] The program described above includes a set of instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrically, optically, acoustically or otherwise propagating signals.

[0027] This disclosure is not limited to the embodiments described above, and may be modified as appropriate without departing from its spirit. [Explanation of symbols]

[0028] 1, 2 Learning system, 11 Client terminal, 12 Server, x t n Client image, x s Server image, T ps (x t n ), T ps (x s ) Proxy image, F g Feature extractor, D n k , D s Discriminator, C n , C g classifier

Claims

[Claim 1] A learning system comprising multiple client terminals and a server, Each client terminal learns a first classifier that distinguishes between the distribution of first features extracted from a client image by a feature extractor and the distribution of second features extracted from a first proxy image obtained by performing a predetermined process on the client image, and transmits the first classifier to the server. The server learns the feature extractor based on the first classifier and learns the first classifier by machine learning using the third features, such that the distribution of the third features extracted from the server image by the feature extractor approaches the distribution of the first features. Learning system.

Images