Learning device, learning method, program, and classification device

Abstract

In order to train a machine learning model such that the inference accuracy becomes higher, a learning device (1): comprises an acquisition unit (11) for acquiring a learning image and a learning unit (12) for training a machine learning model by using the learning image; sets, as a reference positive example, a learning image belonging to a subclass different from a subclass to which an image of interest belongs, and belonging to a class same as a class to which the image of interest belongs; sets, as a negative example, a learning image belonging to a class different from a class to which the image of interest belongs; and trains the machine learning model such that the similarity between a feature amount of the image of interest and a feature amount of the reference positive example becomes greater than the similarity between the feature amount of the image of interest and a feature amount of the negative example.

Description

Technical Field

[0001] The present invention relates to a learning device, a learning method, a program for training a machine learning model (neural network) that performs an image recognition task based on image feature amounts, and a classification device that classifies images.

Background Art

[0002] There is disclosed a method for training a machine learning model that takes an image as an input and outputs an inference result according to the features of the subject included in the image.

[0003] Patent Document 1 discloses a method for training a machine learning model using data related to disease risks as training data with a biological image such as a fundus image as an input.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the learning method as described in Patent Document 1, cross entropy is usually used as a loss function, and learning is performed to minimize the error between the output from the machine learning model and the correct data. As a learning method capable of obtaining invariant feature amounts that are less dependent on the type of image recognition task than this method, contrastive learning is known. In contrastive learning, the machine learning model is trained so that the inner product of the feature vectors of the anchor, which is the target image, and the positive example becomes large, and the inner product of the feature vectors of the anchor and the negative example becomes small.

[0006] Here, as an example, let's consider a case where a machine learning model that infers whether sample cells included in an image are benign or malignant is trained using controlled learning. In this case, images in which the type of sample cells included as the subject is the same as the type of cells included as the subject in the anchor image are considered positive examples. Conversely, images in which the type of sample cells is different from the type of cells included as the subject in the anchor image are considered negative examples. Furthermore, it is desirable to also consider images that have undergone arbitrary image augmentation (data augmentation) such as rotation, vertical / horizontal flipping, scaling, color shifting, and cropping as positive examples.

[0007] In this case, even if the type of sample cell included in the image as the subject is different from the type of benign (or malignant) cell included as the subject in the anchor image, if the sample cell is benign (or malignant), it will still be considered a negative example. That is, even if the cell subcategory (adenocarcinoma, squamous cell carcinoma) is different, but the cell major category (benign or malignant) is the same, it will still be considered a negative example. Such negative examples may have partially common features between the sample cell and the cells included as subjects in the anchor image. However, conventional controlled learning does not anticipate such cases, which leads to the problem that it is not possible to improve inference accuracy.

[0008] One aspect of the present invention has been made in view of the above-mentioned problems, and one example of its objective is to provide a technique for training a machine learning model to improve its inference accuracy. [Means for solving the problem]

[0009] A learning device according to one aspect of the present invention is a learning device that generates feature quantities of an input image and trains a machine learning model that performs an image recognition task, comprising: acquisition means for acquiring a plurality of training images; and learning means for training the machine learning model using the plurality of training images, wherein each training image belongs to one of a plurality of classes classified according to the features of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the features of the subject, and the learning means provides a training image belonging to the same subclass as the subclass to which the focus image belongs to, for a focus image included in the plurality of training images. The machine learning model is trained such that a positive example is used, a training image belonging to a subclass different from the subclass to which the image of interest belongs, but belonging to the same class as the image of interest, is used as a quasi-positive example, and a training image belonging to a class different from the class to which the image of interest belongs is used as a negative example. The machine learning model is trained such that the similarity between the features of the image of interest and the features of the positive example increases, the similarity between the features of the image of interest and the features of the quasi-positive example decreases, the similarity between the features of the image of interest and the features of the negative example decreases, and the similarity between the features of the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example.

[0010] A learning method according to one aspect of the present invention is a learning method executed by a learning device that generates feature quantities of an input image and trains a machine learning model that performs an image recognition task, comprising: acquiring a plurality of training images; and training the machine learning model using the plurality of training images, wherein each training image belongs to one of a plurality of classes classified according to the features of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the features of the subject, and in the training, a focus image included in the plurality of training images belongs to the same subclass as the subclass to which the focus image belongs The machine learning model is trained such that a training image is used as a positive example, a training image belonging to a different subclass from the subclass to which the image of interest belongs, but belonging to the same class as the image of interest, is used as a quasi-positive example, and a training image belonging to a different class from the class to which the image of interest belongs is used as a negative example. The similarity between the features of the image of interest and the features of the positive examples increases, the similarity between the features of the features of the quasi-positive examples decreases, the similarity between the features of the features of the image of interest and the features of the negative examples decreases, and the similarity between the features of the features of the quasi-positive examples is greater than the similarity between the features of the features of the image of interest and the features of the negative examples.

[0011] A program relating to one aspect of the present invention is a program that causes a computer to function as a learning device for generating feature quantities of an input image and training a machine learning model to perform an image recognition task, wherein the program causes the computer to function as an acquisition means for acquiring a plurality of training images and a learning means for training the machine learning model using the plurality of training images, wherein each training image belongs to one of a plurality of classes classified according to the features of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the features of the subject, and the learning means determines the identity of the image of interest contained in the plurality of training images, and the identity of the image of interest belongs to the identity of the image of interest. The machine learning model is trained such that a training image belonging to the same subclass as the subclass is used as a positive example, a training image belonging to a different subclass from the subclass to which the image of interest belongs, but belonging to the same class as the image of interest, is used as a quasi-positive example, and a training image belonging to a different class from the class to which the image of interest belongs is used as a negative example. The model is trained so that the similarity between the features of the image of interest and the features of the positive examples increases, the similarity between the features of the images of interest and the features of the quasi-positive examples decreases, the similarity between the features of the images of interest and the features of the negative examples decreases, and the similarity between the features of the images of interest and the features of the quasi-positive examples is greater than the similarity between the features of the images of interest and the features of the negative examples. [Effects of the Invention]

[0012] According to one aspect of the present invention, a machine learning model can be trained to improve inference accuracy. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram showing the configuration of a learning device according to exemplary embodiment 1 of the present invention. [Figure 2] This is a flowchart showing the flow of the learning method according to exemplary embodiment 1 of the present invention. [Figure 3] This figure shows an example of a class and subclass in Exemplary Embodiment 2 of the present invention. [Figure 4] This figure shows an example of a class, middle class, and subclass in Exemplary Embodiment 2 of the present invention. [Figure 5] This is a block diagram showing the configuration of a learning device according to an exemplary embodiment 2 of the present invention. [Figure 6] This is a block diagram showing the configuration of the classification unit 22 according to exemplary embodiment 2 of the present invention. [Figure 7] This is a block diagram showing an example of the hardware configuration of a learning device according to each exemplary embodiment of the present invention. [Modes for carrying out the invention]

[0014] [Exemplary Embodiment 1] A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of the exemplary embodiments described later.

[0015] (Overview of Learning Device 1) The learning device 1 according to this exemplary embodiment is a learning device that takes an image as input, generates feature quantities of the input image, and trains a machine learning model that performs an image recognition task, outputting the result of recognizing the image based on the feature quantities. An example of an image recognition task is a classification task that identifies which of the predefined labels an object included as a subject in an image belongs to. The machine learning model is, as an example, a neural network.

[0016] In this case, each training image used by the learning device 1 to train the machine learning model belongs to one of several classes classified according to the features of the subject contained in the training image. Furthermore, each training image belongs to one of several subclasses obtained by further classifying each of the multiple classes according to the features of the subject.

[0017] As an example, the learning device 1 trains a machine learning model by contrast learning. In this case, the learning device 1 selects an arbitrary target image (anchor) from the learning images. Depending on the class and subclass to which the selected target image belongs and the class and subclass to which each learning image belongs, each learning image is classified into one of the following: positive example (positive sample), semi-positive example, and negative example (negative sample).

[0018] For the target image, a learning image belonging to the same subclass as the subclass to which the target image belongs is defined as a positive example.

[0019] For the target image, a learning image belonging to a subclass different from the subclass to which the target image belongs and belonging to the same class as the class to which the target image belongs is defined as a semi-positive example.

[0020] For the target image, a learning image belonging to a class different from the class to which the target image belongs is defined as a negative example.

[0021] The learning device 1 trains the machine learning model according to whether the learning image is a positive example, a semi-positive example, or a negative example. As an example, the learning device 1 trains the machine learning model using a loss function whose contribution changes according to whether the learning image is a positive example, a semi-positive example, or a negative example. The method by which the learning device 1 trains the machine learning model will be described later.

[0022] (Configuration of the learning device 1) The configuration of the learning device 1 according to this exemplary embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the learning device 1 according to this exemplary embodiment.

[0023] As shown in FIG. 1, the learning device 1 includes an acquisition unit 11 and a learning unit 12. The acquisition unit 11 and the learning unit 12 are configurations that respectively implement acquisition means and learning means in this exemplary embodiment.

[0024] The acquisition unit 11 acquires multiple training images. The acquisition unit 11 supplies the acquired training images to the learning unit 12.

[0025] The learning unit 12 trains a machine learning model using multiple training images acquired by the acquisition unit 11.

[0026] Specifically, the learning unit 12 trains the machine learning model so that the similarity between the features of the image of interest and the features of positive examples increases, the similarity between the features of the image of interest and the features of near-positive examples decreases, the similarity between the features of the image of interest and the features of negative examples decreases, and the similarity between the features of the image of interest and the features of near-positive examples is greater than the similarity between the features of the image of interest and the features of negative examples.

[0027] As described above, the learning device 1 according to this exemplary embodiment is a learning device 1 that generates feature quantities of an input image and trains a machine learning model to perform an image recognition task, and employs a configuration comprising an acquisition unit 11 that acquires a plurality of training images, and a learning unit 12 that trains the machine learning model using the plurality of training images acquired by the acquisition unit 11.

[0028] Furthermore, each training image belongs to one of several classes classified according to the characteristics of the subject contained in the training image, and each of those classes belongs to one of several subclasses further classified according to the characteristics of the subject.

[0029] Furthermore, the learning unit 12 trains the machine learning model as follows: for each of the multiple training images, it uses training images belonging to the same subclass as the subclass to which the image of interest belongs as positive examples, training images belonging to a different subclass from the subclass to which the image of interest belongs, but belonging to the same class as the class to which the image of interest belongs, as quasi-positive examples, and training images belonging to a different class from the class to which the image of interest belongs as negative examples.

[0030] The learning unit 12 trains the machine learning model so that the similarity between the features of the image of interest and the features of the positive examples increases, the similarity between the features of the image of interest and the features of the near-positive examples decreases, the similarity between the features of the image of interest and the features of the negative examples decreases, and the similarity between the features of the image of interest and the features of the near-positive examples is greater than the similarity between the features of the image of interest and the features of the negative examples.

[0031] Therefore, according to the learning device 1 of this exemplary embodiment, a training image belonging to a subclass different from the subclass to which the image of interest belongs, but belonging to the same class as the image of interest, is used as a quasi-positive example. The machine learning model is trained so that the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example. In this way, the learning device 1 of this exemplary embodiment trains the machine learning model while considering quasi-positive examples, resulting in the effect of training the machine learning model to have higher inference accuracy.

[0032] (Learning process flow) The flow of the learning method S1 according to this exemplary embodiment will be explained with reference to Figure 2. Figure 2 is a flowchart showing the flow of the learning method S1 according to this exemplary embodiment.

[0033] (Step S11) In step S11, the acquisition unit 11 acquires multiple training images. The acquisition unit 11 supplies the acquired training images to the learning unit 12.

[0034] (Step S12) In step S12, the learning unit 12 trains a machine learning model using multiple training images acquired by the acquisition unit 11.

[0035] Specifically, in step S12, the learning unit 12 trains the machine learning model so that the similarity between the features of the image of interest and the features of the positive examples increases, the similarity between the features of the image of interest and the features of the near-positive examples decreases, the similarity between the features of the image of interest and the features of the negative examples decreases, and the similarity between the features of the image of interest and the features of the near-positive examples is greater than the similarity between the features of the image of interest and the features of the negative examples.

[0036] As described above, the learning method S1 according to this exemplary embodiment is a learning method S1 executed by a learning device 1 that generates feature quantities of an input image and trains a machine learning model to perform an image recognition task, wherein in step S11, the acquisition unit 11 acquires a plurality of training images, and in step S12, the learning unit 12 trains the machine learning model using the plurality of training images acquired by the acquisition unit 11.

[0037] Furthermore, in step S12, the learning unit 12 trains the machine learning model so that the similarity between the features of the image of interest and the features of the positive examples increases, the similarity between the features of the image of interest and the features of the near-positive examples decreases, the similarity between the features of the image of interest and the features of the negative examples decreases, and the similarity between the features of the image of interest and the features of the near-positive examples is greater than the similarity between the features of the image of interest and the features of the negative examples.

[0038] Therefore, the learning method S1 according to this exemplary embodiment provides the same effects as the learning device 1 described above.

[0039] [Exemplary Embodiment 2] A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Components having the same function as those described in Exemplary Embodiment 1 will be denoted by the same reference numerals, and their descriptions will be omitted as appropriate.

[0040] (Overview of Classification Device 2) The classification device 2 according to this exemplary embodiment includes a second machine learning model which is trained using a first loss function described later. machineThis device uses a second machine learning model, trained with a second loss function (cross-entropy), to classify images containing a subject into classes or subclasses. Classification device 2 can be used, for example, in cytology during rapid on-site evaluation (ROSE).

[0041] The training images belong to one of several classes, which are classified according to the characteristics of the subjects they contain. Furthermore, each of the several classes is further classified into several subclasses, which are obtained by further classifying each of these classes according to the characteristics of the subjects. An example of classes and subclasses is shown in Figure 3. Figure 3 is a diagram showing an example of classes and subclasses in this exemplary embodiment.

[0042] Figure 3 shows that sample cells included as subjects in an image are classified into classes depending on whether they are benign or malignant. Furthermore, Figure 3 shows that sample cells included as subjects in an image are classified into subclasses depending on the type of cell. In Figure 3, for example, the class "benign cells" is further classified into several subclasses: "EC normal epithelial cells," "IEC inflammatory EC," "M macrophages," "LC lymphocytes," and "WBC leukocytes."

[0043] Furthermore, the training images may belong to any of the multiple middle classes, which are obtained by classifying each class into multiple subclasses, and which are groups of one or more subclasses. An example of further classifying the example shown in Figure 3 into middle classes is shown in Figure 4. Figure 4 is a diagram showing an example of a class, middle class, and subclass in this exemplary embodiment.

[0044] In Figure 4, for example, the class "benign cells" is classified into three middle classes: "normal cells," "normal cells with findings," and "other normal cells." Furthermore, in Figure 4, the subclass "EC normal epithelial cells" is grouped within the middle class "normal cells," and the subclass "IEC inflammatory EC" and "M macrophages" are grouped within the middle class "normal cells with findings."

[0045] Furthermore, Classification Device 2 performs the training of the first machine learning model and the second machine learning model. In other words, Classification Device 2 also functions as a learning device. Details of the process by which Classification Device 2 performs the training of the first machine learning model and the second machine learning model will be described later.

[0046] (Configuration of Classification Device 2) Figure 5 is a block diagram showing the configuration of the classification device 2 according to this exemplary embodiment. As shown in Figure 5, the classification device 2 includes a control unit 21, a storage unit 25, a communication unit 26, an input unit 27, and an output unit 28.

[0047] The memory unit 25 stores data that the control unit 21 references. Examples of data stored in the memory unit 25 include training images and training data (training labels).

[0048] The communication unit 26 is a communication module that communicates with other devices connected via a network. For example, the communication unit 26 receives training images and outputs classification results.

[0049] The input unit 27 is an interface for acquiring data from other connected devices. For example, the input unit 27 acquires training images.

[0050] The output unit 28 is an interface that outputs data to other connected devices. For example, the output unit 28 outputs classification results.

[0051] (Control Unit 21) The control unit 21 controls each component of the classification device 2. The control unit 21 also includes an acquisition unit 11, a learning unit 12, and a classification unit 22, as shown in Figure 5. In this exemplary embodiment, the acquisition unit 11, the learning unit 12, and the classification unit 22 are configured to implement the acquisition means, learning means, and classification means, respectively.

[0052] The acquisition unit 11 acquires data via the communication unit 26 or the input unit 27. Examples of data acquired by the acquisition unit 11 include training images and images to be classified. The acquisition unit 11 stores the acquired data in the storage unit 25.

[0053] The learning unit 12 trains the first machine learning model using the first loss function and the second machine learning model using the second loss function. The first machine learning model is an Encoder (feature analysis model) that takes an input image as input and generates features of the input image, and is used as a pre-trained model for the second machine learning model. The second machine learning model is formed by connecting a Classifier (classifier, second layer group) to the Encoder (first layer group), which is the first machine learning model. As shown in Figure 5, the learning unit 12 comprises a first learning unit 12A and a second learning unit 12B. The first learning unit 12A and the second learning unit 12B are , book In an exemplary embodiment, the learning hand Step This is the configuration that will be implemented.

[0054] The first learning unit 12A trains a first machine learning model using a plurality of training images stored in the memory unit 25 and the corresponding teacher labels for each of the plurality of training images. As mentioned above, the first machine learning model is an encoder (feature analysis model) and is used as a pre-training model for the second machine learning model.

[0055] Details of the process by which the first learning unit 12A trains the first machine learning model will be described later.

[0056] The second learning unit 12B trains a second machine learning model using a plurality of training images stored in the memory unit 25 and teacher labels corresponding to the results of classifying each of the plurality of training images into the aforementioned class or subclass. As mentioned above, the second machine learning model is an Encoder, which is the first machine learning model, with a Classifier connected to it. Known methods are used for the method by which the second learning unit 12B trains the second machine learning model and for the loss function (second loss function). One example is a method that minimizes the error between the output data from the second machine learning model and the teacher labels using cross-entropy. The second learning unit 12B mainly trains the Classifier part, but may also train the Encoder part to fine-tune it as needed.

[0057] The classification unit 22 inputs the image to the second machine learning model, thereby classifying the input image into a class or subclass.

[0058] An example of the configuration of the classification unit 22 will be described with reference to Figure 6. Figure 6 is a block diagram showing the configuration of the classification unit 22 according to this exemplary embodiment.

[0059] As shown in Figure 6, the classification unit 22 includes a first machine learning model which is an Encoder (feature analysis model) and a second machine learning model which has a Classifier connected to the Encoder (feature analysis model).

[0060] The first machine learning model takes an input image containing the subject as input and outputs features (feature vectors) optimized by the first loss function.

[0061] The second machine learning model comprises an Encoder and a Classifier connected to the Encoder, which takes the features of the input image output from the Encoder as input and classifies the input image into a class or subclass. The second machine learning model outputs the class or subclass classified by the Classifier as the classification result.

[0062] (Process 1 for training the first machine learning model) The first learning unit 12A selects an arbitrary target image (anchor) from a plurality of training images. Depending on the class and subclass to which the selected target image belongs, and the class and subclass to which each training image belongs, each training image is classified as either a positive example, a near-positive example, or a negative example, as follows.

[0063] For each image of interest, a training image belonging to the same subclass as the image of interest is used as a positive example.

[0064] For each image of interest, a training image belonging to a different subclass than the one to which the interest image belongs, but belonging to the same class as the interest image, is used as a quasi-positive example.

[0065] Regarding the featured image, attention The training images belonging to a different class than the image to which the original image belongs are used as negative examples.

[0066] Furthermore, it is desirable for the first learning unit 12A to add data-enhanced images obtained by data augmenting the image of interest to the positive examples. Examples of data-enhanced images include images in which the image of interest has been rotated, images in which the subject included in the image of interest has been moved, images in which the subject included in the image of interest has been enlarged or reduced, images in which the image of interest has been flipped in at least one direction (up / down or left / right), images in which a part of the image of interest has been cropped, and images in which the hue, saturation, or brightness of the image of interest has been changed.

[0067] The first learning unit 12A then trains the first machine learning model such that the similarity between the features of the image of interest and the features of the positive examples (i.e., the dot product of the feature vectors) increases, the similarity between the features of the image of interest and the features of the near-positive examples decreases, the similarity between the features of the image of interest and the features of the negative examples decreases, and the similarity between the features of the image of interest and the features of the near-positive examples is greater than the similarity between the features of the image of interest and the features of the negative examples.

[0068] The first learning unit 12A, as an example, trains the first machine learning model so that the value of the following loss function, equation (1), becomes small.

number

[0069] In other words, in equation (1), the feature quantity is a feature vector, and the similarity between the feature vector of the image of interest and the feature vectors of the positive example, semi-positive example, and negative example is based on the inner product of the feature vector of the image of interest and the feature vectors of the positive example, semi-positive example, and negative example.

[0070] Furthermore, in equation (1), it is preferable that the weight coefficient Wq assigned to the quasi-positive example is set to a value greater than the weight coefficient Wn assigned to the negative example. In this case, the value of equation (1) changes more significantly depending on the inner product of the feature vector of the image of interest and the feature vector of the quasi-positive example than on the inner product of the feature vector of the image of interest and the feature vector of the negative example. In other words, the first learning unit 12A can train the first machine learning model such that the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example.

[0071] (Process 2 for training the first machine learning model) As described above, the training images may consist of multiple middle classes, each of which is a subclass, and may belong to any of the multiple middle classes that group one or more subclasses. In this case, in addition to the positive examples, semi-positive examples, and negative examples described above, each training image is classified into either a first semi-positive example or a second semi-positive example, as follows.

[0072] Among the quasi-correct examples, the training image belonging to the same middle class as the image of interest is designated as the first quasi-correct example.

[0073] Among the quasi-correct examples, a training image belonging to a different middle class than the one to which the image of interest belongs is designated as a second quasi-correct example.

[0074] The first learning unit 12A then trains the first machine learning model such that the similarity between the features of the image of interest and the features of the first quasi-positive example is greater than the similarity between the features of the image of interest and the features of the second quasi-positive example. The process by which the first learning unit 12A trains the first machine learning model using positive and negative examples is as described above.

[0075] The first learning unit 12A, as an example, trains the first machine learning model so that the value of the following loss function, equation (2), becomes small.

number

[0076] Furthermore, in equation (2), it is preferable that the weight coefficient Wq1 assigned to the first quasi-positive example is set to a larger value than the weight coefficient Wq2 assigned to the second quasi-positive example. In this case, the value of equation (2) changes more significantly depending on the inner product of the feature vector of the image of interest and the feature vector of the first quasi-positive example than on the inner product of the feature vector of the image of interest and the feature vector of the second quasi-positive example. In other words, the first learning unit 12A can train the first machine learning model such that the similarity between the features of the image of interest and the features of the first quasi-positive example is greater than the similarity between the features of the image of interest and the features of the second quasi-positive example.

[0077] (Effect of Classification Device 2) As described above, in the classification device 2 according to this exemplary embodiment, each training image is further configured to belong to one of a plurality of middle classes, each of which is a plurality of middle classes obtained by classifying each of a plurality of classes into a plurality of subclasses. In the classification device 2 according to this exemplary embodiment, among the quasi-correct examples, a training image belonging to the same middle class as the middle class to which the image of interest belongs is designated as the first quasi-correct example, and among the quasi-correct examples, a training image belonging to a different middle class from the middle class to which the image of interest belongs is designated as the second quasi-correct example. The machine learning model is trained so that the similarity between the features of the image of interest and the features of the first quasi-correct example is greater than the similarity between the features of the image of interest and the features of the second quasi-correct example.

[0078] Therefore, according to the classification device 2 of this exemplary embodiment, the machine learning model is trained by considering a first quasi-correct example and a second quasi-correct example, which are obtained by further classifying the quasi-correct examples. This provides the effect of training the machine learning model to achieve higher inference accuracy.

[0079] [Examples of implementation using software] Some or all of the functions of the learning device 1 and the classification device 2 may be implemented by hardware such as integrated circuits (IC chips), or by software.

[0080] In the latter case, the learning device 1 and the classification device 2 are implemented, for example, by a computer that executes instructions for a program, which is software that implements each function. An example of such a computer (hereinafter referred to as computer C) is shown in Figure 7. Computer C comprises at least one processor C1 and at least one memory C2. The memory C2 stores a program P that causes computer C to operate as learning device 1 and classification device 2. In computer C, the processor C1 reads program P from memory C2 and executes it, thereby implementing each function of learning device 1 and classification device 2.

[0081] Processor C1 can include, for example, a CPU (Central Processing Unit), GPU (Graphic Processing Unit), DSP (Digital Signal Processor), MPU (Micro Processing Unit), FPU (Floating Point Number Processing Unit), PPU (Physics Processing Unit), microcontroller, or a combination thereof. Memory C2 can include, for example, flash memory, HDD (Hard Disk Drive), SSD (Solid State Drive), or a combination thereof.

[0082] Computer C may also be equipped with RAM (Random Access Memory) for loading program P at runtime and for temporarily storing various data. Furthermore, computer C may be equipped with communication interfaces for sending and receiving data with other devices. Additionally, computer C may be equipped with input / output interfaces for connecting input / output devices such as keyboards, mice, displays, and printers.

[0083] Furthermore, program P can be recorded on a non-temporary, tangible recording medium M that is readable by computer C. Such a recording medium M could be, for example, tape, disk, card, semiconductor memory, or programmable logic circuitry. Computer C can acquire program P via such a recording medium M. Program P can also be transmitted via a transmission medium. Such a transmission medium could be, for example, a communication network or broadcast waves. Computer C can also acquire program P via such a transmission medium.

[0084] [Additional Note 1] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the embodiments described above are also included in the technical scope of the present invention.

[0085] [Additional Note 2] Some or all of the embodiments described above may also be described as follows. However, the present invention is not limited to the embodiments described below.

[0086] (Note 1) A learning device for generating feature quantities of an input image and training a machine learning model to perform an image recognition task, comprising: acquisition means for acquiring a plurality of training images; and learning means for training the machine learning model using the plurality of training images, wherein each training image belongs to one of a plurality of classes classified according to the features of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the features of the subject, and the learning means uses a training image belonging to the same subclass as the subclass to which the focus image belongs as a positive example for a focus image included in the plurality of training images, and the focus image A learning device that trains a machine learning model such that a training image belonging to a subclass different from the subclass to which the image in question belongs, but belonging to the same class as the class to which the image in question belongs, is used as a quasi-positive example, and a training image belonging to a class different from the class to which the image in question belongs is used as a negative example, and the similarity between the features of the image in question and the features of the positive example increases, the similarity between the features of the image in question and the features of the quasi-positive example decreases, the similarity between the features of the image in question and the features of the negative example decreases, and the similarity between the features of the image in question and the features of the quasi-positive example is greater than the similarity between the features of the image in question and the features of the negative example.

[0087] (Note 2) The feature quantity is a feature vector, and the similarity between the feature vector of the image of interest and the feature vectors of the positive example, the semi-positive example, and the negative example is based on the inner product of the feature vector of the image of interest and the feature vectors of the positive example, the semi-positive example, and the negative example, as described in Appendix 1 of the learning device.

[0088] (Note 3) Each training image further belongs to one of a plurality of middle classes, each of which is a plurality of subclasses obtained by classifying each of the plurality of classes into a plurality of middle classes, and the learning means further designates a training image from among the quasi-correct examples that belongs to the same middle class as the middle class to which the image of interest belongs as a first quasi-correct example, and a training image from among the quasi-correct examples that belongs to a different middle class from the middle class to which the image of interest belongs as a second quasi-correct example, and trains the machine learning model such that the similarity between the features of the image of interest and the features of the first quasi-correct example is greater than the similarity between the features of the image of interest and the features of the second quasi-correct example, as described in Appendix 1 or 2.

[0089] (Note 4) The learning device according to any one of the appendices 1 to 3, wherein the subject matter included in the learning image is a sample cell, and the sample cell is classified into a class according to whether it is benign or malignant, and further classified into a subclass according to the type of sample cell.

[0090] (Note 5) The learning means further comprises a learning device according to any one of the appendices 1 to 4, wherein the data-enhanced image obtained by data augmenting the image of interest is used as a positive example.

[0091] (Note 6) The machine learning model comprises a first group of layers that take the input image as input and generate feature quantities of the input image, and a second group of layers connected to the first group of layers that take the feature quantities of the input image as input and classify the input image into the class or the subclass, and the learning means further trains the machine learning model using a plurality of training images and teacher labels corresponding to the result of classifying each of the plurality of training images into the class or the subclass. A learning device as described in any one of the appendices 1 to 5.

[0092] (Note 7) A classification device comprising a classification means for classifying an image, including a subject, into the class or subclass by inputting the image into the machine learning model trained by the learning device described in any of the appendices 1 to 6.

[0093] (Note 8) A learning method executed by a learning device that generates feature quantities of an input image and trains a machine learning model to perform an image recognition task, comprising: acquiring a plurality of training images; and training the machine learning model using the plurality of training images, wherein each training image belongs to one of a plurality of classes classified according to the features of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the features of the subject; and in training, for a particular image included in the plurality of training images, a training image belonging to the same subclass as the subclass to which the particular image belongs is used as a positive example. A learning method in which a machine learning model is trained such that a training image belonging to a subclass different from the subclass to which the image of interest belongs, but belonging to the same class as the image of interest, is used as a quasi-positive example, and a training image belonging to a class different from the class to which the image of interest belongs is used as a negative example, and the similarity between the features of the image of interest and the features of the positive example increases, the similarity between the features of the image of interest and the features of the quasi-positive example decreases, the similarity between the features of the image of interest and the features of the negative example decreases, and the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example.

[0094] (Note 9) A program that causes a computer to function as a learning device for training a machine learning model that generates feature quantities of input images and performs an image recognition task, wherein the program causes the computer to function as an acquisition means for acquiring a plurality of training images, and a learning means for training the machine learning model using the plurality of training images, wherein each training image belongs to one of a plurality of classes classified according to the features of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the features of the subject, and the learning means for a focus image included in the plurality of training images to be the same subclass as the subclass to which the focus image belongs A program that trains a machine learning model such that a training image belonging to the class S is used as a positive example, a training image belonging to a different subclass from the subclass to which the image of interest belongs, but belonging to the same class as the image of interest, is used as a quasi-positive example, and a training image belonging to a different class from the class to which the image of interest belongs is used as a negative example, and the similarity between the features of the image of interest and the features of the positive example increases, the similarity between the features of the image of interest and the features of the quasi-positive example decreases, the similarity between the features of the image of interest and the features of the negative example decreases, and the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example.

[0095] [Additional Note 3] Some or all of the embodiments described above can also be expressed as follows:

[0096] A learning device that generates feature quantities of input images and trains a machine learning model to perform an image recognition task comprises at least one processor, the processor performing an acquisition process to acquire multiple training images and training the machine learning model using the multiple training images. training schoolA learning device that performs the following steps, wherein each training image belongs to one of a plurality of classes classified according to the characteristics of the subject contained in the training image, and each of the plurality of classes belongs to one of a plurality of subclasses further classified according to the characteristics of the subject, and in the learning process, for a particular image included in the plurality of training images, a training image belonging to the same subclass as the subclass to which the particular image belongs is used as a positive example, a training image belonging to a different subclass from the subclass to which the particular image belongs, but belonging to the same class as the class to which the particular image belongs, is used as a quasi-positive example, and a training image belonging to a different class from the class to which the particular image belongs is used as a negative example, thereby training the machine learning model such that the similarity between the features of the particular image and the features of the positive examples increases, the similarity between the features of the particular image and the features of the quasi-positive examples decreases, the similarity between the features of the particular image and the features of the negative examples decreases, and the similarity between the features of the features of the particular image and the features of the quasi-positive examples is greater than the similarity between the features of the features of the particular image and the features of the negative examples.

[0097] Furthermore, this learning device may also be equipped with memory, and this memory may store a program that causes the processor to execute the acquisition process and the learning process. This program may also be recorded on a computer-readable, non-temporary, tangible recording medium. [Explanation of Symbols]

[0098] 1. Learning device 2 Classification device 11 Acquisition Department 12. Learning Department 12A First Learning Section 12B Second Learning Section 22 Classification Department

Claims

1. A learning device that generates feature quantities from an input image and trains a machine learning model to perform an image recognition task, A means for acquiring multiple training images, The system includes a learning means for training the machine learning model using the aforementioned plurality of training images, Each training image belongs to one of several classes classified according to the characteristics of the subject contained in the training image, and each of the said classes belongs to one of several subclasses further classified according to the characteristics of the subject. Furthermore, there are multiple middle classes obtained by classifying each of the multiple classes into multiple subclasses, and each belongs to one of the multiple middle classes which is a group of one or more of the above subclasses, The learning means is For the image of interest included in the aforementioned plurality of training images, The training images belonging to the same subclass as the aforementioned image of interest are used as positive examples. A training image belonging to a subclass different from the subclass to which the aforementioned image of interest belongs, and belonging to the same class as the aforementioned image of interest, is used as a quasi-positive example. As negative examples, training images belonging to a different class from the class to which the aforementioned image of interest belongs are used. The similarity between the features of the aforementioned image of interest and the features of the aforementioned positive example increases, The similarity between the feature quantities of the aforementioned image of interest and the feature quantities of the aforementioned quasi-correct example decreases, The similarity between the feature quantities of the aforementioned image of interest and the feature quantities of the aforementioned negative example decreases, and The machine learning model is trained such that the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example, and further, Among the aforementioned quasi-correct examples, the training image belonging to the same middle class as the image of interest is designated as the first quasi-correct example. Among the aforementioned quasi-correct examples, a training image belonging to a different middle class from the middle class to which the image of interest belongs is designated as a second quasi-correct example. The machine learning model is trained such that the similarity between the features of the image of interest and the features of the first quasi-positive example is greater than the similarity between the features of the image of interest and the features of the second quasi-positive example. Learning device.

2. The aforementioned feature is a feature vector, The similarity between the feature vector of the image of interest and the feature vectors of the positive example, the semi-positive example, and the negative example is determined based on the dot product of the feature vector of the image of interest and the feature vectors of the positive example, the semi-positive example, and the negative example. The learning device according to claim 1.

3. The subject matter included in the aforementioned training image is the sample cell. The aforementioned sample cells are classified into classes according to whether they are benign or malignant. The cells are classified into subclasses according to the type of the sample cells. The learning device according to claim 1 or 2.

4. The learning means further, The data-enhanced image obtained by data augmenting the aforementioned image of interest is taken as a positive example. The learning device according to claim 1 or 2.

5. The aforementioned machine learning model, A first group of layers that takes the aforementioned input image as input and generates feature quantities of the aforementioned input image, A second group of layers connected to the first group of layers, which takes the feature quantities of the input image as input and classifies the input image into the class or subclass, Equipped with, The learning means further trains the machine learning model using a plurality of training images and a teacher label corresponding to the result of classifying each of the plurality of training images into the class or subclass. The learning device according to claim 1 or 2.

6. A classification means that, by inputting an image including a subject into the machine learning model trained by the learning device according to claim 1 or 2, classifies the image into the class or the subclass, A classification device equipped with the following features.

7. A learning method executed by a learning device that generates feature quantities of an input image and trains a machine learning model to perform an image recognition task, Acquiring multiple training images, This includes training the machine learning model using the aforementioned plurality of training images, Each training image belongs to one of several classes classified according to the characteristics of the subject contained in the training image, and each of the said classes belongs to one of several subclasses further classified according to the characteristics of the subject. Furthermore, there are multiple middle classes obtained by classifying each of the multiple classes into multiple subclasses, and each belongs to one of the multiple middle classes which is a group of one or more of the above subclasses, In the aforementioned learning process, For the image of interest included in the aforementioned plurality of training images, The training images belonging to the same subclass as the aforementioned image of interest are used as positive examples. A training image belonging to a subclass different from the subclass to which the aforementioned image of interest belongs, and belonging to the same class as the aforementioned image of interest, is used as a quasi-positive example. As negative examples, training images belonging to a different class from the class to which the aforementioned image of interest belongs are used. The similarity between the features of the aforementioned image of interest and the features of the aforementioned positive example increases, The similarity between the feature quantities of the aforementioned image of interest and the feature quantities of the aforementioned quasi-correct example decreases, The similarity between the feature quantities of the aforementioned image of interest and the feature quantities of the aforementioned negative example decreases, and The machine learning model is trained such that the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example, and further, Among the aforementioned quasi-correct examples, the training image belonging to the same middle class as the image of interest is designated as the first quasi-correct example. Among the aforementioned quasi-correct examples, a training image belonging to a different middle class from the middle class to which the image of interest belongs is designated as a second quasi-correct example. The machine learning model is trained such that the similarity between the features of the image of interest and the features of the first quasi-positive example is greater than the similarity between the features of the image of interest and the features of the second quasi-positive example. Learning methods.

8. A program that makes a computer function as a learning device to generate feature quantities of an input image and train a machine learning model to perform an image recognition task, The aforementioned program, the computer, A means for acquiring multiple training images, This is configured to function as a learning means for training the machine learning model using the aforementioned multiple training images. Each training image belongs to one of several classes classified according to the characteristics of the subject contained in the training image, and each of the said classes belongs to one of several subclasses further classified according to the characteristics of the subject. Furthermore, there are multiple middle classes obtained by classifying each of the multiple classes into multiple subclasses, and each belongs to one of the multiple middle classes which is a group of one or more of the above subclasses, The learning means is For the image of interest included in the aforementioned plurality of training images, The training images belonging to the same subclass as the aforementioned image of interest are used as positive examples. A training image belonging to a subclass different from the subclass to which the aforementioned image of interest belongs, and belonging to the same class as the aforementioned image of interest, is used as a quasi-positive example. As negative examples, training images belonging to a different class from the class to which the aforementioned image of interest belongs are used. The similarity between the features of the aforementioned image of interest and the features of the aforementioned positive example increases, The similarity between the feature quantities of the aforementioned image of interest and the feature quantities of the aforementioned quasi-correct example decreases, The similarity between the feature quantities of the aforementioned image of interest and the feature quantities of the aforementioned negative example decreases, and The machine learning model is trained such that the similarity between the features of the image of interest and the features of the quasi-positive example is greater than the similarity between the features of the image of interest and the features of the negative example, and further, Among the aforementioned quasi-correct examples, the training image belonging to the same middle class as the image of interest is designated as the first quasi-correct example. Among the aforementioned quasi-correct examples, a training image belonging to a different middle class from the middle class to which the image of interest belongs is designated as a second quasi-correct example. The machine learning model is trained such that the similarity between the features of the image of interest and the features of the first quasi-positive example is greater than the similarity between the features of the image of interest and the features of the second quasi-positive example. program.

Images