Method for training neural networks through autoencoder and multi-instance training and computing system for carrying out the same

Abstract

The present invention relates to a neural network training method and a computing system for executing the same, and more particularly, a neural network training method and a computing system for executing the same that can improve the performance of a neural network even with a small amount of training data by using an autoencoder and a multi-instance training method are disclosed.

Description

[Technical field]

[0001] The present invention relates to a neural network training method and a computing system for executing the same, and more particularly to a neural network training method and a computing system for executing the same, which can improve the performance of a neural network even with a small amount of training data by using an autoencoder and a multi-instance training method. [Background technology]

[0002] In recent years, with the development of machine learning, there have been many attempts to automate tasks such as image recognition and classification using computer systems. In particular, attempts have been made to automate various classifications and judgments that have been performed by humans using neural networks (e.g., deep learning methods using Convolution Neural Networks (CNNs)), which are a type of machine learning. A representative example is the use of deep learning using neural networks (e.g., CNNs) to read biological images (e.g., slides of a patient's biological tissue) and perform diagnosis to determine the condition or symptoms of a specific disease.

[0003] Meanwhile, multiple instance learning, which is one of the background techniques of the present invention, will be described with reference to FIGS. 1a and 1b.

[0004] Unlike individual instance learning, which uses one instance as a learning unit, multi-instance learning considers a bag, which is a collection of instances, as the learning unit. Therefore, in individual instance learning, a label is attached to an instance, whereas in multi-instance learning, a label is attached to a bag, not an instance. Multi-instance learning is similar to individual instance learning except for the aspect of the learning unit, but has the following additional constraints: When performing binary classification, if a bag is positive, it is assumed that at least one of the instances present in the bag is positive, and if a bag is negative, all the instances in the bag are negative.

[0005] Due to these characteristics, multi-instance learning can be applied to, for example, the field of diagnosing lesions from pathological whole slide images using neural networks. In order to train a neural network for diagnosing lesions, image patches obtained by dividing a whole slide image into fixed sizes are used as training data, and information on the presence or absence of a lesion (i.e., a label) is attached to the whole slide image, not to each patch.

[0006] FIG. 1a shows an example of training data used in multi-instance learning. FIG. 1 shows M bags (B1 to B2) each containing N data instances. M In FIG. 1, bag B i , L i where i is any integer 1≦i≦M, and the data instance D ij Bag B i (where j is any integer such that 1≦j≦N).

[0007] FIG. 1b is a diagram showing a pseudo code showing an example of a process of training a neural network (NN) through a multi-instance training method. FIG. 1b shows a process of training one epoch of training data, and in an actual training process, training for multiple epochs can be performed. In FIG. 1b, it is assumed that training is performed using the training data shown in FIG. 1a.

[0008] Referring to Figure 1b, in multi-instance learning, first, each bag (B1 to B M ) to the training data instances (T1 to T M ) is extracted (S10), followed by a step of training a neural network (NN) with the extracted training data instances (S20).

[0009] To explain the S10 stage in more detail, Bag B i The following process is performed on i, where i is any integer such that 1≦i≦M:

[0010] Neural network (NN) bag B currently undergoing training i Each data instance in is input and the probability that the instance is positive (e.g., a lesion is present) is calculated (S11, S12).

[0011] Bag B i Among all data instances in, the data instance D with the highest positive probability is ik Let T be the training data instance. i (S13), and the training data instance T i The label is bag B i becomes the label assigned to the object (S14).

[0012] In the conventional multi-instance learning, one instance per bag is extracted and used for learning, so there is a problem that a large number of bags are required to improve the performance of the neural network. For example, in order to use the existing multi-instance learning in a neural network for detecting lesions, a large number of whole slide images marked with a lesion presence / absence mark are required.

[0013] In addition, as mentioned above, in multi-instance learning, a neural network before learning is completed is used in the process of extracting training data instances, so if a large number of data instances are extracted from one bag, there is a high possibility that incorrect instances will be extracted. Summary of the Invention [Problem to be solved by the invention]

[0014] The technical problem that the present invention aims to achieve is to provide a method and system that can appropriately train a neural network even with a relatively small amount of data by extracting multiple data instances from a single data bag as training instances. [Means for solving the problem]

[0015] According to one aspect of the present invention, there is provided a neural network training method executed in a computing system including an autoencoder for determining whether an input data instance is in a first state or a second state, and a neural network capable of outputting a probability that the input data instance is in the first state or the second state, the neural network training method including: an extraction step of extracting, for each of a plurality of data bags labeled with either a first state or a second state, a training instance that is a part of the data instances included in the data bag; and a training step of training the neural network based on the training instance corresponding to each of the plurality of data bags, the extraction step including: inputting each data instance included in the data bag to the neural network being trained, and calculating a probability for each data instance included in the data bag; and determining that a part of each data instance included in the data bag is a training instance based on the probability for each data instance included in the data bag and the judgment result of the autoencoder for at least a part of each data instance included in the data bag.

[0016] In one embodiment, the neural network training system can determine whether a data instance input to the autoencoder is in a first state or a second state based on a difference between a data instance input to the autoencoder and output data output from the autoencoder.

[0017] In one embodiment, the autoencoder is pre-trained with only data instances in a first state, and the step of determining that a portion of each data instance included in the data bag is a training instance based on the probability of each data instance included in the data bag and the determination result of the autoencoder for at least a portion of each data instance included in the data bag may include: inputting data instances in order from the most likely to be in a second state to the autoencoder in descending order of the likelihood of the second state when the data bag is labeled in a first state, and determining that a top portion of the data instances determined by the autoencoder to be in the first state are training instances corresponding to the data bag; and inputting data instances in order from the most likely to be in the second state to the autoencoder in descending order of the likelihood of the second state when the data bag is labeled in a second state, and determining that a top portion of the data instances determined by the autoencoder to be in the second state are training instances corresponding to the data bag.

[0018] In one embodiment, the neural network training method may further include repeatedly performing the extraction and training steps for one or more epochs.

[0019] In one embodiment, each of the plurality of data bags may be a whole image, and a data instance included in each of the plurality of data bags may be an image patch obtained by dividing the whole image corresponding to the data bag into a predetermined size.

[0020] According to another aspect of the present invention, there is provided a neural network training method executed in a computing system including an autoencoder for determining whether an input patch is in a first state or a second state (wherein the patch is one of the patches obtained by dividing an image into a predetermined size), and a neural network capable of outputting a probability that the input patch is in the first state or the second state, the neural network including: an extraction step of extracting a training patch that is a part of the patches constituting a plurality of training images labeled with either the first state or the second state; and a training step of training the neural network based on the training patches corresponding to each of the plurality of training images, the extraction step including: inputting each patch constituting the training image into the neural network being trained, and calculating a probability for each patch constituting the training image; and determining that a part of each patch constituting the training image is a training patch based on the probability for each patch constituting the training image and the judgment result of the autoencoder for at least a part of each patch constituting the training image.

[0021] In one embodiment, the autoencoder is trained in advance with only patches that are in a first state, and the step of determining that some of the patches that constitute the training image are training patches based on the possibility of each patch that constitutes the training image and the judgment result of the autoencoder for at least some of the patches that constitute the training image can include a step of inputting the training image to the autoencoder in order of the patch most likely to be in the second state to the patch most likely to be in the second state when the training image is labeled in the first state, and determining that the top part of the patches determined by the autoencoder to be in the first state are training patches corresponding to the training image; and a step of inputting the training image to the autoencoder in order of the patch most likely to be in the second state to the patch most likely to be in the second state to the patch most likely to be in the second state when the training image is labeled in the second state, and determining that the top part of the patches determined by the autoencoder to be in the second state are training patches corresponding to the training image.

[0022] In one embodiment, each of the training images is either an image containing a lesion caused by a specified disease or an image not containing the lesion, and the first state can be a normal state in which the lesion is not present, and the second state can be an abnormal state in which the lesion is present.

[0023] According to another aspect of the invention there is provided a computer program stored on a recording medium for installation in a data processing device to carry out the method as described above.

[0024] According to another aspect of the present invention, there is provided a computer readable recording medium having stored thereon a computer program for carrying out the method described above.

[0025] According to another aspect of the invention, there is provided a computing system comprising a processor and a memory, the memory, when executed by the processor, causing the computing system to perform the method described above.

[0026] According to another aspect of the present invention, there is provided a neural network training system including: a storage module for storing an autoencoder for determining whether an input data instance is in a first state or a second state, and a neural network capable of outputting a probability that the input data instance is in the first state or the second state; an extraction module for extracting, for each of a plurality of data bags labeled with either the first state or the second state, a training instance that is a part of the data instances included in the data bag; and a training module for training the neural network based on the training instances corresponding to each of the plurality of data bags, wherein the extraction module inputs each data instance included in the data bag to the neural network being trained, calculates a probability for each data instance included in the data bag, and determines that a part of each data instance included in the data bag is a training instance based on the probability for each data instance included in the data bag and the judgment result of the autoencoder for at least a part of each data instance included in the data bag.

[0027] In one embodiment, the extraction module may determine whether a data instance input to the autoencoder is in a first state or a second state based on a difference between the data instance input to the autoencoder and output data output from the autoencoder.

[0028] In one embodiment, the autoencoder is pre-trained with only data instances that are in a first state, and the extraction module determines a portion of each data instance included in the data bag as a training instance based on the probability of each data instance included in the data bag and the determination result of the autoencoder for at least a portion of each data instance included in the data bag, by inputting data instances to the autoencoder in order from the most likely to be in a second state to the least likely to be in the second state when the data bag is labeled in a first state, and determining a top portion of data instances determined by the autoencoder to be in the first state as training instances corresponding to the data bag, and by inputting data instances to the autoencoder in order from the most likely to be in the second state to the least likely to be in the second state when the data bag is labeled in a second state, and determining a top portion of data instances determined by the autoencoder to be in the second state as training instances corresponding to the data bag.

[0029] According to another aspect of the present invention, there is provided a neural network learning system including: an autoencoder for determining whether an input patch is in a first state or a second state (wherein the patch is one of the patches obtained by dividing an image into a predetermined size, and the autoencoder is previously trained only with a plurality of patches that are in the first state); a storage module for storing a neural network capable of outputting the possibility that the input patch is in the first state or the second state; an extraction module for extracting a learning patch that is a part of the patches constituting a plurality of learning images for each of the plurality of learning images labeled with either the first state or the second state; and a learning module for training the neural network based on the learning patches corresponding to each of the plurality of learning images, wherein the extraction module inputs each patch constituting the learning image to the neural network during training, calculates a possibility for each patch constituting the learning image, and determines that a part of each patch constituting the learning image is a learning patch based on the possibility for each patch constituting the learning image and the judgment result of the autoencoder for at least a part of each patch constituting the learning image.

[0030] In one embodiment, the autoencoder is pre-trained with only patches that are in a first state, and the extraction module determines a portion of the patches that constitute the training image as training patches based on the possibility of each patch that constitutes the training image and the judgment result of the autoencoder for at least a portion of the patches that constitute the training image, so that when the training image is labeled in a first state, the patches are input to the autoencoder in order from the patch that is most likely to be in a second state to the patch that is least likely to be in the second state, and the top portion of the patches determined by the autoencoder to be in the first state are determined to be training patches corresponding to the training image, and when the training image is labeled in a second state, the patches are input to the autoencoder in order from the patch that is most likely to be in the second state to the patch that is least likely to be in the second state, and the top portion of the patches determined by the autoencoder to be in the second state are determined to be training patches corresponding to the training image. Effect of the Invention

[0031] According to the technical idea of ​​the present invention, multiple data instances can be extracted as training instances from one data bag. In the conventional multi-instance learning method, when multiple training instances are extracted from one data bag, there is a high possibility that an incorrect training data instance is extracted, which may have a negative impact on the training of the neural network. However, according to the method according to the technical idea of ​​the present invention, the training instances are filtered using a pre-trained autoencoder, which has the effect of significantly reducing the extraction of incorrect training data instances. Therefore, it is possible to effectively train the neural network with only a smaller amount of data. [Brief description of the drawings]

[0032] In order to provide a more complete understanding of the drawings referred to in the detailed description of the invention, a brief description of each of the drawings is provided.

[0033] [Figure 1] 1A is a diagram showing an example of training data used in multi-instance learning, and FIG 1B is a diagram showing pseudo code showing an example of a process of training a neural network through the multi-instance learning method. [Diagram 2] FIG. 2 is a diagram showing a schematic configuration of a computing system for executing a neural network training method according to the technical concept of the present invention. [Diagram 3] FIG. 3 is a schematic diagram showing the structure of an autoencoder used in a neural network learning method according to the technical concept of the present invention. [Figure 4] FIG. 4 is a diagram illustrating an example of how a neural network training system according to the technical concept of the present invention extracts training data instances. [Diagram 5] FIG. 5 is a diagram showing an example of a specific process of step S120 of FIG. [Figure 6] FIG. 6 is a diagram showing an example of a specific process of step S121 of FIG. [Figure 7] FIG. 7 is a diagram showing a schematic configuration of a determination system using a neural network according to an embodiment of the present invention. [Figure 8] FIG. 8 shows an example in which the neural network-based determination system according to an embodiment of the present invention outputs a lesion area in a pathology slide image in the form of a matrix map. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] Since the present invention can be modified in various ways and has various embodiments, a specific embodiment will be described in detail with reference to the drawings. However, this is not intended to limit the present invention to the specific embodiment, and it should be understood that the present invention includes all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. In describing the present invention, if it is determined that a detailed description of related known technology may make the gist of the present invention unclear, the detailed description will be omitted.

[0035] Although the terms "first", "second", etc. are used to describe various components, these components should not be limited by the terms. The terms are used only to distinguish one component from another component.

[0036] The terms used in this application are merely used to describe specific embodiments and are not intended to limit the present invention. Unless otherwise clearly stated in the context, singular expressions include plural expressions.

[0037] In this specification, the terms "comprise" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but should be understood as not precluding the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0038] In addition, in this specification, when one component "transmits" data to another component, it means that the component may transmit data directly to the other component, or may transmit data to the other component via at least one other component. Conversely, when one component "transmits" data to another component, it means that data is transmitted from the component to the other component without passing through the other component.

[0039] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will now be described in detail with reference to the accompanying drawings, in which like reference numerals denote like elements throughout the drawings.

[0040] FIG. 2 is a diagram showing a schematic configuration of a computing system for executing a neural network training method according to the technical concept of the present invention.

[0041] The neural network training method according to the teachings of the present invention may be performed by a neural network training system 100 .

[0042] The neural network learning system 100 may be a computing system, which is a data processing device having computing capabilities for realizing the technical ideas of the present invention, and may generally include computing devices such as a server, which is a data processing device to which a client can connect via a network, as well as a personal computer, a mobile terminal, etc.

[0043] The neural network learning system 100 can be realized by any one physical device, but a person skilled in the art of the present invention can easily deduce that, if necessary, multiple physical devices can be organically combined to realize the neural network learning system 100 according to the technical idea of ​​the present invention.

[0044] Referring to FIG. 2, the neural network training system 100 may include a storage module 110, an extraction module 120, and a learning module 130. Some of the above-mentioned components may not necessarily be essential components for implementing the present invention, and the neural network training system 100 may include more components depending on the embodiment. For example, the system 100 may further include a control module (not shown) for controlling functions and / or resources of other components (e.g., the storage module 110, the extraction module 120, the learning module 130, etc.) of the neural network training system 100. Also, depending on the embodiment, the neural network training system 100 may further include a database (DB, 140) for storing various information and / or data required for implementing the technical idea of ​​the present invention.

[0045] The neural network learning system 100 may refer to a logical configuration having hardware resources and / or software necessary to realize the technical idea of ​​the present invention, and does not necessarily refer to one physical component or one device. That is, the system 100 may refer to a logical combination of hardware and / or software provided to realize the technical idea of ​​the present invention, and may be realized as a set of logical configurations for realizing the technical idea of ​​the present invention by performing each function installed in devices separated from each other, if necessary. The system 100 may also refer to a set of configurations separately realized for each function or role for realizing the technical idea of ​​the present invention. For example, each of the storage module 110, the extraction module 120, and the learning module 130 may be located in different physical devices or may be located in the same physical device. Furthermore, depending on the implementation example, the combination of software and / or hardware constituting each of the storage module 110, the extraction module 120, and the learning module 130 may also be located in different physical devices, and the configurations located in the different physical devices may be organically combined with each other to realize each module.

[0046] In addition, in this specification, a module may refer to a functional and structural combination of hardware for implementing the technical idea of ​​the present invention and software for driving the hardware. For example, a module may refer to a logical unit of a certain code and a hardware resource for executing the certain code, and it is easily inferred by those skilled in the art that a module does not necessarily refer to a physically connected code or a single type of hardware.

[0047] The storage module 110 can store a neural network 111 and an autoencoder 112 .

[0048] A neural network, as used herein, may refer to a collection of information representing a set of design considerations that define an artificial neural network, including a multi-layer perceptron model.

[0049] In one embodiment, the neural network 112 may be a convolutional neural network. A convolutional neural network may include an input layer, multiple hidden layers, and an output layer, as is well known. Each of the multiple hidden layers may include a convolutional layer and a pooling layer (or subsampling layer).

[0050] The convolutional neural network may be defined by functions, filters, strides, weight factors, etc. for defining each of these layers. Also, the output layer may be defined as a feed-forward type fully-connected layer.

[0051] The design items for each layer constituting a convolutional neural network are widely known. For example, the number of layers included in the layers, the convolution function for defining the layers, the pooling function, and the activation function may be publicly known functions, or functions defined separately to realize the technical idea of ​​the present invention may be used.

[0052] An example of the convolution function is discrete convolution sum, etc. An example of the pooling function is max pooling, average pooling, etc. An example of the activation function is sigmoid, hyperbolic tangent (tanh), ReLU (rectified linear unit), etc.

[0053] Once the design items of such a convolutional neural network are defined, the convolutional neural network with the defined design items can be stored in a storage device. Then, once the convolutional neural network is trained, the weight factors for each layer can be identified.

[0054] That is, the training of a convolutional neural network may refer to a process in which the weight factors of each layer are determined. Once the convolutional neural network is trained, the trained convolutional neural network can receive input data at an input layer and output output data through a predefined output layer.

[0055] A neural network according to an embodiment of the present invention may be defined by selecting any one or more of the widely known design items as described above, or unique design items may be defined for the neural network.

[0056] The neural network 111 may be a classification neural network that can be used for classification of input data. Preferably, the neural network 111 may be a neural network used for binary classification of input data by outputting a probability of whether the input data is in a predetermined first state or a second state. For example, the neural network 111 may be a neural network for receiving an input image of a living body and determining the probability that the image contains a lesion caused by a predetermined disease (e.g., cancer).

[0057] That is, the neural network 111 can output the possibility that a value input to the neural network 111 is a predetermined first state or a second state. The first state may be either positive or negative, and the second state may be the other of positive or negative. For example, the first state may be a normal state (negative) in which no lesion is present, and the second state may be an abnormal state (positive) in which a lesion is present.

[0058] The likelihood output by the neural network 111 may be a value calculated by a loss function within the neural network 111 (e.g., mean squared error (MSE), cross entropy error (CEE), or a function indicating the distance between two vectors (e.g., Euclidean distance, n-dimensional distance, Manhattan distance, etc.)).

[0059] The autoencoder 112 is a neural network structure that is mainly used in unsupervised learning methodologies. The autoencoder 112 is an unsupervised machine learning model that reduces the dimension of an input value and then restores it, and has a function of learning the features of the values ​​used for learning. More specifically, the autoencoder 112 learns a function that approximates an output value to an input value, extracts features for the input value by an encoder, and reconstructs the input value by a decoder.

[0060] FIG. 3 is a diagram illustrating a schematic structure of the autoencoder 112. Referring to FIG. 3, the autoencoder 112 may include an encoder unit 112-1 including a convolutional layer and a decoder unit 112-2 including a deconvolutional layer. When a data source (x) is input to the encoder 111, the encoder unit 112-1 may encode the data source (x) to generate features (z=E(x)) of the data source (x). The generated features (z) may be decoded by the decoder unit 112-2 to generate restored data (x′=D(z)) corresponding to the data source (x).

[0061] Since an autoencoder is also a type of neural network, it is first trained using a large amount of training data. During the autoencoder training phase, the following steps 1) to 3) are executed for each training data d.

[0062] 1) Learning data d is input to the autoencoder 112, which then undergoes encoding and decoding processes to generate restored data d' corresponding to the learning data d. 2) The error e = L(d, d'), which is the difference between the training data d and the restored data d', is calculated (L is the loss function). 3) The weights in the autoencoder 112 are updated by error backpropagation.

[0063] Alternatively, the autoencoder 112 can be used to determine whether an input value is in a first state or a second state.

[0064] In one embodiment, the autoencoder 112 can be pre-trained using only training data of the first state, and if a predetermined predicted target value is input to the trained autoencoder 112 and the resulting value restored (i.e., output) by the autoencoder 112 differs from the predicted target value by a predetermined threshold or more, the predicted target value can be determined to be in the second state.

[0065] In another embodiment, the autoencoder 112 can be pre-trained using only data in the second state, and if a predetermined predicted target value is input to the trained autoencoder 112 and the resulting value restored (i.e., output) by the autoencoder 112 differs from the predicted target value by a predetermined threshold or more, the predicted target value can be determined to be in the first state.

[0066] Additionally, depending on the embodiment, the autoencoder 112 may include a Variational Autoencoder (VAE).

[0067] 2, DB 140 may store training data used to train neural network 111. The training data may be multi-instance training data as described with reference to Fig. 1a, i.e., each of the training data stored in DB 140 may be a data bag including multiple data instances.

[0068] In one embodiment, each of the plurality of training data may be a whole image, and the data instances constituting each training data may be image patches obtained by dividing the whole image into predetermined sizes. For example, each of the training data may be a complete pathology slide image. In this case, a data bag may be a complete pathology slide image, and the data instances included in the data bag may be individual patches obtained by dividing the pathology slide image into predetermined sizes.

[0069] On the other hand, each of the learning data stored in DB 140 may be labeled with a first state or a second state. For example, when a plurality of learning data are pathology slide images, each of the learning data may be labeled with a diagnosis result for the pathology slide image (e.g., the presence or absence of a lesion, etc.).

[0070] In some embodiments, the learning data may not be stored in DB140, but may be input by a user through an external input means, or may be stored in the form of a file in a storage device such as a HDD or SSD.

[0071] For each of the plurality of data bags labeled with either the first state or the second state, the extraction module 120 may perform an extraction step to extract a portion of the data instances contained in the data bag as training instances. The training instances extracted by the extraction module 120 may then be used to train the neural network 111.

[0072] An example of how the extraction module 120 extracts training data instances is shown in Figure 4. Figure 4 illustrates the case where the training data is similar to that shown in Figure 1a.

[0073] Referring to FIG. 4, the extraction module 120 first extracts each of the data bags B1 to B M Steps S110 to S130 can be performed for the above (S100).

[0074] On the other hand, the extraction module 120 extracts data bag B i For (i is any integer 1≦i≦M), a data bag B i Each data instance D ij (j is an arbitrary integer such that 1≦j≦N) is input to the neural network 111, and a data bag B i Each data instance D ij Possibility of P ij For example, P ij is a possible second state, and the data instance D ij The cross entropy error for the probability P ij It can be calculated as follows:

[0075] Then, the extraction module 120 extracts data bag Bi The probability (P i1 ~P iN ), and each data instance (D i1 ~D iN ), the data bag B i Each data instance (D i1 ~D iN ) can be determined as a training instance (S120), and the determined training instance is denoted as B i Label L i Labeling can be performed (S130).

[0076] Figure 5 is a diagram showing an example of a specific process of step S120 in Figure 4. In the example of Figure 5, it is assumed that the autoencoder 112 has been trained in advance using only the data instances, which are the first state.

[0077] 5, when a data bag is labeled with a first state, the extraction module 120 inputs data instances to the autoencoder 112 in order of the most likely data instance to be in the second state, and determines that a top part of the data instances determined by the autoencoder to be in the first state are learning instances corresponding to the data bag (S121). In this case, the extraction module 120 may determine that the data instance input to the autoencoder 112 is in the first state if the difference between the data instance input to the autoencoder 112 and the output data output by the autoencoder 112 is equal to or greater than a predetermined threshold.

[0078] On the other hand, if the data bag is labeled with a second state, the extraction module 120 can input data instances to the autoencoder 112 in order of most likely to be in the second state to least likely to be in the second state, and determine that a top portion of the data instances determined by the autoencoder 112 to be in the second state are training instances corresponding to the data bag (S122).

[0079] FIG. 6 is a diagram showing an example of a specific process of step S121 of FIG. Referring to FIG. 6, the extraction module 120 extracts data bag B i Data instance in (D i1 ~D iN ) can be sorted in descending order of the likelihood of being in the second state (S1211).

[0080] The extraction module 120 extracts B in the order sorted by step S1211. i Data instance A in k (k is an arbitrary integer between 1 and N) is input to the autoencoder 112, and the input data instance A k The state of data instance A can be determined (S1213). k If it is determined that data instance A is in the first state, k It can be determined that the instance is a learning instance (S1215).

[0081] The extraction module 120 executes steps S1213 to S1215 until the loop ends or the data bag B i This can be repeated until a predetermined number Z of learning instances corresponding to the above are found (see S1212, S1216, and S1217).

[0082] A person skilled in the art to which the present invention pertains should be able to easily derive a specific example of step S122 of Fig. 5 with reference to Fig. 6, and therefore a detailed description thereof will be omitted. Meanwhile, Fig. 5 and Fig. 6 are only an example for implementing step S120 of Fig. 4, and it goes without saying that there are various methods for implementing step S120 of Fig. 4.

[0083] Through the above-mentioned extraction process, the extraction module 120 can extract multiple data instances from one data bag as training instances. In the conventional multi-instance learning method, when multiple training instances are extracted from one data bag, there is a high possibility that an incorrect training data instance is extracted, which may have a negative impact on the training of the neural network. However, according to the method according to the technical idea of ​​the present invention, the training data instance having only one state is filtered using a pre-trained autoencoder, which has the effect of significantly reducing the extraction of an incorrect training data instance.

[0084] Referring again to FIG. 2, the learning module 130 can train the neural network 111 based on the training instances extracted by the extraction module 120 .

[0085] As is well known, the learning module 130 can learn the neural network 111 by backpropagating the loss error between the learning instances input to the neural network 111 and the output values ​​to the neural network 111.

[0086] The neural network learning method treats the learning data instance extraction process performed by the extraction module and the learning process performed by the learning module 130 as one epoch, and repeats the process for multiple epochs to improve the performance of the neural network 111.

[0087] Meanwhile, the neural network learning method according to the technical idea of ​​the present invention can be applied to learning a neural network for image-based disease diagnosis, which can be used for image-based disease diagnosis or diagnostic assistance to help doctors with their diagnosis, and an example of such an application will be described below.

[0088] In this application example, the neural network 111 may be a neural network for diagnosis or diagnosis assistance, which receives image patches obtained by dividing a whole slide image into a predetermined size and determines whether or not the image patches contain a lesion due to a predetermined disease. In this case, the DB 140 may store a plurality of pathological image slides. The pathological slide images are various biological images, including tissue images. Meanwhile, each pathological slide image is labeled with either a first state (normal state) in which no lesion is present, or a second state (abnormal state) in which a lesion is present. In addition, in this application example, the autoencoder 112 may be trained in advance using only image patches in a normal state in which no lesion is present. For example, a learner may select only those pathological slide images labeled as normal from among those stored in the DB 140 for training the neural network 111, divide them into patches, and then train the autoencoder 112 in advance. Alternatively, a learner may collect separate normal state patches not used for training the neural network 111 to train the autoencoder 112 in advance.

[0089] In this application example, the extraction module 120 can perform an extraction step to extract a training patch, which is a part of the patches that constitute the training pathology slide image, for each of a plurality of training pathology slide images that are labeled with either a normal state or an abnormal state, and the learning module 130 can perform a learning step to train the neural network 111 based on the extracted training patches.

[0090] In this case, in the extraction step, the extraction module 120 inputs each patch that constitutes the training pathology slide image into the neural network 111 being trained, and can calculate the probability of each patch that constitutes the training pathology slide image, and can determine that some of the patches that constitute the training pathology slide image are training patches based on the probability of each patch that constitutes the training pathology slide image and the judgment result of the autoencoder 112 for at least a portion of the patches that constitute the training pathology slide image.

[0091] More specifically, when a training pathology slide image is labeled in a normal state, the extraction module 120 can input patches to the autoencoder 112 in order of decreasing probability of being an abnormal state, starting from the patch most likely to be an abnormal state, and determine that a top portion of patches determined by the autoencoder 112 to be in a normal state are training patches corresponding to the training image. Furthermore, when a training image is labeled in an abnormal state, the extraction module 120 can input patches to the autoencoder 112 in order of decreasing probability of being an abnormal state, starting from the patch most likely to be an abnormal state, and determine that a top portion of patches determined by the autoencoder 112 to be in an abnormal state are training patches corresponding to the training image.

[0092] The neural network trained by the above-mentioned neural network training method can be installed in a predetermined judgment system and can perform prediction on input data. In one embodiment, the neural network installed in the judgment system can be used to judge an image. That is, the neural network can be for judging whether a predetermined image is in a first state or a second state. Meanwhile, the judgment system can be a predetermined computing system.

[0093] An example of a determination system incorporating a neural network trained by the above-described neural network training method will be described below with reference to FIG.

[0094] FIG. 7 is a diagram showing a schematic configuration of a determination system using a neural network according to an embodiment of the present invention. Referring to FIG. 7, the neural network-based judgment system 300 may include a storage module 310, a patch-based judgment module 320, and an output module 330.

[0095] The storage module 310 can store the neural network trained by the above-mentioned neural network training method. The storage module 310 can be a storage means capable of storing various information and data including the neural network.

[0096] The patch unit judgment module 320 can input each of a plurality of diagnostic patches obtained by dividing a predetermined judgment target image into a neural network, and obtain judgment results corresponding to each of the plurality of diagnostic patches.

[0097] The output module 330 may output a matrix map of the judgment target image based on the judgment results of each of the plurality of diagnostic patches acquired by the patch-by-patch diagnostic module. The matrix map may mean a map that visually distinguishes the patch area of ​​the first state from the patch area of ​​the second state.

[0098] The neural network-based determination system 300 can be used as a system that receives a pathology slide image and outputs a lesion area. In this case, the determination system 300 can receive a pathology slide image including a lesion caused by a predetermined disease, and the patch unit determination module 320 can divide the pathology slide image into patch units and determine the presence or absence of a lesion for each patch. Thereafter, the output module 330 can output the lesion area of ​​the pathology slide image in the form of a matrix map divided into patch units based on the determination result for each patch.

[0099] Fig. 8 shows an example of a lesion area in a pathology slide image output in the form of a matrix map. In Fig. 8, the area (lesion area) corresponding to the patch determined to be in the second state is displayed in a dark color.

[0100] Meanwhile, the computing device 100 may include a processor and a storage device. The processor means a computing device capable of running a program for implementing the technical idea of ​​the present invention, and may execute a neural network learning method defined by the program and the technical idea of ​​the present invention. The processor may include a single-core CPU or a multi-core CPU. The storage device may mean a data storage means capable of storing a program and various data required for implementing the technical idea of ​​the present invention, and may be implemented by a plurality of storage means depending on the implementation. The storage device may also mean not only a main storage device included in the computing device 100, but also a temporary storage device or memory that may be included in the processor. The memory may include a high-speed random access memory, and may also include a non-volatile memory such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to the memory by the processor and other components may be controlled by a memory controller.

[0101] Meanwhile, the method according to the embodiment of the present invention may be realized in the form of computer readable program instructions and stored in a computer readable recording medium, and the control program and the target program according to the embodiment of the present invention may also be stored in a computer readable recording medium. The computer readable recording medium includes all kinds of recording devices in which data readable by a computer system is stored.

[0102] The program instructions contained in the recording media may be those specially designed and constructed for the present invention, or they may be of the type well known and available to those having skill in the software arts.

[0103] Examples of computer readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions, such as read only memory (ROM), RAM, flash memory, etc. Also, the computer readable recording media may be distributed among computer systems coupled through a network, so that the computer readable code is stored and executed in a distributed manner.

[0104] Examples of program instructions include not only machine code, such as produced by a compiler, but also high level language code executable by a device for processing information electronically, such as a computer, using an interpreter, or the like.

[0105] The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

[0106] The above description of the present invention is merely illustrative, and those skilled in the art will understand that the present invention may be easily modified into other specific forms without changing the technical idea or gist of the present invention. Therefore, it should be understood that the above-described embodiments are illustrative in all respects and are not limiting. For example, each component described as a single type may be implemented in a distributed form, and similarly, each component described as a distributed type may be implemented in a combined form.

[0107] The scope of the present invention is indicated by the claims set forth below rather than by the detailed description, and all modifications and variations derived from the meaning and scope of the claims, as well as their equivalent concepts, should be interpreted as being included in the scope of the present invention. [Industrial Applicability]

[0108] The present invention can be used in a neural network training method through autoencoder and multi-instance training and a computing system for executing the same.

Claims

1. 1. A neural network training method executed on a computing system including an autoencoder for determining whether an input data instance is in a first state or a second state, and a neural network capable of outputting a likelihood that the input data instance is in the first state or the second state, comprising: An extraction step of extracting, for each of a plurality of data bags labeled with either a first state or a second state, a training instance that is a part of the data instances included in the data bag; a training step of training the neural network based on training instances corresponding to each of the plurality of data bags; The extraction step comprises: inputting each data instance included in the data bag to the neural network being trained to calculate a probability for each data instance included in the data bag; determining, based on a probability for each data instance included in the data bag and a judgment result of the autoencoder for at least a portion of each data instance included in the data bag, that a portion of each data instance included in the data bag is an instance for training.

2. The computing system includes:

2. The neural network training method according to claim 1, further comprising: determining whether a data instance input to the autoencoder is in a first state or a second state based on a difference between a data instance input to the autoencoder and output data output from the autoencoder.

3. The autoencoder may include The first state is pre-trained with only data instances, The step of determining that a portion of each data instance included in the data bag is a learning instance based on a probability for each data instance included in the data bag and a determination result of the autoencoder for at least a portion of each data instance included in the data bag includes: If the data bag is labeled with a first state, inputting data instances into the autoencoder in order of the most likely data instance to be in the second state to the least likely data instance to be in the second state, and determining that a top portion of the data instances determined by the autoencoder to be in the first state are training instances corresponding to the data bag; If the data bag is labeled with a second state, and inputting the data instances into the autoencoder in order of the data instances most likely to be in the second state to the data instances least likely to be in the second state, and determining that a top portion of the data instances determined by the autoencoder to be in the second state are training instances corresponding to the data bag.

4. The neural network training method includes:

2. The neural network training method according to claim 1, further comprising the step of repeatedly executing the extraction step and the training step for one or more epochs.

5. Each of the plurality of data bags is a whole image; 2. The neural network training method of claim 1, wherein a data instance included in each of the plurality of data bags is an image patch obtained by dividing a hole image corresponding to the data bag into a predetermined size.

6. A neural network training method executed in a computing system, the method including: an autoencoder for determining whether an input patch is in a first state or a second state, the patch being one of a patch obtained by dividing an image into a predetermined size; and a neural network capable of outputting a possibility that the input patch is in the first state or the second state, An extraction step of extracting a training patch that is a part of the patches constituting the training image for each of a plurality of training images labeled in either the first state or the second state; a training step of training the neural network based on training patches corresponding to each of the plurality of training images; The extraction step comprises: inputting each patch constituting the learning image to the neural network being trained to calculate a probability for each patch constituting the learning image; determining that some of the patches constituting the training image are training patches based on the possibility of each patch constituting the training image and the judgment result of the autoencoder for at least some of the patches constituting the training image.

7. The autoencoder is trained in advance only with patches in a first state, and the step of determining that some of the patches constituting the training image are training patches based on the possibility of each patch constituting the training image and the determination result of the autoencoder for at least some of the patches constituting the training image includes: If the training images are labeled in a first state, inputting the patches to the autoencoder in order of the most likely patch to be in the second state to the least likely patch to be in the second state, and determining that a top portion of the patches determined by the autoencoder to be in the first state are training patches corresponding to the training image; 7. The neural network training method of claim 6, further comprising: when the training image is labeled with a second state, inputting the patches to the autoencoder in order of decreasing probability of being in the second state, starting with the patch most likely to be in the second state, and determining that a top portion of the patches determined by the autoencoder to be in the second state are training patches corresponding to the training image.

8. Each of the learning images is either an image including a lesion caused by a predetermined disease or an image including no lesion, the first state is a normal state in which the lesion is not present, 7. The neural network training method according to claim 6, wherein the second state is an abnormal state in which the lesion is present.

9. A computer program stored on a recording medium, the computer program being installed in a data processing device and adapted to carry out the method according to any one of claims 1 to 8.

10. A computer-readable recording medium having stored thereon a computer program for carrying out the method according to any one of claims 1 to 8.

11. 1. A computing system comprising: A processor and a memory, A computing system, wherein the program instructions stored in the memory, when executed by the processor, cause the computing system to perform a method according to any one of claims 1 to 8.

12. A storage module for storing an autoencoder for determining whether an input data instance is in a first state or a second state and a neural network capable of outputting a probability that the input data instance is in the first state or the second state; an extraction module for extracting, for each of a plurality of data bags labeled with either a first state or a second state, a training instance that is a part of the data instances included in the data bag; a learning module for learning the neural network based on training instances corresponding to each of the plurality of data bags; The extraction module includes: inputting each data instance included in the data bag to the neural network being trained to calculate a probability for each data instance included in the data bag; A neural network training system that determines that some of the data instances included in the data bag are training instances based on the probability for each data instance included in the data bag and the autoencoder's judgment results for at least some of the data instances included in the data bag.

13. The extraction module includes:

13. The neural network training system of claim 12, further comprising: determining whether a data instance input to the autoencoder is in a first state or a second state based on a difference between a data instance input to the autoencoder and output data output from the autoencoder.

14. The autoencoder is pre-trained with only data instances in a first state; The extraction module determines a portion of each data instance included in the data bag as a training instance based on a probability for each data instance included in the data bag and a determination result of the autoencoder for at least a portion of each data instance included in the data bag, If the data bag is labeled with a first state, inputting data instances into the autoencoder in order of the likelihood of the data instances being in the second state from the most likely to be in the second state to the least likely to be in the second state, and determining that a portion of the data instances that are determined by the autoencoder to be in the first state are training instances corresponding to the data bag; If the data bag is labeled with a second state, 13. The neural network training system of claim 12, wherein data instances are input to the autoencoder in order of the most likely data instance to be in the second state to the least likely data instance to be in the second state, and a top portion of the data instances determined by the autoencoder to be in the second state are determined to be training instances corresponding to the data bag.

15. an autoencoder for determining whether an input patch is in a first state or a second state, the patch being one of the patches obtained by dividing an image into a predetermined size; a storage module for storing a neural network capable of outputting a probability that an input patch is in a first state or a second state; an extraction module that extracts, for each of a plurality of training images labeled with either a first state or a second state, a training patch that is a part of the patches that constitute the training image; a learning module that learns the neural network based on training patches corresponding to each of the plurality of training images; The autoencoder is pre-trained with only a plurality of first state patches; The extraction module includes: inputting each patch constituting the training image to the neural network under training to calculate a probability for each patch constituting the training image; A neural network learning system that determines that some of the patches that make up the training image are training patches based on the possibility of each patch that makes up the training image and the judgment results of the autoencoder for at least some of the patches that make up the training image.

16. The autoencoder is trained in advance using only patches in a first state, and the extraction module determines that some of the patches constituting the training image are training patches based on the possibility of each patch constituting the training image and the judgment result of the autoencoder for at least some of the patches constituting the training image, If the training images are labeled in a first state, inputting the patches to the autoencoder in order from the patch most likely to be in the second state to the patch least likely to be in the second state, and determining that a top portion of the patches determined by the autoencoder to be in the first state are training patches corresponding to the training image; If the training images are labeled in the second state, 16. The neural network learning system of claim 15, wherein patches are input to the autoencoder in order of the likelihood of being in the second state, starting with the patch most likely to be in the second state, and a top portion of patches determined by the autoencoder to be in the second state are determined to be learning patches corresponding to the learning image.

Images