Learning device, image processing device, and image processing method
Patent Information
- Application Number
- JP2023139538
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-08-30
- Publication Date
- 2025-03-13
Smart Images
Figure 2025033659000001_ABST
Abstract
Description
[Technical field]
[0001] The embodiments disclosed in this specification and the drawings relate to a learning device, an image processing device, and an image processing method. [Background technology]
[0002] Monitoring the growth state of cells during culture is important for producing high-quality cells. Staining is used as a method for obtaining information reflecting the growth state of cells. However, staining has problems in terms of safety and time and cost. Therefore, when monitoring cells at the culture site, a method that does not require staining is desirable.
[0003] Although cell monitoring technologies that can be performed without staining are being investigated, they may not be able to fully capture the differences in cells. As a result, managers who manage cell cultures may not be able to determine the appropriate culture management method, and cells may not be cultured efficiently and with high quality. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2023-10124 A Summary of the Invention [Problem to be solved by the invention]
[0005] The problem to be solved by the embodiments disclosed in this specification and the drawings is to support efficient cell culture with high quality. However, the problem to be solved by the embodiments disclosed in this specification and the drawings is not limited to the above problem. Problems corresponding to the effects of each configuration shown in the embodiments described later can also be positioned as other problems. [Means for solving the problem]
[0006] The information processing device of the embodiment has an acquisition unit, an image generation unit, a learning unit, a calculation unit, and a parameter determination unit. The acquisition unit acquires a training cell image, which is a cell image to be trained. The image generation unit generates an image from a parameter set using a numerical calculation model. The learning unit learns a machine learning model to output the type of the parameter set in response to input of a generated image, which is the image generated from the parameter set, based on a training data set in which the type of the parameter set is labeled with respect to the generated image. The calculation unit calculates a fitness, which is an index indicating how well the machine learning model is adapted to the training cell image. The parameter determination unit determines the parameter set to be used for learning the machine learning model based on the fitness. [Brief description of the drawings]
[0007] [Figure 1] 1 is a diagram showing an example of the arrangement of an image processing system 1 according to an embodiment. [Diagram 2] FIG. 1 is a diagram illustrating an example of the configuration of a learning device 100 according to an embodiment. [Diagram 3] 4 is a flowchart showing the flow of a series of processes of a processing circuit 120 according to the embodiment. [Figure 4] FIG. 4 is a diagram for explaining a method of extracting a base image IMG2. [Diagram 5] 4 is a flowchart showing the flow of a series of processes of a processing circuit 120 according to the embodiment. [Figure 6] FIG. 1 is a diagram for explaining a learning method of a feature extractor MDL. [Figure 7] FIG. 13 is a diagram for explaining a method for calculating the degree of separation of clusters. [Figure 8] A diagram showing a feature space with high cluster separation. [Figure 9] A diagram showing a feature space with poor separation of clusters. [Figure 10] FIG. 4 is a diagram for explaining a method of searching for a formula parameter set. [Figure 11]FIG. 1 is a block diagram showing an example of the arrangement of an image processing device 200 according to an embodiment. [Figure 12] 4 is a flowchart showing the flow of a series of processes of a processing circuit 220 according to the embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a quantification result displayed on the display 213a. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] Hereinafter, a learning device, an image processing device, and an image processing method according to embodiments will be described with reference to the drawings.
[0009] [Image processing system configuration] 1 is a diagram illustrating an example of the configuration of an image processing system 1 according to an embodiment. The image processing system 1 includes, for example, a learning device 100 and an image processing device 200. These devices are communicatively connected via a communication network NW.
[0010] The communication network NW may refer to any information and communication network that uses electrical communication technology. For example, the communication network NW includes wireless / wired LANs such as hospital backbone LANs (Local Area Networks) and the Internet, as well as telephone communication line networks, optical fiber communication networks, cable communication networks, and satellite communication networks.
[0011] The learning device 100 learns a feature extractor MDL for extracting features from cell images. MDL is simply a code that means a model.
[0012] The image processing device 200 extracts features from the cell image using the feature extractor MDL trained by the learning device 100, and outputs information based on the extracted features. The feature-based information is, for example, information in which features derived from the cell image are quantified.
[0013] [Learning device configuration] 2 is a diagram illustrating an example of a configuration of a learning device 100 according to an embodiment. The learning device 100 includes a communication interface 111, an input interface 112, an output interface 113, a memory 114, and a processing circuit 120, for example.
[0014] The communication interface 111 communicates with external devices via the communication network NW. The communication interface 111 includes, for example, a network interface card (NIC) and an antenna for wireless communication.
[0015] The input interface 112 accepts various input operations from an operator, converts the accepted input operations into electrical signals, and outputs the electrical signals to the processing circuitry 120 .
[0016] For example, the input interface 112 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch panel, etc. The input interface 112 may be, for example, a user interface that accepts audio input from a microphone or the like. When the input interface 112 is a touch panel, the input interface 112 may also have a display function of a display 113a included in the output interface 113 described later.
[0017] In this specification, the input interface 112 is not limited to an interface having physical operation parts such as a mouse, a keyboard, etc. For example, an example of the input interface 112 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs this electrical signal to a control circuit.
[0018] The output interface 113 includes, for example, a display 113a, a speaker 113b, etc. The display 113a displays various types of information.
[0019] For example, the display 113a displays an image generated by the processing circuit 120, a GUI (Graphical User Interface) for receiving various input operations from an operator, etc. For example, the display 113a is an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube) display, an organic EL (Electro Luminescence) display, etc. The speaker 113b outputs information input from the processing circuit 120 as sound.
[0020] The memory 114 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, a hard disk, or an optical disk. These non-transitory storage media may be realized by other storage devices connected via a communication network NW, such as a network attached storage (NAS) or an external storage server device.
[0021] The memory 114 may also include a non-transitory storage medium such as a ROM (Read Only Memory) or a register. The memory 114 stores programs executed by the hardware processor of the processing circuit 120, various calculation results by the processing circuit 120, model definition data, and the like.
[0022] The model definition data is information (program or algorithm) that defines the feature extractor MDL.
[0023] The processing circuit 120 includes, for example, an acquisition function 121, a feature extraction function 122, a parameter determination function 123, an image generation function 124, a learning function 125, a calculation function 126, and an output control function 127. The acquisition function 121 is an example of an "acquisition unit", the feature extraction function 122 is an example of a "feature extraction unit", the parameter determination function 123 is an example of a "parameter determination unit", the image generation function 124 is an example of an "image generation unit", the learning function 125 is an example of a "learning unit", the calculation function 126 is an example of a "calculation unit", and the output control function 127 is an example of an "output control unit".
[0024] The processing circuitry 120 realizes these functions by, for example, a hardware processor (computer) executing a program stored in a memory 114 (storage circuitry).
[0025] The hardware processor in the processing circuit 120 refers to circuitry such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an Application Specific Integrated Circuit (ASIC), a programmable logic device (e.g., a Simple Programmable Logic Device (SPLD) or a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA)), etc.
[0026] Instead of storing the program in memory 114, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes the function by reading and executing the program embedded in the circuit. The above program may be stored in memory 114 in advance, or may be stored in a non-transitory storage medium such as a DVD or CD-ROM, and installed in memory 114 from the non-transitory storage medium by inserting the non-transitory storage medium into a drive device (not shown) of learning device 100.
[0027] The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function. Also, multiple components may be integrated into a single hardware processor to realize each function.
[0028] [Learning device processing flow / preprocessing] Hereinafter, the pre-processing by the processing circuitry 120 of the learning device 100 will be described with reference to a flowchart. Fig. 3 is a flowchart showing the flow of a series of processes by the processing circuitry 120 according to the embodiment. The process of this flowchart is executed before or simultaneously with learning (training) the feature extractor MDL.
[0029] First, the acquisition function 121 acquires a plurality of cell images IMG1 (hereinafter, referred to as a training cell image group) that are training objects (step S100).
[0030] The cell image IMG1 may be, for example, a digital image of a desired cell captured using a microscope or the like.
[0031] The desired cell may be, for example, a cell induced to differentiate from a pluripotent stem cell. Pluripotent stem cells include, for example, embryonic stem cells (ES cells), induced pluripotent stem cells (iPS cells), embryonic stem cells derived from cloned embryos obtained by nuclear transfer (ntES cells), spermatogonial stem cells ("GS cells"), embryonic germ cells ("EG cells"), induced pluripotent stem (iPS) cells, and the like.
[0032] Cells induced to differentiate from pluripotent stem cells include, for example, cells associated with neurodegenerative diseases, such as nerve cells, glial cells, vascular endothelial cells, pericytes, choroid plexus cells, and immune system cells.
[0033] The desired cells may be not only cells induced to differentiate from pluripotent stem cells, but also a group of cells induced by so-called "direct reprogramming," in which the desired cells are directly induced to differentiate without going through pluripotent stem cells.
[0034] In this embodiment, the desired cells do not need to be stained, but may be stained.
[0035] For example, the acquiring function 121 may access a database, which is an external device, via the communication interface 111, and acquire a training cell image group including a plurality of cell images IMG1 from the database. In addition, when a user inputs a training cell image group to the input interface 112, the acquiring function 121 may acquire the training cell image group from the input interface 112. Furthermore, when a training cell image group is stored in the memory 114, the acquiring function 121 may acquire the training cell image group from the memory 114.
[0036] Next, the feature extraction function 122 performs dimensional compression on each of the many cell images IMG1 included in the training cell image group acquired by the acquisition function 121 (step S102).
[0037] Next, the feature extraction function 122 extracts a plurality of basis images IMG2 from the dimension-compressed cell image IMG1 (step S104). The basis images IMG2 are images that represent the external characteristics specific to the cell. This completes the process of this flowchart.
[0038] 4 is a diagram for explaining a method for extracting a base image IMG2. As shown in the figure, the feature extraction function 122 divides each cell image IMG1 included in the training cell image group into a plurality of mesh regions. The feature extraction function 122 performs dimensional compression on each mesh region using a technique such as PCA (Principal Component Analysis) or ICA (Independent Component Analysis). The feature extraction function 122 extracts an image patch obtained by dimensional compression as a base image IMG2.
[0039] In addition to or instead of extracting the basis image IMG2 as the cell-specific appearance feature, the feature extraction function 122 may extract features indicating the frequency characteristics or color characteristics of the cell image IMG1 as the cell-specific appearance feature. The frequency characteristics of the cell image IMG1 may be, for example, the distribution of Fourier coefficients. The color characteristics of the cell image IMG1 may be, for example, the distribution of lightness and chromaticity in the L*a*b color space.
[0040] [Learning device processing flow training] Hereinafter, the training process by the processing circuitry 120 of the learning device 100 will be described with reference to the flowchart. Fig. 5 is a flowchart showing the flow of a series of processes by the processing circuitry 120 according to the embodiment. The process of this flowchart is executed when training the feature extractor MDL.
[0041] First, the image generation function 124 uses a numerical calculation model to computationally generate an image IMG3 from each of a plurality of initialized mathematical expression parameter sets (step S200).
[0042] For example, the image generation function 124 may generate a fractal image, a Perlin noise image, a Bezier curve image, etc., according to a specific mathematical formula. Formula (1) represents a formula for generating a fractal image, that is, a numerical calculation model for generating a fractal image.
[0043]
number
[0044] For example, when generating a fractal image, the formula parameter set includes the weighting coefficient (a i ,b i ,c i ,d i ) and the bias component (e i ,f i In the process described below, these weighting coefficients and bias components are updated from arbitrary initial values to optimal values.
[0045] Furthermore, the image generating function 124 may generate the image IMG3 by a convolution operation of a filter, such as GAN (Generative Adversarial Networks).
[0046] Next, the learning function 125 generates a training data set for an image IMG3 computationally generated from the mathematical parameter set (hereinafter referred to as the generated image IMG3), in which the type of mathematical parameter set used to generate the generated image IMG3 is labeled (step S202).
[0047] For example, assume that 1000 generated images IMG3 are generated using 1000 different mathematical parameter sets. In this case, each generated image IMG3 contains a 1000-dimensional vector [m1, m2, ..., m i ,…,m 1000 ] is labeled. The element m1 corresponds to the first type of formula parameter set among the 1000 types of formula parameter sets, the element m2 corresponds to the second type of formula parameter set, and the element m icorresponds to the i-th type of formula parameter set.
[0048] For example, the generated image IMG3 generated using the first type of mathematical parameter set is labeled with a one-hot vector [1,0,…,0,…,0]. The generated image IMG3 generated using the second type of mathematical parameter set is labeled with a one-hot vector [0,1,…,0,…,0], and the generated image IMG3 generated using the i-th type of mathematical parameter set is labeled with a one-hot vector [0,0,…,1,…,0].
[0049] Next, the learning function 125 learns the feature extractor MDL defined by the model definition data stored in the memory 114, using the training data set (step S204).
[0050] The feature extractor MDL may be implemented, for example, by a neural network, or, instead, by other machine learning models such as a support vector machine, a decision tree, a random forest, or a logistic regression.
[0051] When the feature extractor MDL is implemented by a neural network, the model definition data includes, for example, connection information about how units included in each of the layers constituting the neural network, such as the input layer, one or more hidden layers (intermediate layers), and the output layer, are connected to each other, and weight information about the connection coefficients assigned to the data input and output between the connected units.
[0052] The connection information includes, for example, the number of units included in each layer, information specifying the type of unit to which each unit is connected, the activation function that realizes each unit, and gates provided between units in the hidden layer.
[0053] The activation functions that realize the units may be, for example, ReLU (Rectified Linear Unit) functions, ELU (Exponential Linear Units) functions, clipping functions, sigmoid functions, step functions, hyperpolytangent functions, identity functions, etc. The gates selectively pass or weight data communicated between units, for example, depending on the value (e.g., 1 or 0) returned by the activation functions.
[0054] The combination coefficients include weights that are assigned to output data when data is output from a unit in a layer to a unit in a deeper layer in a hidden layer of a neural network, and may also include bias components specific to each layer.
[0055] 6 is a diagram for explaining a learning method of the feature extractor MDL. For example, the learning function 125 inputs a generated image IMG3 included in the training data set as input data (explanatory variable) to the feature extractor MDL. Then, the learning function 125 learns the feature extractor MDL so that the output result of the feature extractor MDL (multidimensional vector output from the output layer) approaches a multidimensional vector indicating the type of mathematical expression parameter set included in the training data set as correct output data (objective variable). When 1000 types of mathematical expression parameter sets are used, the number of units in the output layer is 1000.
[0056] Specifically, the learning function 125 calculates the difference between a multidimensional vector indicating the type of mathematical formula parameter set included in the training data set as the correct output data (objective variable) and a multidimensional vector output by the feature extractor MDL to which the generated image IMG3 is input. The learning function 125 calculates a loss from the difference based on a predetermined loss function. The loss may be, for example, a cross-entropy error. Then, the learning function 125 adjusts the parameters (weighting coefficients and bias components) of the neural network using a stochastic gradient descent method, a steepest descent method, or the like, so that the loss calculated based on the loss function becomes smaller.
[0057] As a result, the feature extractor MDL outputs a multidimensional vector capable of determining the type of mathematical formula parameter set used in the generated image IMG3 in response to the input of the generated image IMG3. A multidimensional vector indicating the type of mathematical formula parameter set is output from the output layer of the feature extractor MDL, while latent features equivalent to so-called latent variables are output from the intermediate layer before the output layer. The latent features may be interpreted as features obtained during a series of processes (partway through) from when the generated image IMG3 is input to the feature extractor MDL to when the type of mathematical formula parameter set is output by the feature extractor MDL.
[0058] Next, the calculation function 126 calculates an index (hereinafter referred to as adaptability) indicating how well the feature extractor MDL (or the generated image IMG3 used to train the feature extractor MDL) is adapted to the cell image IMG1 included in the training cell image group (step S206).
[0059] For example, the calculation function 126 may use a base image IMG2 extracted from the training cell image group in preprocessing to calculate the similarity of the appearance features between a cell image IMG1 included in the training cell image group and a generated image IMG3, and use the similarity as the fitness. The higher the similarity, the higher the fitness, and conversely, the lower the similarity, the lower the fitness.
[0060] Specifically, the calculation function 126 may calculate the similarity between the base image IMG2 and the generated image IMG3 as the adaptability. The similarity may be, for example, the convolution sum of the base image IMG2 with respect to the generated image IMG3. The similarity may also be the KL divergence of the distribution of Fourier coefficients, or the KL divergence of the distribution of lightness and chromaticity in the L*a*b color space.
[0061] In addition, the calculation function 126 may calculate the degree of separation of clusters in a feature space obtained by applying the cell image IMG1 to the feature extractor MDL, and use the degree of separation as the fitness. The higher the degree of separation, the higher the fitness, and conversely, the lower the degree of separation, the lower the fitness.
[0062] 7 is a diagram for explaining a method for calculating the degree of separation of clusters. First, the feature extraction function 122 inputs each of the multiple cell images IMG1 included in the training cell image group to the feature extractor MDL, thereby extracting latent features corresponding to each cell image IMG1.
[0063] Next, the calculation function 126 distributes the multiple latent features in a feature space (also called a latent space) and clusters the multiple latent features. For example, the calculation function 126 may cluster the multiple latent features using a clustering method such as the k-means method. In the illustrated example, three clusters, CL1, CL2, and CL3, are formed.
[0064] Then, the calculation function 126 calculates the degree of separation between the clustered latent features in the feature space, i.e., between the clusters. The degree of separation may be calculated as, for example, the variance between the clusters / the variance within the cluster.
[0065] Fig. 8 is a diagram showing a feature space with high cluster separation. Fig. 9 is a diagram showing a feature space with low cluster separation. In the example of Fig. 8, the variance within each cluster is small (each cluster is agglomerated), and the variance between clusters is large (the clusters are far apart). In such a case, the degree of separation is high, and as a result, the fitness is high. In other words, it can be evaluated that the feature extractor MDL captures the differences between cells.
[0066] On the other hand, in the example of Figure 9, the variance within each cluster is large (each cluster is spread out), and the variance between clusters is small (the clusters are close to each other). In such a case, the degree of separation is low, and as a result, the fitness is low. In other words, it can be evaluated that the feature extractor MDL does not capture the differences between cells.
[0067] Returning to the explanation of the flowchart, the parameter determination function 123 then updates the mathematical formula parameter set used for training the feature extractor MDL based on the fitness (step S208).
[0068] For example, the parameter determination function 123 searches for a set of formula parameters that provides high fitness for the training cell image group.
[0069] 10 is a diagram for explaining a method for searching for a mathematical expression parameter set. In the illustrated example, a mathematical expression parameter set [a i ,b i ,c i ,d i ,e i ,f i For example, the parameter determination function 123 treats each formula parameter set as an individual and searches for an individual in a direction that increases the reward using a genetic algorithm with fitness as the reward. For example, the parameter determination function 123 updates the formula parameter sets by performing crossover to recombine a part of a first type of formula parameter set (individual 1) with a part of a second type of formula parameter set (individual 2) according to the genetic algorithm, or by performing mutation to replace a part of a formula parameter set with a random number.
[0070] The parameter determination function 123 determines whether or not the fitness has converged to a constant value (step S210), and if the fitness has converged to a constant value, ends the processing of this flowchart.
[0071] On the other hand, if the fitness has not converged to a constant value, the parameter determination function 123 returns the process to S200 described above.
[0072] In response, the image generation function 124 uses the numerical calculation model to computationally generate a generated image IMG3 from each of the updated mathematical parameter sets. The learning function 125 then regenerates the training data set and retrains the feature extractor MDL using the training data set. The calculation function 126 then calculates the fitness again.
[0073] In this manner, a series of processes is repeated until the fitness converges to a constant value.
[0074] When the series of processes is completed, the output control function 127 transmits, via the communication interface 111, to the image processing device 200, model definition data in which the trained feature extractor MDL is defined.
[0075] [Image processing device configuration] 11 is a diagram illustrating an example of a configuration of an image processing device 200 according to an embodiment. The image processing device 200 includes, for example, a communication interface 211, an input interface 212, an output interface 213, a memory 214, and a processing circuit 220.
[0076] The communication interface 211 communicates with external devices via the communication network NW. The communication interface 211 includes, for example, a NIC and an antenna for wireless communication.
[0077] The input interface 212 receives various input operations from an operator, converts the received input operations into electrical signals, and outputs the electrical signals to the processing circuitry 220 .
[0078] For example, the input interface 212 includes a mouse, a keyboard, a trackball, a switch, a button, a joystick, a touch panel, etc. The input interface 212 may be, for example, a user interface that accepts audio input from a microphone or the like. When the input interface 212 is a touch panel, the input interface 212 may also have a display function of a display 213a included in the output interface 213 described later.
[0079] In this specification, the input interface 212 is not limited to an interface having physical operation parts such as a mouse, a keyboard, etc. For example, an example of the input interface 212 also includes an electrical signal processing circuit that receives an electrical signal corresponding to an input operation from an external input device provided separately from the device and outputs the electrical signal to a control circuit.
[0080] The output interface 213 includes, for example, a display 213a, a speaker 213b, etc. The display 213a displays various types of information.
[0081] For example, the display 213a displays images generated by the processing circuit 220, a GUI for receiving various input operations from an operator, etc. For example, the display 213a is an LCD, a CRT display, an organic EL display, etc. The speaker 213b outputs information input from the processing circuit 220 as sound.
[0082] The memory 214 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, a hard disk, or an optical disk. These non-transitory storage media may be realized by other storage devices connected via a communication network NW, such as a NAS or an external storage server device.
[0083] The memory 214 may also include a non-transitory storage medium such as a ROM or a register. The memory 214 stores programs executed by the hardware processor of the processing circuit 220, various calculation results by the processing circuit 220, model definition data in which the trained feature extractor MDL is defined, and the like.
[0084] The processing circuit 220 includes, for example, an acquisition function 221, a feature extraction function 222, a quantification function 223, and an output control function 224. The acquisition function 221 is an example of an "acquisition unit", the feature extraction function 222 is an example of a "feature extraction unit", and the output control function 224 is an example of an "output control unit".
[0085] The processing circuitry 220 realizes these functions by, for example, a hardware processor (computer) executing a program stored in a memory 214 (storage circuitry).
[0086] The hardware processor in the processing circuit 220 refers to circuitry such as a CPU, a GPU, an application specific integrated circuit, a programmable logic device (e.g., a simple programmable logic device or a complex programmable logic device, a field programmable gate array), etc.
[0087] Instead of storing the program in the memory 214, the program may be directly embedded in the circuit of the hardware processor. In this case, the hardware processor realizes the function by reading and executing the program embedded in the circuit. The above program may be stored in the memory 214 in advance, or may be stored in a non-transitory storage medium such as a DVD or CD-ROM, and installed in the memory 214 from the non-transitory storage medium by mounting the non-transitory storage medium in a drive device (not shown) of the image processing device 200.
[0088] The hardware processor is not limited to being configured as a single circuit, but may be configured as a single hardware processor by combining multiple independent circuits to realize each function. Also, multiple components may be integrated into a single hardware processor to realize each function.
[0089] [Image processing device processing flow / runtime] Hereinafter, runtime processing by the processing circuitry 220 of the image processing device 200 will be described with reference to the flowchart. Fig. 12 is a flowchart showing the flow of a series of processes by the processing circuitry 220 according to the embodiment. The process of the flowchart shown in Fig. 12 is executed after the feature extractor MDL has completed learning.
[0090] First, the acquisition function 221 acquires a cell image IMG1 to be tested (hereinafter, referred to as a test cell image) (step S300). The test cell image IMG1 may be the same as the cell image IMG1 included in the training cell image group, or may be a cell image IMG1 different from the training cell image group.
[0091] For example, the acquisition function 221 may access a database, which is an external device, via the communication interface 211 and acquire the test cell image IMG1 from the database. Also, when a user inputs the test cell image IMG1 to the input interface 212, the acquisition function 221 may acquire the test cell image IMG1 from the input interface 212. Furthermore, when the test cell image IMG1 is stored in the memory 214, the acquisition function 221 may acquire the test cell image IMG1 from the memory 214.
[0092] Next, the feature extraction function 222 extracts latent features from the test cell image IMG1 using the trained feature extractor MDL defined by the model definition data stored in the memory 214 (step S302).
[0093] Next, the quantification function 223 determines whether or not latent features have been extracted from all test cell images IMG1 (step S304), and if latent features have not yet been extracted from all test cell images IMG1, returns the process to S300, whereby a new test cell image IMG1 is obtained, and the process of extracting latent features from the new test cell image IMG1 is repeated.
[0094] On the other hand, when latent features are extracted from all of the test cell images IMG1, the quantification function 223 quantifies the extracted multiple latent features (step S306).
[0095] For example, the quantification function 223 may quantify multiple latent features by clustering multiple latent features from the test cell image IMG1 using a clustering technique such as k-means or hierarchical clustering.
[0096] The quantification function 223 may also quantify multiple latent features by projecting the latent features onto axes of a feature space using PCA, linear discriminant analysis, or the like.
[0097] Next, the output control function 224 outputs the quantification result of the latent feature (step S308). For example, the output control function 224 may cause the display 213a to display the quantification result. The output control function 224 may also transmit the quantification result to an external device (for example, a computer used by an administrator who manages cell culture) via the communication interface 211. This ends the processing of this flowchart.
[0098] FIG. 13 is a diagram showing an example of the quantification result displayed on the display 213a. As shown in the figure, the display 213a may display a test cell image IMG1 desired by the user, a cell image IMG1 of a similar cell group, or a cell image IMG1 of a dissimilar cell group. The cell image IMG1 of a similar cell group is another test cell image IMG1 separated into the same cluster as the test cell image IMG1 desired by the user in the feature space. On the other hand, the cell image IMG1 of a dissimilar cell group is another test cell image IMG1 separated into a cluster different from the test cell image IMG1 desired by the user in the feature space. Furthermore, the display 213a may display latent features clustered in the feature space. The manager who manages the cell culture can monitor the state of the cells while checking such quantification results.
[0099] According to the embodiment described above, the processing circuit 120 of the learning device 100 acquires a training cell image IMG1. The processing circuit 120 generates a generated image IMG3 from a mathematical formula parameter set using a numerical calculation model such as Equation (1). The processing circuit 120 trains the feature extractor MDL so as to output a type of mathematical formula parameter set in response to input of the generated image IMG3 based on a training data set. The training data set is a data set in which the type of mathematical formula parameter set is labeled for the generated image IMG3. The processing circuit 120 calculates a fitness, which is an index indicating how well the feature extractor MDL (or the generated image IMG3) is adapted to the training cell image IMG1. The processing circuit 120 determines a mathematical formula parameter set to be used for training the feature extractor MDL based on the fitness. Through such a learning process, a feature extractor MDL capable of extracting latent features from the cell image IMG1 with high accuracy can be generated. By using such a feature extractor MDL, it is possible to support efficient cell cultivation with high quality.
[0100] According to the above-described embodiment, the processing circuitry 220 of the image processing device 200 acquires a test cell image IMG1. The processing circuitry 220 extracts latent features from the test cell image IMG1 using the feature extractor MDL previously trained by the learning device 100. The processing circuitry 220 outputs information in which the latent features are quantified via the output interface 213, or transmits the information to an external device via the communication interface 211. This allows a manager who manages cell culture to monitor the state of the cells while checking such quantification results, thereby enabling cells to be cultured efficiently with high quality.
[0101] (Other embodiments) Other embodiments will be described below. As described above, the learning device 100 and the image processing device 200 in the embodiments have been described as being separate devices, but this is not limited to the above. For example, the processing circuit 120 of the learning device 100 may have some or all of the functions of the processing circuit 220 of the image processing device 200. Conversely, the processing circuit 220 of the image processing device 200 may have some or all of the functions of the processing circuit 120 of the learning device 100.
[0102] Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents as described in the claims, as well as in the scope and spirit of the invention.
[0103] Regarding the above embodiment, the following supplementary notes are disclosed as one aspect and optional features of the invention.
[0104] (Appendix 1) An acquisition unit that acquires a training cell image which is a training subject cell image; an image generator that generates an image from the parameter set using a numerical calculation model; A learning unit that learns a machine learning model to output the type of the parameter set in response to input of a generated image, which is the image generated from the parameter set, based on a training data set in which the type of the parameter set is labeled with respect to the generated image; A calculation unit that calculates a fitness that is an index showing how well the machine learning model is adapted to the training cell image; a parameter determination unit that determines the parameter set to be used for training the machine learning model based on the fitness; A learning device comprising:
[0105] (Appendix 2) The learning device according to claim 1 may further include a feature extraction unit that extracts cell-specific appearance features from the training cell images. Furthermore, the calculation unit may calculate the fitness based on the appearance features.
[0106] (Appendix 3) In the learning device described in Supplementary Note 2, the feature extraction unit may divide the training cell image into a plurality of regions and extract basis images obtained by dimensionally compressing the regions as the appearance features.
[0107] (Appendix 4) In the learning device according to Supplementary Note 3, the calculation unit may calculate a similarity between the base image and the generated image as the fitness.
[0108] (Appendix 5) The learning device described in Supplementary Note 1 may further include a feature extraction unit that extracts latent features from the training cell image. The latent features are features obtained during a series of processes from when the training cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model. Furthermore, the calculation unit may calculate the fitness based on the latent features.
[0109] (Appendix 6) In the learning device described in Supplementary Note 5, the calculation unit may cluster the latent features in a feature space, and calculate a degree of separation between the clustered latent features as the fitness.
[0110] (Appendix 7) In the learning device described in Supplementary Note 1 or 2, the parameter determination unit may search for the parameter set that increases the fitness for the training cell image.
[0111] (Appendix 8) An acquisition unit that acquires a test cell image, which is a cell image to be tested; a feature extraction unit that extracts latent features from the test cell image using a pre-trained machine learning model; an output control unit that outputs information based on the latent feature through an output interface; The machine learning model is trained to output a type of the parameter set in response to input of a generated image, which is an image generated from a parameter set by a numerical calculation model, based on a training data set in which the type of the parameter set is labeled, the parameter set used for learning the machine learning model is determined based on a fitness, which is an index showing how well the machine learning model is adapted to a training cell image, which is a cell image to be trained; The latent features are features obtained during a series of processes from when the test cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model. Image processing device.
[0112] (Appendix 9) 1. A computer-assisted learning method, comprising: Obtaining a training cell image, which is a training subject cell image; generating an image from the parameter set using a computational model; training a machine learning model to output the type of the parameter set in response to input of a generated image, the generated image being the image generated from the parameter set, based on a training dataset in which the type of the parameter set is labeled with respect to the generated image; Calculating a fitness, which is an index showing how well the machine learning model is adapted to the training cell image; determining the parameter set to be used in training the machine learning model based on the fitness; Learning methods including:
[0113] (Appendix 10) 1. A method for image processing using a computer, comprising: obtaining a test cell image, the test cell image being a cell image of a test subject; extracting latent features from the test cell images using a pre-trained machine learning model; outputting information based on the latent features via an output interface; The machine learning model is trained to output a type of the parameter set in response to input of a generated image, which is an image generated from a parameter set by a numerical calculation model, based on a training data set in which the type of the parameter set is labeled, the parameter set used for learning the machine learning model is determined based on a fitness, which is an index showing how well the machine learning model is adapted to a training cell image, which is a cell image to be trained; The latent features are features obtained during a series of processes from when the test cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model. Image processing methods.
[0114] (Appendix 11) A program for causing a computer to execute the program, Obtaining a training cell image, which is a training subject cell image; generating an image from the parameter set using a computational model; training a machine learning model to output the type of the parameter set in response to input of a generated image, the generated image being the image generated from the parameter set, based on a training dataset in which the type of the parameter set is labeled with respect to the generated image; Calculating a fitness, which is an index showing how well the machine learning model is adapted to the training cell image; determining the parameter set to be used in training the machine learning model based on the fitness; Programs including.
[0115] (Appendix 12) A program for causing a computer to execute the program, obtaining a test cell image, the test cell image being a cell image of a test subject; extracting latent features from the test cell images using a pre-trained machine learning model; outputting information based on the latent features via an output interface; The machine learning model is trained to output a type of the parameter set in response to input of a generated image, which is an image generated from a parameter set by a numerical calculation model, based on a training data set in which the type of the parameter set is labeled, the parameter set used for learning the machine learning model is determined based on a fitness, which is an index showing how well the machine learning model is adapted to a training cell image, which is a cell image to be trained; The latent features are features obtained during a series of processes from when the test cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model. program. [Explanation of symbols]
[0116] 1...image processing system, 100...learning device, 111...communication interface, 112...input interface, 113...output interface, 114...memory, 120...processing circuit, 121...acquisition function, 122...feature extraction function, 123...parameter determination function, 124...image generation function, 125...learning function, 126...calculation function, 127...output control function, 200...image processing device, 211...communication interface, 212...input interface, 213...output interface, 214...memory, 220...processing circuit, 221...acquisition function, 222...feature extraction function, 223...quantification function, 224...output control function, MDL...feature extractor
Claims
1. An acquisition unit that acquires a training cell image which is a training subject cell image; an image generator that generates an image from the parameter set using a numerical calculation model; A learning unit that learns a machine learning model to output the type of the parameter set in response to input of a generated image, which is the image generated from the parameter set, based on a training data set in which the type of the parameter set is labeled with respect to the generated image; A calculation unit that calculates a fitness that is an index showing how well the machine learning model is adapted to the training cell image; a parameter determination unit that determines the parameter set to be used for training the machine learning model based on the fitness; A learning device comprising:
2. The method further includes a feature extraction unit that extracts cell-specific external features from the training cell images, The calculation unit calculates the fitness based on the appearance feature. The learning device according to claim 1 .
3. The feature extraction unit divides the training cell image into a plurality of regions, and extracts, as the appearance features, basis images obtained by dimensionally compressing the regions. The learning device according to claim 2 .
4. The calculation unit calculates a similarity between the base image and the generated image as the fitness. The learning device according to claim 3 .
5. A feature extraction unit is further provided for extracting latent features from the training cell images, The latent features are features obtained during a series of processes from when the training cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model, The calculation unit calculates the fitness based on the latent feature. The learning device according to claim 1 .
6. The calculation unit is Clustering the latent features in a feature space; Calculating the degree of separation between the clustered latent features as the fitness. The learning device according to claim 5 .
7. The parameter determination unit searches for the parameter set that increases the fitness for the training cell image. The learning device according to claim 1 or 2.
8. An acquisition unit that acquires a test cell image, which is a cell image to be tested; a feature extraction unit that extracts latent features from the test cell image using a pre-trained machine learning model; an output control unit that outputs information based on the latent feature through an output interface; The machine learning model is trained to output a type of the parameter set in response to input of a generated image, which is an image generated from a parameter set by a numerical calculation model, based on a training data set in which the type of the parameter set is labeled, the parameter set used for learning the machine learning model is determined based on a fitness, which is an index showing how well the machine learning model is adapted to a training cell image, which is a cell image to be trained; The latent features are features obtained during a series of processes from when the test cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model. Image processing device.
9. 1. A method for image processing using a computer, comprising: obtaining a test cell image, the test cell image being a cell image of a test subject; extracting latent features from the test cell images using a pre-trained machine learning model; outputting information based on the latent features via an output interface; The machine learning model is trained to output a type of the parameter set in response to input of a generated image, which is an image generated from a parameter set by a numerical calculation model, based on a training data set in which the type of the parameter set is labeled, the parameter set used for learning the machine learning model is determined based on a fitness, which is an index showing how well the machine learning model is adapted to a training cell image, which is a cell image to be trained; The latent features are features obtained during a series of processes from when the test cell image is input to the machine learning model to when the type of the parameter set is output by the machine learning model. Image processing methods.
Citation Information
Patent Citations
Determination device, determination method, and determination program
JP2023010124A