Image processing method, program, and image processing apparatus

Abstract

To provide an image processing method, device and program to be executed by a computer without the need of preparing an estimator one by one according to the expression or style of a photographic image.SOLUTION: In the S2, a photographed image obtained by photographing cells by a microscope is acquired. To generate an appropriate generated image for performing image conversion and estimation processing in an image conversion processing S3. In the quality evaluation processing S4, estimation processing S5 for estimating a plurality of types of elements included in the cells and outputting processing (display processing) S6 are executed on the generated image for which the evaluation equal to or higher than the reference is obtained.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an image processing method, a program, and an image processing device. [Background technology]

[0002] Conventionally, a method of neuron segmentation using machine learning has been known in the processing of microscopic images (Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Michal Januszewski, Viren Jain, "Segmentation-Enhanced CycleGAN", 2019, bioRxiv, https: / / www.biorxiv.org / content / 10.1101 / 548081v1 (accessed January 15, 2021) [Non-patent document 2] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, Sepp Hochreiter, "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium," arXiv:1706.08500, 2017, https: / / arxiv.org / abs / 1706.08500 (search date January 8, 2021) Summary of the Invention [Problem to be solved by the invention]

[0004] The disclosed technique provides a novel image processing method. [Means for solving the problem]

[0005] One embodiment of the present invention is an image processing method including a conversion process for converting a first image obtained by photographing a cell to generate a second image, and an estimation process for estimating multiple types of elements contained in the cell in the second image. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is an overall configuration diagram of an information processing system according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating a hardware configuration of an image processing apparatus. [Figure 3] FIG. 2 is a diagram illustrating a functional configuration of a server. [Figure 4] FIG. 2 is a diagram illustrating the configuration of a segmenter. [Figure 5] 10 is a flowchart showing a learning process of a segmenter. [Figure 6] FIG. 1 illustrates the relationship between a dataset, a segmenter, and an image transformer. [Figure 7] 1 is a flowchart illustrating an outline of processing according to an embodiment. [Figure 8] 10 is a flowchart illustrating an image conversion process according to the embodiment. [Figure 9] 10 is a flowchart illustrating a quality evaluation process according to the embodiment. [Figure 10] FIG. 10 is a diagram showing an outline of processing up to segmentation. [Figure 11] The formula for calculating the Frechet distance between two images and an example of the criteria are shown below. [Figure 12] The formula for calculating the similarity between two images and an example of the criteria for judgment are shown below. [Figure 13] 10 is an example of an operation screen showing a state in which the segmenter is learning. [Figure 14] 10 is an example of an operation screen showing a state in which image conversion is being performed. [Figure 15] 10 is an example of an operation screen showing a state in which image conversion is completed. [Figure 16] FIG. 10 is a diagram showing an example of an operation screen showing a state in which an output image is displayed. DETAILED DESCRIPTION OF THE INVENTION

[0007] Hereinafter, the present invention will be described based on one embodiment thereof with reference to the drawings.

[0008] 〔composition〕 FIG. 1 shows the configuration of an information processing system 1 according to an embodiment of the present invention. 1 includes a server 10, a terminal 20, and an image capturing device 30. The photographing devices 30 are connected via a network 5 so as to be able to transmit and receive data to and from each other. It has been done.

[0009] The network 5 is a wireless or wired communication means, for example, the Internet. WAN (Wide Area Network), LAN (Local Area Network), public communication network, dedicated The information processing system 1 according to this embodiment is implemented by a plurality of information management devices. However, the present invention does not limit the number of these devices. The processing system 1 is configured by one or more devices that have the following functions: It is possible.

[0010] The server 10 and the terminal 20 acquire images taken by the photographing device 30 and edit the images. The data is collected and analyzed.

[0011] The imaging device 30 is a device that takes pictures (also called images) of biological tissue using a microscope. The cell tissue that is the object of the shadow is assumed to be a cell as an example. The microscope can also be an electron microscope. Examples of electron microscopes include scanning electron microscopes (SEMs) and transmission electron microscopes. Examples of images obtained with optical microscopes include phase contrast images and TEM images. Images, bright field images, differential interference contrast images, confocal microscope images, super-resolution microscope images, fluorescence images, pathology Examples include stained images used for diagnosis.

[0012] FIG. 2 shows hardware (hereinafter referred to as "image processing equipment") used to realize the server 10 and the terminal 20. As shown in the figure, the image processing device 100 is an example of a program. processor 101, main memory device 102, auxiliary memory device 103, input device 104, output device 10 These are connected to each other via a communication means such as a bus (not shown). are communicatively connected to each other.

[0013] It should be noted that the entire configuration of the server 10 does not necessarily need to be realized by hardware. In addition, all or part of the configuration is, for example, a cloud service of a cloud system. It may also be realized by a virtual resource such as a cloud server.

[0014] The processor 101 includes a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and The processor 101 is stored in the main memory 102. By reading and executing the program, the server 10, the terminal 20, and the photographing device 3 The function of 0 is realized.

[0015] The main memory device 102 is a device for storing programs and data, and is a read only memory (ROM). Memory), RAM (Random Access Memory), non-volatile semiconductor memory (NVRAM (Non The auxiliary storage device 103 is a solid state drive (SSD), a Non-volatile memory (NVRAM) such as memory cards, hard disks, Drives, optical storage devices (CD (Compact Disc), DVD (Digital Versatile Disc) etc.), storage space on a cloud server, etc.

[0016] The input device 104 is an interface that accepts input of information, and is, for example, a keyboard. keyboard, mouse, touch panel, card reader, voice input device (microphone, etc.), voice recognition The image processing device 100 exchanges information with other devices via the communication device 106. It may also be configured to accept input.

[0017] The output device 105 is an interface that outputs various types of information, and is, for example, a screen display device (such as a liquid crystal monitor, LCD (Liquid Crystal Display), or graphic card), a printer, an audio output device (such as a speaker), or an audio synthesizer. The image processing device 100 may be configured to output information to and from other devices via the communication device 106. The output device 105 corresponds to the display unit of the present invention.

[0018] The communication device 106 is a wired or wireless communication interface that enables communication with other devices via the network 5, such as a NIC (Network Interface Card), a wireless communication module, a USB (Universal Serial Interface) module, a serial communication module, etc.

[0019] [Functional configuration] 3 shows the main functional configuration of the server 10. As shown in the figure, the server 10 includes a database 114 and a management unit 120.

[0020] The database 114 is stored in the main storage device 102 of the server 10. As shown in FIG. 6 , the database 114 stores a dataset D used for machine learning by the segmenter 116. The dataset D includes a plurality of combinations of a reference image R, a first supervised image L1, and a second supervised image L2. In this embodiment, the database 114 is stored in the main storage device 102, but may also be stored in the auxiliary storage device 103.

[0021] The reference image R is an image obtained by photographing biological tissue with a microscope. The dataset D includes multiple reference images R, each of which has its own unique representation (hereinafter referred to as a "specific representation," which will be described in detail later). The multiple reference images R are, for example, images that were all taken in the same environment.

[0022] The first supervised image L1 is an image obtained by segmenting a certain element of interest (first element of interest) from the reference image R, and the second supervised image L2 is an image obtained by segmenting a second element of interest different from the first element of interest from the reference image R. As shown in Fig. 6, the dataset D holds a numbered set of combinations (indicated by dashed frames in the figure) that associate the reference image R with the first supervised image L1 and the second supervised image L2 obtained by segmenting the reference image R.

[0023] The first and second target elements are selected from cellular components, including cell membranes and organelles. Examples of organelles include mitochondria, nuclei, endoplasmic reticulum, Golgi apparatus, endosomes, chloroplasts, and peroxisomes. For example, the first and second target elements are mitochondria and neuronal membranes. The first and second target elements may be combined in any way, including structurally close organelles, such as mitochondria and endoplasmic reticulum (ER). The extracted images are stored in dataset D as first and second ground-truth images L1 and L2. In the following description, the first and second target elements are assumed to be mitochondria and neuronal membranes, respectively. While two ground-truth images are used in this embodiment, dataset D may also include three or more ground-truth images obtained by segmenting three or more target elements from reference image R.

[0024] In addition to the above functions, the server 10 also has functions such as an operating system, a file system, a device driver, and a DBMS (DataBase Management System).

[0025] The management unit 120 performs processes executed by the server 10, such as image acquisition and management. The functions of the management unit 120 are realized by the processor 101 of the server 10 reading and executing a program stored in the main storage device 102 or the auxiliary storage device 103 of the server 10. The management unit 120 includes the functions of the segmenter 116 and the image converter 118 .

[0026] The segmenter 116 segments two or more focal points from an image of a biological tissue taken using a microscope. A functional section that estimates elements and outputs each of the elements of interest as a segmented image. is.

[0027] The segmenter 116 performs deep learning to learn the image features of the input image. It is an estimator that functions as a segmenter to segment elements of interest in an image. The model used in the segmenter 116 is trained using the data set D. The segmenter 116 is a trained model that is trained on the input image. A neural network is constructed to output information indicating the estimation results. The network is a deep convolutional neural network (DCNN). (Uracial Network).

[0028] The segmenter 116 has an input layer that receives an image input and a segmenter that outputs the estimation result of the element of interest. The input layer, the output layer, and the intermediate layer, which extracts the features of the input image (Fig. 5). Each of the power layer and hidden layer has nodes (shown as white circles in the figure). The nodes are connected by edges (shown by arrows in the figure). The configuration of Mentor 116 is an example, and the number of nodes and edges, the number of intermediate layers, etc. can be changed as appropriate. It is Noh.

[0029] The segmenter 116 is a CNN (Convolutional Neural Network), and in particular in this embodiment In this embodiment, the segmenter 116 uses a U-net. The convolution layer convolves the pixel values ​​of the input image, and the pooling layer maps the pixel values. The input image is processed by the output layer, which is a layer that extracts image features. The neural network has one or more neurons that output the result of estimating the element of interest for the given neural network.

[0030] (Segmenter training) The learning and training of the segmenter 116 is performed according to the flowchart shown in FIG. When the server 10 receives a user instruction via the terminal 20, the processor of the server 10 The program stored in the main memory 102 is started by the server 101. The process is executed by 120 as follows:

[0031] In the following, the process executed by the management unit 120 of the server 10 is simply referred to as "server This may be described as being carried out by "10".

[0032] First, the management unit 120 acquires the data set D from the database 114 (S11). .

[0033] Next, the management unit 120 performs regularization based on the reference image R and the first and second correct images L1 and L2. The regularization term is defined as follows (S12): For example, the mitochondria contained in the first and second correct images L1 and L2 are Based on the shape of the neuron membrane and the image features, the server 10 performs regularization. Define the term.

[0034] In step S13, the management unit 120 calculates the regularization term defined in step S12 and the data The data set D is input to the segmenter 116, and the segmenter 116 is trained (S13 ).

[0035] A regularization term is used in training the segmenter 116, preventing over-training. In addition, since the dataset D includes multiple ground truth images L1 and L2, the segmenter 116 is generated that can estimate multiple elements from biological tissue and segment the estimated elements. In other words, the segmenter 116 is generated as an estimator capable of multi-class classification.

[0036] (Image Converter) The image transformer 118 is, for example, a generative adversarial network (GAN) generated using the segmenter 116 (FIGS. 4 and 6). A user can create multiple image transformers 118 with different hyperparameters. Examples of hyperparameters include the number of filters in the network, the number of layers, and the type of activation function.

[0037] The image converter 118 outputs images from the intermediate layer of the neural network that constitutes the segmenter 116, allowing it to generate images with different styles or expressions, or to convert images in accordance with the characteristics of the data that the segmenter 116 has learned.

[0038] Specifically, when a microscope captured image C is input to the input layer of image converter 118, the image is converted so that captured image C has the representation of reference image R (i.e., the specific representation), and is output as generated image G. Therefore, generated image G has a representation that matches or is similar to the specific representation of reference image R. If there are n image converters 118, n generated images G are output.

[0039] Here, "style" is a concept that encompasses a wide variety of elements, including texture, contrast, image statistics, lighting effects, and color characteristics. Differences in style can also occur due to differences in the imaging environment. Examples of differences in imaging environment include differences in the type of microscope or imaging device (including differences in manufacturer and model), differences between scanning electron microscopes (SEM) and transmission electron microscopes (TEM), differences in imaging conditions including aperture and sensor sensitivity, differences in staining conditions for pathology images, and differences in modalities such as brightfield and phase contrast. When style is being converted, image conversion converts the style of captured image C to that of reference image R without changing the arrangement of objects captured in captured image C. Specifically, the contrast, brightness, and saturation of the captured image are adjusted to resemble reference image R, and colors are replaced.

[0040] In this embodiment, terms such as "expression" and "specific expression" of an image include not only the concept of "artistic style" but also the concept of image characteristics attributable to the subject. For example, differences in expression between images may arise due to differences in the subject's biological species, such as mouse and rat, or the subject's cell type, such as HeLa cells and CHO cells.

[0041] [Image Processing] An example of image processing executed by the information processing system 1 will be described below using the flowcharts of Figures 7 to 9. An overview of the image processing is shown in Figure 10. In this image processing, first, the processor 101 of the server 10 starts a program stored in the main memory device 102, and the processing of the information processing system 1 is executed by the management unit 120 as follows.

[0042] 7, the management unit 120 acquires a photographed image C of biological tissue from the photographing device 30 via the network 5 and stores it in the database 114 (S2). Thereafter, the management unit 120 executes each of the following processes: image conversion process (S3), quality evaluation process (S4), estimation process (S5), and output process (S6). Each process will be described in detail below.

[0043] The representation of the captured image C acquired in step S2 does not necessarily match the representation (specific representation) of the reference image R used for training the segmenter 116. Therefore, first, A conversion process (S3) of the image is executed to generate an image having a specific expression.

[0044] The details of the image conversion process (S3) are shown in Figs. 8 and 10. is acquired from the database 114 (S31), and the captured image C is input to the image converter 118. The image converter 118 converts the representation of the input photographed image C to generate a generated image G (S33). In this embodiment, a plurality of image converters 118 are provided. A plurality of generated images G are generated based on the captured image C. As described above, each of the generated images G has a specific expression.

[0045] Next, the management unit 120 executes a quality evaluation process (S4) to evaluate the quality of each of the multiple generated images G. The quality evaluation is performed (Fig. 9). The quality evaluation is performed on the generated image G, the captured image C, and the reference image R. A method for calculating the similarity between images, or the similarity between the generated image G and a reference value or image A method for calculating the similarity is included (S41).

[0046] As an example of a method for calculating the similarity in step S41, Frechet distance (non-patent document) The quality evaluation method using the method from Reference 2 is shown in Figure 11. In equation (1), the features of the image to be evaluated are Assuming that the vector follows a multivariate normal distribution, we compute the feature vectors in the image x and one generated image G. The Frechet distance of the Torr distribution is calculated. Note that image x includes the captured image C and the reference image R. A segmenter 116 is used as a classifier for the feature vector.

[0047] The quality evaluation may be performed using equation (5) (FIG. 12). In equation (5), the function m is used to The similarity between image y and the comparison image is calculated. The comparison images include the captured image C, the raw image C, and the The segmenter 116 segments a portion of each of the generated image G and the reference image R. The images obtained by the above steps (referred to as image Cp, image Gp, and image Rp, respectively) are used. I can.

[0048] Image y accurately segments a part of the captured image C (the same part as images Cp and Gp). This is the image obtained by the segmentation, and is the image prepared as the correct answer for segmentation. Specifically, a part of the captured image C is subjected to a known segmentation process such as pattern matching. Segmentation is performed using a segmentation technique to create image y.

[0049] Function m includes F1, Jaccard coefficient, Dice coefficient, Simpson coefficient, Hausdorff distance, ran A function that measures the similarity between two images, such as d, is used.

[0050] In step S43, the image with the best quality among the multiple generated images G is designated as generated image G1. Specifically, when measuring the Frechet distance between the captured image C and the generated image G, Then, the generated image G with the largest Frechet distance is selected as the generated image G1. This is because G is judged to be more similar to the reference image R than to the photographed image C. .

[0051] In addition, when measuring the Frechet distance between the generated image G and the reference image R, The resulting image G with the smallest distance is selected as resulting image G1.

[0052] When the quality evaluation is performed based on the formula (5) in step S43, the value of the function m is the highest. First, the image Gp that has the highest similarity to the image y is selected. Furthermore, the generated image G that is the basis of this image Gp is ​​selected as the generated image G1. Since image y is an image created based on the captured image C, the image C and the generated image G are similar. It can also be explained that the generated image G1 is selected based on similarity.

[0053] In the next step S45, the management unit 120 determines whether the selected generated image G1 satisfies the criteria. Examples of the criteria when the generated image G1 is selected using the Fréchet distance are shown in equations (2) to (4) (FIG. 11). Examples of the criteria when the generated image G1 is selected using the function m are shown in equations (6) to (8) (FIG. 12). Any one of these equations may be used as the criteria, or the simultaneous satisfaction of multiple equations may be used as the criteria.

[0054] If the generated image G1 does not satisfy the criteria (S45: NO), the management unit 120 displays an error message to notify the user that estimation processing is not possible (S47). In this case, segmentation processing using existing technology may be performed instead of estimation processing by the segmenter 116. One existing technology is to generate a new segmenter that can directly segment the captured image C. In this case, it is necessary to prepare a large number of images as a new training data set and have the segmenter learn from them.

[0055] If the generated image G1 satisfies the criteria (S45: YES), an estimation process is performed using the segmenter 116 (S5). In the estimation process, the management unit 120 inputs the generated image G1 to the segmenter 116. The segmenter 116 performs segmentation of the generated image G1 into a first element of interest and a second element of interest, and outputs a plurality of output images P1 and P2 as shown in FIG.

[0056] The management unit 120 obtains the output images P1 and P2 from the segmenter 116 and outputs them as the results of the estimation process (S6).

[0057] For example, if the segmenter 116 is an estimator that performs segmentation of mitochondria and neuronal membranes, the output images P1 and P2 will be images that display the segmentation results of mitochondria and neuronal membranes, respectively. The output images P1 and P2 are confirmed by the user via the server 10 or the terminal 20. In this embodiment, the output images P1 and P2 are generated for each segmented element of interest, but the images for each element of interest may also be synthesized into a single image and output in a way that displays them in different colors.

[0058] [Operation screen] The above process can be configured so that each step progresses in response to a user instruction. In this case, the terminal 20 displays operation screens such as those shown in Figs. The operation screen may be generated by the server 10 and acquired by the terminal 20, or may be generated and displayed on the terminal 20.

[0059] The operation screen displays a display frame F1 used when training the segmenter 116, a display frame F2 for the captured image and the generated image G, a display frame F3 for the segmentation result, a progress display frame F4, a button B1 for instructing image conversion, and a button B2 for instructing quality evaluation. In response to instructions from the user via the operation screen, the management unit 120 proceeds with the processing.

[0060] Display frame F1 displays the reference image R and the correct image included in data set D (FIG. 13). When the segmenter 116 is learning, display frame F4 displays a message that learning processing is in progress. In FIGS. 13 to 16, images corresponding to the processing currently being performed are shown in white.

[0061] After the segmenter 116 has completed learning, the user operates the button B1 to perform image conversion processing (FIG. 14).

[0062] 15, a generated image G output by the image conversion process is displayed. When the user operates button B2 on the operation screen shown in FIG. 15, the quality evaluation process (S4) will begin.

[0063] When the output process (S6) is performed, the segmentation data output from the segmenter 116 is The results are displayed on the operation screen. As shown in FIG. 16, the output images P1 and P2 are displayed in a display frame F3 will be displayed.

[0064] <Effects> In the image processing in the above embodiment, a photographed image obtained by photographing cells with a microscope is Image conversion is performed on C (corresponding to the first image of the present invention) to generate a generated image G (corresponding to the second image). Image conversion processing (S3) and estimation of multiple elements contained in the cell in the generated image G and an estimation process (S5).

[0065] In the above configuration, the image is converted by the conversion process to generate a generated image G suitable for the estimation process. Generate a new image G and estimate multiple elements from the generated image G. Estimate multiple elements through image transformation. Therefore, accurate processing can be performed. In addition, since multiple types of elements are estimated, the estimated elements There is no need to execute the estimation process for each image, and the processing efficiency is high.

[0066] In the image conversion process (S3), the representation of the captured image C is converted to generate a generated image having a specific representation. G is generated. The image conversion process (S3) includes a process of converting the style of the captured image C. nothing.

[0067] The estimation process (S5) is a process for estimating a plurality of elements in the generated image G having a specific expression. The generated image G is input to the segmenter 116, and multiple types of elements are input to the segmenter 116. This includes processing to determine the

[0068] In the above configuration, regardless of the expression or style of the photographed image C, the photographed image C is captured using the segmenter 116. In other words, it is possible to perform estimation processing on the captured image C. There is no need to prepare an estimator for each image or style. To generate such an estimator, we prepare a large amount of training data for each expression or style. There is no need to spend a lot of time and effort on machine learning.

[0069] The segmenter 116 performs segmentation of multiple types of intracellular organelles. ,Segmentation can be performed efficiently.

[0070] The segmenter 116 generates a reference image R having a particular expression and a ground truth image corresponding to the reference image R. The learning model is trained using a dataset D with images L. Also, segmenter 1 16 is a trained model trained using the regularization term defined from dataset D. do.

[0071] In addition, the correct image is the first correct image obtained by segmenting the first element of the reference image R, and a second correct answer image obtained by segmenting a second element different from the first element.

[0072] By having the segmenter 116 perform such learning, multi-class segmentation can be achieved. The segmenter 116 may be capable of segmentation.

[0073] The image conversion process (S3) converts the representation of the input image to generate an image having a specific representation. A process of inputting the captured image C to the image converter 118 for output (S33), 118 to output a generated image G having a specific expression (S35). The image transformer 118 is generated using a segmenter 116 that has learned images with specific representations. This is what was done.

[0074] The image converter 118 converts the representation of the input captured image C by extracting an image from the intermediate layer of the segmenter 116. In this way, the image converter 118 is generated based on the segmenter 116, so that the representation of the generated image G can be made similar to or identical to the representation of the reference image R used to train the segmenter 116.

[0075] In the above embodiment, the estimation process (S5) is performed on the generated image G that has received an evaluation above the standard in the quality evaluation process (S4).

[0076] By evaluating the quality of the generated image G, it is possible to improve the accuracy of the estimation process.

[0077] The quality evaluation process includes a process of evaluating the generated image G based on the similarity between the captured image C and the generated image G. The similarity is measured by the Fréchet distance between the captured image C and the generated image G (Equation (1)). Alternatively, the quality evaluation process (S4) includes a process of comparing the result of segmentation of the captured image C with the result of estimation processing of the generated image G (S45, Equations (5) to (8)).

[0078] In the above embodiment, an output process (S6, corresponding to a display process) is executed to display output images P1 and P2 obtained by the estimation process (S3) that estimate multiple types of elements in the cell. Since the estimation results can be visually confirmed via the output images P1 and P2, the user can easily confirm whether the output results are appropriate.

[0079] In the above embodiment, a plurality of terminals are connected to one server 10 to perform the above-described functions. The present invention does not limit the number of servers or the number of terminals, and for example, the above-described functions may be performed by only one device. The number of terminals or the number of servers may also be increased. Furthermore, each function does not necessarily have to be performed by the server 10 or the like, and the functions may be shared and performed by a plurality of devices. In other words, the present invention does not limit the number of control units or devices, or the sharing of functions among devices. [Explanation of symbols]

[0080] 1. Information Processing Systems 10 Servers 20 terminals 30 Imaging equipment

Claims

1. a conversion process in which a first image obtained by photographing a cell or biological tissue is input to a converter, and a second image having a specific representation that is a representation of a reference image included in training data for an estimator that is a trained model is output; a quality evaluation process for evaluating the quality of the second image by determining the similarity between the second image output in the conversion process and the reference image based on distance; and an estimation process for estimating the plurality of types of elements contained in the cells or biological tissues in the second image by inputting the second image evaluated in the quality evaluation process to the estimator, which is a trained model generated using a training dataset including the reference image and a plurality of ground truth images corresponding to each of the plurality of types of elements contained in the reference image. An image processing method executed by a computer.

2. the quality evaluation process evaluates the quality of the second images output by the conversion process; inputting the second image evaluated in the quality evaluation process into the estimator; 2. The image processing method according to claim 1, wherein the image processing method is executed by a computer.

3. the quality evaluation process evaluates the second image, among the plurality of second images, that has the smallest distance from the reference image as the image with the highest quality; 2. The image processing method according to claim 1, wherein the image processing method is executed by a computer.

4. the quality assessment process includes determining a similarity between the second image and the first image based on a distance; the quality assessment process determines that the second image is more similar to the reference image than to the correct image when the distance between the second image and the first image is greater than the distance between the second image and the reference image; 2. The image processing method according to claim 1, wherein the image processing method is executed by a computer.

5. The estimator is a training model generated using the training data set and a regularization term.

2. The image processing method according to claim 1, wherein the image processing method is executed by a computer.

6. The converter is generated using the trained model that has trained an image having the specific expression. The image processing method according to claim 1 .

7. further including a management process of determining whether or not the second image whose quality has been evaluated in the quality evaluation process satisfies a determination criterion.

3. The image processing method according to claim 1.

8. the transforming process includes transforming the first image having a first style to generate the second image having a second style. The image processing method according to any one of claims 1 to 7.

9. the plurality of elements are a plurality of organelles, The image processing method according to claim 1 , wherein the estimator performs segmentation of the plurality of types of organelles.

10. The image processing method according to claim 9, wherein the organelle is selected from the group consisting of mitochondria, nuclei, endoplasmic reticulum, Golgi apparatus, endosomes, chloroplasts, and peroxisomes.

11. The correct image is a first correct image obtained by segmenting a first element of the reference image; The image processing method according to claim 1 , further comprising: a second correct answer image obtained by segmenting a second element different from the first element.

12. The converter is generated using the trained model that has trained an image having the specific expression. The image processing method according to any one of claims 1 to 11.

13. further comprising a quality assessment process for assessing the second image; The image processing method according to claim 1 , wherein the estimation process is performed on the second image that has been evaluated as being equal to or higher than a standard in the quality evaluation process.

14. The image processing method according to claim 1 , further comprising a display process of displaying an image obtained by the estimation process, in which the plurality of types of elements contained in the cell are estimated.

15. A program that causes a computer to execute the image processing method according to any one of claims 1 to 14.

16. An image processing device comprising a processing unit that executes the image processing method according to any one of claims 1 to 15.

Images