Image Processing Method

Abstract

To effectively support teaching operation of a user for machine learning by display-outputting a plurality of learning images obtained by photographing a tissue specimen.SOLUTION: A method extracts a feature of an object in an image by image processing for each of a plurality of learning images obtained by photographing a tissue specimen, configures an image set by selecting a plurality of parts of images from among the learning images on the basis of the extracted features and a prescribed selection reference, outputs, to a display device, a display image in which a plurality of learning images contained in the image set are arranged in a same window, and accepts a teaching input from a user for at least one of the displayed learning image. Then, the method configures a plurality of image sets with mutually different combinations of learning images, displays the display images while switching the plurality of display images respectively corresponding to the plurality of image sets, and accepts a teaching input, thereby, causing one learning image to be contained into the plurality of image sets.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to image processing of images captured of, for example, pathological specimens, and more particularly to a technique for supporting the creation of learning data required when performing image processing using machine learning. [Background technology]

[0002] In recent years, with the development of imaging technology that can easily convert the entire pathological tissue into image data, the accumulation of digitized pathological tissue images has progressed. By applying image processing and machine learning to such pathological tissue images, it has become possible to analyze and quantitatively evaluate the state of cells and structures of interest.

[0003] To perform effective analysis using the results of machine learning, it is necessary to collect a large amount of high-quality learning image data that represents typical examples of cells and structures, and to correctly teach the state of each learning image data. This work requires an expert such as a pathologist to visually observe each image and assign a label according to its state, which places a heavy burden on the worker.

[0004] For example, in the technology described in Patent Document 1, a deep learning algorithm with a neural network structure is used to determine whether or not a tissue image contains a tumor. In this technology, deep learning is performed using training data (learning data) created from images of tissue containing a tumor and images of tissue not containing a tumor, and a trained model having the function of determining the presence or absence of a tumor is constructed by learning the difference in morphological characteristics of tissue depending on whether or not a tumor is present. The determination of the presence or absence of a tumor, which is the basis of the training data, is to be performed by a pathologist. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2019-148473 A Summary of the Invention [Problem to be solved by the invention]

[0006] Such human judgment may also include various factors that lead to erroneous judgment. First, the tissues, cells, structures, etc. that are the subject of judgment vary greatly from specimen to specimen, and quantitative and standardized judgment criteria for them have not been established. There is also variation in the staining of specimens. For this reason, there is a problem that the judgment must be made relative based on the knowledge and experience of the judge. Furthermore, the judgment results may vary even for the same person depending on the judge's personal skills, physical condition, degree of fatigue, and the influence of other images seen before the judgment.

[0007] For these reasons, there is a demand for technology that can automatically or semi-automatically organize and present collected images so that users (judges) can efficiently judge the state of cells and structures contained in images of pathological tissue specimens using stable judgment criteria. However, the above-mentioned conventional technologies have not yet met such demands.

[0008] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a technology that can effectively assist a user in performing teaching work for machine learning by displaying and outputting multiple learning images obtained by photographing tissue specimens in an appropriate display format. [Means for solving the problem]

[0009] In order to achieve the above object, the image processing method according to the present invention includes a step of extracting features of objects in each of a plurality of learning images obtained by capturing an image of a tissue specimen by image processing; a step of selecting a plurality of images from the learning images based on the extracted features and a predetermined selection criterion, and constructing an image set including the selected plurality of learning images; a step of outputting a display image in which the plurality of learning images included in the image set are arranged on the same screen to a display device; and a step of outputting the displayed learning images. each and receiving instruction input from a user regarding the learning images and associating the learning images with the results of the instruction input corresponding thereto, creating a plurality of the image sets each having a different combination of the learning images, switching between and displaying a plurality of the display images corresponding to each of the plurality of image sets, receiving the instruction input each time, and selecting the learning images while changing the selection criteria, thereby including one of the learning images in a plurality of the image sets. Then, each of the learning images is displayed on the display device a plurality of times, and the instruction input is received each time. do.

[0010] In the invention configured in this way, multiple learning images selected according to a predetermined selection criterion are displayed on one screen, so the user can compare the multiple images and perform teaching input. It is easier to obtain a more accurate judgment result when other learning images to compare with are displayed together, rather than judging only one learning image or multiple learning images presented in random order against one's own knowledge.

[0011] Furthermore, in this invention, the same learning image is displayed multiple times while changing the combination with other learning images. As described above, there is variation in user judgment, and even if it is the same learning image, the evaluation may fluctuate depending on the combination with other images displayed at the same time. By displaying the same learning image multiple times while changing the combination and accepting the instruction input, it is possible to statistically converge such fluctuations.

[0012] In this way, by displaying each of the multiple learning images that are to receive evaluation input multiple times in combination with other learning images, and in different combinations, the user is able to make a relative evaluation by actually comparing the images rather than relying solely on memory, and erroneous teaching resulting from fluctuations in judgment can be reduced. Effect of the Invention

[0013] As described above, the present invention can effectively assist a user in performing teaching operations for machine learning by displaying each of a plurality of training images to receive teaching input multiple times in conjunction with other training images, and in varying combinations. [Brief description of the drawings]

[0014] [Figure 1] 1 is a flowchart showing a first embodiment of an image processing method according to the present invention. [Diagram 2] FIG. 13 is a schematic diagram showing an example of classification and arrangement of learning images from the viewpoint of similarity. [Diagram 3] 5 is a flowchart showing a second embodiment of an image processing method according to the present invention. [Figure 4] 11A and 11B are diagrams illustrating examples of display aspects of a display image. [Diagram 5] FIG. 13 is a diagram illustrating an example of a combination of learning images included in one image set. [Figure 6] 10A and 10B are diagrams illustrating examples of instruction input by a user. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] A specific embodiment of the image processing method according to the present invention will be described below. This image processing method is intended to support the creation of training data required for analyzing tissue specimen images using a machine learning model, for example, a deep learning model. More specifically, the creation of training data requires a user to label (teach) each of the collected training images (hereinafter, sometimes simply referred to as "images"). This image processing method displays the training images in an appropriate manner, allowing the user to perform the teaching work efficiently while reducing fluctuations in judgment.

[0016] In order to improve the accuracy of machine learning, it is necessary to collect a large number of learning images and perform appropriate teaching work for each of them. However, there is no absolute standard for judgment, such as the nuclear atypia used to determine tumor cells, and judgment must be made relative to other images. In this case, it is practically impossible to present all of the collected learning images when the number of images is large, and it is also difficult for users to compare a large number of images showing various morphologies and make an accurate evaluation, which is a very time-consuming task.

[0017] For this reason, in reality, a portion of a large number of learning images is selected and presented to the user, and the user is asked to evaluate the images based on a relative comparison among them. In this case, even if a relative evaluation between the displayed images is possible, there may be cases where the evaluation is not correct in relation to other images that are not included in the display. The embodiment of the present invention described below addresses such a problem.

[0018] First Embodiment Fig. 1 is a flow chart showing a first embodiment of an image processing method according to the present invention. In this process, the collection of learning images in step S101 and the instruction input in step S106 are basically performed by a user, but the other processing steps can be executed by a general computer device. Here, it is assumed that these processes are executed by a computer device equipped with input devices such as a mouse and keyboard as a user interface, and an output device such as a display device.

[0019] First, a large number of learning images are collected (step S101). The learning images may be collected by imaging various tissue specimens, or images of previously captured tissue specimens that correspond to the purpose may be obtained from a library.

[0020] It is assumed that the specimens captured as learning images are derived from a tissue or organ and that they have (or do not have) a disease. This is a reasonable assumption given the purpose of collecting tissue as a specimen. Furthermore, the specimens are assumed to have been stained in a manner suitable for visual observation of cells, such as hematoxylin-eosin staining (HE staining). In the following, the word "tissue" can be read as "organ" as necessary, unless otherwise specified.

[0021] The collected learning images are subjected to feature extraction by appropriate image processing (step S102), and an index value relating to similarity is derived based on the features (step S103). For example, image binarization, edge detection, extraction of areas having specific colors or brightness, etc. may be used as the image processing. Areas having specific characteristics may also be extracted by a region segmentation algorithm previously constructed by machine learning. For example, various information quantitatively expressing the tissue specimen, such as the number (or density), shape, and shade of cells or nuclei, may be used as the index value.

[0022] Also, there may be variation in staining for each stained specimen. Such differences in staining can be used as index values, for example, the intensity of the red or blue component. Furthermore, the amount of secondary components in the image can also be used as index values. Here, "secondary components" refers to structures contained in tissue images that are not directly evaluated in the analysis. For example, blood vessels in tissue, artifacts (such as artificial scratches during specimen preparation), and areas where no tissue exists (or are in a state equivalent thereto) are secondary components. The amount of these components contained in the image, for example, the area they occupy, can be used as index values. In addition, when there are images that have already been labeled, such as images obtained from an existing library, it is also possible to treat the information as an index of similarity. For example, it can be said that images with the same label have a relatively high similarity, and images with different labels have a lower similarity. In addition, when there are labels that express the characteristics of the images as a degree such as high, medium, or low, they can be used as an index of similarity.

[0023] When the difference in index value between multiple learning images is relatively small, the similarity between them can be said to be relatively high. On the other hand, when the difference in index value is large, the similarity can be said to be low. In this way, the index value can be used to evaluate the relative similarity between multiple learning images.

[0024] FIG. 2 is a schematic diagram showing an example of classification and arrangement of learning images from the viewpoint of similarity. Note that this diagram is for explaining the principle, and such classification and arrangement are not required in actual image processing. Assume that the collected learning images include tissue types A, B, ... and disease types a, b, c, d, e, ... (among which, diseases a to c are diseases that occur in tissue A, and diseases d and e are diseases that occur in tissue B). For tissues that do not contain diseases, that is, normal tissues, the "normal" state can be treated as one type of disease.

[0025] For each combination of tissue and disease, multiple learning images are collected. Each learning image is associated with information indicating which tissue and disease the image corresponds to, and an index value calculated from the characteristics of the image. In the following description, in order to distinguish each learning image, a reference symbol is assigned to each learning image as shown in the figure. For example, the reference symbols Aa1, Aa2, ... are assigned to learning images corresponding to tissue A and disease a. The numbers 1, 2, ... are consecutive numbers assigned in order of magnitude of index value (ascending or descending). Similarly, the reference symbols Ab1, Ab2, ... are assigned to learning images corresponding to tissue A and disease b, and the reference symbols Ba1, Ba2, ... are assigned to learning images corresponding to tissue B and disease d.

[0026] Returning to FIG. 1, next, an image set consisting of a plurality of images selected from the collected learning images is constructed (step S104). As will be described later, a plurality of images included in one image set are arranged on one screen and displayed and output. Therefore, the user can only simultaneously view the learning images included in one image set.

[0027] However, in this embodiment, the image selection criteria for constructing the image sets are varied so that one learning image is included in multiple image sets, so that when switching between image sets, the combinations of learning images displayed change sequentially, and one learning image is displayed multiple times while changing the combination with other images.

[0028] There are various possible criteria for selecting learning images to construct an image set. For example, an image set can be constructed by collecting learning images corresponding to the same tissue or the same disease. In this case, it is possible to combine images with high similarity, i.e., images with small differences in index values, and to combine images with low similarity, i.e., images with large differences in index values.

[0029] It is also possible to combine training images with different tissue types and / or disease types. In this way, even between specimens of different origins, the features that appear in the images may be similar. By displaying them simultaneously and making it possible to compare them, it is possible to reduce misjudgments between them.

[0030] When multiple index values ​​are derived for each learning image, each of them can be an index of similarity. For example, similarity with cell density as an index value, similarity with stainability as an index value, similarity with secondary component amount as an index value, etc. can be defined. Each of these similarities can be a selection criterion when constructing an image set. In other words, it is possible to construct an image set that is combined focusing on differences in cell density, an image set that is combined focusing on differences in stainability, etc. Therefore, one learning image can be included in various image sets.

[0031] A display image is generated from one of the image sets thus constructed and displayed on the display device (step S105). The display image is a plurality of learning images included in the image set arranged on the same screen.

[0032] Then, for each of the training images thus displayed, instruction input from the user is accepted (step S106). The user compares the displayed training images to determine the state of the tissue or the cells and structures contained therein, and inputs the result as instruction input via an input device. The input result is associated with each training image as a label based on the user's judgment.

[0033] The condition of the tissue needs to be judged comprehensively from various perspectives, such as cell density and shape, the presence or absence of irregular nuclear margins, the presence or absence of multinuclei, the presence or absence of apoptosis, etc. Therefore, the user in this case is required to be a pathologist, researcher, or other person with specialized knowledge and experience.

[0034] Steps S105 and S106 are repeatedly executed while changing the displayed image set until instruction input is completed for all image sets (step S107). Then, based on the result of the instruction input, the instruction result for each learning image, i.e., the label to be assigned to the learning image, is determined (step S108).

[0035] Since one learning image is included in multiple image sets, multiple teaching inputs are performed on the learning image. On the other hand, since the judgment by the user is a relative one based on the comparison between images, the result of multiple teaching inputs on one learning image is not necessarily the same due to the influence of other learning images displayed at the same time. In particular, if the morphological difference between the learning image and other learning images displayed at the same time is not clear, it is difficult to judge their condition. In addition, apparent differences due to differences in staining properties may lead to misjudgment of differences based on the original judgment criteria.

[0036] If the results of the multiple teaching inputs are all the same, the teaching input can be regarded as being highly accurate and can be used as the label of the learning image as is. On the other hand, for learning images with different results of multiple teaching inputs, the teaching result can be determined statistically from those input results. For example, the most frequently inputted result among the multiple teaching inputs can be used as the final teaching result. Also, the results of the multiple teaching inputs can be averaged to determine the final teaching result.

[0037] In this embodiment, an image set is formed from a plurality of learning images selected from a large number of collected learning images, and these learning images are simultaneously displayed on the screen to accept teaching input from the user. At this time, teaching input is received for all learning images while switching between images displayed by changing the image set. One learning image is included in multiple image sets, and therefore receives teaching input multiple times. If the input results do not match, the final teaching result is determined by statistical processing.

[0038] By doing as described above under the condition that the number of learning images that can be displayed at one time is limited, this embodiment makes it possible to accurately label each learning image while suppressing fluctuations in judgment that can occur when there are few images to compare. For the user who provides the teaching input, the workload is greatly reduced because it is sufficient to make a judgment while comparing images that are displayed at the same time (in other words, without considering images that are not currently displayed). In this way, this embodiment makes it possible to support the user in performing efficient and accurate teaching work.

[0039] By providing the thus labeled learning images as training data to an appropriate learning model and performing machine learning, it is possible to construct a trained model that can accurately determine the tissue state of unknown tissue specimen images. By performing learning including specimen images related to various tissues and diseases, a highly versatile learning model that can handle various tissues and diseases can be created.

[0040] <Second embodiment> In the first embodiment described above, display images corresponding to an image set that is preconfigured based on the characteristics of each of the collected learning images are displayed in order, and instruction input for each learning image is accepted. The second embodiment described next has the same basic concept of processing, but differs from the above embodiment in that an image set is dynamically configured in response to the progress of the instruction input.

[0041] 3 is a flowchart showing a second embodiment of an image processing method according to the present invention. The processing contents of steps S201 to S203 are the same as the processing contents of steps S101 to S103 in the first embodiment.

[0042] In step S204, one image set is constructed based on an appropriate selection criterion. The method of constructing the image set may be the same as in the first embodiment. Then, a display image corresponding to the image set is created and displayed (step S205), and a teaching input is accepted for the learning image contained therein (step S206). The basic concept of these steps is the same as in the first embodiment.

[0043] The processes of steps S204 to S206 are repeated until the teaching input has been performed for all learning images (step S207). In the following step S208, it is determined whether or not the result of the teaching input given to the currently displayed learning image has converged. Here, "convergence" means that the variation in the teaching input given multiple times to one learning image has become sufficiently small. The reason for adding such a determination will be described later.

[0044] The processes of steps S204 to S206 are repeated until convergence is achieved for the teaching results for all learning images (YES in both steps S207 and S208). If convergence is not achieved (NO in step S208), in step S204, a new image set is constructed taking into account the results of the teaching input up to that point in addition to the same selection criteria as in the first embodiment. For example, the selection criteria are adjusted so that learning images with large fluctuations in the results of multiple teaching inputs are included in the subsequent image set more frequently, and learning images with small fluctuations are included in the subsequent image set less frequently.

[0045] In this way, images that are difficult to judge are displayed in combination with various other images in sequence, allowing the user to evaluate from multiple perspectives. This reduces the fluctuation of the teaching results and increases the accuracy, that is, it is possible to converge the teaching results. On the other hand, for images with little fluctuation and close to convergence, the frequency of displaying them can be reduced or they can be excluded from the display targets, thereby reducing the effort and time required for the teaching work.

[0046] Also, initially, a learning image may be combined with an image for which the difference from other learning images is easy to understand, i.e., with a low similarity, while as the teaching input progresses, a combination with a learning image with a higher similarity may be adopted. By comparing images with relatively large differences first, and then comparing images with smaller differences, it is possible to increase the accuracy of the teaching result even in a situation where a relative judgment is made between displayed images.

[0047] In this way, by dynamically constructing an image set based on past teaching input results, in this embodiment, it is possible to reduce fluctuations in the teaching results for each learning image and converge to a certain teaching result. The converged teaching result is determined as the teaching result to be associated with each learning image (step S209). The converged teaching result has a high degree of accuracy. Therefore, it is possible to improve the accuracy of the trained model that is constructed as a result of machine learning using this as training data.

[0048] Convergence can be determined, for example, when the number of learning images for which the teaching results change with repeated teaching input becomes equal to or less than a certain number. It is desirable that the teaching results for all learning images are stable, but there may be peculiar images for which the results do not converge. Since it may take a long time to converge such images, it may be determined that the entire process has converged when convergence is observed for most of the images. In addition, an upper limit may be set for the number of repetitions from step S208 to S204, and convergence may be forcibly determined when the number of repetitions reaches the upper limit.

[0049] <Screen display example> Fig. 4 is a diagram showing an example of the display mode of display images. Fig. 4(a) shows an example in which all of a plurality of learning images (six images in this case) contained in one image set are tiled in a window Wa. In addition, a group of buttons Ba, including buttons for switching the display to the previous or next image set and buttons for changing the display magnification, are displayed on the screen. Each learning image is displayed at the same magnification, and when the display magnification is changed, the magnification of all images changes in the same way.

[0050] In Fig. 4(b), some of the learning images in the same image set are displayed in a window Wb, and the remaining images can be displayed by moving them by horizontal scrolling. On the other hand, in Fig. 4(c), the displayed images in a window Wc are moved by vertical scrolling. In these display modes, the button groups Bb and Bc similar to those described above are arranged in appropriate positions. Of course, if all of the learning images in the same image set can be arranged in a row vertically or horizontally on the same screen while ensuring sufficient visibility, it is preferable to do so.

[0051] Fig. 5 is a diagram illustrating a combination of learning images included in one image set. In the example shown in Fig. 5(a), learning images Aa1, Aa2, and Aa3 corresponding to the same tissue and the same disease are arranged in order of index value related to similarity. In such a combination and display mode, images with relatively high similarity are arranged adjacent to each other, making it easy to compare them.

[0052] In addition, the example shown in FIG. 5(b) is a schematic diagram showing the change of the display image accompanying the switching of the image set. First, the learning images Aa1 and Aa5, which correspond to the same tissue and the same disease but have a relatively small degree of similarity, that is, which are easy to judge whether they are different or the same, are displayed side by side. Here, the image Aa5 corresponds to the "first learning image" of the present invention, and the image set at this time corresponds to the "first image set" of the present invention. Then, by switching the image set thereafter, the learning images Aa1 and Aa2, which have a higher degree of similarity and are therefore more difficult to judge, are displayed side by side. Here, the image Aa2 corresponds to the "second learning image" of the present invention, and the image set at this time corresponds to the "second image set" of the present invention. This display mode is particularly useful in the second embodiment in which the subsequent image set is dynamically set based on the previous teaching result. In this case, information that allows the previously input teaching result to be known may be added to each learning image.

[0053] Also, Figure 5(c) shows an example of a combination of images of different tissues and disease types. Here, images Aa1 and Ab2 of the same tissue A but with different disease types are combined with image Bd1 corresponding to tissue B. By including such combinations in the image set, it is ultimately possible to obtain a trained model that can distinguish between specimens of different origins but similar appearances.

[0054] FIG. 6 is a diagram showing a schematic example of instruction input by a user. FIG. 6(a) shows an example using a drop-down list. When one of the displayed images is clicked, candidate labels to be assigned to that image are displayed as a drop-down list. In this example, the level of nuclear atypia (high, medium, low) can be selected as the label. By performing such input for each image displayed in an image set, labels can be assigned to all learning images.

[0055] Figure 6(b) shows an example of rearrangement of the image order. The image numbers shown in Figure 2 are based on specific index values ​​and may not necessarily match the similarity when an expert comprehensively evaluates them from various perspectives. As shown by the dashed line in Figure 6(b), if you can drag an image to swap it with other images, it will be possible to display the images in an order that corresponds to their original similarity.

[0056] Figure 6(c) shows an example of instruction input that sets a "boundary" for images sorted in this way. Between images sorted according to similarity, it is possible to set a boundary that represents the difference in the state of each tissue. For example, when there is a mixture of images that do not contain lesions (normal) and images that contain lesions, sorting them according to similarity is expected to result in the existence of a boundary that distinguishes between images of normal tissue and images that contain lesions. Figure 6(c) shows an operation in which the user specifies such a boundary as instruction input.

[0057] <Other> The present invention is not limited to the above-mentioned embodiment, and various modifications can be made without departing from the spirit of the present invention. For example, in the description of the above-mentioned embodiment, the image selection criteria, image combinations, label types, display modes, etc. for constructing an image set are merely examples for the purpose of explanation, and are not limited to the above-mentioned embodiment. For example, the display example of FIG. 4(b) is cited in the example of instruction input in FIG. 6, but the same can be applied to the display examples shown in FIG. 4(a) and FIG. 4(c).

[0058] As described above by way of specific examples, in the image processing method according to the present invention, the similarity between learning images calculated based on extracted features or the result of instruction input can be used as one of the selection criteria. By comparing images with high and low similarity while changing the combinations, the user can perform more accurate instruction input.

[0059] In this case, for example, a first image set including one learning image and a first learning image having a relatively small similarity thereto, and a second image set including the one learning image and a second learning image having a larger similarity thereto than the first learning image may be created, and the display image corresponding to the second image set may be displayed after the display image corresponding to the first image set. According to this configuration, images having a low similarity and therefore relatively easy to judge whether they are the same or different are compared, and then images having a higher similarity are compared, so that it is expected that a teaching input with a high degree of accuracy will be obtained even by a relative comparison between the displayed images.

[0060] Also, for example, the image set may be dynamically configured according to the result of the previous teaching input. The multiple learning images may include images that are expected to have a clear tissue state and small fluctuations in the teaching input result, and images that are not so different from other learning images that the user may have difficulty in judging. When including learning images in multiple image sets, it is desirable to display images that are difficult to judge in combination with various other images so that a multifaceted judgment can be made, while it is wasteful to make teaching input multiple times for images whose state is clear. By reflecting the results of past teaching inputs in the combination of learning images in subsequent image sets, it is possible to respond to such situations and perform teaching work efficiently.

[0061] For example, the image processing method according to the present invention may further include a step of performing machine learning on a learning model based on a plurality of learning images and teaching inputs associated with the learning images. The trained model thus constructed using the teaching input results with little fluctuation has a function of accurately determining the state of tissue contained in an unknown specimen image.

[0062] For example, the specimen image may be an image of a tissue specimen stained with hematoxylin and eosin (HE) captured in bright field. HE staining can distinguish between cell nuclei and cytoplasm. The image captured in bright field has good compatibility with visual observation by the user. [Industrial Applicability]

[0063] The present invention effectively assists users in constructing a trained model for automatically determining tissue conditions from images, and in particular can improve the efficiency and accuracy of teaching work on training images. [Explanation of symbols]

[0064] Aa1, Aa2, ... Learning images Ba, Bb, Bc buttons Wa, Wb, Wc windows

Claims

1. extracting features of objects in each of a plurality of learning images of a tissue specimen by image processing; selecting a plurality of images from among the learning images based on the extracted features and a predetermined selection criterion, and constructing an image set including the selected plurality of learning images; outputting, to a display device, a display image in which a plurality of the learning images included in the image set are arranged on the same screen; receiving instruction input from a user for each of the displayed learning images and associating the learning image with a corresponding result of the instruction input; Equipped with forming a plurality of image sets each having a different combination of the learning images, switching between and displaying a plurality of the display images corresponding to each of the plurality of image sets, and receiving the instruction input each time; An image processing method in which the learning images are selected while changing the selection criteria, so that one learning image is included in multiple image sets, and each of the learning images is displayed on the display device multiple times while accepting the instruction input each time.

2. 2. The image processing method according to claim 1, wherein a similarity between said learning images calculated based on said features or a result of said instruction input is used as one of said selection criteria.

3. creating a first image set including one of the training images and a first training image, and a second image set including the one of the training images and a second training image, wherein a similarity of the second training image to the one of the training images is greater than that of the first training image; 3. The image processing method according to claim 2, wherein the display image corresponding to the second image set is displayed after the display image corresponding to the first image set.

4. 4. The image processing method according to claim 1, wherein the image set is dynamically constructed in accordance with a result of a previous instruction input.

5. The image processing method according to claim 1 , further comprising a step of causing a learning model to perform machine learning based on a plurality of the learning images and the teaching input associated therewith.

6. 6. The image processing method according to claim 1, wherein the learning images are images obtained by bright-field imaging of the tissue specimen stained with hematoxylin and eosin.

Images