Trained Model Creation Method and Trained Model Creation System

Abstract

This trained model generation method comprises: a step for generating a trained model (50) on the basis of learning data (86) which includes first training image data (86b); a step for determining whether or not an inference region in inference result data (84) is incorrectly inferred; a step for determining a noise region (60) included in the first training image data (86b); a step for updating the first training image data (86b); and a step for re-generating trained model (55).

Description

Technical Field

[0001] The present invention relates to a method for creating a learned model and a system for creating a learned model.

Background Art

[0002] Conventionally, a method for creating a learned model by machine learning used for performing segmentation processing of cell images has been known. Such a method for creating a learned model is disclosed, for example, in the re-published patent WO2019 / 171453.

[0003] In the above re-published patent WO2019 / 171453, in an observation image of pluripotent stem cells such as iPS cells and ES cells, in order to identify regions such as undifferentiated cells and undifferentiated deviated cells that have deviated from the undifferentiated state, it is disclosed that segmentation is performed on the observation image. The segmentation of the observation image is performed by a learned model created by machine learning. By performing segmentation on the input image (observation image) to be processed, a label image in which regions such as an undifferentiated cell region (normal cell region), an undifferentiated deviated cell region (abnormal cell region), and a background region where no cells exist are labeled respectively is output. The above re-published patent WO2019 / 171453 also discloses a method for creating a learning model for creating a learning model used when performing segmentation of an observation image.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, as a result of performing segmentation on the input image, in the label image, there may be a case where the label of the "abnormal cell" region is erroneously assigned to a part that should be labeled as the "background" region. On the other hand, in machine learning, due to learning the part where the label of the "abnormal cell" region can be erroneously assigned as described above as the "background" region, in the label image, there may be a case where the label of the "abnormal cell" region is not correctly assigned to the region that should be labeled as the "abnormal cell" region. That is, there is a problem that the detection target is not correctly detected. In particular, when the "abnormal cells" appear less frequently than the "normal cells", the decrease in the detection accuracy of the "abnormal cells" becomes a significant practical problem.

[0006] The present invention has been made to solve the above-described problems, and one object of the present invention is to provide a learned model creation method and a learned model creation system capable of suppressing a decrease in the detection accuracy of a detection target.

Means for Solving the Problems

[0007] The learned model creation method according to the first aspect of the present invention is to use a learned model used for performing segmentation processing of a cell image by machine learning by the control unitA method of creation, comprising: creating a learned model by performing machine learning using first input image data that is a cell image as an input image and first teacher image data that corresponds to the first input image data and is labeled with at least a first label indicating a detection target and a second label indicating a background as an output image; using the created learned model, using second input image data that is cell image data different from the first input image data as an input image, and second teacher image data that corresponds to the second input image data and is labeled with at least the first label and the second label, performing an inference process of outputting inference result data labeled with at least the first label and the second label as an output image by a segmentation process; creating difference image data indicating the difference between the second teacher image data and the inference result data; whether the labels labeled in the second teacher image data and the inference result data are the same using based on the difference image data, determining whether a region estimated to be a detection target or a background in the inference result data is a misestimation; the control unit in a region determined to be a misestimation in the region estimated in the inference result data, identifying a noise region that does not match the first label and the second label in the difference image data; from the shown difference the control unit identifying the shape of the noise region specified in the inference result data; as the region in the first teacher image data having a shape similar to noise region the control unit determining the noise region; based on the noise region determined in the first teacher image data, further labeling the determined noise region with a third label indicating the noise region and updating the first teacher image data; by the control unit re-creating the learned model by performing additional learning on the learned model using the updated learning data composed of the first input image data and the updated first teacher image data, with the first input image data as the input image and the updated first teacher image data as the output image.

[0008] The learned model creation system in the second aspect of the present invention is a learned model creation system that creates a learned model by machine learning for performing segmentation processing of cell images. The system includes a storage unit that stores learning data composed of first input image data, which is a cell image, and first teacher image data corresponding to the first input image data and labeled with at least a first label indicating a detection target and a second label indicating a background, and inference processing data composed of second input image data, which is cell image data different from the first input image data, and second teacher image data corresponding to the second input image data and labeled with at least the first label and the second label. The system also includes a control unit. The control unit performs control to create a learned model by performing machine learning with the first input image data in the learning data stored in the storage unit as the input image and the first teacher image data as the output image. Then, using the created learned model, the control unit performs inference processing to output, as the output image, inference result data labeled with at least the first label and the second label by segmentation processing, with the second input image data in the inference processing data stored in the storage unit as the input image. The control unit also performs control to create difference image data indicating the difference between the second teacher image data and the inference result data. whether the labels labeled in the second teacher image data and the inference result data are the same using Based on the difference image data, the control unit determines whether a region estimated to be a detection target or a background in the inference result data is a misestimation. In a region where it is determined that the region estimated in the inference result data is a misestimation, the control unit from the shown difference identifies a noise region that does not match the first label and the second label in the difference image data. The shape of the noise region identified in the inference result data as the region in the first teacher image data having a shape similar to noise region asControl for making a determination, based on the noise region determined in the first teacher image data, a third label indicating the noise region is further labeled for the determined noise region, control for updating the first teacher image data, and using the updated learning data composed of the first input image data and the updated first teacher image data, performing additional learning on the learned model with the first input image data as the input image and the updated first teacher image data as the output image, thereby reconstructing the learned model, is configured to be performed.

Effect of the Invention

[0009] As described above, the method for creating a learned model according to the first aspect of the present invention includes a step of determining whether or not a region estimated to be a detection target or a background in the inference result data is a misestimation, a step of identifying a noise region in a region determined to be a misestimation among the estimated regions, a step of determining a noise region included in the first teacher image data, a step of further labeling a third label indicating the noise region for the region determined to be the noise region in the first teacher image data, and updating the first teacher image data, and a step of recreating a learned model based on the updated learning data. Here, it is conceivable to grasp in advance a background region in the input image that may be erroneously labeled as a detection target, and to set the types of labels in segmentation to three types: a detection target region, a background region, and a noise region in advance. However, by checking the label image in the result of segmenting the input image performed by the learned model created by machine learning, it is often possible to recognize for the first time which region is the noise region. That is, it is difficult to recognize in advance which regions can be noise regions. Therefore, there is a problem that it is difficult to create a learned model in advance based on learning data including teacher image data simply labeled with three types: a detection target region, a background region, and a noise region. Therefore, by configuring to identify a noise region in a region determined to be a misestimation in the inference result data, the noise region included in the first teacher image data can be identified, so that the noise region can be detected separately from regions with other labels and relearned. Therefore, a learned model can be recreated based on the updated learning data, and as a result, a decrease in the detection accuracy of the detection target can be suppressed.

[0010] In the learned model creation system according to the second aspect of the present invention, as described above, for the noise region corresponding to the noise region included in the first teacher image data among the regions misestimated as detection targets in the inference result data output by the inference processing execution unit, the updated learning data including the updated first teacher image data further labeled with a third label indicating the noise region is further stored, and based on the updated learning data, the learned model is recreated. By configuring in this way, similar to the first aspect, it is possible to relearn to detect the noise region included in the first teacher image data separately from the regions assigned with other labels. Therefore, the learned model can be recreated based on the updated learning data, and as a result, it is possible to suppress a decrease in the detection accuracy of the detection target.

Brief Description of Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments embodying the present invention will be described with reference to the drawings.

[0013] [First Embodiment] With reference to FIGS. 1 to 6, the configuration of the image analysis system 200 including the learned model creation system 100 according to the first embodiment and the method for creating the learned model will be described.

[0014] (Image Analysis System) The image analysis system 200 shown in FIG. 1 includes a learned model creation system 100 for creating a learned model by machine learning used for performing segmentation processing of the cell image 81. The learned model creation system 100 includes a control unit 11 and a storage unit 12 included in the server 10.

[0015] The image analysis system 200 includes a server 10, a computer 20, and an imaging device 30. FIG. 1 shows an example of the image analysis system 200 constructed in a client-server model. The server 10, the computer 20, and the imaging device 30 are communicably connected to each other via a network 40.

[0016] The computer 20 is a client terminal operated by a user and transmits requests (processing requests) for various processes to the server 10. The computer 20 includes a display unit 21. The computer 20 can acquire the cell image 81 captured by the imaging device 30 from the imaging device 30 and display the acquired cell image 81 on the display unit 21. The display unit 21 is, for example, a liquid crystal display device.

[0017] The server 10 performs various information processes in response to requests (processing requests) from the computer 20. The server 10 includes a control unit 11 and a storage unit 12.

[0018] Server 10 (control unit 11) creates a learned model 50. Also, server 10 (control unit 11) executes inference processing. Inference processing is a process of outputting an inference result based on an input using the learned model 50 generated by machine learning. Also, server 10 (control unit 11) performs image processing on the cell image 81 using the learned model 50. As a result of the image processing, server 10 (control unit 11) generates a processed image 82 of the cell image 81. Server 10 (control unit 11) transmits the generated processed image 82 to computer 20. Computer 20 that has received the information causes the display unit 21 to display the processed image 82.

[0019] Control unit 11 is configured to create a learned model 50 based on learning data 86 (see FIG. 2). Also, control unit 11 is configured to execute inference processing using the created learned model 50 with inference processing data 87 (see FIG. 2). In this embodiment, based on the result of the inference processing for the inference processing data 87 (see FIG. 2), the learning data 86 is corrected and updated as learning data 88. Also, control unit 11 is configured to recreate a learned model 55 (see FIG. 3) based on the updated learning data 88 (see FIG. 2). The recreated learned model 55 (see FIG. 3) is used to perform segmentation processing of the cell image 81.

[0020] Also, control unit 11 is configured to create difference image data 83 (see FIG. 2) indicating the difference between the inference processing data 87 (see FIG. 2) and the inference result data 84 (see FIG. 2). Control unit 11 includes a processor such as a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), or an ASIC (Application Specific Integrated Circuit). The processor executes a predetermined program, thereby performing arithmetic processing as control unit 11.

[0021] The storage unit 12 is a device that stores information. The storage unit 12 stores the learned model 50, the recreated learned model 55 (see Fig. 2), and various image data 80. The storage unit 12 includes a non-volatile storage device. The non-volatile storage device is, for example, a hard disk drive, a solid state drive, or the like.

[0022] The imaging device 30 generates a cell image 81 obtained by imaging cells. The imaging device 30 can transmit the generated cell image 81 to the computer 20 and / or the server 10 via the network 40. The cell image 81 is, for example, an image of cultured cells cultured using a cell culture instrument. The cell image 81 to be imaged is a microscopic image. The imaging device performs imaging by imaging methods such as bright field observation method, dark field observation method, phase contrast observation method, differential interference observation method, etc.

[0023] The network 40 connects the server 10, the computer 20, and the imaging device 30 so that they can communicate with each other. The network 40 can be, for example, a LAN (Local Area Network) constructed within a facility. The network 40 can be, for example, the Internet. When the network is the Internet, the image analysis system 200 can be a system constructed in the form of cloud computing.

[0024] (Specific device configuration of the image analysis system) Next, an example of the specific device configuration of the image analysis system 200 will be described.

[0025] The imaging device 30 is an in-line holographic microscope. The imaging device 30 includes a light source unit 31 including a laser diode or the like and an image sensor 32. At the time of imaging, a culture plate 33 containing a cell colony (or a single cell) is disposed between the light source unit 31 and the image sensor 32. The imaging device 30 generates an IHM (In-line Holographic Microscopy) phase image which is the cell image 81.

[0026] The computer 20 includes a processor 22, a storage unit 23, a display unit 21, and an input unit 24 which is a user interface. The input unit 24 includes input devices such as a mouse and a keyboard.

[0027] The processor 22 is configured to control the operation of the imaging device 30 and process the data acquired by the imaging device 30 by executing a program stored in the storage unit 23. The processor 22 includes an imaging control unit 22a and a cell image creation unit 22b as functional blocks.

[0028] When a culture plate 33 containing cell colonies is set at a predetermined position of the imaging device 30 by the user and a predetermined operation is received via the input unit 24, the processor 22 controls the imaging device 30 by the imaging control unit 22a to acquire hologram data.

[0029] Based on the control of the imaging control unit 22a, the imaging device 30 irradiates coherent light from the light source unit 31. The imaging device 30 acquires, by the image sensor 32, an image formed by interference fringes between the light transmitted through the culture plate 33 and the cell colonies 34 and the light transmitted through the vicinity region of the cell colonies 34 on the culture plate 33. The image sensor 32 acquires hologram data (two-dimensional light intensity distribution data of the hologram formed on the detection surface).

[0030] The cell image creation unit 22b calculates phase information by performing arithmetic processing for phase recovery on the hologram data acquired by the imaging device 30. Then, the cell image creation unit 22b creates an IHM phase image (cell image 81) based on the calculated phase information. Since known techniques can be used for the calculation of phase information and the creation method of the IHM phase image, detailed description is omitted.

[0031] The created cell image 81 is stored in the storage unit 23. Also, the created cell image 81 is transmitted by the processor 22 to the server 10.

[0032] (Server) As shown in FIG. 2, the control unit 11 of the server 10 includes a learned model creation unit 11a, an inference process execution unit 12b, and an image processing unit 11c as functional blocks.

[0033] The storage unit 12 stores the created learned model 50. The storage unit 12 stores the recreated learned model 55. The storage unit 12 stores a plurality of cell images 81 captured by the imaging device 30. For example, all the cell images 81 captured by the imaging device 30 are automatically transmitted to the server 10 and stored in the storage unit 12. Further, the storage unit 12 stores learning data 86 in which the first input image data 86a and the first teacher image data 86b corresponding to the first input image data 86a are associated. The learning data 86 is used to create the learned model 50. The first teacher image data 86b is the correct image to be output as a result of the segmentation process for the first input image data 86a. Further, the storage unit 12 stores inference process data 87 in which the second input image data 87a and the second teacher image data 87b corresponding to the second input image data 87a are associated. The inference process data 87 is used for the inference process. The second teacher image data 87b is the correct image to be output as a result of the inference process for the second input image data 87a. Further, the storage unit 12 stores updated learning data 88 including the first input image data 86a and the updated first teacher image data 88b. Further, the storage unit 12 stores difference image data 83 indicating the difference between the second teacher image data 87b of the inference process data 87 and the inference result data 84 corresponding to the second teacher image data 87b.

[0034] 〈Learned Model Creation Unit〉 The learned model creation unit 11a is configured to create a learned model 50 by performing machine learning using the learning data 86 stored in the storage unit 12. The learned model creation unit 11a creates the learned model 50 based on the learning data 86 including the first input image data 86a and the first teacher image data 86b. Further, the learned model creation unit 11a re-creates the learned model 55 based on the updated learning data 88 including the first input image data 86a and the updated first teacher image data 88b.

[0035] 〈Inference Processing Execution Unit〉 The inference processing execution unit 11b is configured to perform inference processing using the inference processing data 87 stored in the storage unit 12. The inference processing execution unit 11b uses the second input image data 87a included in the inference processing data 87 to execute segmentation inference processing by the learned model 50 and outputs inference result data 84.

[0036] 〈Image Processing Unit〉 The image processing unit 11c is configured to perform image processing on the cell image 81 using the re-created learned model 55 stored in the storage unit 12. The image processing unit 11c uses the cell image 81 selected as the processing target as an input image and inputs it to the re-created learned model 55 to perform segmentation processing on the input image. As a result of the segmentation processing, the image processing unit 11c outputs, as a processed image 82, a label image in which the cell image 81 input to the re-learned learned model 55 is divided into a plurality of label regions. The image processing unit 11c transmits the processed image 82 to the computer 20 and stores it in the storage unit 12.

[0037] In this specification, the "segmentation process" is a process of dividing an input image into a plurality of regions, and is a process of classifying the input image into a plurality of labeled regions by attaching a label indicating the detection target to the region where the detection target appears. A label is information representing the meaning indicated by an image portion. A labeled region is a region (a part of an image) composed of a group of pixels to which a common label is attached in the image.

[0038] As shown in FIG. 3, the reconstructed learned model 55 performs a segmentation process on the input cell image 81 and outputs a processed image 82 divided into a plurality of labeled regions. Segmentation is performed by attaching (labeling) a label to each pixel in the image. The label may be attached in units of a group of a plurality of pixels. The type of label is called a class.

[0039] As shown in FIG. 2, the image processing unit 11c is configured to label the noise region 60 (see FIG. 4) with a third label 63 (see FIG. 5) indicating the noise region 60 for the region determined as the noise region 60 (see FIG. 4) in the first teacher image data 86b based on a user operation on the input unit 24 (see FIG. 1) of the computer 20, and to obtain updated first teacher image data 88b. The noise region 60 (see FIG. 4) will be described later.

[0040] In addition, the image processing unit 11c labels a third label (see FIG. 5) indicating the noise region 60 (see FIG. 4), and when the number of updated first teacher image data 88b is less than a predetermined number, based on a user operation on the input unit 24 (see FIG. 1) of the computer 20, it is configured to further create first teacher image data 88b with the third label 63 (see FIG. 5) labeled. Specifically, based on a user operation on the input unit 24 (see FIG. 1) of the computer 20, a part or all of the noise region 60 (see FIG. 4) and the determined region in the first teacher image data 86b are cut out from the first teacher image data 86b, and the cut-out noise region 60 is embedded in the background of the cell image 81 stored in the storage unit 12, thereby adding first teacher image data 88b with the third label 63 labeled. The above-mentioned predetermined number is the number of updated first teacher image data 88b that is considered to be statistically sufficient and set by the user in advance. Alternatively, the image processing unit 11c, based on a user operation on the input unit 24 (see FIG. 1) of the computer 20, diverts first teacher image data 88b with the third label 63 labeled in other learning data stored in the storage unit 12 and adds first teacher image data 88b with the third label 63 labeled.

[0041] In addition, the image processing unit 11c is configured to create difference image data 83 indicating the difference between the second teacher image data 87b of the inference processing data 87 and the inference result data 84 corresponding to the second teacher image data 87b.

[0042] (Difference image data) As shown in FIG. 4, the difference image data 83 is image data for extracting the misestimated regions in the inference result data 84. The difference image data 83 is image data showing the matching points and differences between the second teacher image data 87b included in the inference processing data 87 (see FIG. 2) and the inference result data 84 corresponding to the second teacher image data 87b. The second teacher image data 87b is labeled with a first label 61 indicating the detection target and a second label 62 indicating the background. Also, the inference result data 84 is also labeled with a first label 61 indicating the detection target and a second label 62 indicating the background based on the inference.

[0043] In FIG. 4, an example is shown in which in the second teacher image data 87b and the inference result data 84, an image (cell image) of pluripotent stem cells such as iPS cells and ES cells is regionally divided into three classes: a “normal cell” region (undifferentiated cell region that is a cell maintaining pluripotency) that is the detection target, an “abnormal cell” region (undifferentiated deviation cell region that is a cell deviating from the undifferentiated state) that is the detection target, and other “background” regions. Note that in the segmentation process, the number of classes and the label contents to be detected may differ depending on the usage purpose of the created learned model. Therefore, in the present invention, the number of classes and the label contents to be detected are not particularly limited.

[0044] In the difference image data 83, in the corresponding regions of the second teacher image data 87b and the inference result data 84, a first region 66 labeled with the first label 61 in both is visibly displayed for identification. Also, a second region 67 labeled with the second label 62 in the second teacher image data 87b and labeled with the first label 61 in the inference result data 84 is visibly displayed for identification. Also, a third region 68 labeled with the first label 61 in the second teacher image data 87b and labeled with the second label 62 in the inference result data 84 is visibly displayed for identification. Also, in the corresponding regions of the second teacher image data 87b and the inference result data 84, a fourth region 69 labeled with the second label 62 in both is visibly displayed for identification.

[0045] (Noise area) For example, scratches on the culture surface shown in the input image data are not objects to be detected. Therefore, in the teacher image data, areas such as scratches on the culture surface are labeled as background areas. That is, the teacher image data does not represent areas such as scratches on the culture surface. However, when performing machine learning using such input image data and teacher image data, the learning model learns the background as non-uniform. When performing segmentation processing using the created learned model, cells in the cell image that have a shape similar to scratches may be labeled as the background area. Also, when performing machine learning using input image data and teacher image data without scratches on the culture surface, the learning model learns the background as uniform. When performing segmentation processing using the created learned model, cells in the cell image that have a shape similar to scratches on the culture surface may be labeled as the detection target area. In such a processed image, background areas that may be erroneously labeled as detection targets can often be recognized only by checking the label image in the segmentation result, and it is difficult to recognize them in advance. Similarly, it is difficult to recognize in advance the detection target areas that may be erroneously labeled as background areas in the processed image. In the present embodiment, such background areas in the processed image that may be erroneously labeled as detection targets, and detection target areas that may be erroneously labeled as background areas in the processed image are referred to as noise areas 60. The noise area 60 shown in FIG. 4 is a background area in the processed image that may be erroneously labeled as a detection target and is a foreign object.

[0046] The noise region 60 is a region that is determined by the image processing unit 11c (see FIG. 2) to be a misestimation in the inference result data 84 output by the inference process, and the region determined to be this misestimation does not match the first label 61 and the second label 62. In the present embodiment, the noise region 60 is a region in which it is determined that the "normal cell" region or the "abnormal cell" region in the inference result data 84 is a misestimation, and the region determined to be this misestimation does not match the first label 61 and the second label 62. Further, the noise region 60 is a region specified by the user as a region having a large (non-ignorable) impact on the detection accuracy of the detection target. For example, an object that does not correspond to "normal cells" and "abnormal cells" included in the "background" region and is erroneously estimated as an "abnormal cell" is a region specified as having a large impact on the detection accuracy of the detection target.

[0047] (Update of the first input image data) Therefore, in the present embodiment, a learned model 50 is created by performing machine learning using learning data 86 including first input image data 86a and first teacher image data 86b. The first input image data 86a may include a noise region 60. Next, inference processing is executed by the created learned model 50 using second input image data 87a included in inference processing data 87, and inference result data 84 is output. The second input image data 87a may also include a noise region 60. Difference image data 83 showing the difference between the inference result data 84 and second teacher image data 87b included in the inference processing data 87 is created. As shown in FIG. 4, the user compares the difference image data 83 and the inference result data 84. At this time, when the noise region 60 is included in the second input image data 87a, a second region 67 appears in the difference image data. Therefore, the user can identify the noise region 60 included in the inference result data 84. The user determines the noise region 60 that may be included in the first input image data 86a. For example, in the first input image data 86a, the user can determine, as the noise region 60, a region whose shape or the like is similar to the noise region 60 specified in the inference result data 84. The user can assign a third label 63 to the noise region 60 determined in the first input image data 86a by the input unit 24. The computer 20 or the image processing unit 11c updates the first input image data 86a to which the third label 63 is assigned, and stores it in the storage unit 12 as updated first input image data 88a.

[0048] (Overview of Machine Learning, Inference Processing, Machine Learning (Relearning), and Segmentation Processing) Referring to FIG. 3, the overview of machine learning, inference processing, machine learning (relearning), and segmentation processing according to the present embodiment will be described.

[0049] 〈Machine Learning〉 The learned model creation unit 11a (see FIG. 2) is configured to perform machine learning using the learning data 86 stored in the storage unit 12. The learned model creation unit 11a acquires the learning data 86 from the storage unit 12. The learning data 86 is configured to include a sufficient number of data. Each piece of learning data 86 includes first input image data 86a that is a processing target and first teacher image data 86b. The learned model creation unit 11a inputs the first input image data 86a to the learning model 51 and outputs the first teacher image data 86b, and learns the conversion process (segmentation process) from the first input image data 86a to the first teacher image data 86b. By performing machine learning, a learned model 50 is generated.

[0050] In the present embodiment, the classes of the first teacher image data 86b are composed of three classes: a "normal cell" region as a detection target (for example, an undifferentiated cell region in pluripotent stem cells such as iPS cells and ES cells), an "abnormal cell" region as a detection target (for example, an undifferentiated deviation cell region deviating from the undifferentiated state in iPS cells and ES cells), and a "background" region.

[0051] As a machine learning method for the learned model 50, any method such as a fully convolutional neural network (FCN), a neural network, a support vector machine (SVM), or boosting can be used. As an example, a convolutional neural network frequently used for semantic segmentation is used for the learned model 50. Such a learned model 50 is configured to include an input layer, a convolutional layer, and an output layer into which an image is input.

[0052] <Inference process> The inference processing execution unit 11b (see FIG. 2) is configured to perform inference processing using the second input image data 87a included in the inference processing data 87 stored in the storage unit 12 by the trained model 50 created. The inference processing execution unit 11b acquires the second input image data 87a included in the inference processing data 87 from the storage unit 12. The inference processing data 87 is configured to include a sufficient number of data. During the inference processing, by inputting the second input image data 87a used for the inference processing into the trained model 50, the segmentation processing of the second input image data 87a can be performed, and the inference result data 84 divided into a plurality of label regions can be output.

[0053] The inference processing data 87 used for the inference processing is data different from the learning data 86 used for machine learning. The classes of the second teacher image data 87b and the inference result data 84 are configured in three classes: a "normal cell" region as a detection target, an "abnormal cell" region as a detection target, and a "background" region, similar to the class of the first teacher image data 86b.

[0054] 〈Machine Learning (Relearning)〉 The trained model creation unit 11a is configured to relearn the trained model 50 using the updated learning data 88 stored in the storage unit 12. The trained model creation unit 11a acquires the updated learning data 88 from the storage unit 12. The updated learning data 88 includes the first input image data 86a to be processed and the updated first teacher image data 88b. The trained model creation unit 11a inputs the first input image data 86a into the trained model 50 and outputs the updated first teacher image data 88b, and learns the conversion processing (segmentation processing) from the first input image data 86a to the updated first teacher image data 88b. By performing machine learning (relearning), a newly created trained model 55 is generated.

[0055] In this embodiment, the classes of the updated first teacher image data 88b are composed of four classes: the "normal cell" region as the detection target, the "abnormal cell" region as the detection target, the "background" region, and the "noise" region.

[0056] <Segmentation Processing> The recreated learned model 55 performs segmentation processing on the input cell image 81 and outputs a processed image 82 divided into a plurality of label regions.

[0057] In an example of the processed image 82 output using the learned model 50 before recreation shown in FIG. 5, the region 64a is a part of the "background" region, and the region 64b is the "abnormal cell" region that is the detection target. Also, in an example of the processed image 82 output using the recreated learned model 55, the region 64a is a part of the "background" region, and the region 64b is the "abnormal cell" region that is the detection target. In the processed image 82 output using the learned model 50 before update, the region 64a is erroneously detected as the "abnormal cell" region, and the region 64b is erroneously detected as the "background" region. On the other hand, in the processed image 82 output using the updated learned model 55, the region 64a is correctly detected as the noise region 60, and the region 64b is correctly detected as the "abnormal cell" region.

[0058] (Machine Learning, Inference Processing, and Machine Learning (Relearning) Processing) With reference to FIG. 6, the machine learning, inference processing, and machine learning (relearning) processing according to this embodiment will be described.

[0059] In step S1, the learned model creation unit 11a acquires learning data 86 including the first input image data 86a and the first teacher image data 86b from the storage unit 12. Then, the process proceeds to step S2.

[0060] In step S2, the trained model creation unit 11a inputs the first input image data 86a into the trained model 50 and outputs the first teacher image data 86b, and trains the conversion process (segmentation process) from the first input image data 86a to the first teacher image data 86b to create the trained model 50. Thereafter, the process proceeds to step S3.

[0061] In step S3, the inference process execution unit 11b executes an inference process by inputting the second input image data 87a stored in the storage unit 12 into the trained model 50 that has been created, and outputs inference result data 84. Thereafter, the process proceeds to step S4.

[0062] In step S4, the image processing unit 11c creates difference image data 83 indicating the difference between the second teacher image data 87b stored in the storage unit 12 and the inference result data 84. Further, the server 10 (control unit 11) transmits the created difference image data 83 and the data of the inference result data 84 to the computer 20, and the computer 20 causes the display unit 21 to display the difference image data 83 and the inference result data 84. Thereafter, the process proceeds to step S5.

[0063] In step S5, based on the operation of the input unit 24 by the user, the control unit 11 obtains a determination result as to whether or not the region estimated to be the detection target in the inference result data 84 output by the inference process based on the difference image data 83 and the inference result data 84 displayed on the display unit 21 is a false estimation. Thereafter, the process proceeds to step S6.

[0064] In step S6, based on the operation of the input unit 24 by the user, the control unit 11 obtains a specific result in which a noise region 60 that does not match the first label 61 and the second label 62 is specified in the region determined to be a false estimation in the region estimated to be the detection target in the inference result data 84. Thereafter, the process proceeds to step S7.

[0065] In step S7, based on the operation of the input unit 24 by the user, a determination result is obtained as to whether the influence of the noise region 60 on the detection accuracy of the detection target is significant. If the determination result is that the influence of the noise region 60 on the detection accuracy of the detection target is significant (Yes in step S7), the process proceeds to step S8. If the determination result is that the influence of the noise region 60 on the detection accuracy of the detection target is not significant (No in step S7), the process ends.

[0066] In step S8, based on the operation of the input unit 24 by the user, a determination result is obtained as to the determined noise region 60 based on the noise region 60 specified in the inference result data 84 and included in the first teacher image data 86b. Then, the process proceeds to step S9.

[0067] In step S9, the user checks the difference image data 83 or checks the learning data 86 including the first teacher image data 86b, and labels the region determined as the noise region 60 in the first teacher image data 86b with the third label 63 indicating the noise region 60. For example, the labeling of the third label 63 can be performed via the input unit 24 of the computer. Note that the labeling method of the third label 63 is not particularly limited. As the third label 63 is labeled, the first teacher image data 86b is updated. Then, the process proceeds to step S10.

[0068] In step 10, based on the operation of the input unit 24 by the user, a determination result is obtained as to whether the number of updated first teacher image data 88b is equal to or greater than the number of updated first teacher image data 88b preset by the user. If the determination result is that the number of updated first teacher image data 88b is equal to or greater than the preset number (Yes in step S10), the process proceeds to step S11. If the determination result is that the number of updated first teacher image data 88b is less than the preset number (No in step S10), the process proceeds to step S12.

[0069] In step S11, the trained model creation unit 11a reconstructs (retrains) the trained model 55 based on the updated training data 88 including the updated first teacher image data 88b and the first input image data 86a. By performing machine learning (retraining), the updated trained model 55 is created. Then, the process ends.

[0070] In step S12, the user further creates (adds) the first teacher image data 88b labeled with the third label 63. Then, the process proceeds to step S11.

[0071] (Effect of the First Embodiment) In the first embodiment, the following effects can be obtained.

[0072] In the first embodiment, the learned model creation method includes, as described above, a step of determining whether or not a region estimated to be a detection target in the inference result data 84 is a misestimation, a step of identifying a noise region 60 in the region determined to be a misestimation of the estimated region and determining the noise region 60 included in the first teacher image data 86b, a step of further labeling the noise region 60 with a third label 63 indicating the noise region 60 for the noise region 60 and the region determined in the first teacher image data 86b, and updating the first teacher image data 86b, and a step of recreating the learned model 55 based on the updated learning data 88. Here, it is conceivable to grasp in advance a background region in which a label may be erroneously assigned as a detection target in the input image, and to set the types of labels in segmentation to three types: a detection target region, a background region, and a noise region in advance. However, by checking the label image in the result of segmenting the input image performed by the learned model created by machine learning, it is often possible to recognize for the first time which region is the noise region 60. That is, it is difficult to recognize in advance which region can be the noise region 60. Therefore, there is a problem that it is difficult to create a learned model in advance based on learning data including teacher image data in which three types of a detection target region, a background region, and a noise region are simply labeled. Therefore, by identifying the noise region 60 in the region determined to be a misestimation in the inference result data 84, the noise region 60 included in the first teacher image data 86b can be determined, so that the noise region 60 can be relearned to be detected separately from regions with other labels assigned. Therefore, the learned model 55 can be recreated based on the updated learning data 88, and as a result, a decrease in the detection accuracy of the detection target can be suppressed.

[0073] In the first embodiment, as described above, as image data for extracting the misestimated region in the inference result data 84, a step of creating difference image data 83 showing the difference between the second teacher image data 87b and the inference result data 84 corresponding to the second teacher image data 87b is further provided. Thereby, the difference between the second teacher image data 87b with the correct label and the inference result data 84 shown in the difference image data 83 can be easily recognized. Therefore, based on the difference image data 83, the misestimated region in the inference result data 84 can be easily recognized. As a result, the noise region 60 can be easily specified.

[0074] In the first embodiment, as described above, the first region 66 labeled with the first label 61, the second region 67 labeled with the second label 62 in the second teacher image data 87b and labeled with the first label 61 in the inference result data 84, and the third region 68 labeled with the first label 61 in the second teacher image data 87b and labeled with the second label 62 in the inference result data 84 are displayed discriminably to create difference image data. Thereby, in the difference image data 83, the first region 66 correctly estimated as the detection target region, the second region 67 where the background region is erroneously estimated as the detection target region, and the third region 68 where the detection target region is erroneously estimated as the background region are displayed discriminably. Therefore, the noise region 60 can be specified more easily.

[0075] In the first embodiment, as described above, the noise region 60 is part of the misestimated region. Thereby, among the regions where the background region is erroneously estimated as the detection target region, those having a large influence on the detection accuracy of the detection target can be specified as the noise region 60. Therefore, it is not necessary to specify all of the regions where the background region is erroneously estimated as the detection target region as the noise region 60, and thus the noise region 60 can be specified efficiently.

[0076] Also, in the first embodiment, as described above, the step of recreating the learned model 55 is based on the updated learning data 88 in which, in addition to the updated learning data 88, the first input image data 86a and the first teacher image data 88b labeled with the third label 63 are further added. Thereby, even when the number of the first teacher image data 86b labeled with the third label 63 in the updated learning data 88 is small, the number of the first teacher image data 88b labeled with the third label 63 can be ensured.

[0077] Also, in the first embodiment, the learned model creation system further acquires updated learning data 88 including updated first teacher image data 88b in which a third label 63 indicating the noise region 60 is further labeled for the noise region 60 included in the first teacher image data 86b corresponding to the noise region 60 among the regions erroneously estimated as detection targets in the inference result data 84 output by the inference processing execution unit 11b as described above. The learned model creation system is configured to recreate the learned model 55 based on the updated learning data 88. Thereby, it is possible to perform relearning so as to detect the noise region 60 included in the first teacher image data 88b separately from the regions assigned with other labels. Therefore, the learned model 55 can be recreated based on the updated learning data 88, and as a result, a decrease in the detection accuracy of the detection target can be suppressed.

[0078] [Second Embodiment] Next, a learned model creation system and a learned model creation method according to a second embodiment of the present invention will be described. Different from the first embodiment configured to determine whether or not a region estimated to be a detection target in the inference result data 84 output by the inference processing is an erroneous estimation, in the second embodiment, it is configured to determine whether or not a region estimated to be a low-occurrence detection target whose appearance frequency included in the detection target in the inference result data 84 is equal to or less than a predetermined threshold is an erroneous estimation.

[0079] The influence on the detection accuracy of the detection target due to the noise region 60 becomes prominent when the appearance frequency of the detection target is low (when it is a low-appearance detection target). For example, assume there are 1000 cell images 81 for performing segmentation processing, and only 5 cell images out of the 1000 have the low-appearance detection target shown. In this case, if there are 5 processed images 82 that are erroneously detected with the noise region 60 being the "low-appearance detection target" region, the correct answer rate of region detection drops to 50%. On the other hand, if the number of processed images erroneously detected with the noise region 60 being the "low-appearance detection target" region can be reduced from 5 to 1, the correct answer rate of region detection can be made 83.3%.

[0080] In the present embodiment, the low-appearance detection target is an "abnormal cell". Also, in the present embodiment, the noise region 60 is a region determined to be an incorrect estimation of the "abnormal cell" region in the inference result data 84, and the region determined to be this incorrect estimation does not match the first label 61 and the second label 62.

[0081] The learned model creation unit 11a shown in FIG. 2 is configured to acquire the appearance frequency of the detection target included in the learning data 86 including the first input image data 86a and the first teacher image data 86b acquired from the storage unit 12. Also, the learned model creation unit 11a is configured to determine whether the acquired appearance frequency of the detection target is equal to or less than a preset threshold. The case of being equal to or less than the preset threshold is, for example, when the first input image data 86a with the detection target shown is 3 or less per 10 first input image data, or when the ratio of the area of the detection target region to the total area of all the first input image data 86a is 10% or less. Note that the preset threshold is not limited to these and can be set as appropriate.

[0082] Further, when the appearance frequency of the detection target is equal to or lower than a preset threshold value, the trained model creation unit 11a determines that there is a low appearance detection target, and is configured to store in the storage unit 12 the fact that there is a low appearance detection target. Further, the server 10 (control unit 11) transmits information indicating that the learning data 86 includes a low appearance detection target to the computer 20, and the computer 20 that has received the information is configured to notify the user via the display unit 21 that the low appearance detection target is included.

[0083] The image processing unit 11c determines whether there is data of the first input image data 86a and the first teacher image data 86b including a low appearance detection target in other learning data 86 stored in the storage unit 12. When there is data of the first input image data 86a and the first teacher image data 86b including a low appearance detection target, the image processing unit 11c adds this data to the learning data 86. When there is no data of the first input image data 86a and the first teacher image data 86b including a low appearance detection target, or when adding the data of the first input image data 86a and the first teacher image data 86b including a low appearance detection target does not reach a preset number, the image processing unit 11c is configured to create data of the first input image data 86a and the first teacher image data 86b including a low appearance detection target. Specifically, based on a user operation on the input unit 24 (see FIG. 1) of the computer 20, the low appearance detection target is cut out from the first input image data 86a or the first teacher image data 86b including the low appearance detection target, and the cut-out low appearance detection target is embedded in the background of the learning data 86 stored in the storage unit 12, thereby adding data of the first input image data 86a and the first teacher image data 86b including the low appearance detection target. Note that the method for creating data of the first input image data 86a and the first teacher image data 86b including a low appearance detection target is not particularly limited, and a known data augmentation method can be used.

[0084] (Machine learning, inference processing, and machine learning (re - learning) processing) Referring to FIG. 7, the machine learning, inference processing, and machine learning (relearning) processing according to this embodiment will be described. Note that the processing different from that of the first embodiment will be described, and the description of the same processing as that of the first embodiment will be omitted.

[0085] In step S21, the trained model creation unit 11a obtains the appearance frequency of the detection target included in the training data 86 including the first input image data 86a and the first teacher image data 86b acquired from the storage unit 12, and determines whether there is a low-appearance detection target. If there is no low-appearance detection target (Yes in step S21), the process proceeds to step S2. If there is a low-appearance detection target (No in step S21), the process proceeds to step S22.

[0086] In step S22, the trained model creation unit 11a stores in the storage unit 12 that there is a low-appearance detection target, and the server 10 (control unit 11) notifies the user via the computer 20 that the low-appearance detection target is included. Then, the process proceeds to step S23.

[0087] In step S23, the image processing unit 11c determines whether there is data of the first input image data 86a and the first teacher image data 86b including the low-appearance detection target in other training data 86 stored in the storage unit 12. If there is data of the first input image data 86a and the first teacher image data 86b including the low-appearance detection target (Yes in step S23), the process proceeds to step S24. If there is no data of the first input image data 86a and the first teacher image data 86b including the low-appearance detection target, or if the preset number of sheets is not reached even after adding the training data 86 (No in step S23), the process proceeds to step S24.

[0088] In step S24, the image processing unit 11c adds the data of the first input image data 86a and the first teacher image data 86b including the low-appearance detection target to the training data 86. Then, the process proceeds to step S2.

[0089] In step S25, based on the operation of the input unit 24 by the user, data of the first input image data 86a and the first teacher image data 86b including the low-occurrence detection target is created (added). Then, the process proceeds to step S2.

[0090] In step S26, based on the operation of the input unit 24 by the user, a determination result as to whether or not the region estimated to be the "abnormal cell" region, which is a low-occurrence detection target in the inference result data 84 output by the inference process based on the difference image data 83 and the inference result data 84 on the display unit 21, is a mis-estimation is acquired. Then, the process proceeds to step S6.

[0091] Other configurations of the second embodiment are the same as those of the first embodiment described above.

[0092] (Effect of the Second Embodiment) In the second embodiment, the following effects can be obtained.

[0093] In the second embodiment, as described above, in the step of determining whether or not it is a mis-estimation, the method for creating a learned model determines whether or not the region estimated to be a low-occurrence detection target whose appearance frequency included in the detection target in the inference result data 84 is equal to or less than a predetermined threshold value is a mis-estimation. Thereby, the region determined to be a mis-estimation among the regions estimated to be low-occurrence detection targets can be specified as the noise region 60. Since the influence on the detection accuracy of the detection target caused by the noise region 60 becomes prominent when the appearance frequency of the detection target is low, a decrease in the detection accuracy of the detection target can be further suppressed.

[0094] Also, in the second embodiment, as described above, the low-occurrence detection target is an abnormal cell. Thereby, since the noise region 60 can be specified based on the region mis-estimated to be the abnormal cell region, a decrease in the detection accuracy of the abnormal cell, which is a low-occurrence detection target, can be further suppressed.

[0095] Also, in the second embodiment, as described above, the steps of obtaining the appearance frequency of the detection target included in the learning data 86 and, when the detection target with low appearance is included in the obtained appearance frequency of the detection target included in the learning data 86, notifying the user that the detection target with low appearance is included are further provided. Thereby, the user can be prompted to add the first teacher image data 86b labeled with the first label 61 indicating the detection target with low appearance. Therefore, it is possible to further suppress a decrease in the detection accuracy of the detection target caused by a noise region that becomes prominent when the appearance frequency of the detection target is low.

[0096] Also, in the second embodiment, as described above, in the step of creating the learned model, the learned model 50 is created based on the learning data 86 to which the learning data 86 including the detection target with low appearance is added. Thereby, it is possible to further suppress a decrease in the detection accuracy of the detection target caused by the noise region 60 that becomes prominent when the appearance frequency of the detection target is low.

[0097] Also, other effects of the second embodiment are the same as those of the first embodiment.

[0098] [Modification Example] It should be considered that the embodiments disclosed this time are illustrative in all respects and not restrictive. The scope of the present invention is shown not by the description of the above embodiments but by the claims, and further includes all changes (modification examples) within the meaning and scope equivalent to the claims.

[0099] For example, in the above-described first embodiment, an example is shown in which the inference processing execution unit 11b executes inference processing (step S3) and determines whether or not a region estimated to be a detection target in the inference result data 84 is a false estimation (step S5). However, the present invention is not limited to this. For example, after executing the inference processing and before determining whether or not it is a false estimation, the false-estimated region in the inference result data 84 may be configured to be displayed on the display unit 21 of the computer 20. The image processing unit 11c may extract, for example, a region that is misestimated as a detection target region in the inference result data 84 and is surrounded by a background region, and display it on the display unit 21 of the computer 20 as a specific example of the noise region 60. As a result, the user can visually recognize a specific example of the noise region 60, so that the noise region 60 can be specified more efficiently.

[0100] Also, when the misestimated region in the above-described inference result data 84 is to be displayed on the display unit 21 of the computer 20, the image processing unit 11c may be configured to display on the display unit 21 of the computer 20 a misestimated region that satisfies at least one condition related to the aspect ratio, occupied area, and shape complexity set in advance. As a result, the user can visually recognize a specific example that is easy to recognize as the noise region 60, so that the noise region 60 can be specified more efficiently. Note that the conditions related to the aspect ratio, occupied area, and shape complexity can be appropriately set according to the detection target and are not particularly limited. Also, the shape complexity may be set based on the perimeter length and occupied area of the noise region 60, or may be set according to other conditions.

[0101] Also, when the misestimated region in the above-described inference result data is to be displayed on the display unit 21 of the computer 20, the image processing unit 11c may label the third label 63 in the region of the second teacher image data 87b corresponding to the extracted noise region 60 in the inference result data 84. As a result, the user can visually recognize a specific example of the third label 63 in the noise region 60.

[0102] Further, in the above-described first embodiment, an example is shown in which in the processed image 82 (see FIG. 5) output using the recreated learned model 55, the region 64a is correctly displayed as the noise region 60. However, the present invention is not limited to this. The processed image 82 output using the recreated learned model 55 shown in FIG. 8 corresponds to the processed image 82 output using the updated learned model 50 shown in FIG. 5, except that the display indicating the noise region 60 is not shown. As a result of relearning the noise region 60, it is possible to suppress a decrease in the detection accuracy of the detection target due to the noise region 60. However, depending on the image analysis target, it may be more useful for the user and result in an analysis image that does not reduce the discriminability not to display the noise region 60 in the processed image 82. Therefore, as shown in FIG. 8, the processed image 82 may not have a display indicating the noise region 60.

[0103] Further, in the above-described first embodiment, an example is shown in which it is configured to determine whether or not a region estimated to be a detection target in the inference result data 84 output by the inference process is a false estimation. However, the present invention is not limited to this. For example, it may be configured to determine whether or not a region estimated to be a background in the inference result data 84 output by the inference process is a false estimation.

[0104] Further, in the above-described second embodiment, an example is shown in which the low-occurrence detection target is "abnormal cells". However, the present invention is not limited to this. For example, the low-occurrence detection target may be "normal cells".

[0105] In the first embodiment, the image processing unit 11c creates the difference image data 83, and the server 10 (control unit 11) is configured to transmit the created difference image data 83 and the inference result data 84 to the computer 20. However, the present invention is not limited to this. For example, the image processing unit 11c may further create overlay image data (not shown) in which the correct region label is overlaid on the inference result data 84 based on the created difference image data 83, and the server 10 (control unit 11) may be configured to transmit the created difference image data 83 and the overlay image data to the computer 20.

[0106] Also, in the first embodiment, the processes of steps S5 to S10 and step S12 in the machine learning, inference processing, and machine learning (relearning) processes are performed by the user, but the present invention is not limited to this. All of the processes of steps S5 to S10 and step S12 may be executed by the server 10 (control unit 11), or any one of the processes of steps S5 to S10 and step S12 may be performed by the user and the rest may be executed by the server 10 (control unit 11).

[0107] [Aspect] Those skilled in the art will understand that the above-exemplified embodiments are specific examples of the following aspects.

[0108] (Item 1) A method for creating a learned model by machine learning used for performing segmentation processing of cell images, creating a learned model based on learning data including first input image data and first teacher image data corresponding to the first input image data and labeled with at least a first label indicating a detection target and a second label indicating a background; executing an inference process using the created learned model with inference process data including second input image data; A step of determining whether an area estimated to be a detection target or background in the inference result data output by the inference process is a mis-estimation; In an area determined to be a mis-estimation in the estimated area, a step of identifying a noise area that does not match the first label and the second label; A step of determining a noise area included in the first teacher image data; In the first teacher image data, for the area determined to be the noise area, a third label indicating the noise area is further labeled, and the first teacher image data is updated; A step of recreating a learned model based on the updated learning data including the updated first teacher image data and the first input image data. A method for creating a learned model.

[0109] (Item 2) After the step of performing the inference process and before the step of determining whether it is a mis-estimation, at least the first label and the second label included in the inference process data are used as image data for extracting the mis-estimated area in the inference result data. A step of creating difference image data indicating the difference between the labeled second teacher image data and the inference result data corresponding to the second teacher image data is further provided. The method for creating a learned model according to Item 1.

[0110] (Item 3) The step of creating the difference image data creates difference image data in which, in corresponding areas of the second teacher image data and the inference result data, a first area in which the first label is labeled in both, a second area in which the second label is labeled in the second teacher image data and the first label is labeled in the inference result data, and a third area in which the first label is labeled in the second teacher image data and the second label is labeled in the inference result data are displayed discriminately. The method for creating a learned model according to Item 2.

[0111] (Item 4) The noise region is a part of the misestimated region, and the method for creating a learned model according to any one of Items 1 to 3.

[0112] (Item 5) The step of recreating the learned model is based on the updated learning data that, in addition to the updated learning data, further includes the first input image data and the learning data including the first teacher image data labeled with the third label, and recreates the learned model according to the method for creating a learned model according to any one of Items 1 to 4.

[0113] (Item 6) After the step of performing the inference process and before the step of determining whether it is a misestimation, the method for creating a learned model according to any one of Items 1 to 5 further includes a step of displaying, on the display unit, the misestimated region in the inference result data.

[0114] (Item 7) The step of displaying the misestimated region on the display unit is to display, on the display unit, the misestimated region that satisfies at least one of the conditions regarding the aspect ratio, the occupied area, and the complexity of the shape, which are preset, according to the method for creating a learned model according to Item 6.

[0115] (Item 8) The step of determining whether it is a misestimation is to determine whether a region estimated to be a low-occurrence detection target, in which the occurrence frequency included in the detection target in the inference result data is equal to or lower than a predetermined threshold, is a misestimation, according to the method for creating a learned model according to any one of Items 1 to 7.

[0116] (Item 9) The low-occurrence detection target is an abnormal cell, according to the method for creating a learned model according to Item 8.

[0117] (Item 10) Before the step of creating the learned model, a step of obtaining the occurrence frequency of the detection target included in the learning data, and When the detection target includes a low-occurrence detection target based on the obtained appearance frequency of the detection target, further comprising the step of notifying the user that the low-occurrence detection target is included, the learned model creation method according to item 9.

[0118] (Item 11) The step of creating a learned model is the learned model creation method according to item 10, which creates a learned model based on learning data to which learning data including a low-occurrence detection target is added.

[0119] (Item 12) A system for creating a learned model by machine learning used for performing segmentation processing of cell images, A storage unit that stores learning data including first input image data, and first teacher image data corresponding to the first input image data and labeled with at least a first label indicating a detection target and a second label indicating a background, and inference processing data including at least second input image data; A learned model creation unit that creates a learned model based on the learning data acquired by the storage unit; An inference processing execution unit that executes inference processing using the inference processing data stored in the storage unit by the created learned model, and is configured to further store updated learning data including the updated first teacher image data in which a third label indicating the noise region is further labeled for the noise region corresponding to the noise region in the first teacher image data that does not match the first label and the second label among the regions misestimated as detection targets in the inference result data output by the inference processing execution unit, and the first input image data, and the learned model creation unit is configured to recreate the learned model based on the updated learning data stored in the storage unit. A learned model creation system.

Explanation of Reference Numerals

Explanation of Reference Numerals

[0120] 11a Learned model creation unit 11b Inference processing execution unit 12 Memory unit 50 Trained model 55 Recreated trained model 60 Noise region 61 First label 62 Second label 63 Third label 66 First region 67 Second region 68 Third region 83 Difference image data 84 Inference result data 86 Training data 86a First input image data 86b First teacher image data 87 Data for inference processing 87a Second input image data 87b Second teacher image data 88 Updated training data 88a Updated first teacher image data 100 Trained model creation system

Claims

1. A method for creating a learned model used for performing segmentation processing of cell images by a control unit through machine learning, comprising: creating a learned model by performing machine learning using learning data composed of first input image data that is cell image data and first teacher image data corresponding to the first input image data and labeled with at least a first label indicating a detection target and a second label indicating a background, with the first input image data as the input image and the first teacher image data as the output image; using the created learned model, taking the second input image data, which is cell image data different from the first input image data, as the input image in the inference processing data composed of the second input image data and second teacher image data corresponding to the second input image data and labeled with at least the first label and the second label, and executing an inference process that outputs inference result data labeled with at least the first label and the second label as the output image by segmentation processing; creating difference image data indicating the difference between the second teacher image data and the inference result data; the control unit determining whether a region estimated to be the detection target or the background in the inference result data is a misestimation based on whether the labels labeled in the second teacher image data and the inference result data are the same using the difference image data; the control unit specifying a noise region in the region determined to be a misestimation in the inference result data that does not match the first label and the second label from the difference shown in the difference image data; the control unit determining, as the noise region, a region in the first teacher image data having a shape similar to the shape of the noise region specified in the inference result data; based on the noise region determined in the first teacher image data, the control unit further labels the determined noise region with a third label indicating the noise region and updates the first teacher image data. Using the updated learning data composed of the first input image data and the updated first teacher image data, for the learned model, using the first input image data as the input image and the updated first teacher image data as the output image, performing additional learning to recreate the learned model.

2. The step of creating the difference image data creates the difference image data in which, in corresponding regions of the second teacher image data and the inference result data, a first region in which the first label is labeled in both, a second region in which the second label is labeled in the second teacher image data and the first label is labeled in the inference result data, a third region in which the first label is labeled in the second teacher image data and the second label is labeled in the inference result data, and a fourth region in which the second label is labeled in both are discriminately displayed. The method for creating a learned model according to claim 1.

3. The noise region is included in the misestimated region. The method for creating a learned model according to claim 1.

4. The step of recreating the learned model is When the number of the updated learning data is less than a predetermined number, adding first additional learning data different from the updated learning data, including the first input image data and the first teacher image data labeled with the third label. Based on the updated learning data and the added first additional learning data, recreating the learned model. The method for creating a learned model according to claim 1.

5. After the step of executing the inference process and before the step of determining whether it is a misestimation, further comprising the step of displaying the misestimated region in the inference result data on a display unit. The method for creating a learned model according to claim 1.

6. The step of displaying the misestimated region on the display unit displays the misestimated region that satisfies at least one of a predetermined condition regarding an aspect ratio, an occupied area, and a complexity of a shape based on a perimeter length and an occupied area of the misestimated region, which are preset, on the display unit. The method for creating a learned model according to claim 5.

7. The step of determining whether it is the misestimation determines, based on whether the labels labeled in the second teacher image data and the inference result data are the same using the difference image data, whether a region estimated to be a low-occurrence detection target with a frequency of appearance among the detection targets in the inference result data being equal to or less than a predetermined threshold is a misestimation, and the control unit determines this. The learned model creation method according to claim 1.

8. The low-occurrence detection target is an abnormal cell. The learned model creation method according to claim 7.

9. Before the step of creating the learned model, obtaining the frequency of appearance of the detection target in the first input image data and the first teacher image data constituting the training data based on the ratio of the first input image data in which the detection target appears to the total number of the first input image data, or the ratio of the area of the region of the detection target to the total area of all the first input image data; When the low-occurrence detection target is included in the detection target based on the obtained frequency of appearance of the detection target, notifying the user that the low-occurrence detection target is included in the detection target. The learned model creation method according to claim 8.

10. The step of creating the learned model is When the number of the first input image data and the first teacher image data including the low-occurrence detection target is less than a preset number, adding second additional training data constituted by the first input image data and the first teacher image data including the low-occurrence detection target; Creating a learned model based on the training data and the added second additional training data. The learned model creation method according to claim 9.

11. A learned model creation system that creates a learned model used for performing segmentation processing of cell images by machine learning. Learning data composed of first input image data which is a cell image, and first teacher image data corresponding to the first input image data and labeled with at least a first label indicating a detection target and a second label indicating a background, second input image data which is cell image data different from the first input image data, and second teacher image data corresponding to the second input image data and labeled with at least the first label and the second label, and a storage unit that stores the inference processing data, a control unit, wherein the control unit, performs control to create a learned model by performing machine learning with the first input image data in the learning data acquired by the storage unit as an input image and the first teacher image data as an output image, performs control to execute an inference process of using the created learned model, taking the second input image data in the inference processing data stored in the storage unit as an input image, and outputting inference result data labeled with at least the first label and the second label by a segmentation process as an output image, performs control to create difference image data indicating the difference between the second teacher image data and the inference result data, performs control to determine whether or not a region estimated to be the detection target or the background in the inference result data is a mis-estimation based on whether or not the labels labeled in the second teacher image data and the inference result data are the same using the difference image data, performs control to identify a noise region that does not match the first label and the second label from the difference shown in the difference image data in a region determined to be a mis-estimation in the region estimated to be mis-estimated in the inference result data, performs control to determine a region in the first teacher image data having a shape similar to the shape of the noise region specified in the inference result data as the noise region, performs control to further label the determined noise region with a third label indicating the noise region based on the noise region determined in the first teacher image data and update the first teacher image data, Using the updated training data composed of the first input image data and the updated first teacher image data, for the learned model, the first input image data is used as the input image, and the updated first teacher image data is used as the output image to perform additional learning, thereby performing control to recreate the learned model, a learned model creation system configured to perform the above.

Images