Learned model generation method and learned model generation system

Abstract

The learned model generation method includes the steps of generating a learned model (50) based on learning data (86) including first training image data (86b); determining whether an estimated region in inference result data (84) is a misestimation; deciding a noise region (60) included in the first training image data (86b); updating the first training image data (86b); and regenerating the learned model (55).

Description

Technical Field

[0001] This invention relates to a method and system for generating learned models. Background Technology

[0002] A method for generating a machine learning-based learned model for cell image segmentation is known in the past. Such a method for generating a learned model is disclosed, for example, in Republication Patent No. WO2019 / 171453.

[0003] The aforementioned Republication Patent WO2019 / 171453 discloses the following: To segment observation images of pluripotent stem cells such as iPS cells and ES cells, regions of undifferentiated cells and undifferentiated escaped cells from an undifferentiated state are identified. The segmentation of the observation image is performed using a learned model generated by machine learning. By segmenting the input image (observation image) as the processing object, a labeled image is output, marking undifferentiated cell regions (normal cell regions), undifferentiated escaped cell regions (abnormal cell regions), and background regions without cells. The aforementioned Republication Patent WO2019 / 171453 also discloses a learning model generation method for generating the learning model used in segmenting the observation image.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Re-publication Patent No. WO2019 / 171453 Summary of the Invention

[0007] The problem the invention aims to solve

[0008] However, as a result of segmenting the input image, the labeled image sometimes incorrectly assigns the label of "abnormal cell" region to areas that should be labeled as "background" regions. On the other hand, sometimes machine learning learns that areas that might be incorrectly labeled as "abnormal cell" regions are "background" regions, resulting in regions in the labeled image not being correctly labeled as "abnormal cell" regions. In other words, there is a problem of failing to correctly detect the target object. Especially when the frequency of "abnormal cells" is lower than that of "normal cells," the decrease in the accuracy of "abnormal cell" detection becomes a significant problem in practical applications.

[0009] This invention was made to solve the above-mentioned problems. One of the objectives of this invention is to provide a learned model generation method and a learned model generation system that can suppress the decline in the detection accuracy of the detected object.

[0010] Solution for solving the problem

[0011] The first aspect of the present invention provides a learned model generation method based on machine learning, wherein the learned model is used for cell image segmentation. The learned model generation method includes the following steps: generating a learned model based on learning data including first input image data and first training image data corresponding to the first input image data, wherein the first training image data is labeled with at least a first label representing a detection object and a second label representing background; performing inference processing using inference processing data including second input image data through the generated learned model; determining whether a region estimated as a detection object or background in the inference result data output by the inference processing is a miscalculation; identifying noise regions that are not applicable to the first and second labels in regions determined to be miscalculations; determining noise regions contained in the first training image data; further labeling regions determined to be noise regions in the first training image data with a third label representing noise regions and updating the first training image data; and regenerating the learned model based on updated learning data including the first input image data and the updated first training image data.

[0012] The second aspect of the present invention is a learned model generation system based on machine learning, wherein the learned model is used for segmentation processing of cell images. The learned model generation system comprises: a storage unit storing learning data including first input image data and first training image data corresponding to the first input image data, and inference processing data including at least second input image data, wherein the first training image data is at least labeled with a first label representing a detection object and a second label representing a background; a learned model generation unit that generates a learned model based on the learning data stored in the storage unit; and an inference processing execution unit that, through the generated learned model... The learning model uses inference processing data stored in the storage unit to perform inference processing. The storage unit is configured to store updated learning data including first input image data and updated first training image data. The updated first training image data is obtained by labeling noise regions contained in the first training image data with a third label representing the noise regions. The noise regions are equivalent to noise regions in the result data output by the inference processing execution unit that are misestimated as detection objects and are not applicable to the first and second labels. The learned model generation unit is configured to regenerate the learned model based on the updated learning data stored in the storage unit.

[0013] The effects of the invention

[0014] The first aspect of the present invention, the method for generating a learned model, includes the following steps as described above: determining whether a region in the inference result data that is estimated to be a detection object or background is a miscalculation; identifying noise regions in the regions where the estimated region is determined to be a miscalculation; determining the noise regions contained in the first training image data; further labeling the regions determined to be noise regions in the first training image data with a third label representing the noise region, and updating the first training image data; and regenerating the learned model based on the updated learning data. Here, it is considered that background regions in the input image that may be incorrectly labeled as detection objects are known in advance, and the types of labels during segmentation are pre-defined as detection object regions, background regions, and noise regions. However, in most cases, it is only by checking the label images in the segmentation results of the input image performed using a learned model generated by machine learning that a noise region can be identified. That is, it is difficult to identify in advance which regions are likely to be noise regions. Therefore, there is a problem that it is difficult to generate a learned model in advance based on learning data that includes training image data simply labeled with the three types: detection object regions, background regions, and noise regions. Therefore, by configuring the model to identify noise regions within areas deemed mis-estimated in the inference result data, it is possible to determine noise regions contained in the first training image data. This allows for relearning to detect noise regions distinct from those assigned other labels. Consequently, the learned model can be regenerated based on the updated learning data, resulting in the suppression of any decrease in the detection accuracy of the detected objects.

[0015] The learned model generation system of the second aspect of the present invention is configured as described above to further store updated learning data including updated first training image data. The updated first training image data is obtained by further labeling noise regions contained in the first training image data with third labels representing noise regions, which correspond to noise regions in the inference result data output by the inference processing execution unit that were misjudged as detection targets. The system is configured to regenerate the learned model based on the updated learning data. By configuring it in this way, similar to the first aspect described above, it is possible to relearn to detect noise regions contained in the first training image data in a way that distinguishes them from regions assigned other labels. Therefore, since the learned model can be regenerated based on the updated learning data, a decrease in the detection accuracy of the detected targets can be suppressed. Attached Figure Description

[0016] Figure 1 This is a block diagram illustrating the outline of the image analysis system according to the first embodiment.

[0017] Figure 2 It is a block diagram used to illustrate the structure of the server.

[0018] Figure 3 This is a diagram that shows an overview of machine learning, inference processing, machine learning (relearning), and segmentation processing.

[0019] Figure 4 This is a schematic diagram used to illustrate differential image data.

[0020] Figure 5 This is a diagram comparing an example of a processed image output using the previously learned model before regeneration with an example of a processed image output using the regenerated previously learned model.

[0021] Figure 6 This is a flowchart illustrating the machine learning, inference processing, and machine learning (relearning) of the first embodiment.

[0022] Figure 7 This is a flowchart illustrating the machine learning, inference processing, and machine learning (relearning) of the second embodiment.

[0023] Figure 8 This is a schematic diagram illustrating an example of a processed image output using a regenerated learned model in a variant example. Detailed Implementation

[0024] The embodiments embodied in the present invention will now be described with reference to the accompanying drawings.

[0025] [First Implementation Method]

[0026] Reference Figures 1-6 The structure of the image analysis system 200, including the learned model generation system 100 of the first embodiment, and the learned model generation method will be explained.

[0027] (Image Analysis System)

[0028] Figure 1 The image analysis system 200 shown includes a learned model generation system 100 for generating a learned model 55 through machine learning, the learned model 55 being used for segmentation processing of cell image 81. The learned model generation system 100 includes a control unit 11 and a storage unit 12 contained in server 10.

[0029] The image analysis system 200 includes a server 10, a computer 20, and a camera device 30. Figure 1 An example of an image analysis system 200 constructed using a client-server model is shown. Server 10, computer 20, and camera device 30 are connected to each other via network 40 in a manner that enables them to communicate with each other.

[0030] Computer 20 is a user-operated client terminal that sends various processing requests to server 10. Computer 20 includes a display unit 21. Computer 20 is capable of acquiring cell images 81 captured by camera device 30 and displaying the acquired cell images 81 on display unit 21. Display unit 21 is, for example, a liquid crystal display device.

[0031] Server 10 performs various information processing based on requests (processing requests) from computer 20. Server 10 includes a control unit 11 and a storage unit 12.

[0032] Server 10 (control unit 11) generates a learned model 50. Additionally, server 10 (control unit 11) performs inference processing. Inference processing is the process of using the learned model 50 generated through machine learning to output an inference result based on the input. Furthermore, server 10 (control unit 11) performs image processing on cell image 81 using the learned model 50. As a result of image processing, server 10 (control unit 11) generates a processed image 82 of cell image 81. Server 10 (control unit 11) sends the generated processed image 82 to computer 20. Computer 20, upon receiving the information, causes display unit 21 to display the processed image 82.

[0033] Control unit 11 is configured based on learning data 86 (see reference). Figure 2 The learned model 50 is generated using the generated learned model 50. Furthermore, the control unit 11 is configured to use inference processing data 87 (see reference 1) on the generated learned model 50. Figure 2 Inference processing is performed based on data 87 (see reference 87) used for inference processing. Figure 2 The learning data 86 is corrected and updated to learning data 88 based on the inference results of the inference process. Furthermore, the control unit 11 is configured to correct the learning data 86 based on the updated learning data 88 (refer to...). Figure 2 To regenerate the learned model 55 (refer to) Figure 3 The regenerated learned model 55 (refer to...) Figure 3 It is used for segmentation processing of cell image 81.

[0034] Furthermore, the control unit 11 is configured to generate and display inference processing data 87 (see reference). Figure 2 ) and inference results data 84 (refer to) Figure 2 The difference image data of 83 (reference) Figure 2The control unit 11 includes processors such as CPU (Central Processing Unit), FPGA (Field-Programmable Gate Array), and ASIC (Application Specific Integrated Circuit). The processor executes a prescribed program to perform computational processing as the control unit 11.

[0035] Storage unit 12 is a device for storing information. Storage unit 12 stores the learned model 50 and the regenerated learned model 55 (see reference). Figure 2 The storage unit 12 includes a non-volatile storage device, such as a hard disk drive or a solid-state drive. It also includes various image data 80.

[0036] The imaging device 30 generates a cell image 81 obtained by photographing the cells. The imaging device 30 can send the generated cell image 81 to the computer 20 and / or server 10 via a network 40. The cell image 81 is, for example, an image of cultured cells grown using cell culture equipment. The captured cell image 81 is a microscopic image. The imaging device performs imaging based on photographic methods such as bright-field observation, dark-field observation, phase difference observation, and differential interferometry.

[0037] Network 40 connects server 10, computer 20, and camera device 30 in a manner that enables them to communicate with each other. For example, network 40 can be a LAN (Local Area Network) built within a facility. For example, network 40 can be the Internet. In the case of an Internet network, image analysis system 200 can be a system built using cloud computing.

[0038] (The specific device structure of the image analysis system)

[0039] Next, an example of the specific device structure of the image analysis system 200 will be described.

[0040] The imaging device 30 is an in-line holographic microscope. The imaging device 30 includes an image sensor 32 and a light source 31 comprising a laser diode and the like. During imaging, a culture plate 33 containing cell colonies (or single cells) is placed between the light source 31 and the image sensor 32. The imaging device 30 generates an IHM (In-line Holographic Microscopy) phase image as a cell image 81.

[0041] The computer 20 includes a processor 22, a storage unit 23, a display unit 21, and an input unit 24 as a user interface. The input unit 24 includes input devices such as a mouse and a keyboard.

[0042] The processor 22 is configured to control the operation of the camera device 30 by executing a program stored in the storage unit 23, and to process the data acquired by the camera device 30. The processor 22 includes a camera control unit 22a and a cell image generation unit 22b in the form of functional blocks.

[0043] When the user places the culture plate 33 containing cell colonies at the designated position of the camera device 30 and the input unit 24 accepts the specified operation, the processor 22 controls the camera device 30 through the camera control unit 22a to acquire holographic data.

[0044] The imaging device 30, under the control of the imaging control unit 22a, causes the light source unit 31 to irradiate coherent light. The imaging device 30 acquires an image based on interference fringes formed by the light passing through the culture plate 33 and cell colonies 34 and the light passing through the vicinity of the cell colonies 34 on the culture plate 33 via the image sensor 32. The image sensor 32 acquires holographic data (two-dimensional light intensity distribution data formed on the detection surface).

[0045] The cell image generation unit 22b calculates phase information by performing phase recovery operations on the holographic data acquired by the imaging device 30. Furthermore, the cell image generation unit 22b generates an IHM phase image (cell image 81) based on the calculated phase information. The methods for calculating phase information and generating the IHM phase image can utilize known techniques, therefore detailed descriptions are omitted.

[0046] The generated cell image 81 is stored in storage unit 23. Additionally, the generated cell image 81 is sent to server 10 by processor 22.

[0047] (server)

[0048] like Figure 2 As shown, the control unit 11 of the server 10 includes a learned model generation unit 11a, an inference processing execution unit 12b, and an image processing unit 11c in the form of functional blocks.

[0049] Storage unit 12 stores the generated learned model 50. Storage unit 12 stores the regenerated learned model 55. Storage unit 12 stores multiple cell images 81 captured by camera device 30. For example, all cell images 81 captured by camera device 30 are automatically sent to server 10 and stored in storage unit 12. In addition, storage unit 12 stores learning data 86 obtained by associating the first input image data 86a with the first training image data 86b corresponding to the first input image data 86a. Learning data 86 is used to generate learned model 50. The first training image data 86b is the correct image that should be output as a result of segmentation processing performed on the first input image data 86a. In addition, storage unit 12 stores inference processing data 87 obtained by associating the second input image data 87a with the second training image data 87b corresponding to the second input image data 87a. Inference processing data 87 is used for inference processing. The second training image data 87b is the correct image that should be output as a result of the inference processing performed on the second input image data 87a. Additionally, the storage unit 12 stores updated learning data 88, including the first input image data 86a and the updated first training image data 88b. Furthermore, the storage unit 12 stores difference image data 83, which shows the difference between the second training image data 87b used for inference processing 87 and the inference result data 84 corresponding to the second training image data 87b.

[0050] <Learned Model Generation Department>

[0051] The learned model generation unit 11a is configured to generate a learned model 50 by performing machine learning using the learning data 86 stored in the storage unit 12. The learned model generation unit 11a generates the learned model 50 based on the learning data 86, which includes the first input image data 86a and the first training image data 86b. Furthermore, the learned model generation unit 11a regenerates the learned model 55 based on updated learning data 88, which includes the first input image data 86a and the updated first training image data 88b.

[0052] <Inference Processing Execution Department>

[0053] The inference processing execution unit 11b is configured to perform inference processing using the inference processing data 87 stored in the storage unit 12. The inference processing execution unit 11b performs segmentation inference processing using the second input image data 87a included in the inference processing data 87 through the learned model 50, and outputs inference result data 84.

[0054] Image Processing Department

[0055] The image processing unit 11c is configured to perform image processing on the cell image 81 using the regenerated learned model 55 stored in the storage unit 12. The image processing unit 11c takes the cell image 81 selected as the processing target as the input image and inputs it into the regenerated learned model 55, thereby performing segmentation processing on the input image. As a result of the segmentation processing, the image processing unit 11c outputs a labeled image, obtained by segmenting the cell image 81 into multiple labeled regions, as a processed image 82. The image processing unit 11c sends the processed image 82 to the computer 20 and causes the storage unit 12 to store the processed image 82.

[0056] In this specification, "segmentation processing" refers to the process of dividing an input image into multiple regions. Specifically, it involves dividing the input image into multiple labeled regions by assigning labels to regions where a detection object is captured. A label is information that indicates the meaning of a portion of the image. A labeled region is a region (a portion of the image) consisting of a group of pixels in the image that have been assigned a common label.

[0057] like Figure 3 As shown, the regenerated learned model 55 segments the input cell image 81 and outputs a processed image 82 segmented into multiple labeled regions. Segmentation is performed by assigning a label to each pixel in the image. Labels can be assigned on a set (group of pixels) basis. The type of label is called a class.

[0058] Figure 2 The image processing unit 11c shown is configured as follows: based on the user's input unit 24 to the computer 20 (see reference 24). Figure 1 The operation performed on the region 60 (refer to) identified as noise in the first training image data 86b Figure 4 The area marker indicates a noise area of ​​60 (refer to...). Figure 4 The third label 63 (refer to) Figure 5 ), and set as the updated first training image data 88b. The noise region 60 (see below) is described later. Figure 4 ).

[0059] Furthermore, the image processing unit 11c is configured such that the noise region 60 is marked (refer to...) Figure 4 The third tag (refer to) Figure 5 If the number of updated first training image data 88b is less than the specified number, the system will adjust the input based on the user's input to the computer 20 (see reference 24). Figure 1 The operation performed further generates a third tag 63 (see reference). Figure 5The first training image data 88b. Specifically, based on the user's input to the computer 20 (refer to...) Figure 1 The operation performed is to cut out the noise region 60 (refer to) from the first training image data 86b. Figure 4 The image processing unit 11c processes part or all of the noise region 60 and embeds the cut-out noise region 60 into the background of the cell image 81 stored in the storage unit 12, thereby adding first training image data 88b labeled with a third tag 63. The aforementioned specified number is a number of updated first training image data 88b that the user pre-sets as statistically sufficient. Alternatively, the image processing unit 11c based on the user's input unit 24 to the computer 20 (see reference 11c). Figure 1 The operation performed uses the first training image data 88b stored in the storage unit 12, which is labeled with the third label 63 in other learning data, and adds the first training image data 88b labeled with the third label 63.

[0060] In addition, the image processing unit 11c is configured to generate differential image data 83, which shows the difference between the second training image data 87b, which shows the inference processing data 87, and the inference result data 84 corresponding to the second training image data 87b.

[0061] (Differential image data)

[0062] like Figure 4 As shown, the differential image data 83 is image data used to extract mis-estimated regions from the inference result data 84. The differential image data 83 is illustrated in the inference processing data 87 (see reference). Figure 2 The second training image data 87b includes image data showing the points of agreement and difference between the second training image data 87b and the inference result data 84 corresponding to the second training image data 87b. The second training image data 87b is labeled with a first label 61 representing the detected object and a second label 62 representing the background. Additionally, the inference result data 84 is also labeled with the inferred first label 61 representing the detected object and the second label 62 representing the background.

[0063] exist Figure 4The diagram illustrates an example of segmenting images (cell images) of pluripotent stem cells such as iPS cells and ES cells into three categories within the second training image data 87b and inference result data 84: "normal cell" regions (cells maintaining pluripotency, i.e., undifferentiated cell regions); "abnormal cell" regions (cells escaping from an undifferentiated state, i.e., undifferentiated escaped cell regions); and the "background" regions outside of these categories. Furthermore, the number of categories and the content of the detection labels may vary depending on the intended use of the generated learned model during segmentation. Therefore, in this invention, there are no particular limitations regarding the number of categories or the content of the detection labels.

[0064] In the differential image data 83, a first region 66, which is labeled with the first label 61 in both the corresponding regions of the second training image data 87b and the inference result data 84, is displayed in a recognizable manner. Additionally, a second region 67, which is labeled with the second label 62 in the second training image data 87b and with the first label 61 in the inference result data 84, is displayed in a recognizable manner. Furthermore, a third region 68, which is labeled with the first label 61 in the second training image data 87b and with the second label 62 in the inference result data 84, is displayed in a recognizable manner. Finally, a fourth region 69, which is labeled with the second label 62 in both the corresponding regions of the second training image data 87b and the inference result data 84, is displayed in a recognizable manner.

[0065] (Noise area)

[0066] For example, defects on the culture surface captured in the input image data are not detection targets. Therefore, in the training image data, regions with defects on the culture surface are labeled as background regions. That is, regions that do not represent defects on the culture surface in the training image data. However, when machine learning is performed using such input and training image data, the learning model learns the background non-uniformly. When segmentation is performed using the generated learned model, cells in the cell image that have shapes very similar to defects may be labeled as background regions. Furthermore, when machine learning is performed using input and training image data without defects on the culture surface, the learning model learns the background uniformly. When segmentation is performed using the generated learned model, cells in the cell image that have shapes very similar to defects on the culture surface may be labeled as detection target regions. These background regions that may be incorrectly labeled as detection targets in the processed image can usually only be identified by checking the label images in the segmentation results, making them difficult to identify in advance. Similarly, regions that may be incorrectly labeled as background regions in the processed image are also difficult to identify in advance. In this embodiment, the region of background that may be incorrectly assigned as a label of a detection object in the processed image, and the region of the detection object that may be incorrectly assigned as a background region in the processed image, are referred to as noise region 60. Figure 4 The noise region 60 shown is a background area in the processed image that may be incorrectly labeled as a detection object; it is a foreign object.

[0067] Furthermore, the noise region 60 is the region estimated to be the detection object in the inference result data 84 output by the inference processing, which is then processed by the image processing unit 11c (see reference). Figure 2 The noise region 60 is the region in the inference result data 84 where the "normal cell" region or "abnormal cell" region is determined to be a miscalculation, and this miscalculated region is not applicable to the first label 61 and the second label 62. Furthermore, the noise region 60 is a region that the user determines has a significant (non-negligible) impact on the detection accuracy of the detected object. For example, it is a region in the "background" region where objects that do not conform to "normal cells" and "abnormal cells" are incorrectly estimated as "abnormal cells."

[0068] (Update of the first input image data)

[0069] Therefore, in this embodiment, a learned model 50 is generated by performing machine learning using learning data 86, which includes first input image data 86a and first training image data 86b. The first input image data 86a may contain noisy regions 60. Next, using the generated learned model 50, inference processing is performed using second input image data 87a included in inference processing data 87, and inference result data 84 is output. The second input image data 87a may also contain noisy regions 60. Difference image data 83 is generated, showing the difference between the inference result data 84 and the second training image data 87b included in the inference processing data 87. (User, such as...) Figure 4 The differential image data 83 is compared with the inference result data 84 as shown. If the second input image data 87a contains a noise region 60, a second region 67 appears in the differential image data. Therefore, the user can identify the noise region 60 contained in the inference result data 84. The user determines the noise region 60 that may be contained in the first input image data 86a. The user can identify regions in the first input image data 86a that are similar in shape, etc., to the noise region 60 identified in the inference result data 84 as noise regions 60. The user can assign a third label 63 to the noise region 60 identified in the first input image data 86a via the input unit 24. The computer 20 or the image processing unit 11c updates the first input image data 86a assigned the third label 63 and stores it in the storage unit 12 as the updated first input image data 88a.

[0070] (An overview of machine learning, inference processing, machine learning (relearning), and segmentation processing)

[0071] Reference Figure 3 This section outlines the machine learning, inference processing, machine learning (relearning), and segmentation processing based on this implementation method.

[0072] Machine Learning

[0073] Learned model generation unit 11a (refer to) Figure 2 The learning model generation unit 11a is configured to perform machine learning using the learning data 86 stored in the storage unit 12. The learning data 86 is configured to include a sufficient amount of data. Each piece of learning data 86 includes first input image data 86a and first training image data 86b as the processing object. The learning model generation unit 11a causes the learning model 51 to learn the transformation process (segmentation process) from the first input image data 86a to the first training image data 86b by taking the first input image data 86a as input and the first training image data 86b as output. A learned model 50 is generated by performing machine learning.

[0074] In this embodiment, the first training image data 86b is composed of three categories: the "normal cell" region (e.g., undifferentiated cell region in pluripotent stem cells such as iPS cells and ES cells) which is the object of detection, the "abnormal cell" region (e.g., undifferentiated cells that have escaped from the undifferentiated state in iPS cells and ES cells) which is the object of detection, and the "background" region.

[0075] The machine learning method used for the learned model 50 can employ any method, such as Fully Convolutional Networks (FCN), neural networks, Support Vector Machines (SVM), or Boosting. As an example, the learned model 50 uses a convolutional neural network, frequently used in semantic segmentation. This learned model 50 is structured to include an input layer, a convolutional layer, and an output layer, all derived from the input image.

[0076] <Inference Processing>

[0077] Inference Processing Execution Unit 11b (refer to) Figure 2 The configuration is as follows: The learned model 50 performs inference processing using the second input image data 87a included in the inference processing data 87 stored in the storage unit 12. The inference processing execution unit 11b retrieves the second input image data 87a included in the inference processing data 87 from the storage unit 12. The inference processing data 87 is configured to include a sufficient amount of data. During inference processing, by inputting the second input image data 87a used for inference processing into the learned model 50, segmentation processing of the second input image data 87a can be performed, and inference result data 84 segmented into multiple label regions can be output.

[0078] The inference processing data 87 used for inference processing is different from the learning data 86 used for machine learning. The classes of the second training image data 87b and the inference result data 84 are the same as those of the first training image data 86b, consisting of three classes: "normal cell" regions as detection objects, "abnormal cell" regions as detection objects, and "background" regions.

[0079] Machine Learning (Relearning)

[0080] The learned model generation unit 11a is configured to relearn the learned model 50 using the updated learning data 88 stored in the storage unit 12. The learned model generation unit 11a retrieves the updated learning data 88 from the storage unit 12. The updated learning data 88 includes first input image data 86a as the processing target and updated first training image data 88b. The learned model generation unit 11a causes the learned model 50 to learn the transformation process (segmentation process) from the first input image data 86a to the updated first training image data 88b by taking the first input image data 86a as input and the updated first training image data 88b as output. A regenerated learned model 55 is generated by performing machine learning (relearning).

[0081] In this embodiment, the updated first training image data 88b consists of four categories: "normal cell" region as the detection object, "abnormal cell" region as the detection object, "background" region, and "noise" region.

[0082] <Segmentation Processing>

[0083] The regenerated learned model 55 segments the input cell image 81 to output a processed image 82 segmented into multiple label regions.

[0084] exist Figure 5 In one example of the processed image 82 output using the pre-regenerated learned model 50, region 64a is part of the "background," and region 64b is an "abnormal cell" region that is the target of detection. Similarly, in another example of the processed image 82 output using the regenerated learned model 55, region 64a is part of the "background," and region 64b is an "abnormal cell" region that is the target of detection. In the processed image 82 output using the pre-updated learned model 50, region 64a is incorrectly detected as an "abnormal cell" region, and region 64b is incorrectly detected as a "background" region. In contrast, in the processed image 82 output using the updated learned model 55, region 64a is correctly detected as noise region 60, and region 64b is correctly detected as an "abnormal cell" region.

[0085] (Machine learning, inference processing, and machine learning (relearning) processing)

[0086] Reference Figure 6 This will explain the machine learning, inference processing, and machine learning (relearning) processing based on this implementation method.

[0087] In step S1, the learned model generation unit 11a obtains learning data 86, including the first input image data 86a and the first training image data 86b, from the storage unit 12. Then, the process proceeds to step S2.

[0088] In step S2, the learned model generation unit 11a uses the first input image data 86a as input and the first training image data 86b as output to learn the transformation process (segmentation process) from the first input image data 86a to the first training image data 86b, thereby generating the learned model 50. Afterwards, the process proceeds to step S3.

[0089] In step S3, the inference processing execution unit 11b performs inference processing by inputting the second input image data 87a stored in the storage unit 12 into the generated learned model 50, and outputs inference result data 84. Afterwards, the process proceeds to step S4.

[0090] In step S4, the image processing unit 11c generates difference image data 83, which shows the difference between the second training image data 87b stored in the storage unit 12 and the inference result data 84. Additionally, the server 10 (control unit 11) sends the generated difference image data 83 and inference result data 84 to the computer 20, and the computer 20 causes the display unit 21 to display the difference image data 83 and inference result data 84. Afterwards, the process proceeds to step S5.

[0091] In step S5, based on the user's operation on the input unit 24, the control unit 11 acquires a determination result regarding whether the region estimated as the detection object in the inference result data 84 output by the inference processing is a miscalculation, based on the differential image data 83 and inference result data 84 displayed on the display unit 21. Then, the process proceeds to step S6.

[0092] In step S6, based on the user's operation on the input unit 24, the control unit 11 obtains a determination result that identifies a noise region 60 that is not applicable to the first label 61 and the second label 62 in the region where the region estimated as the detection object in the inference result data 84 is determined to be a miscalculation. Then, the process proceeds to step S7.

[0093] In step S7, based on the user's operation on the input unit 24, a determination result is obtained regarding whether the noise region 60 has a significant impact on the detection accuracy of the detected object. If the determination result is that the noise region 60 has a significant impact on the detection accuracy of the detected object ("Yes" in step S7), the process proceeds to step S8; if the determination result is that the noise region 60 has little impact on the detection accuracy of the detected object ("No" in step S7), the process ends.

[0094] In step S8, based on the user's operation on the input unit 24, a determination result is obtained that identifies the noise region 60 contained in the first training image data 86b, based on the noise region 60 determined in the inference result data 84. Then, the process proceeds to step S9.

[0095] In step S9, the user confirms the differential image data 83 or the learning data 86 including the first training image data 86b, and labels the regions determined to be noise regions 60 in the first training image data 86b with a third label 63 representing the noise regions 60. For example, the labeling of the third label 63 can be performed via the computer's input unit 24. Furthermore, there are no particular limitations on the method for labeling the third label 63. The third label 63 is labeled, and the first training image data 86b is updated. After that, the process proceeds to step S10.

[0096] In step 10, based on the user's operation on the input unit 24, a determination result is obtained regarding whether the number of updated first training image data 88b is greater than or equal to the number of updated first training image data 88b preset by the user. If the determination result is greater than or equal to the number of updated first training image data 88b preset by the user ("Yes" in step S10), the process proceeds to step S11; if the determination result is less than the number of updated first training image data 88b preset by the user ("No" in step S10), the process proceeds to step S12.

[0097] In step S11, the learned model generation unit 11a regenerates (relearns) the learned model 55 based on the updated learning data 88, which includes the first input image data 86a and the updated first training image data 88b. Through machine learning (relearning), an updated learned model 55 is generated. Afterward, the process ends.

[0098] In step S12, the user also generates (adds) first training image data 88b labeled with the third tag 63. Afterwards, the process proceeds to step S11.

[0099] (Effects of the first implementation method)

[0100] In the first embodiment, the following effects can be obtained.

[0101] In the first embodiment, as described above, the learned model generation method includes the following steps: determining whether a region estimated as a detection object in the inference result data 84 is a miscalculation; identifying a noise region 60 in the region determined to be a miscalculation, and determining the noise region 60 included in the first training image data 86b; further updating the first training image data 86b by labeling the region determined as a noise region 60 in the first training image data 86b with a third label 63 representing the noise region 60; and regenerating the learned model 55 based on the updated learning data 88. Here, it is considered that background regions in the input image that may be incorrectly labeled as detection objects are known in advance, and the types of labels during segmentation are set in advance as three types: detection object region, background region, and noise region. However, in most cases, it is only by checking the label image in the segmentation result of the input image performed by the learned model generated by machine learning that it is possible to identify which region is the noise region 60. That is, it is difficult to identify in advance which region may become the noise region 60. Therefore, the following problem exists: it is difficult to generate a learned model in advance based on training image data that includes training image data simply labeled with three types of regions: the target region, the background region, and the noise region. Therefore, by identifying noise regions in regions determined to be mis-estimated in the inference result data 84, the noise region 60 contained in the first training image data 86b can be determined, and thus it is possible to relearn the detection of the noise region 60, distinguishing it from regions assigned other labels. Therefore, the learned model 55 can be regenerated based on the updated training data 88, resulting in the suppression of the decrease in the detection accuracy of the target region.

[0102] Furthermore, in the first embodiment, as described above, the method further includes the step of generating difference image data 83 showing the difference between the second training image data 87b and the inference result data 84 corresponding to the second training image data 87b, as image data for extracting mis-estimated regions in the inference result data 84. This allows for easy identification of the difference between the second training image data 87b and the inference result data 84, which has been correctly labeled, as shown in the difference image data 83. Therefore, it is possible to easily identify mis-estimated regions in the inference result data 84 based on the difference image data 83. Consequently, the noise region 60 can be easily determined.

[0103] Furthermore, in the first embodiment, as described above, differential image data is generated that displays the first region 66 (labeled with the first tag 61), the second region 67 (labeled with the second tag 62 in the second training image data 87b and labeled with the first tag 61 in the inference result data 84), and the third region 68 (labeled with the first tag 61 in the second training image data 87b and labeled with the second tag 62 in the inference result data 84) in a recognizable manner. Thus, the first region 66, correctly estimated as the detection target region, the second region 67, incorrectly estimated as the background region, and the third region 68, incorrectly estimated as the background region, are displayed in the differential image data 83 in a recognizable manner. Therefore, the noise region 60 can be more easily identified.

[0104] Furthermore, in the first embodiment, as described above, the noise region 60 is part of the mis-estimated region. Therefore, the region within the area where the background region is incorrectly estimated as the target area, and which has a significant impact on the detection accuracy of the target, can be identified as the noise region 60. Thus, the entire area where the background region is incorrectly estimated as the target area can be avoided being identified as the noise region 60, thereby enabling efficient identification of the noise region 60.

[0105] Furthermore, in the first embodiment, as described above, in the step of regenerating the learned model 55, the learned model 55 is regenerated based on the updated learning data 88, which includes the first input image data 86a and the first training image data 88b labeled with the third label 63, in addition to the updated learning data 88. Therefore, even if the number of first training image data 86b labeled with the third label 63 in the updated learning data 88 is small, the number of first training image data 88b labeled with the third label 63 can be ensured.

[0106] Furthermore, in the first embodiment, as described above, the system for generating the learned model is configured to: acquire updated learning data 88, and regenerate the learned model 55 based on the updated learning data 88. The updated learning data 88 includes updated first training image data 88b that also labels noise regions 60 contained in the first training image data 86b with a third label 63 representing the noise region 60. The noise region 60 corresponds to the noise region 60 in the inference result data 84 output by the inference processing execution unit 11b that was misestimated as a detection target. Therefore, it is possible to relearn to detect the noise region 60 contained in the first training image data 88b in a way that distinguishes it from regions assigned other labels. Thus, the learned model 55 can be regenerated based on the updated learning data 88, resulting in the suppression of a decrease in the detection accuracy of the detected target.

[0107] [Second Implementation]

[0108] Next, a learned model generation system and a learned model generation method based on a second embodiment of the present invention will be described. Unlike the first embodiment, which is configured to determine whether a region estimated as a detection object in the inference result data 84 output by the inference process is a miscalculation, the second embodiment is configured to determine whether a region in the inference result data 84 that is estimated as a low-occurrence detection object with an occurrence frequency below a predetermined threshold and is included in the detection object is a miscalculation.

[0109] The impact of noise region 60 on the detection accuracy of the target object becomes significant when the frequency of the target object is low (in the case of a low-occurrence target object). For example, suppose there are 1000 cell images 81 used for segmentation, and only 5 of these images contain cells with low-occurrence targets. In this case, if there are 5 processed images 82 where noise region 60 is incorrectly detected as a "low-occurrence target object" region, the region detection accuracy drops to 50%. In contrast, if the number of processed images where noise region 60 is incorrectly detected as a "low-occurrence target object" region can be reduced from 5 to 1, the region detection accuracy can be set to 83.3%.

[0110] In this embodiment, the low-occurrence detection target is "abnormal cells". Additionally, in this embodiment, noise region 60 is defined as the region in the inferred result data 84 where the "abnormal cell" region is determined to be a miscalculation, and this miscalculated region is not applicable to the first label 61 and the second label 62.

[0111] Figure 2The learned model generation unit 11a shown is configured to acquire the frequency of occurrence of the detected object contained in the learning data 86, which includes the first input image data 86a and the first training image data 86b, acquired from the storage unit 12. Furthermore, the learned model generation unit 11a is configured to determine whether the acquired frequency of occurrence of the detected object is below a preset threshold. Examples of a preset threshold being below this threshold include situations where the detected object is captured in 3 or fewer out of every 10 first input image data, or where the area of ​​the detected object region accounts for 10% or less of the total area of ​​all first input image data 86a. However, the preset threshold is not limited to these cases and can be appropriately set.

[0112] Furthermore, the learned model generation unit 11a is configured such that if the frequency of occurrence of a detected object is below a preset threshold, it determines that a low-occurrence detected object exists and stores the existence of a low-occurrence detected object in the storage unit 12. Additionally, the server 10 (control unit 11) is configured to send information that the learning data 86 includes a low-occurrence detected object to the computer 20, and the computer 20, upon receiving the information, notifies the user of the existence of the low-occurrence detected object via the display unit 21.

[0113] The image processing unit 11c determines whether the other learning data 86 stored in the storage unit 12 contains data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object. If the data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object exists, the image processing unit 11c appends this data to the learning data 86. If the data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object does not exist, or if even if the data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object is appended, the image processing unit 11c is configured to generate data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object. Specifically, based on the user's input to the computer 20's input unit 24 (see reference 24), Figure 1 The operation involves cutting out the low-occurrence detection object from the first input image data 86a or the first training image data 86b, which includes the low-occurrence detection object, and embedding the cut-out low-occurrence detection object into the background of the learning data 86 stored in the storage unit 12, thereby adding data including the first input image data 86a and the first training image data 86b containing the low-occurrence detection object. Furthermore, there is no particular limitation on the method for generating the data including the first input image data 86a and the first training image data 86b containing the low-occurrence detection object; known data expansion methods can be used.

[0114] (Machine learning, inference processing, and machine learning (relearning) processing)

[0115] Reference Figure 7 This document will explain the machine learning, inference, and machine learning (relearning) processes based on this embodiment. Furthermore, processes that differ from those in the first embodiment will be described, while processes identical to those in the first embodiment will be omitted from the description.

[0116] In step S21, the learned model generation unit 11a acquires the frequency of occurrence of detection objects contained in the learning data 86, which includes the first input image data 86a and the first training image data 86b, acquired from the storage unit 12, and determines whether there are low-occurrence detection objects. If there are no low-occurrence detection objects ("Yes" in step S21), the process proceeds to step S2; if there are low-occurrence detection objects ("No" in step S21), the process proceeds to step S22.

[0117] In step S22, the learned model generation unit 11a stores the cases of low-occurrence detection objects in the storage unit 12, and the server 10 (control unit 11) notifies the user of this case via the computer 20. Afterwards, the process proceeds to step S23.

[0118] In step S23, the image processing unit 11c determines whether the other learning data 86 stored in the storage unit 12 contains data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object. If the data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object exists (yes in step S23), the process proceeds to step S24. If the data including the first input image data 86a and the first training image data 86b of the low-occurrence detection object does not exist, or if the preset number of images is not reached even if additional learning data 86 is added (no in step S23), the process proceeds to step S24.

[0119] In step S24, the image processing unit 11c appends data including the first input image data 86a and the first training image data 86b, which include the low-occurrence detection object, to the learning data 86. Afterward, the processing proceeds to step S2.

[0120] In step S25, based on the user's operation on the input unit 24, data including first input image data 86a and first training image data 86b, which includes low-occurrence detection objects, is generated (added). Then, the process proceeds to step S2.

[0121] In step S26, based on the user's operation on the input unit 24, the display unit 21 acquires a determination result on whether the region estimated as a low-occurrence detection object, i.e., an "abnormal cell" region, in the inference result data 84 output by the inference processing, is a miscalculation. After that, the process proceeds to step S6.

[0122] The other structures of the second embodiment are the same as those of the first embodiment described above.

[0123] (Effects of the second implementation method)

[0124] In the second embodiment, the following effects can be obtained.

[0125] In the second embodiment, in the learned model generation method, as described above, in the step of determining whether it is a misestimation, it is determined whether a region in the inference result data 84 that is estimated to be a low-occurrence detection object with an occurrence frequency below a predetermined threshold is a misestimation. Therefore, the region determined to be a low-occurrence detection object that is a misestimation can be defined as a noise region 60. Since the impact of the noise region 60 on the detection accuracy of the detection object becomes significant when the occurrence frequency of the detection object is low, the decrease in the detection accuracy of the detection object can be further suppressed.

[0126] Furthermore, in the second embodiment, as described above, the low-occurrence detection target is abnormal cells. Therefore, the noise region 60 can be determined based on the region that is mistakenly estimated to be an abnormal cell region, thus further suppressing the decrease in detection accuracy of the low-occurrence detection target, i.e., abnormal cells.

[0127] Furthermore, in the second embodiment, as described above, the method further includes the following steps: acquiring the occurrence frequency of the detected objects included in the learning data 86; and, if the detected objects include low-occurrence detected objects based on the acquired occurrence frequency of the detected objects included in the learning data 86, notifying the user that low-occurrence detected objects are included. This allows the user to add first training image data 86b labeled with a first tag 61 representing low-occurrence detected objects. Therefore, it is possible to further suppress the significant decrease in detection accuracy of detected objects due to noise regions when the occurrence frequency of detected objects is low.

[0128] Furthermore, in the second embodiment, as described above, in the step of generating the learned model, the learned model 50 is generated based on the learning data 86, which has been supplemented with learning data 86 including low-occurrence detection objects. This further suppresses the significant decrease in detection accuracy caused by the noise region 60 when the occurrence frequency of the detection object is low.

[0129] Furthermore, the other effects of the second embodiment are the same as those of the first embodiment.

[0130] [Variation Example]

[0131] Furthermore, the embodiments disclosed herein should be considered illustrative in all respects and not restrictive. The scope of the invention is defined not by the description of the embodiments above but by the claims, and includes all modifications (variations) within the meaning and scope equivalent to the claims.

[0132] For example, the first embodiment described above shows an example where the inference processing execution unit 11b is configured to perform inference processing (step S3) and determine whether the region estimated as a detection object in the inference result data 84 is a miscalculation (step S5), but the present invention is not limited thereto. For example, it may be configured such that the display unit 21 of the computer 20 displays the miscalculated region in the inference result data 84 after the inference processing is performed and before determining whether it is a miscalculation. The image processing unit 11c may, for example, extract the region that is miscalculated as a detection object region and is surrounded by a background region from the inference result data 84 and display it on the display unit 21 of the computer 20 as a specific example of a noise region 60. As a result, the user can visually confirm the specific example of the noise region 60, and thus the noise region 60 can be determined more efficiently.

[0133] Furthermore, when the display unit 21 of the computer 20 displays the mis-estimated area in the aforementioned inference result data 84, the image processing unit 11c can be configured to display the mis-estimated area that satisfies a preset condition related to at least one of aspect ratio, proprietary area, and shape complexity. This allows the user to visually confirm specific examples that are easily identifiable as noise area 60, thus enabling more efficient identification of the noise area 60. Moreover, the conditions related to aspect ratio, proprietary area, and shape complexity can be appropriately set according to the object being detected, without particular limitation. Additionally, the shape complexity can be set based on the perimeter and proprietary area of ​​the noise area 60, or it can be set based on other conditions.

[0134] Furthermore, when the display unit 21 of the computer 20 displays the mis-estimated region in the aforementioned inference result data, the image processing unit 11c can label the region in the second training image data 87b corresponding to the noise region 60 extracted from the inference result data 84 with a third label 63. Thus, the user can visually confirm the specific example of the third label 63 of the noise region 60.

[0135] Furthermore, in the first embodiment described above, a processed image 82 output using the regenerated learned model 55 is shown (see reference). Figure 5Region 64a in the diagram is correctly shown as an example of noise region 60, but the invention is not limited thereto. Figure 8 The processed image 82 shown, output by the regenerated learned model 55, is otherwise identical to the image except that it does not display the noise region 60. Figure 5 The image shown corresponds to the processed image 82 output by the updated learned model 50. As a result of relearning the noise region 60, it is possible to suppress the decrease in detection accuracy of the detected object caused by the noise region 60. However, depending on the image analysis object, if the noise region 60 is not displayed in the processed image 82, it may become an analysis image that is useful to the user and does not reduce recognizability. Therefore, as Figure 8 As shown, image 82 can be processed without displaying the noise region 60.

[0136] Furthermore, in the first embodiment described above, an example was shown where the region estimated to be a detection object in the inference result data 84 output by the inference process is a miscalculation, but the present invention is not limited thereto. For example, it may be configured to determine whether a region estimated to be background in the inference result data 84 output by the inference process is a miscalculation.

[0137] Furthermore, the second embodiment described above illustrates an example where the low-occurrence detection target is an "abnormal cell," but the present invention is not limited thereto. For example, the low-occurrence detection target could be a "normal cell."

[0138] Furthermore, in the first embodiment, an example is shown in which the image processing unit 11c generates differential image data 83, and the server 10 (control unit 11) sends the generated differential image data 83 and the inference result data 84 to the computer 20; however, the present invention is not limited to this. For example, it may be configured such that the image processing unit 11c further generates overlay image data (not shown) based on the generated differential image data 83, which correctly covers the region labels of the inference result data 84, and the server 10 (control unit 11) sends the generated differential image data 83 and the overlay image data to the computer 20.

[0139] Furthermore, in the first embodiment, steps S5 to S10 and S12 of the machine learning, inference, and machine learning (relearning) processes are performed by the user, but the present invention is not limited thereto. Alternatively, all processes in steps S5 to S10 and S12 may be executed by the server 10 (control unit 11), or any one of steps S5 to S10 and S12 may be performed by the user, while the rest may be executed by the server 10 (control unit 11).

[0140] [Way]

[0141] Those skilled in the art should understand that the above exemplary embodiments are specific examples of the following methods.

[0142] (Project 1)

[0143] A learned model generation method is a machine learning-based method for generating learned models, wherein the learned model is used for cell image segmentation. The learned model generation method includes the following steps:

[0144] A learned model is generated based on learning data including first input image data and first training image data corresponding to the first input image data, wherein the first training image data is labeled with at least a first label representing the detected object and a second label representing the background.

[0145] Using the generated learned model, inference processing is performed using inference processing data including the second input image data;

[0146] Determine whether the regions estimated as the detected object or background in the inference result data output by the inference processing are mis-estimated;

[0147] In areas where the estimated region is determined to be a miscalculation, noise regions that are not applicable to the first and second labels are identified.

[0148] Determine the noise region contained in the first training image data;

[0149] Furthermore, regions identified as noise regions in the first training image data are labeled with third labels representing the noise regions, and the first training image data is updated; and

[0150] The learned model is regenerated based on the updated learning data, which includes the first input image data and the updated first training image data.

[0151] (Project 2)

[0152] In the learned model generation method described in Project 1, after the inference processing step and before the step of determining whether it is a misestimation, the following steps are also included: generating difference image data showing the difference between the second training image data and the inference result data corresponding to the second training image data, as image data for extracting the misestimated region in the inference result data, wherein the second training image data is image data included in the inference processing data and is at least labeled with a first label and a second label.

[0153] (Project 3)

[0154] In the learned model generation methods described in Project 2,

[0155] In the step of generating differential image data, differential image data is generated in a manner that can identify the first region, the second region, and the third region. The first region is the region that is labeled with a first label in both the corresponding regions of the second training image data and the inference result data. The second region is the region that is labeled with a second label in the second training image data and labeled with a first label in the inference result data. The third region is the region that is labeled with a first label in the second training image data and labeled with a second label in the inference result data.

[0156] (Project 4)

[0157] In any of the learned model generation methods described in Items 1-3, the noisy region is part of the mis-estimated region.

[0158] (Project 5)

[0159] In any of the methods for generating a learned model described in Items 1 to 4, in the step of regenerating the learned model, the learned model is regenerated based on the updated learning data, which includes the first input image data and the first training image data labeled with a third label, in addition to the updated learning data.

[0160] (Project 6)

[0161] In any of the learned model generation methods described in items 1 to 5, after the inference processing step and before the step of determining whether it is a miscalculation, the following step is also included: displaying the miscalculated area in the inference result data on the display unit.

[0162] (Project 7)

[0163] In the learned model generation method described in Project 6, in the step of displaying the mis-estimated region on the display unit, the mis-estimated region that satisfies a pre-set condition related to at least one of aspect ratio, proprietary area, and shape complexity is displayed on the display unit.

[0164] (Project 8)

[0165] In any of the learned model generation methods described in Items 1 to 7, in the step of determining whether it is a misestimation, it is determined whether the region in the inference result data that is estimated to be a low-occurrence detection object whose frequency of occurrence is below a specified threshold is a misestimation.

[0166] (Project 9)

[0167] In the learned model generation method described in Project 8, the low-occurrence detection target is abnormal cells.

[0168] (Project 10)

[0169] The learned model generation method described in Project 9 also includes the following steps:

[0170] Before generating the learned model, obtain the frequency of occurrence of the detected objects included in the learning data; and

[0171] If a low-occurrence detection object is included in the detection object based on the frequency of occurrence of the detected object, the user is notified that a low-occurrence detection object is included.

[0172] (Project 11)

[0173] In the learned model generation method described in Project 10, in the step of generating the learned model, the learned model is generated based on learning data that has been supplemented with learning data including low-occurrence detection objects.

[0174] (Project 12)

[0175] A learned model generation system is a machine learning-based system for generating learned models, wherein the learned model is used for cell image segmentation. The learned model generation system comprises:

[0176] The storage unit stores learning data including first input image data and first training image data corresponding to the first input image data, as well as inference processing data including at least second input image data, wherein the first training image data is at least labeled with a first label representing a detected object and a second label representing a background.

[0177] The learned model generation unit generates a learned model based on the learning data obtained from the storage unit; and

[0178] The inference processing execution unit performs inference processing using the inference processing data stored in the storage unit, based on the generated learned model.

[0179] The storage unit is configured to store updated learning data, including first input image data and updated first training image data. The updated first training image data is obtained by labeling noise regions contained in the first training image data with a third label representing the noise regions. The noise regions are equivalent to noise regions in the inference result data output by the inference processing execution unit that are misestimated as detection objects and are not applicable to the first and second labels. The learned model generation unit is configured to regenerate the learned model based on the updated learning data stored in the storage unit.

[0180] Explanation of reference numerals in the attached figures

[0181] 111a: Learned model generation unit; 11b: Inference processing execution unit; 12: Storage unit; 50: Learned model; 55: Regenerate learned model; 60: Noisy region; 61: First label; 62: Second label; 63: Third label; 66: First region; 67: Second region; 68: Third region; 83: Differential image data; 84: Inference result data; 86: Learning data; 86a: First input image data; 86b: First training image data; 87: Inference processing data; 87a: Second input image data; 87b: Second training image data; 88: Update learning data; 88a: Update first training image data; 100: Learned model generation system.

Claims

1. A method for generating a learned model, which is a machine learning-based method for generating a learned model, wherein the learned model is used for segmentation processing of cell images, and the method for generating the learned model includes the following steps: A learned model is generated based on learning data including first input image data and first training image data corresponding to the first input image data, wherein the first training image data is labeled with at least a first label representing the detected object and a second label representing the background. Using the generated learned model, inference processing is performed using inference processing data including the second input image data; Determine whether the region estimated as the detected object or the background in the inference result data output by the inference process is a mis-estimation; In areas where the estimated region is determined to be a miscalculation, noise regions that are not applicable to the first and second labels are identified; The region in the first training image data that has a shape similar to the noise region determined in the inference result data is identified as the noise region; The system also labels the regions identified as noise regions in the first training image data with a third label, and updates the first training image data. as well as The learned model is regenerated based on the updated learning data, which includes the first input image data and the updated first training image data.

2. The learned model generation method according to claim 1, wherein, After performing the inference processing step and before determining whether it is a misestimation step, the method further includes the following steps: generating difference image data showing the difference between the second training image data and the inference result data corresponding to the second training image data, as image data for extracting the misestimated region in the inference result data, wherein the second training image data is image data included in the inference processing data and at least labeled with the first label and the second label.

3. The learned model generation method according to claim 2, wherein, In the step of generating the differential image data, the differential image data is generated in a manner that enables the identification of the first region, the second region, and the third region. The first region is a region in which the first label is marked in both the corresponding regions of the second training image data and the inference result data. The second region is a region in which the second label is marked in the second training image data and the first label is marked in the inference result data. The third region is a region in which the first label is marked in the second training image data and the second label is marked in the inference result data.

4. The learned model generation method according to claim 1, wherein, The noise region is part of the mis-estimated region.

5. The learned model generation method according to claim 1, wherein, In the step of regenerating the learned model, the learned model is regenerated based on the updated learning data, which includes the first input image data and the first training image data labeled with the third label, in addition to the updated learning data.

6. The learned model generation method according to claim 1, wherein, After performing the inference processing step and before determining whether it is a miscalculation step, the method further includes the following step: displaying the miscalculated area in the inference result data on the display unit.

7. The learned model generation method according to claim 6, wherein, In the step of displaying the mis-estimated area on the display unit, the mis-estimated area that satisfies a pre-set condition related to at least one of aspect ratio, proprietary area, and shape complexity is displayed on the display unit.

8. The method for generating a learned model according to claim 1, wherein, In the step of determining whether it is a misestimation, it is determined whether the region in the inference result data that is estimated to be a low-occurrence detection object whose occurrence frequency is below a specified threshold is a misestimation.

9. The learned model generation method according to claim 8, wherein, The low-occurrence detection target is abnormal cells.

10. The learned model generation method according to claim 9 further includes the following steps: Before generating the learned model, the frequency of occurrence of the detected object included in the learning data is obtained; as well as If the low-occurrence detection object is included in the detection object based on the frequency of occurrence of the detected object, the user is notified that the low-occurrence detection object is included.

11. The learned model generation method according to claim 10, wherein, In the step of generating the learned model, the learned model is generated based on the learning data that has been supplemented with the learning data including the low-occurrence detection object.

12. A learned model generation system, which is a machine learning-based learned model generation system, wherein the learned model is used for cell image segmentation processing, and the learned model generation system comprises: The storage unit stores learning data including first input image data and first training image data corresponding to the first input image data, as well as inference processing data including at least second input image data, wherein the first training image data is at least labeled with a first label representing a detected object and a second label representing a background. The learned model generation unit generates a learned model based on the learning data obtained by the storage unit; as well as The inference processing execution unit performs inference processing using the inference processing data stored in the storage unit, based on the generated learned model. The storage unit is configured to further store updated learning data, including the first input image data and the updated first training image data. The updated first training image data is obtained by further labeling noise regions with third labels representing the noise regions. The noise regions correspond to noise regions in the inference result data output by the inference processing execution unit that are misjudged as detection targets and are not applicable to the first and second labels. The noise regions are regions in the first training image data that have a shape similar to the noise regions in the inference result data. The learned model generation unit is configured to regenerate the learned model based on the updated learning data stored in the storage unit.

Images