Estimation device, learning method, and program

Abstract

The estimation model is trained based on training images created automatically. [Solution] The image acquisition unit 11 acquires a training image IMG. The estimation unit 12 outputs an output image OUT[k] representing the segmentation result of the training image IMG based on the training image IMG and the training image. The initial training image assignment unit 13 assigns an initial training image LBL, which is used for training in the estimation unit 12, in which the pseudo-correct labels of the objects contained in the training image IMG are constructed by a mathematical model. INI The estimation unit 12 is given the data. The control unit 14 causes the estimation unit 12 to perform learning multiple times. In the first learning, the control unit 14 uses the initial training image LBL. INI The system is trained using the provided image as the training image. For subsequent training sessions, the output image produced as a result of the previous training is used as the training image. Once the number of training sessions reaches a predetermined number, the training in the estimation unit 12 is stopped.

Description

[Technical Field]

[0001] This disclosure relates, for example, to estimation devices, learning methods, and programs. [Background technology]

[0002] In recent years, the use of machine learning has advanced in various fields, including robotics, medicine, image understanding, automobiles, and speech recognition. For example, in the medical field, it is being applied to assist in the interpretation of medical images that visualize the inside of living organisms.

[0003] One known medical imaging technique is the CT (Computed Tomography) scanner. CT scanners can image the X-ray absorption patterns of the human body, creating images of internal tissues and morphology. Another known imaging technique is the Magnetic Resonance Imaging (MRI) scanner. MRI scanners apply a magnetic field to, for example, human tissue and utilize the resulting nuclear magnetic resonance (NMR) phenomenon to acquire two-dimensional or three-dimensional image information. MRI scanners have superior features, such as the ability to image tissues that cannot be imaged with CT scanners and the absence of radiation exposure. Methods for automatically performing diagnoses based on the acquired data have also been proposed for these devices (Patent Document 1).

[0004] Attempts are being made to estimate lesions contained in images by inputting images acquired with CT and MRI scanners into models pre-built through machine learning. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Special Publication No. 2020-524018 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, in order to build a model that performs estimation using medical images such as CT scans as input data through supervised learning, it is necessary to prepare ground truth data that identifies the location and extent of objects within the image. However, creating ground truth data requires manual annotation work, which results in a great deal of effort.

[0007] For example, if we were to build a model that outputs an image estimating the location and extent of organs from CT scans of the human body, we would need to create ground truth data that identifies the location and extent of organs in the images being trained. In this case, medical professionals such as doctors would need to review the images and manually perform annotation work, such as tracing the outlines of the organs.

[0008] Even with annotation-assisting tools, annotation by a physician takes approximately 30 minutes per case for 3D organs. Training a model requires around 5000 cases of training data. Therefore, a total of 2500 hours of annotation work is necessary. Even if one physician works 4 hours a day, this would take 125 weeks, or about two and a half years.

[0009] This disclosure is made in light of the circumstances described above, and aims to enable the training of an estimation model based on training images created without human intervention. [Means for solving the problem]

[0010] An estimation device according to one aspect of the present disclosure includes: an image acquisition unit that acquires training images; an estimation unit that outputs an output image representing the segmentation result of the training images based on the training images and a teacher image; an initial teacher image assignment unit that provides the estimation unit with an initial teacher image in which pseudo-correct labels of objects included in the training images are constructed by a mathematical model for use in learning by the estimation unit; and a control unit that causes the estimation unit to perform learning multiple times. The control unit, in the first learning, uses the initial teacher image provided to the estimation unit by the initial teacher image assignment unit as the teacher image and causes the estimation unit to perform learning; in the second and subsequent learning, uses the output image output as the result of the previous learning as the teacher image and causes the estimation unit to perform learning; and stops learning in the estimation unit when the number of learning sessions reaches a predetermined number.

[0011] An estimation device according to one aspect of the present disclosure is the estimation device described above, wherein the pseudo-correct label of the initial training image is preferably a region represented by a closed curve with respect to a point set inside.

[0012] An estimation apparatus according to one aspect of the present disclosure is the estimation apparatus described above, wherein the pseudo-correct label of the initial training image is preferably a region represented by a combination of a plurality of closed curves.

[0013] An estimation device according to one aspect of the present disclosure is the estimation device described above, wherein the pseudo-correct label of the initial training image is preferably a figure drawn in advance by the user of the estimation device, the average shape of an object, or a figure output by inputting the training image into a segmentation model in an estimation means different from the estimation device. The estimation device according to claim 1.

[0014] An estimation device according to one aspect of the present disclosure is the estimation device described above, wherein, in the second and subsequent learning sessions, the control unit preferably causes the estimation unit to perform learning using an image with adjusted contrast of the output image output as a result of the previous learning session as the training image.

[0015] An estimation device according to one aspect of the present disclosure is the above-described estimation device, and it is desirable that the pseudo-correct label of the initial teacher image is an area represented by an envelope of the plurality of closed curves.

[0016] An estimation device according to one aspect of the present disclosure is the above-described estimation device, and it is desirable that pixel values of the pseudo-correct label decrease as going from the inside of the pseudo-correct label toward the edge of the pseudo-correct label.

[0017] An estimation device according to one aspect of the present disclosure is the above-described estimation device, and it is desirable that the predetermined number is a predetermined value.

[0018] An estimation device according to one aspect of the present disclosure is the above-described estimation device, and when an error between the teacher image and the output image becomes smaller than a predetermined error, it is desirable that the control unit stops learning in the estimation unit assuming that the number of learning times has reached the predetermined number.

[0019] A learning method according to one aspect of the present disclosure includes acquiring a learning image, causing an estimation unit that outputs an output image representing a segmentation result of the learning image based on the learning image and a teacher image to perform learning a plurality of times. In the plurality of times of learning, in the first learning, an initial teacher image in which a pseudo-correct label of an object included in the learning image is constructed by a mathematical model is given to the estimation unit, and learning is performed in the estimation unit using the initial teacher image as the teacher image. In the learning after the second time, learning is performed in the estimation unit using, as the teacher image, an output image output from the estimation unit as a result of the previous learning. When the number of learning times reaches a predetermined number, learning in the estimation unit is stopped. [[ID=二十]]

[0020] A program in one aspect of this disclosure causes a computer to perform the following processes: acquiring training images; causing an estimation unit, which outputs an output image representing the segmentation result of the training images based on the training images and a teacher image, to perform training multiple times; in the multiple training sessions, in the first training session, the estimation unit is given an initial teacher image in which pseudo-correct labels of objects included in the training images are constructed by a mathematical model, and the estimation unit is given the initial teacher image to perform training; in the second and subsequent training sessions, the output image output from the estimation unit as a result of the previous training is used as the teacher image to perform training; and when the number of training sessions reaches a predetermined number, the training in the estimation unit is stopped. [Effects of the Invention]

[0021] According to this disclosure, an estimation model can be trained based on training images created without human intervention. [Brief explanation of the drawing]

[0022] [Figure 1] This is a schematic block diagram showing the configuration of the estimation device according to Embodiment 1. [Figure 2] This flowchart shows the learning process in the estimation device according to Embodiment 1. [Figure 3] This figure shows an example of an initial training image. [Figure 4] This figure shows the learning progress in the estimation device according to Embodiment 1. [Figure 5] This figure shows an example of a computer configuration for realizing an estimation device. [Modes for carrying out the invention]

[0023] The following describes specific embodiments in detail with reference to the drawings. However, the invention is not limited to the embodiments described below. Also, for clarity, the following descriptions and drawings have been simplified as appropriate. Furthermore, the same elements are denoted by the same reference numerals, and redundant explanations are omitted.

[0024] Embodiment 1 This section will describe the estimation device according to this embodiment and the learning process in the estimation device. Figure 1 is a schematic block diagram showing the configuration of the estimation device according to Embodiment 1. The estimation device 10 includes an image acquisition unit 11, an estimation unit 12, an initial training image assignment unit 13, and a control unit 14.

[0025] The image acquisition unit 11 reads images from an external device to be used for training in the estimation unit 12. It then outputs the read images to the estimation unit 12. The image acquisition unit 11 can read images from various devices such as storage devices, CT scanners, and MRI scanners.

[0026] The estimation unit 12 is configured as a model that performs so-called segmentation, identifying objects in the input image and generating an output image in which the identified objects are displayed separately from other areas. For example, the estimation unit 12 is configured as an estimation model that identifies a specific object in the training image IMG and outputs an output image OUT[k] in which the identified area is displayed with white pixels and the other areas with black pixels. Note that the segmentation in the output image OUT[k] may use various display methods, such as multi-color coding or gradients, in addition to the binary image described above.

[0027] If the input image is a CT scan image of the human abdomen, the estimation unit 12 may output an output image in which, for example, specific organs in the image are represented by white pixels and other areas by black pixels.

[0028] As described later, the estimation unit 12 outputs an output image corresponding to the single training image IMG by performing multiple training iterations using a pre-prepared training image IMG and multiple training image data. Then, by performing similar training on multiple training images, the estimation unit 12 is constructed as a model that can output an output image corresponding to an object contained in an input image to be estimated. The estimation unit 12 may be, for example, a deep learning model. The deep learning model may be a Massive-Training Artificial Neural Network (MTANN). Alternatively, the deep learning model may be various models such as U-Net, V-Net, Convolutional Neural Network (CNN), Fully CNN (FCNN), Graph CNN, Vision Transformer, Res-Net, Generative Adversarial Networks (GAN), Shift-Invariant Neural Networks, or Fourier CNN.

[0029] The initial training image assignment unit 13 assigns the initial training image LBL that the estimation unit 12 uses first when performing supervised learning on the training image IMG. INI This is assigned to the estimation unit 12.

[0030] The control unit 14 provides the estimation unit 12 with a training image IMG and a teacher image, causing it to perform a predetermined number of training cycles. This allows the estimation unit 12 to train to the extent that it can output an output image in which the objects contained in the training image IMG are appropriately segmented.

[0031] Next, the learning process in the estimation device 10 according to Embodiment 1 will be explained based on a specific example. Figure 2 is a flowchart showing the learning process in the estimation device according to Embodiment 1. In this embodiment, as an example, we will explain the case in which the estimation unit 12 receives a CT tomography image of the patient's abdomen as input and outputs an output image in which the liver visible in the CT tomography image has been segmented.

[0032] In this case, the estimation unit 12 uses CT tomography images of the human abdomen as training images (IMG) for learning. The control unit 14 also controls the initial training image assignment unit 13 to provide the initial training image LBL, which represents pseudo-correct labels indicating the location and extent of the liver pixels using mathematical methods, so that the estimation unit 12 can perform its initial learning. INI This is provided to the estimation unit 12.

[0033] Step S1 The control unit 14 issues a control command CON1 to the image acquisition unit 11 in order to load the training image IMG into the image acquisition unit 11. As a result, the image acquisition unit 11 loads the training image IMG. The image acquisition unit 11 then outputs the loaded training image IMG to the estimation unit 12. The image acquisition unit 11 may load the training image IMG directly from the CT scanner, or it may load the training image IMG stored in a storage device (not shown).

[0034] Step S2 The control unit 14 sets the count value k, which indicates the number of learning iterations in the estimation unit 12, to 1. k is an integer between 2 and N. N will be described later.

[0035] Step S3 The control unit 14 sends the initial training image LBL to the estimation unit 12. INI To provide this, the control command CON3 is given to the initial training image assignment unit 13. As a result, the initial training image assignment unit 13 assigns the initial training image LBL INI The output is sent to the estimation unit 12. The estimation unit 12 uses the initial training image LBL. INI This is set as the LBL (Level-Based Learning) image to be used for training.

[0036] Step S4 The control unit 14 issues a control command CON2 to the estimation unit 12 in order to cause it to perform learning. As a result, the estimation unit 12 executes a learning process using the learning image IMG and the training image LBL.

[0037] Step S5 The control unit 14 determines whether the count value k has reached the learning count upper limit value N. If the count value k has reached the learning count upper limit value N, the learning in the estimation unit 12 is terminated.

[0038] Step S6 When the count value k is less than the learning count upper limit value N, the control unit 14 receives the output image OUT[k] output by the estimation unit 12 corresponding to the learning image IMG. Then, the control unit 14 sets the received output image OUT[k] as the teacher image LBL in the estimation unit 12. Note that the control unit 14 may control the estimation unit 12 so that the output image OUT[k] held internally by the estimation unit 12 is set as the teacher image LBL without receiving the output image OUT[k] from the estimation unit 12.

[0039] Step S7 The control unit 14 increments the count value k by 1. Then, the control unit 14 returns the process to step S4.

[0040] Through the above processing procedure, the estimation unit 12 can repeatedly perform the learning process using the output image obtained as the result of learning as the teacher image for the next learning.

[0041] Next, the progress of learning in the above-described estimation device 10 will be described. FIG. 3 is a diagram showing an example of an initial teacher image. In this example, the initial teacher image LBL INI is given as an image that mathematically represents a pseudo correct label in a region of a size that fits in a region where the liver is generally estimated to exist in a CT tomographic image of the abdomen of the human body. This pseudo correct label represents the region where the liver is estimated to exist in the initial teacher image LBL INI In FIG. 3, the initial teacher image LBL INI is represented as a binary image. The white region represents the region where the liver is estimated to exist, and the black region represents the other regions. Note that the liver as the object may be the black region and the other regions may be the white region. In general, in order to suitably perform the learning by the estimation unit 12, the initial teacher image LBL INIIt is desirable that the pseudo-correct label in the image is at least 50% larger than the true correct label that indicates the region where the liver is located.

[0042] For the sake of explanation, Figure 3 shows the true correct label in the lower left and a comparison of the sizes of the true correct label and the false correct label in the upper right. These are shown to facilitate understanding of the relationship between the true correct label and the false correct label, and do not mean that these images are created by the estimation device 10.

[0043] Initial teacher image LBL INI The pseudo-correct labels in this model may be constructed as the internal region of a closed curve expressed by an arbitrary function, with a certain point (marked with an "x" in Figure 3) as the reference point. For example, the mathematical model may be constructed using a single arbitrary closed curve, such as a smooth closed curve like a circle or an ellipse, or a closed curve containing a polyline, such as a polygon. Alternatively, the mathematical model may be constructed by combining two or more arbitrary closed curves such as circles, ellipses, and polygons. In this case, the mathematical model may be constructed using the envelope of multiple overlapping regions represented by closed curves.

[0044] Note that in Figure 3, due to limitations in representation, the initial training image LBL is shown. INI The image shown is a binary image, but this is merely an example. For example, using a probability distribution function where the probability of the liver being present is highest at an arbitrary reference point and decreases as the distance from the reference point increases, the initial training image LBL could be used. INI Pseudo-correct labels may be represented. In this case, a gradient may be applied to the pixel values ​​by applying a filter that represents a probability distribution, such as a Gaussian filter, to the edges of the pseudo-correct labels.

[0045] Figure 4 shows the learning progress in the estimation device according to Embodiment 1. In Figure 4, to facilitate understanding of the learning progress, the initial training image LBL is shown. INIThe pseudo-correct labels in this diagram are represented by ellipses. However, this is for illustrative purposes only; in actual training, it is preferable to use pseudo-correct labels with a shape and size that more closely approximates a typical liver, as shown in Figure 3.

[0046] First training process As shown in Figure 4, in the first training, the estimation unit 12 uses the training image IMG and the initial training image LBL. INI Learning is performed based on the following. In the output image OUT[1] obtained as a result of learning, the region estimated to contain the liver is the initial training image LBL. INI Rather than using pseudo-ground truth labels, the region is represented as one that approximates the shape and size of the region in the training image IMG where the liver exists (i.e., the true ground truth label). The output image OUT[1] obtained in the first training process is used as the training image LBL in the subsequent second training process.

[0047] The liver, which is the target of the training image IMG, is the initial training image LBL. INI If it is greater than the pseudo-correct label in the initial training image LBL, the estimation unit 12 will INI A region wider than the pseudo-correct label in the image is output as a region with high pixel values ​​(white region) where the liver is estimated to be present, as indicated by the white color. On the other hand, the liver, which is the target object of the training image IMG, is in the initial training image LBL. INI If it is smaller than the pseudo-correct label in the initial training image LBL, the estimation unit 12 will INI In the image, areas narrower than the pseudo-correct label are output as regions with high pixel values ​​(white regions) where the liver, shown in white, is estimated to be present. This is because the regions containing objects in the training image IMG contain grayscale patterns that characterize the objects, and the estimation unit 12 learns to output high pixel values ​​for regions containing these characteristic grayscale patterns. As a result, high pixel values ​​are also output for regions outside the pseudo-correct label.

[0048] Therefore, in the output image OUT[1], areas wider (or narrower) than the pseudo-ground truth label will have high pixel values ​​(white areas). As a result, the areas with high pixel values ​​in this output image OUT[1] will be closer to the true ground truth label (i.e., the liver in the training image IMG) than to the pseudo-ground truth label. Therefore, as will be explained below, by using the output image as the training image for the next second training, the initial training image LBL can be improved. INI Compared to the previous method, training can be performed based on training images with improved pseudo-correct labels.

[0049] Generally, if more than 50% of the model used as a pseudo-ground truth label overlaps with the target object, it is considered that the model can correctly learn the patterns characteristic of the object. Therefore, in order to improve the pseudo-ground truth label through learning, it is a condition that more than 50% of the model used as a pseudo-ground truth label overlaps with the target object (i.e., the true ground truth label).

[0050] By repeating this type of learning in subsequent learning iterations, the pseudo-correct labels of the training images can be gradually brought closer to the true correct labels.

[0051] Second training process In the second training, the estimation unit 12 performs training based on the training image IMG and the output image OUT[1] which is set as the training image LBL. In the output image OUT[2] obtained as a result of training, the region estimated to contain the liver is represented as a region whose shape and size approximate the region containing the liver in the training image IMG (i.e., the true ground truth label) rather than the pseudo-ground truth label in the training image LBL, output image OUT[1].

[0052] Furthermore, in subsequent learning processes, the control unit 14 may provide the estimation unit 12 with an image whose contrast has been adjusted from the output image that the estimation unit 12 can output as a result of the previous learning process, as the training image LBL. This allows the estimation unit 12 to learn more efficiently based on an image suitable as a training image with adjusted contrast. Note that various general contrast adjustment processes may be applied to adjust the contrast of the output image, or the initial training image LBL may be used. INI The pixel values ​​may also be normalized using the maximum and minimum values ​​in the image.

[0053] j-th learning process Here, j is an integer between 3 and N-1. In the j-th training, the estimation unit 12 performs training based on the training image IMG and the output image OUT[j-1] set as the training image LBL. In the output image OUT[j] obtained as a result of training, the region estimated to contain a liver is represented as a region whose shape and size approximate the region containing a liver in the training image IMG (i.e., the true ground truth label) rather than the pseudo-ground truth label in the output image OUT[j-1] which is the training image LBL.

[0054] Nth training process In the final Nth training iteration, the estimation unit 12 performs training based on the training image IMG and the output image OUT[N-1] set as the training image LBL. In the output image OUT[N] obtained as a result of training, the region estimated to contain the liver is represented as a region whose shape and size approximate the region containing the liver in the training image IMG (i.e., the true ground truth label) rather than the pseudo-ground truth label in the output image OUT[N-1] which is the training image LBL.

[0055] The upper limit of the number of training iterations N is set so that the error between the region in the training image IMG where the liver is present (i.e., the true ground truth label) and the region in the output image OUT[N] where the liver is estimated to be present (i.e., the false ground truth label) is sufficiently small. Therefore, the output image OUT[N] is output as an image in which the liver is suitably segmented from the training image IMG.

[0056] The upper limit N for the number of training iterations may be determined in advance by any method. For example, the count value k when the error between output image OUT[k] and output image OUT[k-1] becomes less than or equal to a predetermined value may be used as the upper limit N for the number of training iterations.

[0057] Alternatively, the estimation device 10, which has been trained by varying the upper limit of the number of training iterations, may be input with verification input images, and the number of training iterations that minimizes the error between the output image and the verification ground truth image may be determined as the upper limit of the number of training iterations.

[0058] However, these methods for determining the upper limit of the number of training iterations N are merely examples, and other methods may be used to determine the upper limit of the number of training iterations N as appropriate. Furthermore, multiple methods for determining the upper limit of the number of training iterations N may be combined.

[0059] As described above, according to the learning method of this embodiment, by using the output image made possible by supervised learning of images as the training image for the next learning session and performing repeated learning, the error of the output image obtained by segmenting the training image can be reduced to a desired value.

[0060] Furthermore, in the learning method according to this embodiment, the first learning session uses initial training images that indicate the region where the liver is located using a mathematical model. Therefore, there is no need to create training images in advance through manual annotation. Consequently, supervised learning can be performed without relying on manual annotation.

[0061] Initial teacher image LBL INIThe pseudo-correct labels inside may be figures roughly drawn in advance by a user of the estimation device 10, the average shape of an object, or figures output by inputting a training image (IMG) into a segmentation model using an estimation means different from the estimation device 10.

[0062] Furthermore, even when performing the learning process according to this embodiment on multiple training images, the same initial training image containing pseudo-correct labels expressed using the same mathematical model can be used. Therefore, learning using multiple training images can be performed efficiently without relying on manual intervention. As a result, a larger number of training images can be trained compared to learning that relies on manual annotation. This makes it possible to construct an estimation device capable of more accurate segmentation based on input images.

[0063] Other embodiments It should be noted that the present invention is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. In the embodiments described above, estimation related to CT images was explained, but the images to be estimated are not limited to CT images, but may also be other medical images such as MRI images, X-ray images, ultrasound images, and nuclear medicine images. Furthermore, it can be applied to the estimation of images in fields other than medical images.

[0064] Furthermore, the images are not limited to the medical field; they may also be images from various fields, such as manufacturing or other industrial sectors. For example, they may be images from surveillance cameras, in-vehicle cameras, radar images, microscope images, or electron microscope images.

[0065] In addition to the above, the estimation unit 12 may perform any processing that contributes to improving the model's performance, such as regularization and structural optimization, in order to improve the quality of the output image.

[0066] In the embodiments described above, the estimation device according to this disclosure has been described mainly as a hardware configuration, but is not limited thereto. The estimation device according to this disclosure can be realized by having a computer execute a computer program to perform any processing. These processing may be realized by having a computer, which includes at least one processor (e.g., a microprocessor, CPU, GPU, MPU, or DSP (Digital Signal Processor)), execute a program. Specifically, one or more programs containing a set of instructions for having a computer perform algorithms related to these transmission signal processing or reception signal processing can be created and supplied to the computer.

[0067] Computer programs can be stored and supplied to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (random access memory)). Programs may also be supplied to a computer using various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable media can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0068] The following shows an example of a computer configuration for realizing the estimation device according to the above embodiment. Figure 5 is a diagram showing an example of a computer configuration for realizing the estimation device. The estimation device can be realized by a computer 9000 such as a dedicated computer or a personal computer (PC). However, the computer does not need to be physically single; there may be multiple computers when performing distributed processing. As shown in Figure 5, the computer 9000 has, for example, a processor 9001, a ROM (Read Only Memory) 9002, a RAM (Random Access Memory) 9003, a storage unit 9004, a communication interface 9005, and a user interface 9006.

[0069] The processor 9001, ROM 9002, RAM 9003, memory unit 9004, communication interface 9005, and user interface 9006 are interconnected via bus 9007, enabling them to communicate with each other. While the operating system software necessary to run the computer is not described here, it will be implemented in the computer 9000 as appropriate.

[0070] ROM is composed of, for example, non-volatile semiconductor memory devices. ROM 9002 stores information such as various programs used in computer 9000.

[0071] The storage unit 9004 is composed of various storage devices, such as hard disks and solid-state disks. Furthermore, the storage unit 9004 is not limited to storage devices installed in the computer 9000, but may also be external storage devices. External storage devices may include various communication means, such as cloud storage connected to the computer 9000 via a network. The storage unit 9004 stores information such as various programs and data used by the computer 9000.

[0072] RAM 9003 is composed of volatile semiconductor memory devices. Programs and data used by the processor 9001 are loaded into RAM 9003 as needed from either ROM 9002 or memory unit 9004, or both.

[0073] The processor 9001 may be composed of, for example, a CPU (Central Processing Unit). Alternatively, the processor 9001 may include a GPU (Graphics Processing Unit) in addition to the CPU. A GPU is suitable for parallel processing of routine tasks, and can improve processing speed compared to a CPU, for example, by being used in neural network processing. The processor 9001 executes various processes based on various programs stored in the ROM 9002, or various programs and data held in the RAM 9003. The processor 9001 may also store the data created by the processing in the RAM 9003 or the memory unit 9004 as appropriate.

[0074] The communication interface 9005 is an interface that connects the computer 9000 to a communication network such as the Internet or an intranet via various wired or wireless communication means. This allows the computer 9000 to communicate with other devices, systems, and sensors connected to the communication network.

[0075] The user interface 9006 includes, for example, a display unit that provides information so that the user can perceive it, such as through a display device, and an audio output unit that provides audio. The user interface 9006 also includes an input unit that allows the user to input information into the computer 9000 through user operation, such as a keyboard, mouse, and touch panel. Furthermore, the user interface 9006 may include devices such as sensors that acquire information useful to the user.

[0076] Here, the computer 9000 is described as a single device, but this is merely an example. The computer 9000 may consist of multiple physically separated devices. Some of these devices may be portable, while others may be stationary.

[0077] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments rather than with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps shown in any of the drawings may be changed as appropriate. [Explanation of Symbols]

[0078] 10 Estimation device 11 Image acquisition unit 12 Estimation part 13. Initial training image assignment unit 14 Control Unit 9000 Computers 9001 Processor 9002 ROM 9003 RAM 9004 Storage section 9005 Communication Interface 9006 User Interface 9007 Bus CON1~CON3 Control Signals IMG (Image for Learning) LBL Teacher Image LBL INI Initial training image OUT[k] Output image

Claims

1. An image acquisition unit that acquires training images, An estimation unit that outputs an output image representing the segmentation result of the training image based on the training image and the teacher image, An initial training image assignment unit provides the estimation unit with initial training images in which pseudo-correct labels for objects included in the training images are constructed by a mathematical model, which are used for training in the estimation unit. The system comprises a control unit that causes the estimation unit to perform learning multiple times, The control unit, In the first learning phase, the initial training image provided to the estimation unit by the initial training image assignment unit is used as the training image, and the estimation unit is instructed to perform learning. In subsequent learning sessions, the output image produced as a result of the previous learning session is used as the training image, and the estimation unit is instructed to perform the learning. If the number of learning iterations reaches a predetermined number, the learning in the estimation unit is stopped. Estimation device.

2. The pseudo-correct labels of the initial training images are figures drawn in advance by the user of the estimation device, the average shape of an object, or figures output by inputting the training images into a segmentation model in an estimation means different from the estimation device. The estimation device according to claim 1.

3. In the second and subsequent learning sessions, the control unit uses an image with adjusted contrast from the output image output as a result of the previous learning session as the training image, and causes the estimation unit to perform the learning. The estimation device according to claim 1 or 2.

4. The pseudo-correct labels of the initial training image are regions represented by closed curves with reference to points set within them. The estimation device according to claim 1 or 2.

5. The pseudo-correct labels of the initial training image are regions represented by a combination of multiple closed curves. The estimation device according to claim 4.

6. The pseudo-correct labels of the initial training image are regions represented by the envelopes of the plurality of closed curves. The estimation device according to claim 5.

7. As you move from the inside of the pseudo-correct label towards the edge of the pseudo-correct label, the pixel values ​​of the pseudo-correct label become smaller. The estimation device according to claim 1 or 2.

8. The aforementioned predetermined number is a value that has been set in advance. The estimation device according to claim 1 or 2.

9. When the error between the training image and the output image becomes smaller than a predetermined error, the control unit considers that the number of learning iterations has reached the predetermined number and stops the learning in the estimation unit. The estimation device according to claim 1 or 2.

10. Obtain training images, The estimation unit, which outputs an output image representing the segmentation result of the training image based on the training image and the target image, is made to perform training multiple times. In the multiple learning sessions mentioned above, In the first training, the estimation unit is given an initial training image in which the pseudo-correct labels of the objects included in the training image are constructed by a mathematical model, and the estimation unit is instructed to perform training using the initial training image as the training image. In subsequent learning sessions, the output image produced by the estimation unit as a result of the previous learning session is used as the training image, and the estimation unit is instructed to perform learning. If the number of learning iterations reaches a predetermined number, the learning in the estimation unit is stopped. Learning methods.

11. The process of acquiring training images, The computer is instructed to perform the following process: an estimation unit, which outputs an output image representing the segmentation result of the training image based on the training image and the teacher image, is made to perform training multiple times; In the multiple learning sessions mentioned above, In the first training, the estimation unit is given an initial training image in which the pseudo-correct labels of the objects included in the training image are constructed by a mathematical model, and the estimation unit is instructed to perform training using the initial training image as the training image. In subsequent learning sessions, the output image produced by the estimation unit as a result of the previous learning session is used as the training image, and the estimation unit is instructed to perform learning. If the number of learning iterations reaches a predetermined number, the learning in the estimation unit is stopped. program.

Images