Teacher data correction device, teacher data correction method, and storage medium

Abstract

A teacher data correction device 1X includes an acquisition means 31X and a teacher data correction means 33X. The acquisition means 31X acquires training input data input to a machine learning model during training of the machine learning model, and teacher data corresponding to the training input data. The teacher data correction means 33X corrects the teacher data on the basis of the training input data, the teacher data, and a trained model. The trained model is a model which has been machine-trained to output an inference result related to input data when the input data and a prompt which is a suggestion related to the input data are input. The teacher data correction means 33X corrects the teacher data on the basis of the inference result output by the trained model when input data based on the training input data and a prompt based on the teacher data are input to the trained model.

Description

Teacher data correction device, teacher data correction method, and storage medium

[0001] The present disclosure relates to a teacher data correction device, a teacher data correction method, and a storage medium that correct teacher data used in machine learning.

[0002] A method for performing machine learning robustly even when teacher data (labels) contain noise has been proposed. For example, Patent Literature 1 discloses a method for learning a model partway through for teacher labels given on an image-by-image basis, determining whether the given teacher labels are correct using the model, and assigning pseudo labels to incorrect teacher labels.

[0003] International Publication WO2021 / 055904

[0004] In the method of Patent Document 1, if the reliability of the training data is low overall, the quality of the model that is trained up to a certain point will be low, and it will be impossible to accurately determine whether the training data is correct, resulting in a decrease in the quality of the model that is ultimately obtained. As such, in the past, in order to reduce the influence of noise that occurs in the training data, it was necessary to prepare a considerable amount of highly reliable training data.

[0005] In view of the above-mentioned problems, the object of the present disclosure is to provide a teacher data correction device, a teacher data correction method, and a storage medium that can suitably correct teacher data used in machine learning.

[0006] One aspect of a teacher data correction device is a teacher data correction device comprising: an acquisition means for acquiring training input data to be input to a machine learning model when training the machine learning model, and teacher data corresponding to the training input data; and a teacher data correction means for correcting the teacher data based on the training input data, the teacher data, and a trained model, wherein the trained model is a model trained by machine learning so as to output an inference result regarding the input data when input data and a prompt that is suggestive regarding the input data are input; and the teacher data correction means corrects the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.

[0007] One aspect of the teacher data correction method is a teacher data correction method in which a computer acquires training input data to be input to a machine learning model when training the machine learning model, and teacher data corresponding to the training input data, corrects the teacher data based on the training input data, the teacher data, and a trained model, the trained model being a model trained by machine learning to output an inference result regarding the input data when input data and a prompt that is suggestive of the input data are input, and corrects the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.

[0008] One aspect of the storage medium is a storage medium storing a program that: acquires training input data to be input to a machine learning model when training the machine learning model, and teacher data corresponding to the training input data; causes a computer to execute a process to correct the teacher data based on the training input data, the teacher data, and a trained model; the trained model is a model that has been machine-learned to output an inference result regarding the input data when input data and a prompt that is suggestive of the input data are input; and causes the computer to execute a process to correct the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.

[0009] According to the present disclosure, it is possible to suitably correct training data used in machine learning.

[0010] 1 shows a schematic configuration of a teacher data correction system. FIG. 2 shows the hardware configuration of a teacher data correction device. FIG. 3 shows an overview of the correction process for teacher data registered in a training dataset. FIG. 4 is a functional block diagram of a teacher data correction device. FIG. 5 is an example of a display screen displayed by an output device. FIG. 6 is an example of a flowchart showing an overview of the process performed by the teacher data correction device. FIG. 7 is a schematic configuration diagram of a teacher data correction system in which a base model is executed by a device separate from the teacher data correction device. FIG. 8 is an overview of the correction process for teacher data. FIG. 9 is an example of a functional block diagram of a teacher data correction device. FIG. 10 is an example of a flowchart showing an overview of the process performed by the teacher data correction device. FIG. 11 is an example of a functional block diagram that adaptively determines the number of times corrected teacher data is generated. FIG. 12 is an example of a functional block diagram that takes into account the process of generating prompts. FIG. 13 is a block diagram of a teacher data correction device. FIG. 14 is an example of a flowchart showing the processing procedure of a teacher data correction device.

[0011] Hereinafter, embodiments of a teacher data correction device, a teacher data correction method, and a storage medium will be described with reference to the drawings.

[0012] <First embodiment> (1) System configuration Fig. 1 shows a schematic configuration of a teacher data correction system 100. As shown in Fig. 1, the teacher data correction system 100 is a system that corrects teacher data used in machine learning of a machine learning model such as a deep learning model, and mainly includes a teacher data correction device 1, a storage device 2, an output device 3, and an input device 4.

[0013] The teacher data correction device 1 generates teacher data by correcting the teacher data contained in the training dataset D1 based on the training dataset D1 stored in the memory device 2 and the basic model information D2, and stores the generated corrected teacher data in the training dataset D1.

[0014] The storage device 2 is a memory that stores various information necessary for the processing of the teacher data correction device 1. The storage device 2 has a training dataset D1 and base model information D2.

[0015] The training dataset D1 is a dataset used for machine learning of a machine learning model. The training dataset D1 has, for example, multiple records, and each record has a training input image and a teacher label (also called ground truth or teacher label). The training input image is input data to the machine learning model that is the target of machine learning. The teacher data is data that indicates the correct answer that should be output by the machine learning model when a corresponding training input image is input to the machine learning model. The teacher data is generated by annotation work by an operator.

[0016] In this embodiment, as an example, the training input image is a medical image of a patient's organ, and the teacher data includes at least correct region information for the lesion region within the training input image. Here, the "region information" may be a mask image indicating the lesion region, or information indicating a bounding box surrounding the lesion region. The teacher data may further include other information, such as class information into which the lesion region is classified. In this case, the "class information" indicates the class (category) into which the region indicated by the paired region information is classified, and may indicate whether or not it is a lesion region, or may indicate the type of lesion if it is a lesion region in addition to whether or not it is a lesion region.

[0017] Here, we will provide additional information about the teacher data included in the training dataset D1. Generally, when a doctor or other worker annotates medical images to assign a correct answer (generates teacher data), it is difficult to uniquely determine the correct answer, and so the generated teacher data is prone to contain noise. Therefore, the teacher data generated in this manner may contain variations due to individual differences between workers, their condition, and the like. Therefore, the teacher data correction system 100 of this embodiment appropriately corrects teacher data included in the training dataset D1 whose accuracy is not sufficiently guaranteed. Thus, an example of teacher data in the present disclosure is teacher data that can be used to train a machine learning model related to healthcare.

[0018] The training dataset D1 may be used as a validation set to tune the hyperparameters of a machine-learned model, or as a test set to confirm the accuracy of the machine-learned model.

[0019] The base model information D2 is information about the base model, which is a machine learning model, and includes trained parameters of the base model. The base model is a large-scale deep learning neural network trained using a diverse and large-scale dataset, and is an AI model known as a generative AI. The base model in this embodiment is a generative AI that generates an image (e.g., a mask image) showing a segmentation result from an image input to the base model based on a prompt, which is information representing an instruction (suggestion) input by a user. Examples of such generative AI include SAM (Segment Anything Model) and SEEM (Segment Everything Everywhere All at Once). For example, SAM is a generative AI that receives an image and a prompt (segmentation prompt) suggesting an object to be segmented in the image as input, and generates a mask image (segmentation mask) representing the region of an arbitrary object as an inference result. SAM accepts prompts such as points on an image suggesting an object to be segmented, a bounding box (rectangular frame), a mask image, text, or any combination thereof. The base model in this embodiment accepts at least a mask image suggesting an area to be segmented as a prompt. Details of SAM are described in, for example, the following literature: Kirillov A, Mintun E, Ravi N, Mao HZ, Rolland C, Gustafson L, Xiao TT, Whitehead S, Berg AC, Lo WY, Dollar P, Girshick R. Segment anything. arXiv preprint arXiv:2304.02643, 2023

[0020] The output device 3 outputs information based on the control of the teacher data correction device 1. Examples of the output device 3 include display devices such as a display and a projector, and sound output devices such as a speaker. The output device 3 displays information and / or outputs sound based on the output information supplied from the teacher data correction device 1.

[0021] The input device 4 is an interface that accepts user input, which is external input, and corresponds to, for example, a touch panel, buttons, a keyboard, a voice input device, etc. The input device 4 supplies input information generated based on the user input to the teacher data correction device 1.

[0022] The configuration of the teacher data correction system 100 shown in Figure 1 is an example, and various modifications may be made to this configuration. For example, the teacher data correction device 1, the storage device 2, the output device 3, and the input device 4 may be integrated into any combination. The teacher data correction device 1 may also be composed of multiple devices. In this case, the multiple devices that make up the teacher data correction device 1 exchange information necessary to execute pre-assigned processing between these multiple devices.

[0023] (2) Hardware Configuration Fig. 2 shows the hardware configuration of the teacher data correction device 1. The teacher data correction device 1 includes, as hardware, a processor 11, a memory 12, and an interface 13. The processor 11, the memory 12, and the interface 13 are connected via a data bus 19.

[0024] The processor 11 executes predetermined processes by executing programs stored in the memory 12. The processor 11 is a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or a TPU (Tensor Processing Unit). The processor 11 may be composed of multiple processors. The processor 11 is an example of a computer.

[0025] The memory 12 is composed of various types of volatile and non-volatile memories such as RAM (Random Access Memory) and ROM (Read Only Memory). The memory 12 also stores programs for the teacher data correction device 1 to execute various processes. The memory 12 is also used as a working memory, and temporarily stores information obtained from the storage device 2. The memory 12 may also function as the storage device 2. Similarly, the storage device 2 may also function as the memory 12 of the teacher data correction device 1. The programs executed by the teacher data correction device 1 may be stored in a storage medium other than the memory 12.

[0026] The interface 13 is an interface for electrically connecting the teacher data correction device 1 to other devices. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, or may be hardware interfaces for connecting to other devices via cables or the like.

[0027] The hardware configuration of the teacher data correction device 1 is not limited to the configuration shown in Fig. 2. For example, the teacher data correction device 1 may include at least one of the output device 3 or the input device 4. The teacher data correction device 1 may also be connected to or built into a sound output device such as a speaker.

[0028] (3) Teacher Data Correction Process Figure 3 is a diagram showing an overview of the correction process for the teacher data registered in the training dataset D1. Hereinafter, the original teacher data (i.e., before correction) registered in the training dataset D1 will be referred to as "initial teacher data," and the original teacher data corrected by the teacher data correction device 1 will be referred to as "corrected teacher data."

[0029] The teacher data correction device 1 sequentially extracts records (i.e., pairs of training input images and initial teacher data) from the training dataset D1. The teacher data correction device 1 then inputs the extracted training input images into a base model configured based on base model information D2, and inputs the extracted initial teacher data into the base model as a prompt. The teacher data correction device 1 then obtains the inference results (segmentation results) output by the base model in response to the above inputs.

[0030] Next, the teacher data correction device 1 generates corrected teacher data based on the mask image output by the base model. In the example of FIG. 3 , the base model outputs a mask image as an inference result regarding the segmentation of the lesion area, and the teacher data correction device 1 regards the output mask image as corrected teacher data. The teacher data correction device 1 then associates the corrected teacher data with the used training input image and registers it in the training dataset D1. In this case, the teacher data correction device 1 may delete the initial teacher data corresponding to the used training input image, or may register the corrected teacher data in the training dataset D1 while leaving the initial teacher data. Furthermore, the teacher data correction device 1 may determine whether or not to register the corrected teacher data in the training dataset D1 based on user input, as described below.

[0031] In this way, the teacher data correction device 1 corrects the teacher data using a base model that accepts the initial teacher data as a prompt. As a result, even if the initial teacher data registered in the training dataset D1 is low-accuracy teacher data that has variations due to individual differences between workers or the condition of the workers, the teacher data correction device 1 can generate high-accuracy corrected teacher data in which the above-mentioned variations have been suitably reduced.

[0032] Figure 4 shows an example of functional blocks of the teacher data correction device 1. As shown in Figure 4, the processor 11 of the teacher data correction device 1 functionally includes an acquisition unit 31, a base model execution unit 32, and a teacher data correction unit 33. In Figure 4, blocks where data is exchanged are connected by solid lines, but the combination of blocks where data is exchanged is not limited to Figure 4. The same applies to other functional block diagrams described below. Note that Figure 4 shows, as an example, a configuration for determining whether or not corrected teacher data has been registered in the training dataset D1 based on user input.

[0033] The acquisition unit 31 sequentially extracts records (pairs of training input images and initial teacher data) included in the training dataset D1, and supplies the extracted training input images and initial teacher data to the base model execution unit 32. The acquisition unit 31 also supplies the extracted training input images and initial teacher data to the teacher data correction unit 33.

[0034] The base model execution unit 32 acquires the training input image and initial teacher data supplied from the acquisition unit 31, and inputs the training input image into the base model constructed based on the base model information D2. At this time, the base model execution unit 32 inputs the initial teacher data into the base model as a prompt. The base model execution unit 32 then acquires the inference result output by the base model in response to the input, and supplies the acquired inference result to the teacher data correction unit 33.

[0035] The teacher data correction unit 33 generates corrected teacher data by correcting the initial teacher data based on the inference results of the base model supplied from the base model execution unit 32.

[0036] For example, if the inference result of the base model indicates a single mask image, the teacher data correction unit 33 uses the mask image as the corrected teacher data. Note that if it is necessary to generate teacher data indicating a bounding box within an image, the teacher data correction unit 33 regards, for example, the smallest rectangular area within the image that includes the area indicated by the inference result of the base model as the bounding box, and generates corrected teacher data indicating that rectangular area.

[0037] Furthermore, when the base model outputs multiple mask images (e.g., three mask images in the case of SAM) as inference results, a mask image that integrates the inference results output by the base model is generated as modified teacher data. In this case, the teacher data correction unit 33 may generate an overlapping region (so-called AND region) or a merged region (so-called OR region) of multiple mask images as modified teacher data, or may use a mask image selected according to a predetermined rule or randomly as modified teacher data. Furthermore, when it is necessary to generate teacher data indicating a bounding box within an image, the teacher data correction unit 33 identifies the bounding box from the integrated mask image and generates modified teacher data indicating the bounding box.

[0038] In addition, the teacher data correction unit 33 may present the initial teacher data and the corrected teacher data to the user and accept external input based on the user's decision-making regarding the suitability of the corrected teacher data.

[0039] 5 is an example of a display screen that the teacher data correction unit 33 displays on the output device 3. In this case, the teacher data correction unit 33 generates a display signal based on the corrected teacher data supplied from the base model execution unit 32 and the training input image and initial teacher data supplied from the acquisition unit 31, and supplies the generated display signal to the output device 3, thereby causing the output device 3 to display the display screen shown in FIG.

[0040] The teacher data correction unit 33 displays the training input image, the initial teacher data labeled "GT before correction," and the corrected teacher data labeled "GT after correction" on the display screen. The teacher data correction unit 33 also provides a first selection button 70 labeled "Reflect correction" and a second selection button 71 labeled "Do not correct" on the display screen. When the teacher data correction unit 33 detects that the first selection button 70 has been selected based on an input signal supplied from the input device 4, it registers the displayed corrected teacher data into the training dataset D1. On the other hand, when the teacher data correction unit 33 detects that the second selection button 71 has been selected based on an input signal supplied from the input device 4, it discards the displayed corrected teacher data without registering it into the training dataset D1. After the first selection button 70 or the second selection button 71 is selected, the teacher data correction unit 33 causes the output device 3 to display a display screen that accepts input regarding the appropriateness of the corrected teacher data for the record next acquired by the acquisition unit 31.

[0041] In this way, the teacher data correction unit 33 arranges the initial teacher data and the corrected teacher data so that the user can compare them, and accepts external input based on the user's decision on the appropriateness of the corrected teacher data. This allows the user to make a decision in an appropriate manner and accept input based on the decision. Furthermore, the teacher data correction unit 33 can accurately determine whether or not to use each of the generated corrected teacher data as the training dataset D1.

[0042] Here, each of the components of the acquisition unit 31, the base model execution unit 32, and the teacher data correction unit 33 can be realized, for example, by the processor 11 executing a program. Alternatively, each component may be realized by recording the necessary program on any non-volatile storage medium and installing it as needed. Note that at least a portion of these components may not necessarily be realized by software programs, but may also be realized by any combination of hardware, firmware, and software. Furthermore, at least a portion of these components may be realized using a user-programmable integrated circuit, such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, this integrated circuit may be used to realize a program composed of the above components. Furthermore, at least a portion of each component may be configured by an ASSP (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). In this way, each component may be realized by various hardware. The same applies to other embodiments described below. Furthermore, each of these components may be realized by the cooperation of multiple computers, for example, using cloud computing technology.

[0043] FIG. 6 is an example of a flowchart showing an outline of the processing executed by the teacher data correction device 1.

[0044] First, the teacher data correction device 1 acquires unprocessed records (pairs of training input images and initial teacher data) from the training dataset D1 (step S11). Then, the teacher data correction device 1 executes a foundation model based on the foundation model information D2 using the training input images as input and the initial teacher data as prompts (step S12). As a result, the teacher data correction device 1 acquires the inference results of the foundation model.

[0045] Then, the teacher data correction device 1 generates corrected teacher data based on the inference result of the base model and stores it in the storage device 2 (step S13). In this case, the teacher data correction device 1 may display the display screen shown in Figure 5 on the output device 3, and determine whether or not the generated corrected teacher data needs to be registered in the training dataset D1 based on an input signal output from the input device 4 operated by the user.

[0046] Then, the teacher data correction device 1 determines whether or not to end the processing of the flowchart (step S14). For example, if the teacher data correction device 1 determines that the processing of steps S11 to S13 has been completed for all records registered in the training dataset D1, it determines that the processing of the flowchart should be ended, and otherwise it determines that the processing of the flowchart should be continued.

[0047] (4) Modifications Next, suitable modifications of the above-described embodiment will be described. The following modifications may be applied in combination to the above-described embodiment.

[0048] (Modification 1-1) The base model may be executed by a device other than the teacher data correction device 1.

[0049] 7 is a schematic diagram of a teacher data correction system 100A in which the base model is executed by a device separate from the teacher data correction device 1. For simplicity, the storage device 2, output device 3, input device 4, etc. are not shown. The teacher data correction system 100A includes a server device 5 that stores base model information D2. The server device 5 may be composed of multiple devices. The teacher data correction system 100A also includes a teacher data correction device 1 that is capable of data communication with the server device 5 via a network.

[0050] The teacher data correction device 1 transmits training input images and initial teacher data to the server device 5, and in response receives the inference results of the base model from the server device 5. When the server device 5 receives training input images and initial teacher data from the teacher data correction device 1, it inputs them into a base model constructed based on base model information D2, and transmits the inference results output by the base model in response to the input to the teacher data correction device 1. In this case, the interface 13 of each teacher data correction device 1 includes a communication interface such as a network adapter for communication.

[0051] In this way, even if the base model is executed by a device other than the teacher data correction device 1, the teacher data correction device 1 can preferably obtain the inference results of the base model and generate corrected teacher data.

[0052] (Variation 1-2) The training data is not limited to region information indicating a mask image or a bounding box, but may be data in any format that matches the output format of a machine learning model that is the target of machine learning using the training dataset D1.

[0053] For example, if the training dataset D1 is a dataset used to train a machine learning model that outputs classes as classification results, the initial teacher data will be classification labels indicating the classes. In this case, for example, the base model will be a machine learning model that accepts a classification label as a prompt and outputs, as an inference result, a classification label that corrects the class indicated by the classification label input as the prompt or a classification label indicating a class classified more precisely than the class. An example of such a base model is SEEM. Note that SEEM accepts a classification label as a prompt and outputs a classification label and a segmentation mask.

[0054] In another example, if the training dataset D1 is a dataset used to train a machine learning model that outputs text information indicating captions (documents related to input images), the initial training data will be text information indicating the captions. In this case, for example, the base model will be a machine learning model that accepts text information as a prompt and outputs, as an inference result, text information that has been corrected based on the text information input as the prompt in accordance with the image input to the base model. Examples of such base models include Caption Anything. Using such a base model makes it possible to generate corrected training data in which notational variations in the training data in document format are reduced.

[0055] Here, we provide additional information about machine learning of the foundation model for realizing the foundation model described above. Large-scale data sets containing a large number of input images, such as landscape paintings, labeled with labels such as masks, bounding boxes (rectangles), or text information (captions) are prepared. The labels comprehensively represent various objects and features in the input images (e.g., a set of rectangles or mask images attached to all objects, or captions indicating what appears where in the input image). Then, prompts (e.g., region information, text information identifying the object, etc.) are randomly generated, and corresponding correct answers (training data) are extracted and defined from the labels. The foundation model (neural network) is trained to output the correct answer from a pair of input images and prompts. The correct answer may take the form of a class, bounding box (rectangle), mask image, text information, etc. During training, the parameters of the foundation model are determined using, for example, gradient descent or backpropagation so as to minimize the error (loss) between the output of the foundation model and the correct answer when an input image and prompt are input.

[0056] (Modification 1-3) In place of the training input images, data in any format other than images may be registered in the training data set D1.

[0057] In this case, each record of the training dataset D1 includes training input data based on speech data or text information and corresponding teacher data. Here, the training input data is, for example, data in which speech data or text information is encoded, and is in a tensor format that is compatible with the input formats of the base model and the machine learning model to be trained.

[0058] The training input data may also be 3D images containing depth information for each pixel, such as 3D point cloud data output by a lidar.

[0059] In this way, even if training input data in any data format is registered in the training dataset D1, the teacher data correction device 1 can generate corrected teacher data by correcting the teacher data of the training input data based on the base model.

[0060] Note that instead of inputting the training input data directly to the base model or the machine learning model to be learned, the training input data may be input to the above-mentioned model after undergoing a predetermined process (e.g., feature extraction process). In this way, input data based on the training input data may be input to the base model or the machine learning model to be learned.

[0061] (Variation 1-4) The teacher data correction device 1 may further use the inference results output by the base model as a prompt when inputting them into the base model, and may recursively execute the base model on the same training input image.

[0062] In this case, for example, when the teacher data correction device 1 executes the base model N times (N is an integer greater than or equal to 2), it generates corrected teacher data based on the inference result of the Nth base model (i.e., the most recent inference result). In another example, the teacher data correction device 1 may generate corrected teacher data based on the inference results of the first to Nth base model (which may further include the initial teacher data). In this case, for example, the teacher data correction device 1 may generate an overlapping region (so-called AND region) or a merged region (so-called OR region) of mask images indicated by multiple inference results as corrected teacher data, or may use a mask image selected according to a predetermined rule or randomly as corrected teacher data.

[0063] Here, the number of times "N" may be a predetermined default value or may be an adaptively set value. When the number of times "N" is adaptively set, for example, the teacher data correction device 1 repeats generating corrected teacher data until it determines that a predetermined condition is satisfied.

[0064] The predetermined condition is set, for example, as a condition for determining that the fluctuation of the corrected teacher data has converged. In this case, for example, the teacher data correction device 1 determines whether the predetermined condition is satisfied based on the average fluctuation rate of the corrected teacher data for the entire training data set D1. The average fluctuation rate is an example of the "degree of fluctuation."

[0065] In this case, for example, the teacher data correction device 1 compares the average fluctuation rate of the corrected teacher data for the entire training dataset D1 with a predetermined threshold. Here, when corrected teacher data has been generated n times (n is an integer greater than or equal to 1), the convergence determination unit 34A calculates the fluctuation rate for each record of the training dataset D1 based on the corrected teacher data generated the nth time and the corrected teacher data generated the n-1th time (or the initial teacher data when n is 1). In this case, an index such as IoU (Intersection Over Union) may be used as the fluctuation rate. Then, the teacher data correction device 1 calculates the average of the fluctuation rates calculated for each record of the training dataset D1 over all records as the average fluctuation rate.

[0066] If the average fluctuation rate is equal to or greater than the threshold, the teacher data correction device 1 determines that the predetermined condition is not satisfied and continues generating corrected teacher data. On the other hand, if the average fluctuation rate is less than the threshold, the teacher data correction device 1 determines that the predetermined condition is satisfied and determines that generation of corrected teacher data should be terminated. In this case, as described above, the teacher data correction device 1 may adopt the last generated corrected teacher data for each record of the training dataset D1 as the final teacher data, or may generate final teacher data based on all generated corrected teacher data (which may further include the initial teacher data).

[0067] Second Embodiment In the second embodiment, the teacher data correction device 1 differs from the first embodiment in that the teacher data correction device 1 inputs the inference results output by the machine learning model trained based on the training dataset D1 as prompts to the base model. As a result, the teacher data correction device 1 uses data with reduced variability from the teacher data as prompts to the base model to generate more accurate corrected teacher data. Hereinafter, the machine learning model trained based on the training dataset D1 will also be referred to as the "main model."

[0068] Hereinafter, components of the teacher data correction system 100 that are similar to those in the first embodiment will be appropriately designated by the same reference numerals as in the first embodiment, and their description will be omitted. The hardware configuration of the teacher data correction device 1 according to the second embodiment is the same as the hardware configuration of the teacher data correction device 1 shown in Figure 2, and the functional block configuration of the processor 11 of the teacher data correction device 1 according to the second embodiment is the same as the functional block configuration shown in Figure 3.

[0069] FIG. 8 is a diagram showing an outline of the correction process for the training data.

[0070] First, the teacher data correction device 1 constructs a main model by training a machine learning model using a training dataset D1. In this case, the teacher data correction device 1 uses each record of the training dataset D1 to update the parameters of the main model by gradient descent, backpropagation, or the like, so as to minimize the error (loss) between the inference result output by the main model when a training input image is input and the initial teacher data corresponding to the input training input image. Note that the machine learning model trained as the main model is a machine learning model having any architecture, such as a neural network or a support vector machine. Examples of main model architectures include Feature Pyramid Network, Featured image pyramid, Fully Convolutional Network, SegNet, U-Net, V-Net, Mask R-CNN, DeepLab, AlexNet, VGG, ResNet, SqueezeNet, DenseNet, Inception, GoogleNet, ShuffleNet, MobileNet, ResNeXt, Wide ReNet, NASNet, etc.

[0071] Next, the teacher data correction device 1 inputs each training input image used in training the main model into the main model and obtains the inference result (here, a mask image) output by the main model in response to the input. The teacher data correction device 1 then inputs the training input images input to the main model into the base model, and inputs the inference result output by the main model as a prompt into the base model, obtaining the inference result output by the base model in response to these inputs. The teacher data correction device 1 then generates modified teacher data based on the inference result of the base model and registers the generated modified teacher data in the training dataset D1 as teacher data for the training input images input to the base model. In this case, the teacher data correction device 1 may delete the initial teacher data for the same record as the used training input image, or may register the modified teacher data in the training dataset D1 while leaving the initial teacher data. Furthermore, as in the first embodiment, the teacher data correction device 1 may accept user input regarding the suitability of the modified teacher data for the training dataset D1.

[0072] By doing this, the teacher data correction device 1 can use the inference results of the main model, in which the noise contained in the initial teacher data has been reduced, as a prompt for the base model, thereby generating more accurate corrected teacher data.

[0073] Figure 9 shows an example of functional blocks of the teacher data correction device 1. As shown in Figure 9, the processor 11 of the teacher data correction device 1 functionally includes a learning unit 30A, a main model execution unit 31A, a base model execution unit 32A, and a teacher data correction unit 33A. Note that Figure 9 shows, as an example, a configuration for determining whether or not corrected teacher data has been registered in the training dataset D1 based on user input.

[0074] The learning unit 30A performs machine learning of the main model based on the training dataset D1 and stores the parameters of the training dataset D1 obtained by machine learning as main model information D3 in the storage device 2. The learning unit 30A updates the parameters of the main model registered in the main model information D3 for each record of the training dataset D1 so as to minimize the error between the inference result output by the main model when a training input image is input to the main model and the corresponding initial teacher data. If the main model is configured using a neural network, the main model information D3 includes various parameters such as the layer structure, the neuron structure of each layer, the number and filter size of filters in each layer, and the weight of each element of each filter. Instead of providing the learning unit 30A, the main model information D3 storing the parameters of the main model that has been machine-learned may be stored in advance in the storage device 2.

[0075] The main model executing unit 31A sequentially extracts training input images used in learning the main model from the training data set D1, and executes the main model constructed with reference to the main model information D3. In this case, the main model executing unit 31A supplies the inference results output by the main model by inputting the training input images to the main model to the base model executing unit 32A.

[0076] The base model execution unit 32A uses the inference result of the main model supplied from the main model execution unit 31A as a prompt, and inputs the training input image input to the main model into the base model constructed based on the base model information D2. The base model execution unit 32A then acquires the inference result output by the base model in response to the input, and supplies the acquired inference result to the teacher data correction unit 33A.

[0077] The teacher data correction unit 33A generates corrected teacher data by correcting the initial teacher data based on the inference results of the base model supplied from the base model execution unit 32A. In this case, for example, if the inference result of the base model indicates a single mask image, the teacher data correction unit 33A uses the mask image as the corrected teacher data. If the base model outputs multiple mask images as the inference result, the teacher data correction unit 33A generates, for example, a mask image that integrates the inference results output by the base model as the corrected teacher data. Note that the teacher data correction unit 33A may identify a bounding box from the mask image and generate information indicating the bounding box as the corrected teacher data. Furthermore, the teacher data correction unit 33A may present the initial teacher data and the corrected teacher data to the user by, for example, displaying a display screen such as that shown in FIG. 5 on the output device 3, and allow the user to select whether or not to correct the initial teacher data based on input to the input device 4. The processing performed by the teacher data correction unit 33A is the same as the processing performed by the teacher data correction unit 33 in the first embodiment.

[0078] FIG. 10 is an example of a flowchart showing an outline of the processing executed by the teacher data correction device 1.

[0079] First, the teacher data correction device 1 performs machine learning to generate a main model, which is a machine learning model, based on the training dataset D1 (step S21). As a result, the teacher data correction device 1 acquires parameters of the machine-learned main model.

[0080] Next, the teacher data correction device 1 sequentially acquires the training input images used in the machine learning in step S21, and executes the main model using the training input images (step S22). By inputting the training input images into the main model to which the machine-learned parameters have been applied, the device 1 acquires the inference results output by the main model.

[0081] Next, the teacher data correction device 1 executes the base model using the inference result of the main model as a prompt (step S23). In this case, the training input image used in step S22 is input to the base model, and the inference result of the main model is input to the base model as a prompt, thereby obtaining the inference result output by the base model.

[0082] Then, the teacher data correction device 1 generates corrected teacher data based on the inference result of the base model and stores it in the storage device 2 (step S24). In this case, the teacher data correction device 1 may display the display screen shown in Figure 5 on the output device 3, and determine whether or not the generated corrected teacher data needs to be registered in the training dataset D1 based on an input signal output from the input device 4 operated by the user.

[0083] Then, the teacher data correction device 1 determines whether or not to end the processing of the flowchart (step S25). For example, if the teacher data correction device 1 determines that the processing of steps S22 to S24 has been completed for all records registered in the training dataset D1, it determines that the processing of the flowchart should be ended, and otherwise it determines that the processing of the flowchart should be continued.

[0084] Next, a description will be given of preferred modifications of the second embodiment. In the second embodiment, in addition to (Modification 1-1) to (Modification 1-3) described in the first embodiment, any combination of modifications described below may be applied.

[0085] (Variant 2-1) The teacher data correction device 1 may generate corrected teacher data for each record of the training data set D1, reflect the generated corrected teacher data in the training data set D1, and then learn the main model based on the training data set D1 after the correction has been reflected.

[0086] 9 , after the processing of the teacher data correction unit 33A for each record of the training dataset D1 is completed, the teacher data correction device 1 performs machine learning of the main model by the learning unit 30A based on the training dataset D1 reflecting the corrected teacher data. In this case, the learning unit 30A may additionally learn the main model that was initially machine-learned based on the teacher data based on the training dataset D1 reflecting the corrected teacher data, or may newly machine-learn the main model based on the training dataset D1 reflecting the corrected teacher data (i.e., machine learning after initializing the main model information D3).

[0087] The main model execution unit 31A then executes the main model using each training input image of the training dataset D1 as input, based on the main model information D3 obtained by machine learning based on the training dataset D1 reflecting the modified teacher data. The base model execution unit 32A and the teacher data correction unit 33A then generate second (i.e., second) modified teacher data based on the inference result of the main model, and update the training dataset D1. The teacher data correction device 1 may also regard the inference result of the main model as the second modified teacher data and update the training dataset D1.

[0088] (Variation 2-2) The teacher data correction device 1 may generate N pieces of corrected teacher data (N is an integer greater than or equal to 2) for each record by repeating the process of generating corrected teacher data based on (Variation 2-1). In this case, the teacher data correction device 1 may adopt the Nth piece of corrected teacher data obtained as the final teacher data, or may generate the final teacher data based on the first through Nth pieces of corrected teacher data obtained. In this case, the teacher data correction device 1 may adaptively determine the number of iterations of the process of generating corrected teacher data based on the generated corrected teacher data. In this case, the teacher data correction device 1 may repeat the generation of corrected teacher data until it is determined that a predetermined condition is met, for example, as in (Variation 1-4). The predetermined condition is set, for example, to a condition that determines that fluctuations in the corrected teacher data have converged.

[0089] 11 shows a functional block diagram of a processor 11 that adaptively determines the number of times corrected teacher data is generated. The processor 11 has a convergence determination unit 34A. Note that the same components as those in the block diagram shown in FIG. 9 are appropriately designated by the same reference numerals, and their description will be omitted.

[0090] The convergence determination unit 34A compares the average fluctuation rate of the corrected teacher data for the entire training dataset D1 with a predetermined threshold. Here, when corrected teacher data has been generated n times (n is an integer greater than or equal to 1), the convergence determination unit 34A calculates the fluctuation rate for each record of the training dataset D1 based on the corrected teacher data generated the nth time and the corrected teacher data generated the (n-1)th time (or the initial teacher data when n is 1). The convergence determination unit 34A then calculates the average of the fluctuation rates calculated for each record of the training dataset D1 as the average fluctuation rate.

[0091] If the average fluctuation rate is equal to or greater than the threshold, the convergence determination unit 34A determines that the process of generating modified teacher data should be continued. In this case, the learning unit 30A, the main model execution unit 31A, the base model execution unit 32A, and the teacher data correction unit 33A execute the process of generating modified teacher data based on the latest base model information D2. On the other hand, if the average fluctuation rate is less than the threshold, the convergence determination unit 34A determines that the generation of modified teacher data should be terminated. In this case, the teacher data correction device 1 may adopt the last generated modified teacher data as the final teacher data for each record of the training dataset D1, or may generate the final teacher data based on all generated modified teacher data (which may include the initial teacher data).

[0092] According to this modification, the teacher data correction device 1 can preferably generate teacher data with reduced variability.

[0093] (Variant 2-3) When generating the second or subsequent modified teacher data in (Variant 2-2), the teacher data correction device 1 may determine the prompt based on the inference results of the main model machine-learned based on the latest training dataset D1, as well as the inference results of the main model obtained in the past, the modified teacher data already generated, the initial teacher data, etc.

[0094] Fig. 12 shows an example of a functional block configuration of the processor 11 that takes into account the process of generating a prompt. The processor 11 shown in Fig. 12 includes a prompt determination unit 35A. Note that the same components as those in the block configurations shown in Fig. 9 or 11 are appropriately designated by the same reference numerals, and their description will be omitted.

[0095] In generating the nth iteration of corrected teacher data, the prompt determination unit 35A generates a prompt by integrating the inference result of the main model most recently generated by the main model execution unit 31A and the inference results of the main model generated by the main model execution unit 31A in the first through (n-1)th iterations of corrected teacher data. In this case, the teacher data correction unit 33A generates, for example, an overlapping region (so-called AND region) or a merged region (so-called OR region) of multiple inference results (mask images) as a prompt. The teacher data correction unit 33A may also generate a prompt by integrating the inference result of the main model most recently generated by the main model execution unit 31A with the initial teacher data, or may generate a prompt by integrating the inference result of the main model most recently generated by the main model execution unit 31A with the corrected teacher data generated in the first through (n-1)th iterations. The teacher data correction unit 33A may also generate a prompt by combining the above-mentioned integration techniques.

[0096] The prompt determination unit 35A then supplies the determined prompt to the base model execution unit 32A, and the base model execution unit 32A executes the base model using the prompt determined by the prompt determination unit 35A.

[0097] According to this modification, the teacher data correction device 1 can generate highly accurate corrected teacher data by using statistically processed prompts.

[0098] 13 is a block diagram of a teacher data correction device 1X. The teacher data correction device 1X includes an acquisition unit 31X and a teacher data correction unit 33X. The teacher data correction device 1X may be composed of multiple devices.

[0099] The acquisition unit 31X acquires training input data to be input to the machine learning model during training of the machine learning model and teacher data corresponding to the training input data. The acquisition unit 31X can be, for example, the acquisition unit 31 in the first embodiment or the learning unit 30A and the main model execution unit 31A in the second embodiment.

[0100] The teacher data correction means 33X corrects the teacher data based on the training input data, the teacher data, and the trained model. Here, the trained model is a model trained by machine learning to output an inference result regarding the input data when the input data and a prompt suggesting the input data are input. The teacher data correction means 33X then corrects the teacher data based on the inference result output by the trained model when input data based on the training input data and a prompt based on the teacher data are input to the trained model. The "input data based on the training input data" may be the training input data itself, or may be data generated using the training input data. The "prompt based on the teacher data" may be the teacher data itself (e.g., the prompt in the first embodiment) or may be data generated using the teacher data (e.g., the prompt in the second embodiment). The teacher data correction means 33X may be the base model execution unit 32 and the teacher data correction unit 33 in the first embodiment, or the base model execution unit 32A and the teacher data correction unit 33A in the second embodiment.

[0101] FIG. 14 is an example of a flowchart showing the processing steps of the teacher data correction device 1X. First, the acquisition means 31X acquires training input data to be input to a machine learning model during training of the machine learning model and teacher data corresponding to the training input data (step S31). Next, the teacher data correction means 33X corrects the teacher data based on the training input data, the teacher data, and the trained model (step S32). Here, the trained model is a model trained by machine learning to output an inference result regarding the input data when input data and a prompt suggesting the input data are input. Then, the teacher data correction means 33X corrects the teacher data based on the inference result output by the trained model when input data based on the training input data and a prompt based on the teacher data are input to the trained model.

[0102] According to the third embodiment, the teacher data correction device 1X can preferably correct the teacher data.

[0103] In each of the above-described embodiments, the program can be stored using various types of non-transitory computer-readable media and supplied to a computer processor, etc. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic storage media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical storage media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, and random access memories (RAMs). The program may also be supplied to a computer by various types of transient computer-readable media. Examples of transient computer-readable media include electric signals, optical signals, and electromagnetic waves. The transient computer-readable medium can supply the program to a computer via a wired communication path such as an electric wire or optical fiber, or via a wireless communication path.

[0104] In addition, part or all of the above-described embodiments (including variations, the same applies below) may also be described as, but are not limited to, the following supplementary notes. Furthermore, not only the devices, methods, and storage media described in the supplementary notes, but also various hardware, software, various recording means for recording software, or systems may be made to depend on part or all of the configurations described in the supplementary notes, as long as they do not deviate from the above-described embodiments.

[0105] [Supplementary Note 1] A teacher data correction device comprising: an acquisition means for acquiring training input data to be input to a machine learning model when training the machine learning model, and teacher data corresponding to the training input data; and teacher data correction means for correcting the teacher data based on the training input data, the teacher data, and a trained model, wherein the trained model is a model trained by machine learning to output an inference result related to the input data when the input data and a prompt suggestive of the input data are input, and the teacher data correction means corrects the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model. [Supplementary Note 2] The teacher data correction device according to Supplementary Note 1, further comprising learning means for performing machine learning on the machine learning model based on the training input data and the teacher data, and wherein the teacher data correction means determines the prompt based on the inference result output by the machine learning model after the machine learning has been performed, based on the training input data. [Supplementary Note 3] The teacher data correction device according to Supplementary Note 1, wherein the teacher data correction means corrects the corrected teacher data based on corrected teacher data obtained by correcting the teacher data, the training input data, and the trained model. [Supplementary Note 4] The teacher data correction device according to Supplementary Note 3, wherein the teacher data correction means repeatedly executes a process of correcting the corrected teacher data based on the corrected teacher data, the training input data, and the trained model until a predetermined condition is satisfied. [Supplementary Note 5] The teacher data correction device according to Supplementary Note 4, wherein the process of correcting the corrected teacher data further includes learning means that performs machine learning on the machine learning model based on the training input data and the corrected teacher data, and the teacher data correction means determines the prompt based on an inference result output by the machine learning model on which the machine learning has been performed, based on the training input data. [Supplementary Note 6] The teacher data correction device according to Supplementary Note 4, wherein the predetermined condition is a condition related to the degree of change in the corrected teacher data due to the process of correcting the corrected teacher data.[Supplementary Note 7] The teacher data correction device according to Supplementary Note 3, wherein the teacher data correction means, when correcting the corrected teacher data, determines the prompt based on at least one of the teacher data and the corrected teacher data. [Supplementary Note 8] The teacher data correction device according to Supplementary Note 1, wherein the teacher data correction means displays the teacher data and the corrected teacher data obtained by correcting the teacher data, and accepts external input based on decision-making regarding the appropriateness of the corrected teacher data. [Supplementary Note 9] The teacher data correction device according to Supplementary Note 1, wherein the training input data is a medical image, and the teacher data is information regarding a lesion site present in the medical image. [Supplementary Note 10] A teacher data correction method, comprising: a computer acquiring training input data to be input to a machine learning model in training the machine learning model and teacher data corresponding to the training input data; correcting the teacher data based on the training input data, the teacher data, and a trained model; the trained model being a model trained by machine learning to output an inference result regarding the input data when the input data and a prompt suggestive of the input data are input; and correcting the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model. [Supplementary Note 11] A storage medium storing a program that acquires training input data to be input to a machine learning model in training the model, and teacher data corresponding to the training input data, causes a computer to execute a process of correcting the teacher data based on the training input data, the teacher data, and a trained model, wherein the trained model is a model that has been machine-learned to output an inference result regarding input data when input data and a prompt that is suggestive regarding the input data are input, and causes the computer to execute a process of correcting the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.[Supplementary Note 12] The teacher data correction method of Supplementary Note 10, comprising: performing machine learning of the machine learning model based on the training input data and the teacher data; and determining the prompt based on an inference result output by the machine learning model on which the machine learning has been performed based on the training input data. [Supplementary Note 13] The teacher data correction method of Supplementary Note 10, comprising correcting the corrected teacher data based on corrected teacher data obtained by correcting the teacher data, the training input data, and the trained model. [Supplementary Note 14] The teacher data correction method of Supplementary Note 13, comprising repeatedly performing a process of correcting the corrected teacher data based on the corrected teacher data, the training input data, and the trained model until a predetermined condition is satisfied. [Supplementary Note 15] The teacher data correction method of Supplementary Note 14, wherein in the process of correcting the corrected teacher data, machine learning of the machine learning model is performed based on the training input data and the corrected teacher data; and determining the prompt based on an inference result output by the machine learning model on which the machine learning has been performed based on the training input data. [Supplementary Note 16] The teacher data correction method according to Supplementary Note 14, wherein the predetermined condition is a condition related to the degree of change in the corrected teacher data due to the process of correcting the corrected teacher data. [Supplementary Note 17] The teacher data correction method according to Supplementary Note 13, wherein when correcting the corrected teacher data, the prompt is determined based on at least one of the teacher data and the corrected teacher data. [Supplementary Note 18] The teacher data correction method according to Supplementary Note 10, wherein the teacher data and the corrected teacher data obtained by correcting the teacher data are displayed, and an external input based on a decision-making regarding the appropriateness of the corrected teacher data is accepted. [Supplementary Note 19] The teacher data correction method according to Supplementary Note 10, wherein the training input data is a medical image, and the teacher data is information related to a lesion site present in the medical image.[Supplementary Note 20] The storage medium of Supplementary Note 11, storing the program that causes the computer to perform machine learning of the machine learning model based on the training input data and the teacher data, and to determine the prompt based on an inference result output by the machine learning model on which the machine learning has been performed based on the training input data. [Supplementary Note 21] The storage medium of Supplementary Note 11, storing the program that causes the computer to perform machine learning of the machine learning model based on the training input data and the teacher data, and to determine the prompt based on an inference result output by the machine learning model on which the machine learning has been performed based on the training input data. [Supplementary Note 22] The storage medium of Supplementary Note 11, storing the program that causes the computer to perform a process of correcting the corrected teacher data based on corrected teacher data obtained by correcting the teacher data, the training input data, and the trained model. [Supplementary Note 23] The storage medium of Supplementary Note 22, storing the program causing the computer to execute a process of repeatedly correcting the corrected teacher data based on the corrected teacher data, the training input data, and the trained model until a predetermined condition is satisfied. [Supplementary Note 24] The storage medium of Supplementary Note 23, storing the program causing the computer to execute a process of correcting the corrected teacher data, comprising: performing machine learning of the machine learning model based on the training input data and the corrected teacher data; and determining the prompt based on an inference result output by the machine learning model based on the training input data. [Supplementary Note 25] The storage medium of Supplementary Note 23, wherein the predetermined condition is a condition related to the degree of change in the corrected teacher data due to the process of correcting the corrected teacher data. [Supplementary Note 26] The storage medium of Supplementary Note 22, storing the program causing the computer to execute a process of determining the prompt based on at least one of the teacher data and the corrected teacher data when correcting the corrected teacher data.[Supplementary Note 27] The storage medium according to Supplementary Note 11, which stores the program that causes the computer to execute a process of displaying the teacher data and corrected teacher data obtained by correcting the teacher data, and accepting external input based on a decision-making process regarding the appropriateness of the corrected teacher data. [Supplementary Note 28] The storage medium according to Supplementary Note 11, wherein the training input data is medical images, and the teacher data is information regarding a lesion site present in the medical images.

[0106] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various modifications within the scope of the present invention that would be understood by those skilled in the art can be made to the configuration and details of the present invention. In other words, the present invention naturally includes various modifications and alterations that would be possible for those skilled in the art based on the entire disclosure, including the claims, and the technical ideas. Furthermore, the disclosures of the above-cited patent and non-patent documents are incorporated herein by reference.

[0107] 1, 1X Teacher data correction device 2 Storage device 3 Output device 4 Input device 11 Processor 12 Memory 13 Interface 100, 100A Teacher data correction system

Claims

1. A teacher data correction device comprising: an acquisition means for acquiring training input data to be input to a machine learning model when training the model, and teacher data corresponding to the training input data; and a teacher data correction means for correcting the teacher data based on the training input data, the teacher data, and a trained model, wherein the trained model is a model trained by machine learning so as to output an inference result regarding the input data when input data and a prompt that is suggestive of the input data are input, and the teacher data correction means corrects the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.

2. The teacher data correction device of claim 1, further comprising a learning means for performing machine learning of the machine learning model based on the training input data and the teacher data, wherein the teacher data correction means determines the prompt based on an inference result output by the machine learning model on which the machine learning has been performed based on the training input data.

3. The teacher data correction device described in claim 1, wherein the teacher data correction means corrects the modified teacher data based on modified teacher data obtained by correcting the teacher data, the training input data, and the learned model.

4. The teacher data correction device described in claim 3, wherein the teacher data correction means repeatedly executes the process of correcting the modified teacher data based on the modified teacher data, the training input data, and the learned model until a predetermined condition is satisfied.

5. The teacher data correction device of claim 4, further comprising a learning means for performing machine learning of the machine learning model based on the training input data and the modified teacher data in the process of correcting the modified teacher data, wherein the teacher data correction means determines the prompt based on an inference result output by the machine learning model after the machine learning is performed based on the training input data.

6. The teacher data correction device according to claim 4, wherein the predetermined condition is a condition related to the degree of change in the corrected teacher data due to the process of correcting the corrected teacher data.

7. The teacher data correction device according to claim 3, wherein said teacher data correction means, when correcting said corrected teacher data, determines said prompt based on at least one of said teacher data and said corrected teacher data.

8. The teacher data correction device described in claim 1, wherein the teacher data correction means displays the teacher data and corrected teacher data obtained by correcting the teacher data, and accepts external input based on a decision-making process regarding the appropriateness of the corrected teacher data.

9. The teacher data correction device according to claim 1, wherein the training input data is a medical image, and the teacher data is information relating to a lesion site present in the medical image.

10. A method for correcting teacher data, comprising: a computer acquiring training input data to be input to a machine learning model in training the model, and teacher data corresponding to the training input data; correcting the teacher data based on the training input data, the teacher data, and a trained model; the trained model being a model trained by machine learning to output an inference result regarding the input data when input data and a prompt suggestive of the input data are input; and correcting the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.

11. A storage medium storing a program which acquires training input data to be input to a machine learning model in training the model, and teacher data corresponding to the training input data, causes a computer to execute a process of correcting the teacher data based on the training input data, the teacher data, and a trained model, the trained model being a model trained by machine learning to output an inference result regarding the input data when input data and a prompt suggestive of the input data are input, and causes the computer to execute a process of correcting the teacher data based on the inference result output by the trained model when the input data based on the training input data and the prompt based on the teacher data are input to the trained model.

12. The teacher data correction method described in claim 10, further comprising: performing machine learning of the machine learning model based on the training input data and the teacher data; and determining the prompt based on an inference result output by the machine learning model based on the training input data.

13. A teacher data correction method as described in claim 10, wherein the corrected teacher data is corrected based on corrected teacher data obtained by correcting the teacher data, the training input data, and the learned model.

14. A teacher data correction method as described in claim 13, wherein the process of correcting the modified teacher data based on the modified teacher data, the training input data, and the learned model is repeatedly executed until a predetermined condition is satisfied.

15. The teacher data correction method described in claim 14, wherein in the process of correcting the corrected teacher data, machine learning is performed on the machine learning model based on the training input data and the corrected teacher data, and the prompt is determined based on an inference result output by the machine learning model on which the machine learning has been performed based on the training input data.

16. The teacher data correction method according to claim 14, wherein the predetermined condition is a condition related to the degree of change in the corrected teacher data due to the process of correcting the corrected teacher data.

17. The teacher data correction method according to claim 13, wherein when the corrected teacher data is corrected, the prompt is determined based on at least one of the teacher data and the corrected teacher data.

18. A method for correcting teacher data as described in claim 10, further comprising displaying the teacher data and corrected teacher data obtained by correcting the teacher data, and accepting external input based on a decision-making process regarding the suitability of the corrected teacher data.

19. A method for correcting training data as described in claim 10, wherein the training input data is a medical image, and the training data is information relating to a lesion site present in the medical image.

20. The storage medium of claim 11, further comprising a program for causing the computer to execute a process of performing machine learning of the machine learning model based on the training input data and the teacher data, and determining the prompt based on an inference result output by the machine learning model after the machine learning has been performed based on the training input data.

Images