Method for generating learning data set

Abstract

To provide a method for generating a learning data set capable of efficiently preparing a learning data set including a large amount of learning data and correct answer labels.SOLUTION: A method for generating a learning data set according to the present disclosure includes an acquisition step of acquiring first medical image data and a first correct answer label that is a correct answer label of the first medical image data, a conversion step of converting the first medical image data into second medical image data by performing style conversion of the first medical image data in a style to which the first correct answer label is applicable, and an output step of generating a pair of the second medical image data and the first correct answer label as a learning data set.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a method for generating a training dataset. [Background technology]

[0002] In recent years, there have been attempts in the medical field to use machine learning models to assist in image diagnosis, etc. Machine learning uses large amounts of data to teach machines patterns and correlations in the data, allowing them to perform tasks such as identification, recognition, detection, and prediction.

[0003] Patent Document 1 discloses an image processing device that generates a second radiographic image with reduced noise compared to the first radiographic image by inputting the first radiographic image acquired by an acquisition unit into a trained model obtained by training using training data including a radiographic image obtained by adding noise with attenuated high-frequency components. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2022-39989 Summary of the Invention [Problem to be solved by the invention]

[0005] In order to create a highly accurate machine learning model (classifier) ​​through machine learning, the amount of training data used for training and the quality of the correct labels corresponding to the training data are important.

[0006] Depending on the application of a machine learning model, it may be difficult to prepare a large amount of training data. Even if a large amount of training data can be prepared, it takes a great deal of time and effort to assign high-quality correct labels to each piece of the training data. For this reason, there is a demand for an efficient way to prepare a training dataset consisting of pairs of training data and correct labels.

[0007] The present disclosure aims to provide a method for generating a training dataset that can efficiently prepare a training dataset that includes a large amount of training data and correct answer labels. [Means for solving the problem]

[0008] A method for generating a training dataset according to one embodiment of the present disclosure includes a computer executing an acquisition step of acquiring first medical image data and a first correct label that is a correct label for the first medical image data, a conversion step of converting the first medical image data into second medical image data by style conversion in a style to which the first correct label can be applied, and an output step of generating a pair of the second medical image data and the first correct label as a training dataset. [Effects of the Invention]

[0009] The present disclosure makes it possible to efficiently prepare a training dataset containing a large amount of training data and correct labels. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing a configuration of an image diagnostic system according to an embodiment of the present disclosure. [Figure 2] A block diagram showing an example of the configuration of a learning dataset generation device. [Figure 3] Flowchart for explaining an example of operation of the learning dataset generation device [Figure 4] FIG. 1 is a diagram illustrating an example of the hardware configuration of a learning dataset generation device. [Figure 5] Block diagram illustrating the configuration of a machine learning device [Figure 6] Block diagram illustrating the configuration of an imaging diagnostic apparatus DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0012] [overview] 1 is a diagram illustrating an example of a configuration of an image diagnostic system 100 according to an embodiment of the present disclosure. As illustrated in FIG. 1, the image diagnostic system 100 includes a training dataset generation device 10, a machine learning device 20, and an image diagnostic device 30.

[0013] The training dataset generation device 10 acquires second medical image data generated by a predetermined image conversion process on first medical image data, and generates a pair of the second medical image data and a first correct answer label that is the correct answer label of the first medical image data as a training dataset. The training dataset generated by the training dataset generation device 10 is input to a machine learning device 20.

[0014] The first medical image data is medical image data input to the training dataset generation device 10. A first correct answer label is associated with the first medical image data in advance.

[0015] The predetermined image conversion process in the training dataset generation device 10 is executed by, for example, a predetermined image conversion model that has been subjected to machine learning. The image conversion model that executes the predetermined image conversion process may be included in the training dataset generation device 10, or may be included in an image conversion device (not shown) different from the training dataset generation device 10.

[0016] A correct answer label is information that indicates a correct answer and is assigned to training data. The content of the correct answer label varies depending on the content of the processing that a machine learning model trained by machine learning using a training dataset composed of pairs of training data and correct answer labels performs on input data. In this embodiment, the correct answer label is at least one of information indicating the position of a region of interest included in medical image data that is training data (e.g., coordinates, region, boundary, etc.), information indicating the structure of the region of interest (e.g., bone, blood vessel, muscle, nerve, affected area, etc.), and information indicating whether the region of interest is normal or abnormal (there is or is not a disease).

[0017] The machine learning device 20 outputs a trained image diagnostic model obtained by performing machine learning using the training dataset input from the training dataset generation device 10. The trained image diagnostic model output by the machine learning device 20 is input to the image diagnostic device 30.

[0018] The image diagnostic device 30 inputs the third medical image data, which is new medical image data, into the trained image diagnostic model and outputs the inference result as the image diagnostic result. For example, a doctor or the like can perform an accurate image diagnosis by referring to the image diagnostic result output by the image diagnostic device 30.

[0019] In this embodiment, the diagnostic imaging device 30 is, for example, an ultrasound imaging device (diagnostic imaging device) that has an ultrasound probe (probe) and generates ultrasound image data based on reflected ultrasound waves from ultrasound waves transmitted to a subject. When the diagnostic imaging device 30 is an ultrasound imaging device, the first to third medical image data are ultrasound image data.

[0020] Generally, in the medical field, completely different images are obtained from different modalities. For example, MRI images taken with a nuclear magnetic resonance (NMR) imaging device and ultrasound images taken with an ultrasound device are completely different. For this reason, when performing machine learning on an image diagnosis model that performs image diagnosis based on ultrasound image data, for example, medical image data other than ultrasound image data (e.g., MRI image data) generated by a modality other than an ultrasound imaging device cannot be used as training data.

[0021] Furthermore, even in ultrasound imaging devices, if the type of ultrasound probe (probe) that transmits and receives ultrasound waves or the imaging mode is changed, the angle of view, resolution, depth-wise signal-to-noise ratio, etc. of the obtained ultrasound image will be completely different. For this reason, when machine learning is applied to an image diagnosis model that performs image diagnosis based on ultrasound image data obtained using one type of ultrasound probe, ultrasound image data obtained using other types of ultrasound probes or imaging modes cannot be used as training data.

[0022] Therefore, for example, in order to prepare a training dataset for an image diagnosis model that performs image diagnosis based on ultrasound image data in a specific imaging mode using a specific ultrasound probe, it is necessary to capture a large number of ultrasound images in a specific imaging mode using the specific ultrasound probe and then assign a correct answer label to each image. However, capturing a large number of ultrasound images in a specific imaging mode and assigning high-quality correct answer labels to each image requires a lot of time and effort.

[0023] Furthermore, depending on the type of medical image data used in the image diagnostic model, invasiveness such as medical exposure may become an issue, and in the case of such medical image data, it becomes increasingly difficult to prepare large amounts of specific types of medical image data as learning data.

[0024] In the image diagnostic system 100 according to an embodiment of the present disclosure, a training dataset generation device 10 generates a large amount of training datasets with relatively little effort, by performing image conversion processing on various types of medical image data, for efficiently training an image diagnostic model that performs image diagnosis using certain types of medical image data. Then, a machine learning device 20 uses the training dataset generated by the training dataset generation device 10 to efficiently train an image diagnostic model that can perform image diagnosis with high accuracy. Furthermore, the image diagnostic device 30 can output image diagnostic results based on new medical image data using the trained image diagnostic model, which is extremely useful for medical professionals and the like.

[0025] [Details of each configuration] Each component of the diagnostic imaging system 100 will be described in detail below.

[0026] <Learning Dataset Generation Device 10> Fig. 2 is a block diagram showing an example of the configuration of the training dataset generation device 10. In the example shown in Fig. 2, the training dataset generation device 10 includes a dataset acquisition unit 11, an image conversion unit 12, an image data acquisition unit 13, and a generation unit 14.

[0027] The dataset acquisition unit 11 acquires a dataset that is a pair of first medical image data and first correct labels from an external source. The external source may be, for example, various modalities that generate various medical image data, or a database that includes various medical image data and correct labels.

[0028] The image conversion unit 12 stores a trained image conversion model and performs a predetermined image conversion process on input first medical image data. The image conversion model is, for example, an image conversion model using a convolutional neural network or an attention mechanism. In this embodiment, medical image data input to the image conversion unit 12 is referred to as first medical image data, and medical image data converted by the image conversion unit 12 is referred to as second medical image data. The first medical image data may include not just one type of image data, but various image data. The various image data include, for example, medical image data captured with various modalities, ultrasound image data captured with various ultrasound probes (e.g., sector, linear, convex, etc.) in an ultrasound imaging device, ultrasound image data captured in various imaging modes, etc.

[0029] 2 illustrates a case where the training dataset generation device 10 includes an image conversion unit 12 that includes an image conversion model, but as described above, the present disclosure is not limited to this. For example, the training dataset generation device 10 may acquire second medical image data that is output by an external image conversion device based on input of first medical image data.

[0030] The image conversion process performed by the image conversion unit 12 is, for example, called style conversion, which converts the style of an image. Style conversion is a technique for extracting style information from an image and synthesizing it by transferring only the style and texture of another image while leaving most of the original form of the image intact. An existing model such as a cycleGAN or a diffusion model may be used as the image conversion model for performing style conversion.

[0031] The image conversion unit 12 performs style conversion on various types of input first medical image data and outputs second medical image data that looks like a specific type of ultrasound image data. In this case, the image conversion model held by the image conversion unit 12 is, for example, a machine learning model that has been subjected to machine learning to convert various types of first medical image data into ultrasound image data obtained using a linear probe. The image conversion unit 12 can output various types of second medical image data by exchanging the machine learning model. For example, by exchanging the image conversion model held by the image conversion unit 12, the image conversion unit 12 may output second medical image data that looks like an MRI image for various types of input first medical image data. Furthermore, the image conversion unit 12 may have multiple types of image conversion models in advance and may be configured to switch the image conversion model to be used as needed (for example, by user specification).

[0032] A specific example of the image conversion process that the image conversion model of the image conversion unit 12 applies to the first image data will be described.

[0033] As a first example, the image converter 12 converts first medical image data generated using various types of ultrasound probes in an ultrasound imaging device into second medical image data generated using a specific ultrasound probe. Medical image data generated using different types of ultrasound probes have different center frequencies and different image qualities, i.e., different spatial resolutions and signal-to-noise ratios. The image converter 12 converts first medical image data generated using one of three types of ultrasound probes, for example, sector, linear, or convex, into second medical image data that looks like medical image data generated using a specific ultrasound probe, such as a linear ultrasound probe. For example, according to the first example, when preparing ultrasound image data obtained using a less common type of ultrasound probe, the ultrasound image data obtained using the less common type of ultrasound probe can be easily prepared by converting the ultrasound image data obtained using a relatively common type of ultrasound probe.

[0034] As a second example, the image conversion unit 12 converts first medical image data generated using various types of modalities into second medical image data generated using a specific modality. Examples of various types of modalities include an ultrasound imaging device, a nuclear magnetic resonance imaging device, a CT (Computed Tomography) device, and an X-ray imaging device. As described above, for example, ultrasound image data generated using an ultrasound imaging device and MRI image data generated using a nuclear magnetic resonance imaging device are completely different images. The image conversion unit 12 converts, for example, MRI image data generated using a nuclear magnetic resonance imaging device or CT image data generated using a CT device into second image data that looks like ultrasound image data.

[0035] MRI images generally have a better signal-to-noise ratio for image depth than ultrasound images. Therefore, using MRI image data as the first medical image data may result in a more accurate first correct label than when the first medical image data is ultrasound image data. The image conversion unit 12 converts the first medical image data, which is MRI image data, into second medical image data that resembles ultrasound image data. The generation unit 14, described below, pairs the second medical image data with the first correct label to generate a training dataset, thereby improving the quality of the correct label in the generated training dataset. This improves the generalization performance of the image diagnostic model and improves diagnostic accuracy. Furthermore, converting the first medical image data, which is ultrasound image data, into second medical image data that resembles an X-ray image allows for the acquisition of a large training dataset without having to consider factors such as medical radiation exposure to the subject.

[0036] As a third example, the image conversion unit 12 converts first medical image data generated using the same type of modality (e.g., ultrasound imaging devices) manufactured by various manufacturers into second medical image data generated using a modality manufactured by a specific manufacturer. Modalities manufactured by different manufacturers may generate medical image data with completely different image quality, even if they are the same type of modality or image the same subject. The image conversion unit 12 converts first medical image data generated using modalities manufactured by, for example, companies A, B, C, and D, into second medical image data that appears to have been generated using a modality manufactured by a specific company, for example, company C. This improves the generalization performance of the image diagnostic model and improves diagnostic accuracy. In other words, by including the second medical image data in the training dataset, the image diagnostic model can be trained to accommodate a wider range of image quality and a wider range of anatomical patterns of subjects.

[0037] As a fourth example, the image converter 12 converts first medical image data generated under various image processing conditions, i.e., first medical image data with different gains, spatial frequencies, etc., into second medical image data that appears to have been generated under specific image processing conditions. This improves the generalization performance of the image diagnostic model and improves diagnostic accuracy. In other words, by including the second medical image data in the training dataset, the image diagnostic model can be trained to accommodate more image processing conditions (such as gains and spatial frequencies) and more anatomical patterns of subjects.

[0038] As a fifth example, the image converter 12 converts first medical image data generated by various signal processing, i.e., first medical image data generated at different focuses in an ultrasound imaging device, into second medical image data that appears to have been generated at a specific focus. This improves the generalization performance of the image diagnostic model and improves diagnostic accuracy. In other words, by including the second medical image data in the training dataset, the image diagnostic model can be trained to better accommodate differences in focus.

[0039] As a sixth example, the image converter 12 converts multiple first medical image data sets, each containing subjects with different BMIs (Body Mass Indexes), into second medical image data sets that appear to depict subjects with a specific BMI. This allows for the conversion of first medical image data sets containing a mixture of thin subjects, normal-sized subjects, and obese subjects into second medical image data sets that appear to depict an obese subject. This improves the generalization performance of the diagnostic imaging model and improves diagnostic accuracy. In other words, by including the second medical image data sets in the training dataset, the diagnostic imaging model can be trained to better accommodate differences in image quality due to the subject's body shape.

[0040] As a seventh example, the image conversion unit 12 converts multiple first medical image data generated at different transmission voltages into second medical image data that appear to have been generated at a specific transmission voltage. For example, in an ultrasound imaging device, medical image data generated by a device with a relatively low transmission voltage (e.g., a portable device) may have lower image quality than medical image data generated by a device with a relatively high transmission voltage (e.g., a stationary device). This improves the generalization performance of the image diagnostic model and improves diagnostic accuracy. In other words, by including the second medical image data in the training dataset, the image diagnostic model can be trained to better accommodate differences in image quality due to transmission voltage.

[0041] As an eighth example, the image converter 12 converts the first medical image data into second medical image data by deleting specific objects from among various objects captured within the imaging range of the first medical image data. Examples of various objects captured within the imaging range of the first medical image data include artifacts, artificial structures such as a puncture needle, an anesthetic solution, a contrast agent, or a lesion.

[0042] For example, by performing a conversion to remove artificial structures and the like that exist outside the region of interest indicated by the first correct label associated with the first medical image data, second medical image data that does not include the artificial structures can be generated. As a result, when the generation unit 14 (described later) generates a training dataset, it is not necessary to assign a new correct label to the second medical image data from which the artificial structures have been removed from the first medical image data. This eliminates the need to spend time assigning correct labels. Furthermore, even if various objects outside the region of interest are captured in the first medical image data, correct labels are assigned without being affected by the captured objects, resulting in a high-quality training dataset. By training using the training dataset generated in this way, an image diagnostic model that can accurately respond can be obtained even if various objects (e.g., artifacts, artificial structures such as a puncture needle, anesthetic fluid, contrast agent, or lesions) that are captured within the imaging range of the first medical image data are not captured.

[0043] In the above example, a case where artificial structures and the like existing outside the region of interest are deleted is described, but for example, the image conversion unit 12 may delete specific structures that appear inside the region of interest.

[0044] As a ninth example, the image conversion unit 12 converts the first medical image data into second medical image data by adding a specific object not captured within the imaging range. For example, by performing image conversion to add a new image of a lesion within a region of interest in the first medical image data, new second medical image data capturing the specific lesion can be generated. Furthermore, a training dataset can be prepared for training to recognize the same region of interest regardless of whether or not a lesion is present within the region of interest. This is more preferable because it improves the quality of the training dataset generated by the generation unit 14. Note that the specific object not captured within the imaging range is not limited to a lesion, but may also be an artificial structure such as a puncture needle, an artifact (e.g., an artifact caused by the presence of a puncture needle), an anesthetic fluid, or a contrast agent. By including the second medical image data obtained by converting the first medical image data to add these objects in the training dataset, the image diagnostic model can be trained to accurately respond even when artifacts, anesthetic fluid, a contrast agent, or a lesion is captured in the image.

[0045] In this embodiment, the image conversion unit 12 performs style conversion without changing the contour positions of the input first medical image data. As a result, for various types of input first medical image data, the image conversion unit 12 can obtain second medical image data that is converted into one type of medical image data that differs only in the style or texture of part of the image or the entire image, without changing the positions of boundaries of tissues such as bones, muscles, and blood vessels.

[0046] The image data acquisition unit 13 acquires the second medical image data output by the image conversion unit 12 and inputs it to the generation unit 14. In an example where the learning dataset generation device 10 does not have the image conversion unit 12, the image data acquisition unit 13 acquires the second medical image data from an external image conversion device.

[0047] The generation unit 14 generates a training dataset for training an image diagnostic model by pairing the second medical image data with the first correct label. That is, the training dataset generated by the generation unit 14 includes the second medical image data obtained by style conversion of the first medical image data, and the first correct label associated with the first medical image data. The generation unit 14 outputs the generated training dataset to the machine learning device 20.

[0048] As described above, the contour position of the image is not changed in the style conversion in the image conversion unit 12. Therefore, the first correct answer label indicating information related to the region of interest in the first medical image data can be applied as is to the second medical image data obtained by performing style conversion on the first medical image data.

[0049] In this way, when a pair of first medical image data and a first correct label is input, the training dataset generation device 10 performs image conversion processing, including style conversion, on the first medical image data to generate second medical image data of a specific type, and then outputs the pair of the second medical image data and the first correct label as a training dataset. Here, the type of input first medical image data does not matter, making it easy to prepare a large amount of first medical image data. As a result, even if the second medical image data after the image conversion processing is a specific type of medical image data for which it is difficult to collect a large training dataset, a large amount of second medical image data can be obtained by preparing a large amount of diverse first medical image data regardless of type. This makes it possible to generate a large amount of second medical image data of a specific type.

[0050] Furthermore, since the first correct answer labels corresponding to the first medical image data before conversion can be applied as they are to the second medical image data converted in the image conversion process, there is no need to prepare new correct answer labels even when a large amount of second medical image data is generated. This makes it possible to obtain a large amount of training data set without requiring much effort.

[0051] Furthermore, the training dataset generation device 10 does not include the first medical image data among the input datasets (pairs of first medical image data and first correct answer labels) in the training dataset to be output. This makes it possible to generate a training dataset that includes second medical image data that is completely different in type from the first medical image data (for example, different modality, different type of ultrasound probe, etc.). This means that the training dataset output by the training dataset generation device 10 includes only a large number of medical image data of a specific type, thereby improving the efficiency and quality of learning when learning using such a training dataset.

[0052] FIG. 3 is a flowchart illustrating an example of the operation of the training dataset generation device 10.

[0053] In step S1, the training dataset generation device 10 externally acquires a dataset that is a pair of first medical image data and a first correct answer label.

[0054] In step S2, the training dataset generation device 10 performs a predetermined image conversion process on the first medical image data to generate second medical image data.

[0055] In step S3, the training dataset generation device 10 acquires the second medical image data generated in step S2.

[0056] In step S4, the training dataset generation device 10 generates a pair of the second medical image data and the first correct label as a training dataset.

[0057] In the flowchart shown in FIG. 3, the operation of the training dataset generation device 10 stops after the process of step S4, but the operation of FIG. 3 may be repeated, for example, at predetermined intervals.

[0058] The training dataset generation device 10 can be realized by a computer such as a server device, a personal computer (PC), a smartphone, a tablet terminal, etc. Furthermore, the various functions of the training dataset generation device 10 are realized by the computer executing a program.

[0059] 4 is a diagram illustrating an example of the hardware configuration of the training dataset generation device 10. The training dataset generation device 10 includes a drive device 101, a storage device 102, a memory device 103, a processor 104, a user interface (UI) device 105, and a communication device 106, which are interconnected via a bus B.

[0060] A program or instructions for implementing various functions and processes described below in the training dataset generation device 10 may be stored in a removable storage medium such as a CD-ROM (Compact Disk-Read Only Memory) or a flash memory. When the storage medium is set in the drive device 101, the program or instructions are installed from the storage medium into the storage device 102 or the memory device 103 via the drive device 101. However, the program or instructions do not necessarily have to be installed from the storage medium, and may be downloaded from an external device via a network or the like.

[0061] The storage device 102 is realized by a hard disk drive or the like, and stores installed programs or instructions as well as files, data, etc. used to execute the programs or instructions.

[0062] The memory device 103 is realized by a random access memory, a static memory, or the like, and when a program or instruction is activated, reads and stores the program, instruction, data, or the like from the storage device 102. The storage device 102, the memory device 103, and the removable storage medium may be collectively referred to as a non-transitory tangible storage medium.

[0063] The processor 104 may be realized by one or more CPUs (Central Processing Units), GPUs (Graphics Processing Units), processing circuitry, etc., which may be composed of one or more processor cores, and performs various functions and processes of the training dataset generation device 10 described below in accordance with programs, instructions, data such as parameters required to execute the programs or instructions, etc. stored in the memory device 103.

[0064] The user interface (UI) device 105 may be composed of input devices such as a keyboard, mouse, camera, and microphone, output devices such as a display, speaker, headset, and printer, and input / output devices such as a touch panel, and realizes an interface between a user and the training dataset generation device 10. For example, a user operates the training dataset generation device 10 by operating a keyboard, mouse, and the like to use a GUI (Graphical User Interface) displayed on a display or touch panel.

[0065] The communication device 106 is realized by various communication circuits that execute wired and / or wireless communication processing with external devices, the Internet, a LAN (Local Area Network), a cellular network, or other communication networks.

[0066] However, the above-described hardware configuration is merely an example, and the training dataset generation device 10 according to the present disclosure may be realized by any other appropriate hardware configuration.

[0067] <Machine Learning Device 20> Next, we will explain the machine learning device 20. Fig. 5 is a block diagram illustrating an example of the configuration of the machine learning device 20. As shown in Fig. 5, the machine learning device 20 includes a learning dataset acquisition unit 21, a learning target model storage unit 22, and a learning unit 23.

[0068] The training dataset acquisition unit 21 acquires the training dataset output from the training dataset generation device 10.

[0069] The learning target model storage unit 22 stores a machine learning model (image diagnosis model) that is the target of learning using a learning data set.

[0070] The learning unit 23 uses the learning dataset to train the image diagnosis model stored in the learning model storage unit 22. As a result, the image diagnosis model is trained to output an image diagnosis result as an inference result in response to input of, for example, medical image data.

[0071] The learning unit 23 outputs the learned image diagnosis model to the image diagnosis device 30.

[0072] [Imaging Diagnostic Device 30] Next, the image diagnostic device 30 will be described. The image diagnostic device 30 is an example of an image processing device of the present disclosure. FIG. 6 is a block diagram illustrating the configuration of the image diagnostic device 30. As shown in FIG. 6, the image diagnostic device 30 includes an image diagnostic model acquisition unit 31, an image generation unit 32, and an inference unit 33.

[0073] The image diagnosis model acquisition unit 31 acquires the trained image diagnosis model output from the machine learning device 20.

[0074] The image generation unit 32 generates new third medical image data. The third medical image data generated by the image generation unit 32 is image data of the same type as the second medical image data included in the training dataset used to train the image diagnostic model. In other words, the training dataset generation device 10 determines the type of second medical image data to convert the first medical image data into, in accordance with the type of third medical image data generated by the image generation unit 32 of the image diagnostic device 30.

[0075] When the image generation unit 32 generates ultrasound image data as, for example, the third medical image data, the second medical image data included in the training dataset generated by the training dataset generation device 10 is also ultrasound image data. In this case, the image diagnosis model on which the machine learning device 20 performs machine learning is a machine learning model that outputs an image diagnosis result as an inference result for input of ultrasound image data.

[0076] The inference unit 33 performs image diagnosis on the third medical image data generated by the image generation unit 32 using the trained image diagnosis model acquired by the image diagnosis model acquisition unit 31. Specifically, the inference unit 33 inputs the third medical image data to the image diagnosis model, acquires and outputs an image diagnosis result as an inference result. The image diagnosis result includes, for example, the position of the region of interest in the ultrasound image data, the position and type of the lesion, etc.

[0077] In this embodiment, the imaging diagnostic device 30 has been described as a modality capable of generating third medical image data, but the present disclosure is not limited thereto. For example, the imaging diagnostic device 30 may not generate the third medical image data. In this case, the imaging diagnostic device 30 acquires the third medical image data from an external source and outputs an imaging diagnosis result using an imaging diagnostic model. Even in this case, the imaging diagnostic device 30 is preferable because it can assist medical professionals in performing imaging diagnosis.

[0078] <Actions and Effects> As described above, according to the image diagnostic system 100 according to the embodiment of the present disclosure, the training dataset generation device 10 generates second medical image data that appears to be the same type of medical image data as the third medical image data generated by the image diagnostic device 30 from first medical image data, which is a different type of medical image data, and can use the second medical image data for training by the machine learning device 20. Therefore, even if the third medical image data is a type of medical image data for which it is difficult to prepare a large amount of training data, a large amount of second medical image data can be obtained with little effort by converting various other types of first medical image data into the second medical image data.

[0079] Furthermore, the training dataset generation device 10 generates a training dataset by applying a first correct answer label corresponding to the first medical image data, which is the source of the conversion, to the second medical image data obtained by the image conversion process. That is, a new training dataset can be generated using existing correct answer labels as they are, thereby enabling efficient acquisition of a training dataset. Since a large number of training datasets can be prepared with relatively little effort in this manner, an image diagnosis model with high diagnostic performance can be generated by performing machine learning using the training dataset in the machine learning device 20. This also improves the accuracy of image diagnosis by the image diagnosis device 30, which performs image diagnosis using a trained image diagnosis model.

[0080] In particular, when the third medical image data is medical image data generated using a specific ultrasound probe, for example, medical image data generated using an ultrasound probe that is not a major technique is very rare, and the database size is small. It is very difficult to efficiently train an image diagnostic model that performs image diagnosis based on medical image data generated using such an ultrasound probe. According to the image diagnostic system 100 according to an embodiment of the present disclosure, when performing machine learning on an image diagnostic model that performs image diagnosis based on medical image data generated using such an uncommon ultrasound probe, a large amount of training data set can be easily generated based on medical image data generated using a major ultrasound probe. This significantly improves the efficiency of machine learning and efficiently trains an image diagnostic model that can perform highly accurate image diagnosis.

[0081] Alternatively, if the third medical image data is ultrasound image data, it may be difficult to visualize the subject or perform labeling (such as specifying whether the image contains a lesion or encircling a specific area within the image) due to reasons such as a poor signal-to-noise ratio or difficulty in applying the probe. In such cases, the effort required to assign a first correct label to the first medical image data can be reduced by using, for example, MRI image data generated using a nuclear magnetic resonance imaging device, which is likely to have a higher signal-to-noise ratio, as the first medical image data. Furthermore, a high-quality training dataset can be generated by performing image conversion processing on the first medical image data to generate second medical image data that looks like ultrasound image data, and using the pair with the first correct label as a training dataset.

[0082] The embodiment of the present disclosure described above is a training dataset generation device that generates a training dataset, a machine learning device that performs machine learning on an image diagnostic model using the training dataset, an image diagnostic device that outputs an image diagnostic result using the trained image diagnostic model, and an image diagnostic system that includes each of these components. The image diagnostic system of the present disclosure may be, for example, an ultrasound image diagnostic system that can handle ultrasound image data.

[0083] Although examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and changes are possible within the scope of the gist of the present disclosure as set forth in the claims. [Explanation of symbols]

[0084] 100 Diagnostic Imaging System 10. Learning dataset generation device 11 Dataset Acquisition Section 12 Image conversion section 13 Image data acquisition unit 14 Generation part 20 Machine Learning Device 21 Learning dataset acquisition unit 22 Learning model storage section 23 Learning Department 30 Image Diagnosis Device 31 Image Diagnosis Model Acquisition Unit 32 Image Generation Unit 33 Inference Unit

Claims

1. The computer, an acquisition step of acquiring first medical image data and a first correct label that is a correct label of the first medical image data; a conversion step of converting the first medical image data into second medical image data by style conversion in a style to which the first correct label can be applied; an output step of generating a pair of the second medical image data and the first correct label as a training dataset; A method for generating a training dataset.

2. The first correct label is information regarding a region of interest included in the first medical image data. The method for generating a training dataset according to claim 1 .

3. The converting step does not change the style of the tissue position corresponding to the region of interest of the first medical image data. The method for generating a training dataset according to claim 2 .

4. the first medical image data is ultrasound image data, the converting step converts the first medical image data based on reflected ultrasound waves transmitted from a first ultrasound probe into a style of a second ultrasound imaging device different from that of the first ultrasound imaging device; The method for generating a training dataset according to claim 1 .

5. the first medical image data is ultrasound image data, the conversion step converts the reflected ultrasound on which the first medical image data is based into a style of a second ultrasound probe different from a style of a first ultrasound probe that transmits ultrasound. The method for generating a training dataset according to claim 1 .

Images