Learning device, learning system, machine learning model learning method and program

Abstract

To provide a learning apparatus, a learning system, a learning method of a machine learning model and a program, enabling learning of the machine learning model more properly even with little data.SOLUTION: A learning apparatus includes: an acquisition part for acquiring a first medical image; an image generation part for generating a second medical image in which a feature according to feature information related to non-recognition-target areas of the first medical image is brought close to a feature according to reference feature information used as a predetermined reference; and a learning part for promoting learning of a machine learning model by using the second medical image.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a learning device, a learning system, a learning method for a machine learning model, and a program. [Background technology]

[0002] Conventionally, diagnosis has been widely performed by detecting and recognizing lesions and abnormalities related to diseases from medical images. However, for doctors and medical technicians to visually inspect images individually to detect and recognize lesions, it is time-consuming and depends on the skill of the individual doctor, and there is a possibility that something may be overlooked.

[0003] Meanwhile, research, development, and use of computer-based image recognition technology are expanding. One well-known image recognition technology is a machine learning model that uses convolutional neural networks. Machine learning models are used by training them to identify targets based on a large amount of image data, some of which may contain the target and some of which may not. However, if there is insufficient image data for training, the model may become sensitive to differences in areas other than the target, potentially resulting in an incorrect diagnosis. When using machine learning models for specialized applications such as medical imaging, it is not always possible to obtain sufficient images containing targets, such as lesions.

[0004] Patent Document 1 discloses a technique for generating adversarial features in a subspace that contributes to learning in a multidimensional space related to features, thereby optimizing the threshold for judgment. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2018 / 167900 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in the conventional technology, the amount of learning is increased by adding data that includes data of dimensions that do not contribute to learning, which poses a problem that the learning efficiency is not necessarily good.

[0007] An object of the present invention is to provide a learning device, a learning system, a learning method and a program for a machine learning model that can more appropriately learn a machine learning model even with a small amount of data. [Means for solving the problem]

[0008] In order to achieve the above object, the invention described in claim 1 is as follows: an acquisition unit that acquires a plurality of first medical images; A feature according to feature information relating to a non-target portion of the plurality of first medical images is The only a control unit that controls to generate a plurality of second medical images that are closer to features corresponding to the reference feature information that serves as a reference; a learning unit that uses the plurality of second medical images to train a machine learning model; It is a learning device equipped with the above.

[0009] The invention described in claim 2 is the learning device described in claim 1, The non-recognition target portion is Multiple This is at least a part of the area other than the recognition target in the second medical image.

[0010] The invention described in claim 3 is the learning device described in claim 1 or 2, The characteristic information includes Multiple The image acquisition information includes at least one of manufacturer information of the imaging device for the first medical image, type information of the imaging device, model information, imaging condition information, imaging facility information, and imaging target information.

[0011] The invention described in claim 4 is the learning device described in claim 3, The image capture information is associated with image quality characteristics including at least one of sharpness, contrast, noise, density, gradation, and resolution.

[0012] The invention of claim 5 is a learning device according to any one of claims 1 to 4, The aforementioned Multiple The first medical image is any one of an X-ray image, an ultrasound image, a nuclear magnetic resonance image, and a positron emission tomography image, or a combination of two or more of these.

[0013] The invention of claim 6 is a learning device according to any one of claims 1 to 5, The aforementioned Multiple The second medical image is Multiple The feature corresponding to at least one of the feature information related to the first medical image is made to approximate the feature corresponding to the reference feature information.

[0014] The invention described in claim 7 is the learning device described in claim 6, The aforementioned Multiple The second medical image is Multiple The feature corresponding to at least one of the feature information related to the first medical image is matched with the feature corresponding to the reference feature information.

[0015] The invention of claim 8 is a learning device according to any one of claims 1 to 7, The aforementioned Multiple a storage unit that stores conversion details of the features corresponding to a difference between feature information related to the first medical image and the reference feature information; The aforementioned control The unit acquires the conversion content from the storage unit according to the feature information, Multiple applying the transformation to the first medical image Multiple Generate a second medical image Control .

[0016] The invention of claim 9 is a learning device according to any one of claims 1 to 7, The aforementioned control The unit has a trained model that has been trained to output, in response to an input of a medical image, a medical image that approximates the medical image to features corresponding to the reference feature information.

[0017] The invention described in claim 10 is as follows: an acquisition unit that acquires a plurality of first medical images; A feature according to feature information relating to a non-target portion of the plurality of first medical images is The only a control unit that controls to generate a plurality of second medical images that are closer to features corresponding to the reference feature information that serves as a reference; a learning unit that uses the plurality of second medical images to train a machine learning model; It is a learning system that includes the following.

[0018] The invention described in claim 11 is as follows: an acquiring step of acquiring a plurality of first medical images; A feature according to feature information relating to a non-target portion of the plurality of first medical images is The only a control step of controlling to generate a plurality of second medical images having features that are closer to those according to the reference feature information serving as a reference; a learning step of performing learning of a machine learning model using the plurality of second medical images; This is a method for training a machine learning model including:

[0019] The invention described in claim 12 is as follows: Computer, an acquisition means for acquiring a plurality of first medical images; A feature according to feature information relating to a non-target portion of the plurality of first medical images is The only a control means for controlling the generation of a plurality of second medical images having features that are closer to those according to the reference feature information serving as a reference; a learning means for learning a machine learning model using the plurality of second medical images; It is a program that functions as a [Effects of the Invention]

[0020] According to the present invention, it is possible to effectively train a machine learning model even with a small amount of data. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a diagram illustrating an outline of the overall configuration of an information system according to an embodiment of the present invention. [Figure 2] 1 is a block diagram showing a functional configuration of an information processing apparatus according to a first embodiment; [Figure 3] FIG. 10 is a diagram for schematically explaining differences relating to portions outside the recognition target; [Figure 4] 5 is a flowchart showing a control procedure of a model learning control process executed in the information processing apparatus of the first embodiment. [Figure 5] 10 is a flowchart showing a control procedure for image diagnosis control processing using the obtained trained model. [Figure 6] FIG. 10 is a block diagram showing the functional configuration of an information processing apparatus according to a second embodiment. [Figure 7] 10 is a flowchart showing a control procedure of a model learning control process executed in an information processing apparatus according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram illustrating an outline of the overall configuration of an information system 100 according to this embodiment. The information system 100 includes an information processing device 1, an imaging device 2, a data server 3, and the like that are connected to the information processing device 1 so as to be able to send and receive data via a communication network N. The communication network N may be within a specific LAN (Local Area Network) or VPN (Virtual Private Network), or may be connected via the Internet (including cases where authentication is required for connection).

[0023] The information processing device 1 is a learning device of this embodiment, and generates a machine learning model for diagnosing an image based on acquired captured image data.

[0024] The imaging device 2 is a modality that captures images for medical purposes and generates and outputs the captured images. In other words, the captured images are medical images, and the imaging range includes a diagnostic target area such as an injury or disease in the human body. The imaging device 2 may be, for example, an X-ray imaging device, an ultrasound imaging device, a magnetic resonance imaging (MRI), a positron emission tomography (PET), etc. (i.e., the captured medical images include X-ray images, ultrasound images, magnetic resonance imaging (MRI), positron emission tomography (PET), etc.), but is not limited to these. X-ray imaging devices may include imaging devices that generate digital data related to simple imaging such as CR and DR, and imaging devices related to CT (computed tomography). Multiple imaging devices 2 may be connected to the communication network N, and the multiple imaging devices 2 may include multiple types of imaging devices of different types or multiple imaging devices of the same type. Multiple imaging devices of the same type may be the same model from the same manufacturer, different manufacturers, or different models from the same manufacturer.

[0025] The data server 3 stores and holds the photographed image data and photographing information taken by the photographing device 2, as well as the patient's diagnostic information corresponding to the photographed image. There may be multiple data servers 3. The data servers 3 may correspond one-to-one to each photographing device 2, or may aggregate image data from multiple photographing devices 2. The photographing devices 2 do not need to be directly accessible from the information processing device 1 via the communication network N. The photographed image data may be acquired and held by the data server 3 once, and then acquired by the information processing device 1 through communication between the data server 3 and the information processing device 1.

[0026] [First embodiment] FIG. 2 is a block diagram showing the functional configuration of the information processing device 1 according to the first embodiment. The information processing device 1 is, for example, a normal computer (PC) and includes a control unit 10 (acquisition unit, image generation unit, learning unit), a memory unit 20, a communication unit 30, a display unit 40, an operation reception unit 50, etc.

[0027] The control unit 10 controls the overall operation of the information processing device 1. The control unit 10 includes a central processing unit (CPU) 11 and a random access memory (RAM) 12. The CPU 11 is a hardware processor that performs various arithmetic operations. The RAM 12 provides the CPU 11 with a working memory space and stores temporary data. The temporary data includes expanded data and setting data for the control program, and the CPU 11 executes the program contents based on the expanded data. The CPU 11 does not need to be a single CPU; multiple CPUs 11 may process the same process or process multiple processes in parallel. In addition to the CPU 11, the control unit 10 may also include a hardware logic circuit dedicated to a specific process. The RAM 12 may have separate working memory for the CPU 11 and memory for storing various data, or a common memory may be dynamically allocated to each process as needed.

[0028] The storage unit 20 stores and holds the above-mentioned control program 21, machine learning model 23, setting data, etc. The storage unit 20 can also store acquired medical image data and a conversion table 22 (conversion content) which is the conversion data thereof. The storage unit 20 includes a non-volatile memory, such as, but not limited to, a flash memory. The storage unit 20 may include not only a storage unit built into the information processing device 1, but also an external auxiliary storage device, etc. The auxiliary storage device may also be located on a network as a cloud server, etc.

[0029] The program 21 includes a control processing program for training the machine learning model 23. The machine learning model 23 is initial data for a machine learning model (trained model) that has been trained to output a recognition result of a medical abnormality such as a lesion in response to input medical image data. The algorithm of the machine learning model 23 is not particularly limited as long as it is capable of image recognition. For example, deep learning using a convolutional neural network is widely used as a machine learning method suitable for image recognition. Furthermore, for example, semantic segmentation may be used to identify and recognize lesion locations.

[0030] The conversion table 22 is data that stores conversion contents for converting and generating medical image data (data of a second medical image) from original medical image data (data of a first medical image) to be input to the machine learning model 23, in association with feature information (described later) related to the original medical image data. The conversion table 22 will be described later.

[0031] The storage unit 20 may also include a volatile memory for temporarily storing large amounts of medical image data. Alternatively, the medical image data and its processed data may be stored in a nonvolatile memory, similar to programs. The nonvolatile memory may include a hard disk drive (HDD), and may also include a flash memory or the like in addition to or instead of the HDD.

[0032] The communication unit 30 controls data communication between the device and external devices via a communication network N or the like. The communication unit 30 may have, for example, a network card that controls communication based on a communication standard (such as TCP / IP) related to communication in a LAN or the like. Furthermore, communication is not limited to wired communication such as a LAN, and the communication unit 30 may have a network card that controls wireless communication via Wi-Fi or the like. The communication unit 30 may also include a driver for reading data from a portable storage medium such as a CDR, a driver for direct communication with an external device via a Universal Serial Bus (USB) or the like, and the like.

[0033] The display unit 40 has a display screen and performs display based on the control of the control unit 10. The display screen includes a digital display screen such as a liquid crystal display (LCD), and can display statuses related to the control operations of the control unit 10, menus for accepting input operations by the user, and the like. The display screen may also be capable of displaying images represented by image data, as described below. The display unit 40 may also include an image forming device (printer) that forms images of the displayed content.

[0034] The operation reception unit 50 receives an operation from an external device such as a user, generates an operation signal according to the type of the received operation, and outputs the operation signal to the control unit 10. The operation reception unit 50 may have input devices such as switch buttons that receive physical switching operations, such as push button switches, slide switches, and rocker switches, various keyboards, and / or pointing devices such as a mouse. In addition to or instead of these, the operation reception unit 50 may have a touch panel. The touch panel is positioned so as to overlap the display screen of the display unit 40, and outputs an operation signal indicating the touch position, the continuation status of the touch operation, and the like as the type of operation, thereby allowing the control unit 10 to identify the content of the received operation in conjunction with the content displayed on the display screen.

[0035] The information processing device as a computer of this embodiment is only required to have the control unit 10 as a minimum configuration and to be capable of acquiring the first medical image.

[0036] Next, the characteristics of medical images will be described. A medical image (first medical image) that is an image captured by the imaging device 2 is an image of a specific region captured by an imaging device suitable for capturing images of that region for medical diagnosis or the like, as described above, or an image that has undergone primary processing. The primary processing may include processing to superimpose (combine) or subtract images of the results of capturing images of the same region by multiple imaging devices 2 or the results of capturing images captured by the same imaging device 2 at different times. For example, image data that has undergone processing to superimpose and fuse a CT image and a PET image may be used as the data of the medical image.

[0037] Even medical images of the same subject captured using the same type of imaging device have unique image quality characteristics related to parameters related to imaging information, such as the manufacturer, model, and device. These characteristics primarily relate to image quality, such as noise levels and characteristics, resolution, sharpness, contrast, density, and gradation. Even when the same model of device is used, the image quality can vary depending on the imaging conditions. Imaging conditions may vary depending on the settings made by the photographer, the photographer's skill, and the conditions of the imaging room. Furthermore, when primary processing of medical image data is performed, image quality can vary depending on the processing content (e.g., algorithms, parameters, and other characteristics). Image acquisition information (imaging information, imaging conditions, and primary processing information) that can affect these characteristic differences is attached to or associated with the medical image data as metadata (ancillary information). Therefore, by identifying (associating) the correspondence between image acquisition information and image quality characteristics in advance, it is possible to estimate the characteristics corresponding to the image acquisition information of the medical image data without analyzing the medical image data itself each time.

[0038] Furthermore, although not particularly limited, in this case, in order to perform supervised learning of the machine learning model 23, the metadata may include, as training data, information such as medical abnormalities in the medical image (i.e., the portions to be recognized in the medical image by the trained model obtained by training the machine learning model 23). Note that this training data may be added after the medical image has been converted as described below and before machine learning.

[0039] FIG. 3 is a diagram for explaining the difference relating to the portion outside the recognition target. Medical images contain features such as the amount of noise, noise patterns, and gradations corresponding to the noise, relative to the surroundings (background) of the recognition target area (shown here as a star-shaped area in the center). These features also appear in the gradation characteristics of the recognition target. These features, which vary due to factors unrelated to the recognition target (related to the non-recognition target area that is outside the range of recognition targets by the trained model), tend to depend on the image acquisition information (manufacturer information, type information, model information, imaging condition information, imaging facility information), which includes parameters such as the manufacturer and type of imaging device, the model of the imaging device, individual imaging devices, the imaging location (facility information), and the photographer (imaging conditions). In other words, they can generally be identified by referencing the image acquisition information.

[0040] Furthermore, when a wider area than the diagnostic target is captured, the diagnostic target (image capture target; Region of Interest; ROI) may be extracted from these images during diagnosis. The learning data can be extracted together with other images regardless of the actual ROI (image capture target information). Furthermore, when extraction is performed, the resulting resolution and sharpness of the extracted portion tend to be smaller (lower) than the resolution and sharpness when the image is captured by narrowing the focus to the extracted portion from the beginning.

[0041] Even if these features are unrelated to the appearance of characteristic areas due to the lesions of the recognition target, they may be detected in large numbers in conjunction with the recognition target and may be mistakenly learned as being related to the recognition target, especially if there is insufficient training data. In particular, when using a trained model, even if abnormal areas are photographed with the same size and shape as shown in FIG. 3, it is necessary to minimize the possibility that they may be judged as symptoms of different severity, particularly milder symptoms than they actually are, or may be overlooked and not recognized as a characteristic lesion pattern, depending on the imaging conditions, etc. (especially in the case of medical image data with other image quality characteristics not used in training). In this embodiment, an image (second medical image) is generated from the original medical image (first medical image) in such a way that the differences in image features that are not directly related to the recognition target (related to parts outside the recognition target), i.e., not useful for training, are reduced. By using the data of the generated medical image for training, the features of the image portion of the recognition target can be more appropriately learned even with a small amount of training data.

[0042] In this way, features corresponding to image acquisition information related to imaging information, primary processing information, etc., which are not useful for machine learning, are identified, for example, as differences from predetermined reference features (features corresponding to reference feature information). Then, the amount of image conversion corresponding to this difference (difference) is associated with the image acquisition information and stored and held as a conversion table 22 (conversion content) in the storage unit 20, and is acquired based on the image acquisition information, so that features that are not useful for recognizing the recognition target in a medical image can be easily processed to approximate those corresponding to the reference feature information.

[0043] The reference feature information may be determined, for example, based on the average of images captured with basic settings by one or more models of imaging devices 2 with a large market share. The amount of transformation required to bring the features of a medical image corresponding to various image acquisition information closer to the features associated with the reference feature information may be determined by determining a process that increases the similarity between the medical image and a reference image having features (reference features) associated with the reference feature information, based on the similarity (quantitative evaluation of the degree of similarity between the images). For example, the features of the image data may be represented as multidimensional feature vectors, and the transformation amount (conversion content) may be determined to increase the similarity using cosine similarity or Euclidean distance between the feature vectors as a similarity measure. Here, the feature vector is not particularly limited, and may be an array of intermediate features obtained by well-known image recognition techniques, such as a convolutional neural network (CNN).

[0044] Furthermore, without using feature vectors, medical image data may be directly compared with reference image data corresponding to reference feature information, and a similarity indicating the degree of similarity in pixel value (brightness value) arrangement and distribution (histogram) may be calculated.

[0045] Alternatively, rather than using similarity as a scalar value, each of the above features related to image quality may be quantitatively evaluated (which may include well-known values ​​such as S / N ratio), and a conversion may be performed so that the evaluation value approaches the reference evaluation value related to the reference feature information.

[0046] The conversion amount may be determined so that, in the feature vector, the magnitude of at least one of the characteristic components of the non-target portion of the medical image approaches the magnitude of the characteristic component of the corresponding reference feature information (increasing the overall similarity), more preferably so that at least one parameter of the characteristic information matches. In this case, the conversion amount must be determined so as not to apply unnecessary conversion to the target image portion. The determined conversion amount is stored in the storage unit 20 as a conversion table 22 in association with the characteristic information (image acquisition information) of the target image data.

[0047] The feature information includes multiple types of parameters related to the image acquisition information as described above, and a conversion amount may be determined for each variable that can be set for each parameter, and the conversion amounts for the multiple parameters may be calculated and integrated, or a conversion amount may be determined for each combination of multiple parameters. Even in these cases, the overall conversion amount may be determined so as to ultimately increase the similarity.

[0048] Furthermore, the portion of the image acquisition information that depends on the shooting information and the portion that depends on the individual shooting conditions and primary processing conditions may be adjusted separately. That is, if the image quality (e.g., contrast, gradation, density, sharpness, resolution, etc.) is adjusted during the shooting or primary processing of each image, and if the image quality information (image quality information) is included in the metadata, a process may be performed to adjust the conversion amount depending on the difference from the image quality set in the image acquisition information. Furthermore, in the case of a medical image that undergoes adjustments not included in the metadata, the difference between the medical image and a reference image may be analyzed, and if this difference does not match the conversion amount within a reference range, a process may be performed to calculate a separate conversion amount, or the corresponding captured image may simply be deleted from the medical images used for training.

[0049] FIG. 4 is a flowchart showing a control procedure by the control unit 10 (CPU 11) of the model learning control process executed by the information processing device 1 of this embodiment. The model learning control process as an embodiment of a machine learning model learning method is started, for example, when an operation receiving unit receives a predetermined input operation by a user or the like, or when a start command is obtained via a communication unit.

[0050] When the model learning control process is started, the control unit 10 (CPU 11) acquires medical image data from outside via the communication network N (step S101; processing as an acquisition unit, acquisition step, acquisition means). The acquired medical image data is stored in the storage unit 20, which may be external or located on the network as described above. The control unit 10 analyzes the image information of each medical image (including simply referring to metadata) and acquires feature information (particularly image acquisition information) related to the generation of a converted image (step S102).

[0051] The control unit 10 refers to the conversion table 22 and acquires and determines the conversion content (amount of conversion) based on the acquired feature information (step S103).The control unit 10 generates medical image data by converting the original medical image data based on the conversion content (step S104; processing as an image generation unit, image generation step, image generation means).

[0052] When the process of generating medical image data converted from all acquired medical image data is completed (as described above, the generation of medical image data related to medical image data for which conversion is not performed appropriately may be stopped), the control unit 10 inputs the generated medical image data in order to the machine learning model 23, and causes the machine learning model 23 to learn (step S105; processing as a learning unit, learning step, learning means). Although there are no particular limitations on the learning, learning is performed by feeding back the difference between the teacher data indicating the range of the lesion included in the medical image data as supplementary information as described above and the output result of the machine learning model 23.

[0053] When the input of all medical image data has been completed and learning is complete, the control unit 10 stores and saves data including settings such as parameters related to the learned model in the storage unit 20 (step S106). Then, the control unit 10 ends the model learning control process.

[0054] 5 is a flowchart showing the control procedure for image diagnosis control processing using the trained model obtained as described above. This image diagnosis control processing is executed and controlled by a control unit of an arbitrary information processing device (such as a PC) separate from the above-described information processing device 1, in which the trained model and conversion table 22 are installed. The information processing device that recognizes abnormalities from medical images using this trained model may be connected to a communication network N and be able to access a data server 3, etc.

[0055] When the image diagnosis control process is started, the control unit acquires medical image data to be recognized (step S201). The control unit analyzes image information (mainly image acquisition information) of the medical image data to acquire feature information of the medical image data (step S202).

[0056] The control unit specifies the image transformation content that brings the feature information closer to the reference feature information (step S203). The control unit generates medical image data by transforming the original medical image data according to the specified transformation content (step S204).

[0057] The control unit inputs the generated medical image data into the trained model (step S205). The control unit acquires the recognition result output from the trained model (step S206). If necessary, the control unit may output the acquired recognition result as is and / or generate document data or image data based on the recognition result together with identification information that can identify the original medical image data to the data server 3 or the like, or may display it on a display screen or the like. Then, the control unit terminates the image diagnosis control process. In addition, the medical image data generated for input into the trained model may be deleted after the recognition results are obtained.

[0058] That is, in image diagnosis using this trained model, appropriate recognition results can be obtained by converting the captured image data of the diagnosis target into image data that is similar to the reference feature information, as in the case of generating the trained model, and then inputting the converted image data, rather than inputting the captured image data directly into the trained model. However, even if medical image data including the recognition target area is input into the trained model without the above conversion, appropriate diagnostic results can be obtained as long as the trained model has been trained so as not to have sensitivity to areas outside the recognition target. Therefore, input of converted medical image data is not necessarily required.

[0059] [Second embodiment] Next, an information processing device 1a, which is a learning device according to the second embodiment, will be described. FIG. 6 is a block diagram showing the functional configuration of an information processing apparatus 1a according to the second embodiment. This information processing device 1a differs from the information processing device 1 of the first embodiment in that a conversion trained model 22a (trained model) is stored in the storage unit 20 instead of the conversion table 22. The other configurations are the same between the first and second embodiments, and the same configurations are assigned the same reference numerals and detailed description thereof will be omitted.

[0060] In the information processing device 1 of the first embodiment, a conversion process was performed to bring the image features closer together using similarity, etc., but in the information processing device 1a of this embodiment, a trained model for conversion 22a trained by machine learning is also used for this conversion process. That is, by inputting medical image data to this trained model for conversion 22a, medical image data with features closer to those corresponding to the reference feature information is generated and output.

[0061] The machine learning algorithm of this trained conversion model 22a used in the conversion process may be, for example, a generative adversarial network (GAN). The machine learning model is trained by inputting a set of captured images having differences related to parts outside the recognition target in advance. The training can be performed so that the converted and generated medical image can be distinguished from a reference image (real) corresponding to the reference feature information (for example, a binary classifier outputs 1 (real)), that is, so that the difference between the features of parts that do not contribute to identifying the diagnostic target, such as the background of the converted and generated medical image, and the features related to the reference feature information is minimized.

[0062] FIG. 7 is a flowchart showing a control procedure by the control unit 10 of the model learning control process executed in the information processing device 1a of the second embodiment. In this model learning control process, the processes of steps S102 to S104 of the model learning control process in the above embodiment are replaced with processes of steps S111 and S112. The other processes are the same, and the same process contents are assigned the same reference numerals and detailed explanations are omitted.

[0063] After step S101, the control unit 10 (CPU 11) inputs each acquired medical image into the trained model for conversion 22a (step S111). This trained model for conversion 22a is a trained model trained using the above-mentioned GAN or the like. The control unit 10 acquires each medical image data generated and output by the trained model for conversion 22a (step S112). At this time, the control unit 10 may display each acquired medical image on the display unit 40 to allow the user to confirm whether the conversion has been successful. The control unit 10 may leave the converted medical images for which an approval operation has been accepted by the operation accepting unit 50, and delete the medical images for which an approval operation has not been accepted (for which a rejection operation has been accepted). Then, the processing of the control unit 10 proceeds to step S105.

[0064] As described above, the information processing device 1, which is the learning device of this embodiment, includes a control unit 10 (CPU 11), and the control unit 10 functions as an acquisition unit to acquire a first medical image, as an image generation unit to generate a second medical image in which features corresponding to feature information relating to parts of the first medical image outside the recognition target are made closer to features corresponding to reference feature information that serves as a predetermined standard, and as a learning unit to use the second medical image to train a machine learning model 23 that outputs a recognition result for the recognition target. In this way, in the information processing device 1, by aligning the image quality of the non-target parts of the second medical image that are not within the range of the target to be recognized by the trained machine learning model 23 and training the machine learning model 23, systematic variations in image quality due to the effects of photography and primary processing, etc., are prevented from occurring, which is not only useless for recognizing medical abnormalities but also leads to erroneous judgments, and a trained model that can more appropriately recognize the above-mentioned abnormalities is obtained even if sufficient training image data is not available.

[0065] The feature information also includes image acquisition information, which includes at least one of manufacturer information of the imaging device 2 of the first medical image, type information of the imaging device 2, model information, imaging condition information, imaging facility information, and imaging target information. These generate image quality features such as noise systematically rather than randomly, so learning using an insufficient amount of learning image data can result in incomplete recognition and learning, and depending on the degree of variation in image quality features, it may become impossible to accurately recognize abnormalities. Therefore, by performing preprocessing to reduce systematic differences depending on the image acquisition information (especially the image acquisition information) and then performing machine learning related to recognizing abnormalities, a trained model that can more appropriately recognize abnormalities can be obtained.

[0066] Furthermore, the image acquisition information is associated with image quality characteristics including at least one of sharpness, contrast, noise, density, gradation, and resolution. In this way, image quality is made uniform by converting the image characteristics according to the image acquisition information, making it possible to more accurately recognize abnormal areas.

[0067] The first medical image is one of an X-ray image, an ultrasound image, a nuclear magnetic resonance image, and a positron emission tomography image, or a combination of two or more of these. In this way, in the diagnosis of medical images that capture two-dimensional images, there are few examples of images of specific symptoms, and it is difficult to accurately recognize them by excluding slight differences in image quality, such as background, from machine learning. In this way, by training features of parts that are not effective in recognizing abnormalities such as lesions, it is possible to effectively use a variety of captured images for training, thereby improving recognition accuracy.

[0068] Furthermore, the second medical image is one in which the features corresponding to at least one of the feature information related to the first medical image are made to approximate the features corresponding to the quasi-feature information. That is, whether the overall similarity is determined for each parameter of the feature information or the degree of similarity is determined for each piece of feature information, by converting the image into the second medical image in accordance with the content of a parameter of any of the feature information, it is possible to target and reduce the variation in image quality related to at least that parameter.

[0069] Furthermore, in particular, if the second medical image can match the features corresponding to at least one of the feature information related to the first medical image with the features corresponding to the reference feature information, the variation related to the parameter will be eliminated, and the types of variation in image quality can be reduced by the amount of the match. Therefore, it is possible to reduce the factors that cause erroneous judgment in machine learning and improve the learning accuracy in machine learning using an insufficient amount of training image data.

[0070] The information processing device 1 also includes a storage unit 20 that stores feature conversion details corresponding to the difference between feature information related to the first medical image and reference feature information. The control unit 10, as an image generation unit, acquires the conversion details from the storage unit 20 according to the feature information related to the first medical image and generates a second medical image by applying the conversion details to the first medical image. By predetermining conversion details for each piece of patterned feature information in this way, the information processing device 1 can easily reduce image quality variations that are not useful for machine learning, thereby suppressing an increase in the effort required for training a machine learning model related to the recognition of abnormalities such as lesions.

[0071] Alternatively, in the information processing device 1a, the control unit 10 has, as an image generation unit, a trained model that has been trained to output, in response to an input medical image, a medical image that approximates the medical image to features corresponding to reference feature information. That is, by separately preparing and using a trained model that preprocesses the training data of a machine learning model for recognizing abnormalities such as lesions, it is possible to appropriately reduce differences in the image that are not attributable to the recognition target, thereby suppressing an increase in the effort required for training a machine learning model for recognizing abnormalities such as lesions.

[0072] Furthermore, a learning system in which at least some of the configurations of the control unit 10 as the acquisition unit, the image generation unit, and the learning unit are distributed can also provide a trained model that can more appropriately recognize the abnormal areas even if sufficient training image data is not available.

[0073] Furthermore, the training method for the machine learning model 23 of this embodiment includes an acquisition step of acquiring a first medical image, an image generation step of generating a second medical image in which features according to feature information relating to a portion of the first medical image outside the recognition target are made to resemble features according to predetermined reference feature information, and a training step of training the machine learning model 23 using the second medical image, which outputs a recognition result for the recognition target. That is, even if the processing of each step is not performed by a single information processing device 1, 1a, the machine learning model 23 can be trained more appropriately with a small amount of training image data.

[0074] Furthermore, the program 21 of this embodiment causes a computer to function as an acquisition means for acquiring a first medical image, an image generation means for generating a second medical image in which features according to feature information relating to a portion of the first medical image outside the recognition target are made to resemble features according to predetermined reference feature information, and a learning means for using the second medical image to train a machine learning model that outputs a recognition result for the recognition target. In this way, because preprocessing of machine learning is performed in software by installing the program 21, no dedicated information processing device is required, and training data for the machine learning model 23 used to identify abnormal areas such as lesions can be easily prepared, and the machine learning model 23 can be trained more appropriately even without sufficient training data.

[0075] The present invention is not limited to the above-described embodiment, and various modifications are possible. For example, in the above embodiment, a second medical image is generated from a first medical image using the conversion table 22, but a conversion formula or the like may be stored instead of the conversion table 22, and the conversion amount may be calculated. Alternatively, the conversion amount may be determined directly or appropriately from the difference in feature amounts, without preparing a conversion table or conversion formula in advance.

[0076] Furthermore, in the above embodiment, it was described that at least some of the characteristic information of the medical image, particularly each parameter of the image acquisition information, are brought closer to the image quality indicated in the reference characteristic information, but as long as the similarity of multiple parameters as a whole increases, it may also include cases where the image quality for some parameters does not ultimately approach the image quality indicated in the reference characteristic information.

[0077] Furthermore, the input of the obtained trained model may not only be medical images but also include additional information such as age and gender, etc. Furthermore, the output of the trained model may not only be the recognition result but also other additional information such as the size and severity of symptoms.

[0078] Furthermore, in the above embodiment, the image conversion process and the process related to the learning of the machine learning model 23 are described as being performed by the same information processing device 1, 1a, but they may be performed by separate information processing devices. That is, in a learning system having two information processing devices, after the image conversion is performed by the first information processing device, the machine learning model 23 may be trained by the second information processing device using the converted image data. Also, some or all of the processes may be performed using hardware resources such as a cloud server or a rental server.

[0079] Furthermore, the information processing devices 1 and 1a do not need to be devices dedicated to learning the machine learning model 23. Other processing may be performed. For example, the primary processing of each first medical image may be performed within the same information processing device. In this case, the medical image data may be passed on directly to the generation processing of the second medical image without being output to the outside.

[0080] Furthermore, the parameters and variables related to image acquisition information and the types of image quality characteristics shown in the above embodiments are not limited to those exemplified. The former may be any parameter that can be set and identified for an imaging device capable of capturing medical images, and the latter may include any parameter that can quantitatively express image quality.

[0081] Furthermore, the reference feature information may be set arbitrarily. It is not limited to the image quality obtained by an actual imaging device, but may be ideal information that makes it easier for doctors and others to make diagnoses. In this case, not only the original first medical image but also the converted second medical image data itself may be saved and used for diagnosis, examination, and the like by doctors.

[0082] In the above description, the storage unit 20, which is composed of a nonvolatile memory such as an HDD or flash memory, has been used as an example of a computer-readable medium for storing the program 21 related to the model learning control of the present invention, but this is not limited to this. Other computer-readable media that can be used include other nonvolatile memories such as MRAM, and portable storage media such as CD-ROMs and DVD discs. Furthermore, a carrier wave can also be used as a medium for providing data for the program related to the present invention via a communication line. In addition, the specific configurations, contents and procedures of the processing operations, etc. shown in the above embodiments can be modified as appropriate without departing from the spirit of the present invention. The scope of the present invention includes the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0083] 1, 1a Information processing device 2. Imaging equipment 3 Data Server 10 Control Unit 11 CPU 20 Memory section 21 Programs 22 Conversion Table 22a Pre-trained model for conversion 23 Machine Learning Models 30 Communications Department 40 Display section 50 Operation reception section 100 Information Systems N Communication Network

Claims

1. an acquisition unit that acquires a plurality of first medical images; a control unit that controls to generate a plurality of second medical images by approximating features according to feature information related to non-target portions of the plurality of first medical images to features according to reference feature information that serves as a predetermined unique reference; a learning unit that uses the plurality of second medical images to train a machine learning model; A learning device comprising:

2. The learning device according to claim 1 , wherein the non-recognition target portion is at least a part of the range other than the recognition target in the plurality of second medical images by the machine learning model used for the learning.

3. A learning device as described in claim 1 or 2, wherein the feature information includes image acquisition information including at least one of manufacturer information of the imaging device of the plurality of first medical images, type information of the imaging device, model information, imaging condition information, imaging facility information, and imaging subject information.

4. 4. The learning device according to claim 3, wherein the image acquisition information is associated with image quality characteristics including at least one of sharpness, contrast, noise, density, gradation, and resolution.

5. The learning device according to any one of claims 1 to 4, wherein the plurality of first medical images are any one of X-ray images, ultrasound images, nuclear magnetic resonance images, and positron emission tomography images, or a combination of two or more of these.

6. A learning device described in any one of claims 1 to 5, wherein the plurality of second medical images are obtained by approximating features corresponding to at least one of the feature information relating to the plurality of first medical images to features corresponding to the reference feature information.

7. The learning device according to claim 6, wherein the plurality of second medical images are obtained by matching features corresponding to at least one of the feature information relating to the plurality of first medical images with features corresponding to the reference feature information.

8. a storage unit that stores conversion details of the features corresponding to differences between feature information related to the plurality of first medical images and the reference feature information; The control unit acquires the transformation content from the storage unit according to the feature information, and controls the generation of the plurality of second medical images by applying the transformation content to the plurality of first medical images. The learning device according to any one of claims 1 to 7.

9. The control unit has a trained model trained to output a medical image in response to an input of a medical image, the medical image being closer to features corresponding to the reference feature information. The learning device according to any one of claims 1 to 7.

10. an acquisition unit that acquires a plurality of first medical images; a control unit that controls to generate a plurality of second medical images by approximating features according to feature information related to non-target portions of the plurality of first medical images to features according to reference feature information that serves as a predetermined unique reference; a learning unit that uses the plurality of second medical images to train a machine learning model; A learning system that includes:

11. an acquiring step of acquiring a plurality of first medical images; a control step of controlling to generate a plurality of second medical images by approximating features according to feature information relating to non-recognition target portions of the plurality of first medical images to features according to reference feature information serving as a predetermined unique reference; a learning step of performing learning of a machine learning model using the plurality of second medical images; How to train machine learning models, including:

12. Computer, an acquisition means for acquiring a plurality of first medical images; a control means for controlling the generation of a plurality of second medical images in which features corresponding to feature information relating to non-recognition target portions of the plurality of first medical images are made closer to features corresponding to reference feature information serving as a predetermined unique reference; a learning means for learning a machine learning model using the plurality of second medical images; A program that functions as a

Images