Image conversion program, method, and device

Abstract

To provide an image conversion program, a method, and a device allowing medical images acquired in various imaging conditions to be efficiently identified.SOLUTION: A computer converts a first image to a third image by applying a second filter applied to a second filter used as a teacher data of a machine learning model after specifying the first filter applied to the first image and eliminating features of the first filter. In this way, the computer allows medical images acquired in various imaging conditions to be efficiently identified.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an image conversion program, method, and device. [Background technology]

[0002] There is a growing burden on doctors due to a shortage of doctors and an increase in the number of images taken per examination, so there is a demand for greater efficiency in medical imaging examinations. It is expected that by allowing doctors to refer to the names of diseases and medical records of similar cases in the past, the time required for imaging examinations can be shortened and the efficiency improved.

[0003] In order to refer to similar cases from the past, it is conceivable to use machine learning to characterize cases from images and search for similar cases from the past based on those characteristics. However, machine learning that characterizes cases within images cannot identify images that are not included in the learning. Therefore, for medical images such as computed tomography (CT) images, whose image quality varies depending on the shooting conditions, it is necessary to generate a large number of pairs of images with various variations taken under expected shooting conditions and correct labels that indicate cases.

[0004] One method for generating a large number of pairs of images and correct labels is for doctors to manually create correct labels for cases in CT images based on their knowledge. Another method is known that generates images with a wide variety of variations by performing image processing such as rotation, inversion, translation, scale change, color change, and contrast change on training data that has already been correctly labeled. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2018-180729 A [Patent Document 2] JP 2019-56957 A [Non-patent literature]

[0006] [Non-Patent Document 1] Respiratory medicine explained through pictograms (https: / / respiresi.exblog.jp / 23965463 / ) [Non-Patent Document 2] Kenta Miura, Takuya Kita, "Construction of Flower Image Dataset and Convolutional Neural Networks", DEIM Forum 2017. [Non-Patent Document 3] A. Dosovitskiy, JT Springenberg, and T. Brox, “Unsupervisedfeature learning by augmenting single images,” arXiv, 2014. Summary of the Invention [Problem to be solved by the invention]

[0007] However, the conventional method has a problem that it may not be possible to efficiently generate pairs of appropriate learning images and correct labels. For example, a method in which a doctor manually creates correct labels requires manual work and a sufficient amount of images to be prepared in advance, and it may not be possible to efficiently generate pairs of images and correct labels. In addition, a method of generating images by image processing may generate images that do not exist as captured images, and it may not be possible to improve the recognition accuracy.

[0008] In one aspect, an object of the present invention is to provide an image conversion program, method, and device that can efficiently identify medical images captured under various imaging conditions. [Means for solving the problem]

[0009] In one embodiment, the image conversion program causes a computer to perform a process of converting a first image into a third image by identifying a first filter applied to a first image, canceling characteristics of the first filter, and applying a second filter applied to a second image used as training data for a machine learning model. Effect of the Invention

[0010] According to one aspect, medical images captured under various imaging conditions can be efficiently identified. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining a method for searching for similar cases. [Diagram 2] FIG. 2 is a diagram illustrating an example of the configuration of an image conversion system. [Diagram 3] FIG. 3 is a diagram illustrating an example of the image identification process. [Figure 4] FIG. 4 is a diagram illustrating an example of a data structure of the learning data. [Diagram 5] FIG. 5 is a diagram illustrating an example of the data structure of the shooting condition data. [Figure 6] FIG. 6 is a diagram illustrating an example of a data structure of the identification data. [Figure 7] FIG. 7 is a flowchart showing the flow of the image conversion and classification processing. [Figure 8] FIG. 8 is a flowchart showing the flow of the PSF estimation process. [Figure 9] FIG. 9 is a flowchart showing the flow of the frequency conversion process. [Figure 10] FIG. 10 is a flowchart showing the flow of the image conversion process. [Figure 11] FIG. 11 is a flowchart showing the flow of the image identification process. [Figure 12] FIG. 12 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the image conversion program, method, and device according to the present invention will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. Furthermore, each embodiment can be appropriately combined within a range that does not cause inconsistency.

[0013] First, an overview of the method for searching for similar cases using CT images will be explained using Figure 1. Figure 1 explains the method for searching for similar cases. As shown in Figure 1, cases are characterized using machine learning and image analysis from the CT image that serves as the search key, and candidates for abnormal shadows are extracted. The extracted candidates for abnormal shadows are then compared with the shadow data of past cases that have been saved in a database (DB), and a list of images similar to the CT image that is the search key is obtained as the search result.

[0014] Here, the accuracy of the search results for similar cases depends on the accuracy of the characterization of the cases from the CT images, which are compared with the similar cases. The characterization of cases from CT images is performed using a machine learning model constructed by learning training data in which the CT images are input and the correct labels indicating the cases are the correct answers. However, since CT images are taken under various imaging conditions, for example, a machine learning model constructed by learning a large number of CT images taken under imaging conditions different from the CT image to be identified cannot correctly identify the CT image to be identified.

[0015] Therefore, in the present invention, the characteristics of the imaging conditions of the CT image to be identified are cancelled, and the CT image to be identified is identified by applying the same imaging conditions as the CT image used in machine learning.

[0016] Next, an image conversion system for carrying out the present invention will be described. FIG. 2 is a diagram showing an example of the configuration of an image conversion system. As shown in FIG. 2, the image conversion system 1 has an image conversion device 10, a storage device 20, an image input device 30, and a computer 40. For example, the computer 40 is a terminal such as a personal computer operated by a user. Also, for example, the image conversion device 10, the storage device 20, and the image input device 30 are server computers. Also, some or all of the functions of each device of the image conversion system 1 may be configured as an integrated unit.

[0017] The image conversion device 10 cancels the characteristics of the shooting conditions of the image to be classified input from the image input device 30, such as a camera device, and applies the same shooting conditions as the image used in machine learning. This aims to equalize the image quality of the image to be classified and the image used in machine learning. Furthermore, the image conversion device 10 refers to information stored in the storage device 20 as necessary. The image conversion device 10 may perform processing in response to an operation performed via the computer 40.

[0018] The image conversion device 10 includes an estimation unit 101, a conversion unit 102, a classification unit 103, and a determination unit 104. The storage device 20 stores learning data 201, shooting condition data 202, and classification data 203.

[0019] The estimation unit 101 identifies a filter applied to a CT image to be identified. Here, the filter in this embodiment is, for example, a PSF (Point Spread Function). The PSF is applied to the image by a convolution operation. Therefore, for example, the estimation unit 101 identifies the PSF applied to the CT image by a convolution operation.

[0020] In addition, the PSF is associated with the shooting conditions. Therefore, the PSF applied to the captured image can be specified by estimating the shooting conditions when the captured image was captured. The shooting conditions include, for example, the type of imaging equipment and the reconstruction function.

[0021] As a specific example of a method for identifying a PSF, the estimation unit 101 can extract the name (type) of imaging equipment and the reconstruction function from tags attached to a DICOM (Digital Imaging and Communications in Medicine) image. DICOM is an international standard for the format of medical images such as CT images and the communication protocol for communicating medical image information. The "Manufacturer's Model Name" of the DICOM tag (0008, 1090) corresponds to the name of the imaging equipment. Also, the "Convolution Kernel" of the DICOM tag (0018, 1210) corresponds to the reconstruction function. The estimation unit 101 can identify the associated PSF from the imaging equipment and reconstruction function, which are the extracted imaging conditions.

[0022] A conventional estimation method can be used as a method for estimating the specified PSF. For example, the estimation unit 101 can estimate the PSF by capturing an image of a metal wire (phantom) (reference: Katsuhiro Ichikawa, Takanori Hara, Shinji Niwa, Kazuya Ohashi, "Method of Measuring MTF by Metal Wire in CT", Journal of the Japanese Radiological Society, 2008). As another example, the estimation unit 101 can estimate the PSF by using blood vessels in a CT image (reference: Yoshiki Kawata, Yoshihiro Nakaya, Noboru Niki, Hironobu Omatsu, Kenji Eguchi, Masahiro Kaneko, Noriyuki Moriyama, "Method of Measuring PSF from CT Images", Transactions of the Institute of Electronics, Information and Communication Engineers, 2008).

[0023] Furthermore, the estimation unit 101 may specify the PSF applied to the CT image used as training data of the machine learning model by the above method. Here, the machine learning model is a machine learning model constructed by inputting a CT image and learning training data in which a correct answer label indicating a case projected onto the CT image is used as a correct answer.

[0024] The conversion unit 102 converts the CT image to be classified into a CT image that can be classified by the machine learning model by canceling the characteristics of the applied PSF to the CT image to be classified and applying the PSF applied to the CT image used as training data for the machine learning model. To this end, the conversion unit 102 converts the PSF or CT image in the spatial domain into the spatial frequency domain by Fourier transform. The conversion unit 102 also converts the PSF or CT image in the spatial frequency domain into the spatial domain by inverse Fourier transform.

[0025] More specifically, the transform unit 102 transforms I(x, y), which is a CT image in the spatial domain, into the spatial frequency domain as shown in equation (1). F{} indicates that the value in the brackets is Fourier transformed.

[0026]

number

[0027] The transform unit 102 transforms P(x, y), which is the PSF in the spatial domain, into the spatial frequency domain as shown in equation (2).

[0028]

number

[0029] Here, for a CT image I(x,y), the image data obtained by the CT device before applying the PSF is O(x,y). The PSF is P(x,y). In this case, I(x,y) is expressed as in equation (3), where * is the convolution operator (filtering process).

[0030]

number

[0031] Then, as shown in equation (4), the conversion unit 102 cancels the PSF by dividing F{O(x,y)}·F{P(x,y)}, which indicates the Fourier transformed CT image, by F{P(x,y)}, which is the Fourier transform of the PSF applied to the CT image.

[0032]

number

[0033] Furthermore, the conversion unit 102 applies the PSF by multiplying F{P'(x,y)}, which indicates the PSF applied to the CT image used in the machine learning model, as shown in equation (4), and converts it to F{I'(x,y)}, which indicates a CT image that can be identified by the machine learning model.

[0034] In this way, a convolution operation in the spatial domain can be expressed as a multiplication in the spatial frequency domain, where a Fourier transformed PSF can be cancelled by dividing it and a Fourier transformed PSF can be applied by multiplying it.

[0035] Furthermore, the transform unit 102 transforms the transformed F{I'(x,y)} from the spatial frequency domain to the spatial domain by inverse Fourier transform, as shown in equation (5), to obtain I'(x,y).

[0036]

number

[0037] Here, we prove that the convolution operation in the spatial domain can be expressed as a multiplication in the spatial frequency domain, as in equation (6).

[0038]

number

[0039] First, the convolution operation is defined as shown in equation (7).

[0040]

number

[0041] Moreover, the discrete-time Fourier transform is defined as shown in equation (8).

[0042]

number

[0043] In this case, it is proven by equation (9) that the convolution operation transformed by the discrete-time Fourier transform can be expressed as a multiplication.

[0044]

number

[0045] The identification unit 103 identifies the CT image to be identified, which has been converted by the conversion unit 102, using a machine learning model constructed by inputting the CT image and learning training data in which the correct answer is the correct label indicating the case projected onto the CT image.

[0046] Fig. 3 is a diagram showing an example of image classification processing. As shown in Fig. 3, the classification unit 103 inputs the CT image to be classified, which has been converted by the conversion unit 102, to a classifier 50 having at least a machine learning model, thereby obtaining a classification label output from the classifier 50. The classifier 50 may be provided in the image conversion device 10, or may be provided in a device separate from the image conversion device 10.

[0047] The CT image to be identified here is I'(x,y) in formula (5). Since the PSF of the CT image used as training data for the machine learning model is applied to I'(x,y), I'(x,y) can be identified with high accuracy using the machine learning model. For example, even if the CT images used as training data for the machine learning model do not contain any images or contain only a few images taken under the same shooting conditions as the original image of I'(x,y), I'(x,y) can be identified without collecting new training data for the machine learning model.

[0048] Furthermore, the classification unit 103 classifies a plurality of CT images using a machine learning model in order to determine a PSF to be applied to the CT image to be classified.

[0049] The determination unit 104 determines the PSF applied to the CT image with the highest classification accuracy among the multiple CT images classified by the classification unit 103 as the PSF to be applied to the CT image to be classified. For example, assume that the classification accuracy of CT images taken under imaging conditions A to D is 78%, 80%, 90%, and 84%, respectively, as a result of classification by the machine learning model. In this case, since the classification accuracy of the imaging condition C is the highest, the PSF associated with the imaging condition C is determined as the PSF to be applied to the CT image to be classified. This allows the CT image to be classified with higher accuracy. Note that the classification accuracy of the machine learning model changes according to re-learning of the machine learning model, so the determination unit 104 may periodically determine the PSF to be applied. In addition, the determination unit 104 may determine the PSF to be applied in advance, rather than when the CT image to be classified is classified. This allows the CT image to be classified more quickly.

[0050] The learning data 201 includes information about learning images used for learning or relearning a machine learning model. The learning images are, for example, data of CT images captured by a CT device or the like to be used as teacher data for the machine learning model. Note that the learning images include images captured under various imaging conditions, and therefore the PSFs applied to each image are also various.

[0051] Fig. 4 is a diagram showing an example of a data structure of learning data. As shown in Fig. 4, learning data 201 includes a file name and a link. "File name" is the name of an image file. "Link" is the path to the storage location of the image file. In addition to the data items in Fig. 4, learning data 201 can also include, for example, a correct answer label indicating a case projected onto a CT image.

[0052] The imaging condition data 202 includes information on imaging conditions. For example, the imaging conditions are the type of CT equipment that captured the image and the reconstruction function. The imaging conditions may also include tube voltage and the like.

[0053] FIG. 5 is a diagram showing an example of the data structure of the imaging condition data. As shown in FIG. 5, the imaging condition data 202 includes an imaging device, a reconstruction function, spatial data, a link (spatial data), spatial frequency data, a link (spatial frequency data), and a transformation. The "imaging device" and the "reconstruction function" correspond to the imaging conditions. The "spatial data" is a flag for managing the presence or absence of a PSF in the spatial domain. The "link (spatial data)" is a storage location for the PSF data in the spatial domain. The "spatial frequency data" is a flag for managing the presence or absence of a PSF in the spatial frequency domain. The "link (spatial frequency data)" is a storage location for the PSF data in the spatial frequency domain. The storage location may be a location where the data is stored, or a location where the corresponding data is to be stored when it is generated. The "transformation" is a flag for managing the use in the transformation of the CT image to be identified. For example, in the example of FIG. 5, the imaging condition in which the imaging device is "D2" and the reconstruction function is "F4" indicates that the PSF in the spatial domain and the spatial frequency domain exists and is used in the transformation of the CT image to be identified.

[0054] The identification data 203 includes information about an image to be identified. The image to be identified is, for example, data of a CT image captured by a CT device or the like for a doctor to perform an examination. The images to be identified are images captured under various imaging conditions, and therefore various PSFs are applied.

[0055] Fig. 6 is a diagram showing an example of the data structure of identification data. As shown in Fig. 6, identification data 203 includes a file name and a link. The file name is the name of the image file. The link is the path to the storage location of the image file.

[0056] Next, a flow of the image conversion and classification processing executed by the image conversion device 10 will be described. Fig. 7 is a flow chart showing the flow of the image conversion and classification processing.

[0057] As shown in FIG. 7, first, the estimation unit 101 of the image conversion device 10 executes a PSF estimation process (step S1). Next, the conversion unit 102 of the image conversion device 10 executes a frequency conversion process (step S2). Then, the conversion unit 102 executes an image conversion process (step S3). Then, the classification unit 103 of the image conversion device 10 executes an image classification process (step S4) and outputs the classification result (step S5). Details of each process will be described.

[0058] First, the PSF estimation process will be described. Fig. 8 is a flowchart showing the flow of the PSF estimation process. The process in Fig. 8 corresponds to the PSF estimation process (step S1) in Fig. 7, which is executed by the estimation unit 101 of the image conversion device 10.

[0059] 8, the estimation unit 101 acquires an image used for learning the machine learning model from the storage device 20 (step S101). The image is, for example, a CT image captured in advance by a CT device and stored in a storage location in the storage device 20 indicated in the "link" of the learning data 201.

[0060] Next, the estimation unit 101 identifies the imaging conditions from the DICOM tag of the image acquired in step S101 (step S102). The imaging conditions include, for example, the type of CT equipment that captured the image, the reconstruction function, and the tube voltage.

[0061] Next, if there are other images used for learning (step S103: Yes), the estimation unit 101 returns to step S102 and repeats the process. On the other hand, if there are no other images used for learning (step S103: No), the estimation unit 101 proceeds to step S107.

[0062] In parallel with or before or after steps S101 to S103, the estimation unit 101 acquires an image of the classification target from the storage device 20 (step S104), and identifies the imaging conditions from the DICOM tag of the image (step S105). The image is, for example, a CT image captured in advance by a CT device for a doctor to perform an examination, and stored in a storage location of the storage device 20 indicated in the "link" of the identification data 203.

[0063] Furthermore, if there are other images to be classified (step S106: Yes), the estimation unit 101 returns to step S105 and repeats the process, and if there are no other images to be classified (step S106: No), the estimation unit 101 proceeds to step S107.

[0064] Proceeding to the No route of steps S103 and S106, the estimation unit 101 determines whether or not data in which the "spatial data" flag of the PSF corresponding to the shooting condition is 0 exists in the shooting condition data 202 in the storage device 20 (step S107).

[0065] If there is data with the "spatial data" flag set to 0 (step S107: Yes), the corresponding PSF has not yet been identified, so the estimation unit 101 identifies the corresponding PSF, sets the "spatial data" flag to 1 (step S108), and proceeds to step S109. On the other hand, if there is no data with the "spatial data" flag set to 0 (step S107: No), the corresponding PSF has already been identified, so the process proceeds to step S109.

[0066] Next, if there is a shooting condition for which processing has not been performed (step S109: Yes), the estimation unit 101 returns to step S107 and repeats the process. On the other hand, if there is no shooting condition for which processing has not been performed (step S109: No), the determination unit 104 determines a PSF for converting the image to be classified (step S110).

[0067] Here, the PSF for converting the image to be classified is the PSF applied to the image used for learning the machine learning model acquired in step S101. As described above, for example, the PSF applied to the image with the highest classification accuracy of the machine learning model among the multiple images is determined as the PSF for converting the image to be classified. By applying the determined PSF to the image to be classified, the image to be classified by the machine learning model can be classified.

[0068] Then, the determining unit 104 sets the "conversion" flag of the PSF to be used to 1 (step S111). After step S111, the processing in FIG. 8 ends.

[0069] Note that the PSF determination process (steps S110 and S111) by the determination unit 104 does not need to be performed in the PSF estimation process, and can be performed in advance. For example, in the PSF estimation process, only the PSF corresponding to the shooting conditions of the image to be classified is specified, and the PSF for converting the image to be classified, that is, the PSF applied to the image used for learning the machine learning model, can be determined in advance as a separate process. When performed as a separate process, steps S101 to S103 are performed, and the PSF of each image is specified, and then S110 and S111 are performed. In this case, the PSF estimation process (step S1) does not need to include steps S101 to S103, S110, and S111 that are performed as separate processes.

[0070] Next, the frequency conversion process will be described. Fig. 9 is a flowchart showing the flow of the frequency conversion process. The process in Fig. 9 corresponds to the frequency conversion process (step S2) in Fig. 7, which is executed by the conversion unit 102 of the image conversion device 10.

[0071] First, as shown in FIG. 9, the conversion unit 102 acquires an image to be classified from the storage device 20 (step S201), performs frequency conversion, and stores the image in the storage device 20 (step S202).

[0072] Next, if there is an image that has not been processed (step S203: Yes), the conversion unit 102 returns to step S202 and repeats the process. On the other hand, if there is no image that has not been processed (step S203: No), the process of FIG. 9 ends.

[0073] In parallel with or before or after steps S201 to S203, the conversion unit 102 acquires the shooting conditions from the storage device 20 (step S204), and determines whether the “conversion” flag of the PSF corresponding to the shooting conditions is 1 and the “spatial frequency data” flag is 0 (step S205).

[0074] If the "conversion" flag of the PSF corresponding to the shooting conditions is 1 and the "spatial frequency data" flag is 0 (step S205: Yes), the conversion unit 102 frequency-converts the corresponding PSF, stores it in the storage device 20, and sets the "spatial frequency data" flag to 1 (step S206). Hereinafter, the stored PSF that has been converted into the frequency domain is referred to as spatial frequency data. On the other hand, if the "conversion" flag of the PSF corresponding to the shooting conditions is not 1 or the "spatial frequency data" flag is not 0 (step S205: No), the process proceeds to step S207.

[0075] Next, if there is a shooting condition for which processing has not been performed (step S207: Yes), the conversion unit 102 returns to step S205 and repeats the process. On the other hand, if there is no shooting condition for which processing has not been performed (step S207: No), the process of FIG. 9 is terminated.

[0076] Next, the image conversion process will be described. Fig. 10 is a flowchart showing the flow of the image conversion process. The process in Fig. 10 corresponds to the image conversion process (step S3) in Fig. 7, which is executed by the conversion unit 102 of the image conversion device 10.

[0077] 10, the conversion unit 102 acquires a frequency conversion result of an image to be classified from the storage device 20 (step S301). The classification unit 103 acquires a frequency conversion result of a PSF corresponding to the shooting conditions of the image to be classified from the storage device 20 (step S302).

[0078] Next, the conversion unit 102 cancels the characteristics of the shooting conditions of the image to be classified (step S303). As described above, this can be canceled by performing a Fourier transform on the image to be classified and the identified PSF, and dividing the image to be classified by the identified PSF.

[0079] Next, the conversion unit 102 acquires the shooting conditions from the shooting condition data 202 in the storage device 20 (step S304), and determines whether the "conversion" flag is 1 (step S305).

[0080] If the "conversion" flag is not 1 (step S305: No), the process proceeds to step S308. On the other hand, if the "conversion" flag is 1 (step S305: Yes), the conversion unit 102 acquires spatial frequency data corresponding to the shooting conditions (step S306). The acquired spatial frequency data is the PSF converted to the frequency domain and stored in the storage device 20 in step S206 of the frequency conversion process in FIG. 9.

[0081] Next, the conversion unit 102 applies the spatial frequency data to the image of the classification target from which the characteristics of the shooting conditions have been cancelled in step S303, and stores the result in the storage device 20 (step S307). As described above, this can be applied by multiplying the image of the classification target from which the characteristics of the shooting conditions have been cancelled by the Fourier transform with the spatial frequency data.

[0082] Next, if there is an unprocessed shooting condition (step S308: Yes), the conversion unit 102 returns to step S305 and repeats the process. On the other hand, if there is no unprocessed shooting condition (step S308: No), the conversion unit 102 proceeds to step S309.

[0083] Next, if there is an image to be classified that has not been processed (step S309: Yes), the classification unit 103 returns to step S301 and repeats the process. On the other hand, if there is no image to be classified that has not been processed (step S309: No), the process of FIG. 10 ends.

[0084] Next, the image identification process will be described. Fig. 11 is a flowchart showing the flow of the image identification process. The process in Fig. 11 corresponds to the image identification process (step S4) in Fig. 7 executed by the identification unit 103 of the image conversion device 10.

[0085] First, the classification unit 103 acquires an image to be classified obtained by the image conversion process from the storage device 20 (step S401), as shown in Fig. 11. The image acquired here is an image to which a PSF that can be classified by a machine learning model has been applied, which has been stored in the storage device 20 in step S307 of the image conversion process in Fig. 10.

[0086] Next, the classification unit 103 acquires the classifier 50 generated by machine learning from the storage device 20 (step S402). As described above, the classifier 50 includes a machine learning model constructed by inputting, for example, a CT image and learning training data in which a correct answer label indicating a case projected onto the CT image is taken as a correct answer.

[0087] Next, the classification unit 103 classifies the image to be classified, obtained by the image conversion process, by inputting the image to the classifier 50 (step S403), and obtains a classification label output from the classifier 50 as a classification result.

[0088] Next, if there is an image to be classified that has not been processed (step S404: Yes), the classification unit 103 returns to step S401 and repeats the process. On the other hand, if there is no image to be classified that has not been processed (step S404: No), the process of FIG. 11 ends.

[0089] [effect] As described above, the image conversion device 100 converts the first image into a third image by identifying a first filter applied to the first image, canceling the characteristics of the first filter, and applying a second filter applied to a second image used as training data for a machine learning model.

[0090] In this way, the image conversion device 100 applies a filter of an image used as training data of a machine learning model to an image to be classified. As a result, the image conversion device 100 can classify an image to be classified without collecting new training data for a machine learning model even if there are no images or only a few images taken under the same shooting conditions as the image to be classified among the images used as training data of the machine learning model. In other words, the image conversion device 100 can efficiently classify images taken under various shooting conditions.

[0091] Moreover, the image conversion device 100 identifies a third image using a machine learning model.

[0092] This allows the image conversion device 100 to efficiently distinguish between images captured under various shooting conditions.

[0093] Moreover, the image conversion device 100 inputs a medical image including the second image, and identifies a third image using a machine learning model constructed by learning training data that takes a correct answer label indicating a case as a correct answer.

[0094] This allows the image conversion device 100 to efficiently distinguish between medical images captured under various imaging conditions.

[0095] Moreover, the image conversion device 100 specifies a first PSF as a first filter, and applies a second PSF as a second filter.

[0096] As a result, the image conversion device 100 can efficiently classify medical images even if the medical images used as training data for the machine learning model contain few medical images captured under the same shooting conditions as the medical image to be classified.

[0097] Furthermore, the image conversion device 100 converts the first image into the spatial frequency domain by a Fourier transform, and cancels the characteristics of the first filter by dividing the first image converted into the spatial frequency domain by a Fourier transform of the first PSF, and applies the second filter by multiplying the first image converted into the spatial frequency domain by a Fourier transform of the second PSF.

[0098] This enables the image conversion device 100 to more efficiently classify medical images even when the medical images used as training data for the machine learning model contain few medical images captured under the same shooting conditions as the medical image to be classified.

[0099] Moreover, the image conversion device 100 identifies a first filter associated with the first shooting condition, cancels the characteristics of the first filter, and applies a second filter associated with the second shooting condition, thereby converting the first image into a third image.

[0100] As a result, the image conversion device 100 can efficiently classify the image to be classified even if the images used as training data for the machine learning model contain few images captured under the same shooting conditions as those of the image to be classified.

[0101] Furthermore, the image conversion device 100 identifies a first filter associated with a first shooting condition including at least a first shooting device and a first reconstruction function, cancels the characteristics of the first filter, and applies a second filter associated with a second shooting condition including at least a second shooting device and a second reconstruction function, thereby converting the first image into a third image.

[0102] This makes it possible to efficiently identify the image to be classified even if the images used as training data for the machine learning model include few images captured using the same imaging equipment and reconstruction function as the imaging conditions of the image to be classified.

[0103] Moreover, the image conversion device 100 acquires the first imaging device and the first reconstruction function from the tag attached to the first image, which is a DICOM image.

[0104] This enables the image conversion device 100 to more easily estimate the imaging conditions and PSF of a medical image.

[0105] Furthermore, the image conversion device 100 classifies a plurality of images using a machine learning model, and determines the third filter applied to an image having the highest classification accuracy among the plurality of images as the second filter.

[0106] This allows the image conversion device 100 to classify the image to be classified with higher accuracy.

[0107] The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified. In addition, the specific examples, distributions, values, etc. described in the embodiments are merely examples and can be changed as desired.

[0108] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads and usage conditions. Furthermore, each processing function performed by each device can be realized in whole or in any part by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0109] Fig. 12 is a diagram for explaining an example of a hardware configuration. As shown in Fig. 12, an image conversion device 10 has a communication interface 10a, a HDD (Hard Disk Drive) 10b, a memory 10c, and a processor 10d. Moreover, each unit shown in Fig. 12 is connected to each other by a bus or the like.

[0110] The communication interface 10a is a network interface card or the like, and communicates with other servers. The HDD 10b stores programs and DBs that operate the functions shown in FIG.

[0111] The processor 10d is a hardware circuit that reads out a program that executes the same processes as the respective processing units shown in FIG. 2 from the HDD 10b or the like and loads it in the memory 10c, thereby operating a process that executes each function described in FIG. 2 or the like. That is, this process executes the same functions as the respective processing units of the image conversion device 10. Specifically, the processor 10d reads out a program having the same functions as the estimation unit 101, the conversion unit 102, the classification unit 103, and the determination unit 104 from the HDD 10b or the like. Then, the processor 10d executes a process that executes the same processes as the estimation unit 101, the conversion unit 102, the classification unit 103, the determination unit 104, and the like.

[0112] In this way, the image conversion device 10 operates as an information processing device that executes a learning method by reading and executing a program. The image conversion device 10 can also realize functions similar to those of the above-mentioned embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the image conversion device 10. For example, the present invention can be similarly applied to cases where another computer or server executes a program, or where these cooperate to execute a program.

[0113] This program can be distributed via a network such as the Internet. In addition, this program can be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a MO (Magneto-Optical disk), or a DVD (Digital Versatile Disc), and can be executed by being read out from the recording medium by a computer.

[0114] The following supplementary notes are further disclosed regarding the embodiments including the above examples.

[0115] (Appendix 1) To the computer, Identifying a first filter applied to a first image; Transforming a first image into a third image by applying a second filter that cancels the characteristics of the first filter and is applied to a second image that is used as training data for a machine learning model. 2. An image conversion program for executing a process.

[0116] (Appendix 2) The image conversion program according to appendix 1, further comprising causing a computer to execute a process of identifying a third image using a machine learning model.

[0117] (Appendix 3) The image conversion program according to appendix 1, further comprising having a computer input a medical image including a second image, and further execute a process of identifying a third image using a machine learning model constructed by learning training data in which a correct label indicating a case is the correct answer.

[0118] (Appendix 4) To the computer, A first PSF (Point Spread Function) is specified as a first filter; Apply the second PSF as the second filter 2. The image conversion program according to claim 1,

[0119] (Appendix 5) To the computer, Transforming the first image into the spatial frequency domain by a Fourier transform; canceling the characteristic of the first filter by dividing the first image converted into the spatial frequency domain by a Fourier transform of the first PSF; A second filter is applied to the first image, transformed into the spatial frequency domain, by multiplying the Fourier transform of the second PSF. 5. The image conversion program according to claim 4, further comprising:

[0120] (Appendix 6) To the computer, Identifying a first filter associated with a first photographing condition; The first image is converted into a third image by canceling the characteristics of the first filter and applying a second filter corresponding to a second photographing condition. 2. The image conversion program according to claim 1, which causes a process to be executed.

[0121] (Appendix 7) To the computer, Identifying a first filter associated with a first imaging condition including at least a first imaging device and a first reconstruction function; The first image is converted into a third image by applying a second filter that cancels the characteristics of the first filter and corresponds to a second imaging condition including at least a second imaging device and a second reconstruction function. 2. The image conversion program according to claim 1, which causes a process to be executed.

[0122] (Appendix 8) The image conversion program according to appendix 7, further comprising causing a computer to execute a process of acquiring a first imaging device and a first reconstruction function from a tag attached to a first image, which is a DICOM (Digital Imaging and Communications in Medicine) image.

[0123] (Appendix 9) A computer uses a machine learning model to identify multiple images, The image conversion program according to claim 1, further comprising a process of determining a third filter applied to an image having the highest classification accuracy among a plurality of images as the second filter.

[0124] (Appendix 10) The computer: Identifying a first filter applied to a first image; Transforming a first image into a third image by applying a second filter that cancels the characteristics of the first filter and is applied to a second image that is used as training data for a machine learning model. A method comprising:

[0125] (Appendix 11) The method of appendix 10, further comprising the computer performing a process of identifying a third image using a machine learning model.

[0126] (Appendix 12) The method described in Appendix 10, characterized in that the computer further performs a process of identifying a third image using a machine learning model constructed by inputting a medical image including the second image and learning training data in which a correct label indicating a case is the correct answer.

[0127] (Appendix 13) The computer: A first point spread function (PSF) is identified as a first filter; Apply the second PSF as the second filter 11. The method according to claim 10,

[0128] (Appendix 14) The computer: Transforming the first image into the spatial frequency domain by a Fourier transform; canceling the characteristic of the first filter by dividing the first image converted into the spatial frequency domain by a Fourier transform of the first PSF; A second filter is applied to the first image, transformed into the spatial frequency domain, by multiplying the Fourier transform of the second PSF. 14. The method of claim 13, further comprising performing processing.

[0129] (Appendix 15) The computer: Identifying a first filter associated with a first photographing condition; The first image is converted into a third image by canceling the characteristics of the first filter and applying a second filter corresponding to a second photographing condition. 11. The method of claim 10, further comprising:

[0130] (Appendix 16) The computer: Identifying a first filter associated with a first imaging condition including at least a first imaging device and a first reconstruction function; The first image is converted into a third image by applying a second filter that cancels the characteristics of the first filter and corresponds to a second imaging condition including at least a second imaging device and a second reconstruction function. 11. The method of claim 10, further comprising:

[0131] (Appendix 17) The method described in Appendix 16, characterized in that the computer further performs a process of obtaining a first imaging device and a first reconstruction function from a tag attached to a first image that is a DICOM (Digital Imaging and Communications in Medicine) image.

[0132] (Appendix 18) The computer uses a machine learning model to identify a plurality of images, The method described in Appendix 10, further comprising a process of determining a third filter applied to an image having the highest classification accuracy among the multiple images as the second filter.

[0133] (Supplementary Note 19) An estimation unit that identifies a first filter applied to a first image; a conversion unit that converts the first image into a third image by applying a second filter that cancels out the characteristics of the first filter and is applied to a second image that is used as training data for the machine learning model; An apparatus comprising:

[0134] (Supplementary Note 20) The device described in Supplementary Note 19, further comprising an identification unit that identifies a third image using a machine learning model.

[0135] (Appendix 21) The apparatus described in Appendix 19, further comprising an identification unit that identifies a third image using a machine learning model constructed by inputting a medical image including the second image and learning training data in which a correct label indicating a case is the correct answer.

[0136] (Supplementary Note 22) The estimation unit specifies a first PSF (Point Spread Function) as a first filter; The transform unit applies a second PSF as a second filter. 20. The device of claim 19.

[0137] (Appendix 23) The conversion unit further: Transforming the first image into the spatial frequency domain by a Fourier transform; canceling the characteristic of the first filter by dividing the first image converted into the spatial frequency domain by a Fourier transform of the first PSF; A second filter is applied to the first image, transformed into the spatial frequency domain, by multiplying the Fourier transform of the second PSF. 23. The apparatus of claim 22.

[0138] (Supplementary Note 24) The estimation unit specifies a first filter associated with a first shooting condition; The conversion unit converts the first image into a third image by canceling the characteristics of the first filter and applying a second filter associated with a second photographing condition. 20. The device of claim 19,

[0139] (Supplementary Note 25) The estimation unit specifies a first filter associated with a first imaging condition including at least a first imaging device and a first reconstruction function; The conversion unit converts the first image into a third image by canceling the characteristics of the first filter and applying a second filter associated with a second imaging condition including at least a second imaging device and a second reconstruction function. 20. The device of claim 19.

[0140] (Appendix 26) The device described in Appendix 25, characterized in that the estimation unit further obtains the first imaging device and the first reconstruction function from a tag attached to the first image, which is a DICOM (Digital Imaging and Communications in Medicine) image.

[0141] (Appendix 27) The identification unit uses a machine learning model to identify a plurality of images, The device described in Supplementary Note 19, further comprising a determination unit that determines a third filter applied to an image having the highest classification accuracy among the multiple images as the second filter.

[0142] (Appendix 28) A processor; a memory operatively connected to the processor; An information processing device comprising: an estimator for identifying a first filter applied to a first image; a conversion unit that converts the first image into a third image by applying a second filter that cancels out the characteristics of the first filter and is applied to a second image that is used as training data for the machine learning model; An apparatus comprising: [Explanation of symbols]

[0143] 1 Image conversion system 10 Image conversion device 20 Storage device 30 Image input device 40 calculator 50 Classifier 101 Estimation part 102 Conversion unit 103 Identification unit 104 Decision Section 201 Training Data 202 Shooting condition data 203 Identification Data

Claims

1. On the computer, Identifying a shooting condition of a first image that is to be identified by the machine learning model learned by machine learning; Executing a process of converting the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model, and that has uniform image quality between an image to be identified and an image used for machine learning, by canceling out an influence of the shooting conditions; The process of specifying specifying a first PSF (Point Spread Function) as a first filter associated with a photographing condition of the first image; The conversion process includes:

2. An image conversion program comprising: a computer program product for converting an image into a second image; and a computer program product for converting an image into a second image.

2. The image conversion program according to claim 1 , further comprising causing the computer to execute a process of identifying the third image using the machine learning model.

3. The image conversion program according to claim 1, further comprising: inputting a medical image including the second image to the computer; and using the machine learning model constructed by learning the training data in which a correct label indicating a case is a correct answer, the computer is caused to execute a process of identifying the third image.

4. The computer includes: Transforming the first image into the spatial frequency domain by a Fourier transform; canceling the characteristic of the first filter by dividing the first image transformed into the spatial frequency domain by a Fourier transform of the first PSF; applying the second filter by multiplying the first image transformed into the spatial frequency domain by a Fourier transform of the second PSF; 2. The image conversion program according to claim 1, further comprising: a program for executing a process.

5. A computer comprising: Identifying a shooting condition of a first image that is to be identified by the machine learning model learned by machine learning; Executing a process of converting the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model, and that has uniform image quality between an image to be identified and an image used for machine learning, by canceling out an influence of the shooting conditions; The process of specifying Identifying a first filter associated with a first photographing condition that is a photographing condition of the first image; The conversion process includes: an image conversion program for converting the first image into a third image by canceling out characteristics of the first filter and applying a second filter corresponding to second shooting conditions which are shooting conditions of the second image.

6. A computer comprising: Identifying a shooting condition of a first image that is to be identified by the machine learning model learned by machine learning; Executing a process of converting the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model, and that has uniform image quality between an image to be identified and an image used for machine learning, by canceling out an influence of the shooting conditions; The process of specifying Identifying a first filter associated with a first imaging condition, which is an imaging condition of the first image, the first filter including at least a first imaging device and a first reconstruction function; The conversion process includes: an image conversion program for converting the first image into a third image by canceling a characteristic of the first filter and applying a second filter associated with a second photographing condition that is a photographing condition of the second image, the second filter including at least a second photographing device and a second reconstruction function.

7. 7. The image conversion program according to claim 6, further causing the computer to execute a process of acquiring the first imaging device and the first reconstruction function from a tag attached to the first image, which is a DICOM (Digital Imaging and Communications in Medicine) image.

8. The computer includes: Using the machine learning model to identify a plurality of images; 6. The image conversion program according to claim 5, further comprising a process of determining, as the second filter, a third filter applied to the image having the highest classification accuracy among the plurality of images.

9. The computer Identifying a shooting condition of a first image that is to be identified by the machine learning model learned by machine learning; Executing a process of converting the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model, and in which image quality of an image to be identified and an image used for machine learning are uniformed, by canceling out an influence of the shooting conditions of the first image; The process of specifying specifying a first PSF (Point Spread Function) as a first filter associated with a photographing condition of the first image; The conversion process includes: applying a second PSF as a second filter associated with a capture condition of the second image.

10. an estimation unit that specifies a shooting condition of a first image that is an object to be classified by a machine learning model learned by machine learning; a conversion unit that converts the first image into a third image, which is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model by canceling out an influence of the shooting conditions, and which equalizes image quality between an image to be classified and an image used for machine learning; The estimation unit is specifying a first PSF (Point Spread Function) as a first filter associated with a photographing condition of the first image; The conversion unit is and applying a second PSF as a second filter associated with a capture condition of the second image.

11. A computer comprising: Identifying a shooting condition of a first image that is to be identified by the machine learning model learned by machine learning; Executing a process of converting the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model, and in which image quality of an image to be identified and an image used for machine learning are uniformed, by canceling out an influence of the shooting conditions of the first image; The process of specifying Identifying a first filter associated with a first photographing condition that is a photographing condition of the first image; The conversion process includes: and converting the first image into a third image by canceling the characteristics of the first filter and applying a second filter associated with second image capture conditions that are image capture conditions of the second image.

12. An estimation unit that identifies a shooting condition of a first image that is an object to be identified by a machine learning model trained by machine learning; a conversion unit that converts the first image into a third image, which is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model by canceling out an influence of the shooting conditions, and which equalizes image quality between an image to be classified and an image used for machine learning; The estimation unit is Identifying a first filter associated with a first photographing condition that is a photographing condition of the first image; The conversion unit is and converting the first image into a third image by canceling the characteristics of the first filter and applying a second filter associated with second shooting conditions that are shooting conditions of the second image.

13. A computer comprising: Identifying a shooting condition of a first image that is to be identified by the machine learning model learned by machine learning; Executing a process of converting the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model, and in which image quality of an image to be identified and an image used for machine learning are uniformed, by canceling out an influence of the shooting conditions of the first image; The process of specifying Identifying a first filter associated with a first imaging condition, which is an imaging condition of the first image, the first filter including at least a first imaging device and a first reconstruction function; The conversion process includes: converting the first image into a third image by canceling a characteristic of the first filter and applying a second filter associated with a second imaging condition, the second imaging condition including at least a second imaging device and a second reconstruction function.

14. An estimation unit that identifies a shooting condition of a first image that is an object to be identified by a machine learning model trained by machine learning; A conversion unit converts the first image into a third image that is an image captured under the same shooting conditions as a second image used as training data for the machine learning of the machine learning model by canceling out the influence of the shooting conditions, and that equalizes image quality between an image to be classified and an image used for machine learning, The estimation unit is Identifying a first filter associated with a first imaging condition, which is an imaging condition of the first image, the first filter including at least a first imaging device and a first reconstruction function; The conversion unit is and converting the first image into a third image by applying a second filter corresponding to a second imaging condition, the second imaging condition including at least a second imaging device and a second reconstruction function, and canceling a characteristic of the first filter.

Images