Information Processing Apparatus, Information Processing Method, and Information Processing Program

Abstract

To provide a device, method and program for processing information, which allow for utilizing even an image not suitable for extraction of a region of interest for diagnosis.SOLUTION: An information processing device 10 disclosed herein comprises at least one processor configured to train a learning model for extracting a region of interest from an input image by inputting, as learning data, a pseudo image generated by modifying pixel values of at least a portion of an original image obtained by photographing an object.SELECTED DRAWING: Figure 17

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, a doctor generally makes a diagnosis based on medical images obtained by an image acquisition apparatus such as a CT (Computed Tomography) apparatus and an MRI (Magnetic Resonance Imaging) apparatus. In addition, a technique (so-called CAD (Computer Aided Detection / Diagnosis)) in which a computer supports the detection and diagnosis of abnormal shadows and structures such as tissues included in medical images is also known. For example, Patent Document 1 describes analyzing a medical image using a discriminator learned by machine learning and specifying the type of tissue or lesion included in the medical image, that is, the type of finding.

[0003] Also, for example, Non-Patent Document 1 describes not performing CAD when at least a part of a medical image is unclear in order to avoid non-detection and misdiagnosis of abnormal shadows. Unclear medical images can be obtained, for example, when they are not taken in an appropriate environment and when the photography technique of the photographer is low.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Non-Patent Documents

[0005]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] By the way, in medical institutions in emerging countries, for example, it may be difficult to obtain clear medical images because an appropriate environment for taking medical images cannot be created or the imaging technique of the imager is low. Therefore, in recent years, there has been a demand for a technology that can be used for diagnosis even with unclear images, that is, images that are not suitable for extracting regions of interest that may include abnormal shadows and structures such as tissues.

[0007] The present disclosure provides an information processing apparatus, an information processing method, and an information processing program that can be used for diagnosis even with images that are not suitable for extracting regions of interest.

Means for Solving the Problems

[0008] A first aspect of the present disclosure is an information processing apparatus including at least one processor, the processor being a pseudo-image generated by changing pixel values of at least a part of a region including at least a region of interest in an original image obtained by photographing a subject, and by inputting a plurality of pseudo-images generated by changing pixel values in different regions for one original image as learning data, a learning model for extracting a region of interest from the input image is learned The learning model takes as input an image that includes a blurred region where at least one of the hue, saturation, brightness, and lightness indicated by each pixel does not meet a predetermined reference value, and outputs a region of interest included in the input image. .

[0009] In the first aspect, the processor may learn the learning model by inputting a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image as learning data.

[0010] In the first aspect described above, when the learning model fails to extract the region of interest from the input pseudo-image, the processor may re-train the learning model by re-inputting the pseudo-image as learning data.

[0011] In the first aspect described above, when the learning model fails to extract the region of interest from the input pseudo-image, the processor may re-train the learning model by re-inputting a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image as learning data.

[0012] In the first aspect described above, the pseudo-image may be an image generated by changing pixel values so that the image quality deteriorates in at least a part of the original image.

[0013] In the first aspect described above, the pseudo-image may be an image generated by changing at least a part of the pixel values of the original image while maintaining the presence or absence of the structure included in the original image.

[0016] In the first aspect described above, the pseudo-image may be an image generated based on the original image using a GAN (Generative Adversarial Networks).

[0017] In the first aspect described above, the pixel value may be a value indicating at least one of the hue, saturation, brightness, and lightness shown by each pixel.

[0018] In the first aspect described above, the region of interest may be a region including at least one of a subject, a part of the tissue included in the subject, and an abnormal part included in the subject or the tissue.

[0019] In the first aspect described above, the original image may be an image obtained by at least one of a radiation imaging device, a magnetic resonance imaging device, an ultrasonic device, an ophthalmic imaging device, and an endoscope. In the first aspect, the processor inputs a plurality of pseudo-images as learning data to train the learning model through unsupervised learning. When the learning model fails to extract a region of interest from the input pseudo-image, the processor retrains the learning model through supervised learning by re-inputting as learning data a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image.

[0020] A second aspect of the present disclosure is an information processing method, which is a pseudo-image generated by changing pixel values of at least a partial region including at least a region of interest in an original image obtained by photographing a subject, and a plurality of pseudo-images generated by changing pixel values in different regions for one original image are used as learning data to input, thereby training a learning model for extracting a region of interest from the input image. The learning model takes as input an image that includes a blurred region where at least one of the hue, saturation, brightness, and lightness indicated by each pixel does not meet a predetermined reference value, and outputs a region of interest included in the input image. The processing is executed by a computer.

[0021] In the above second aspect, the computer may execute a process of training the learning model by inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image.

[0022] In the above second aspect, when the learning model fails to extract the region of interest from the input pseudo-image, the computer may execute a process of re-training the learning model by re-inputting the pseudo-image as learning data.

[0023] In the above second aspect, when the learning model fails to extract the region of interest from the input pseudo-image, the computer may execute a process of re-training the learning model by re-inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image.

[0024] A third aspect of the present disclosure is an information processing program, which is a pseudo-image generated by changing pixel values of at least a partial region including at least a region of interest in an original image obtained by photographing a subject, and a plurality of pseudo-images generated by changing pixel values in different regions for one original image are used as learning data to input, thereby training a learning model for extracting a region of interest from the input image. The learning model takes as input an image that includes a blurred region where at least one of the hue, saturation, brightness, and lightness indicated by each pixel does not meet a predetermined reference value, and outputs a region of interest included in the input image. It is for causing a computer to execute the processing.

[0025] In the above-described third aspect, a process of causing a computer to train a learning model may be performed by inputting a pair of a pseudo image and information indicating a region of interest included in the pseudo image as learning data.

[0026] In the above-described third aspect, a process of causing a computer to re-train a learning model may be performed by re-inputting the pseudo image as learning data when the learning model fails to extract a region of interest from the input pseudo image.

[0027] In the above-described third aspect, a process of causing a computer to re-train a learning model may be performed by re-inputting a pair of the pseudo image and information indicating a region of interest included in the pseudo image as learning data when the learning model fails to extract a region of interest from the input pseudo image.

Advantages of the Invention

[0028] According to the above aspect, the information processing apparatus, information processing method, and information processing program of the present disclosure can be utilized for diagnosis even with an image unsuitable for extraction of a region of interest.

Brief Description of the Drawings

[0029]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Mode for Carrying Out the Invention

[0030] Hereinafter, with reference to the drawings, an example of a mode for carrying out the technology of the present disclosure will be described in detail.

[0031] [First Embodiment] With reference to FIG. 1, an example of the configuration of the information processing system 1 according to the present embodiment will be described. As shown in FIG. 1, the information processing system 1 includes an information processing apparatus 10 and an image acquisition apparatus 2. The information processing apparatus 10 and the image acquisition apparatus 2 are capable of communicating with each other by wired or wireless communication.

[0032] The image acquisition apparatus 2 is a device (so-called modality) that acquires an image obtained by photographing a subject. In the present embodiment, as an example of a specific image, it will be described that the image acquisition apparatus 2 acquires a medical image. As the image acquisition apparatus 2, at least one of a radiation image imaging apparatus, a magnetic resonance imaging apparatus, an ultrasonic apparatus, a fundus imaging apparatus, and an endoscope can be applied, and these may be applied in appropriate combination.

[0033] By the way, for example, in order to clearly photograph the fundus of the eye, the photographing should be performed in a dark place. However, in medical institutions in emerging countries, etc., it may not be possible to create a sufficient dark place, and unclear medical images may be taken. Also, for example, in mammography, in order to clearly photograph the breast, the breast should be sufficiently compressed for photographing. However, depending on the positioning technique of the photographer and the shape of the breast, the breast may not be sufficiently compressed, and unclear medical images may be taken. In recent years, there has been a demand for a technology that can utilize such unclear images for diagnosis.

[0034] Therefore, the information processing apparatus 10 according to the present embodiment has a function of determining whether or not an unclear region is included in a medical image. Note that "unclear" means a case where pixel values such as hue, saturation, luminance, and lightness indicated by each pixel of the medical image do not satisfy a predetermined reference value. For example, an unclear region may occur when the pixel becomes darker than the reference value due to insufficient light amount, or when the pixel becomes brighter than the reference value due to ambient light. Hereinafter, an example of the configuration of the information processing apparatus 10 according to the present embodiment will be described.

[0035] First, with reference to FIG. 2, an example of the hardware configuration of the information processing apparatus 10 according to the present embodiment will be described. As shown in FIG. 2, the information processing apparatus 10 includes a CPU (Central Processing Unit) 21, a nonvolatile storage unit 22, and a memory 23 as a temporary storage area. The information processing apparatus 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F 26 that performs wired or wireless communication with the image acquisition device 2 and an external network (not shown). The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected to each other via a bus 28 such as a system bus and a control bus so as to be able to exchange various information.

[0036] The storage unit 22 is realized by a storage medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash memory. The information processing program 27 according to the present embodiment is stored in the storage unit 22. The CPU 21 reads the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of the processor of the present disclosure. As the information processing apparatus 10, various computers such as a console of the image acquisition apparatus 2, a workstation, a server computer, and a personal computer can be applied.

[0037] Next, with reference to FIG. 3, an example of the functional configuration of the information processing apparatus 10 according to the present embodiment will be described. As shown in FIG. 3, the information processing apparatus 10 includes an acquisition unit 11, an extraction unit 12, a determination unit 13, a detection unit 14, and a presentation unit 15. By executing the information processing program 27, the CPU 21 functions as the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, and the presentation unit 15.

[0038] The acquisition unit 11 acquires a medical image obtained by photographing a subject from the image acquisition apparatus 2. The medical image in the present embodiment includes a region of interest including at least one of a subject, a part of the tissue included in the subject, and an abnormal part included in the subject or the tissue, as a structure of an object for which detection and diagnosis are desired. Examples of the subject include a human body and various organs of the human body such as the fundus of the eye, the lung, the breast, the stomach, the liver, the heart, and the brain. Examples of the tissue include elements constituting various organs such as blood vessels, nerves, and muscles. Examples of the abnormal part include lesions and abnormalities such as tumors, injuries, defects, nodules, and inflammations.

[0039] FIG. 4 shows a medical image G0 as an example of a clear medical image. The medical image G0 is a clear fundus image obtained by a fundus imaging device. In the fundus image, since an abnormal part such as a lesion may be included in the entire imaging range including the fovea centralis, the macula, the optic disc, and blood vessels, the entire imaging range corresponds to the region of interest A0. In the example of FIG. 4, as an example of the abnormal part, abnormal shadows S1 and S2 are included in the region of interest A0.

[0040] On the one hand, FIG. 5 shows a first image G1 as an example of an unclear medical image. The first image G1 is an image of the fundus of the eye similar to the medical image G0, but a part of the imaging range is unclear, and there is an incompatible region N1 that is not suitable for extracting the original first region of interest A1 (i.e., the entire imaging range). The incompatible region N1 originally contains an abnormal shadow S1, but it is difficult to detect due to its unclarity. Hereinafter, it will be described assuming that the acquisition unit 11 has acquired the first image G1.

[0041] The extraction unit 12 extracts the first region of interest A1 from the first image G1 acquired by the acquisition unit 11. As a method for extracting the first region of interest A1, a method using known image processing, a method using AI (Artificial Intelligence) technology, etc. can be appropriately applied. For example, the first image G1 may be binarized, the background may be removed and the edges of each structure may be emphasized, and the contour of the imaging range may be specified to extract the first region of interest A1. Also, for example, a learned model that has been trained to input an image of the fundus of the eye and extract and output the imaging range may be used to extract the imaging range (i.e., the first region of interest A1) from the first image G1.

[0042] The detection unit 14 detects abnormal shadows included in the first region of interest A1 extracted by the extraction unit 12. In the example of the first image G1 in FIG. 5, the abnormal shadow S2 is detected by the detection unit 14, but the abnormal shadow S1 is not detected because it is included in the unclear incompatible region N1. As a method for detecting abnormal shadows, known CAD technology can be appropriately applied.

[0043] The determination unit 13 determines whether there is an incompatible region N1 that is incompatible with the extraction for the first region of interest A1 extracted by the extraction unit 12. Various methods can be applied as the determination method. For example, the determination unit 13 may perform the determination according to the degree of similarity between the shape of the first region of interest A1 extracted by the extraction unit 12 and a reference shape predetermined for the first region of interest A1. In the case of a fundus image, since the region of interest is the entire imaging range, the reference shape can be predetermined as a substantially circular shape. Therefore, when the degree of similarity between the shape of the contour of the first region of interest A1 extracted by the extraction unit 12 and the substantially circular reference shape is equal to or less than a predetermined threshold value (that is, the deviation is large), it may be determined that there is an incompatible region N1. Note that, as the determination method according to the degree of similarity, known matching techniques such as matching based on feature amounts and template matching can be appropriately applied.

[0044] Also, for example, the AI technology may be applied to the determination, and the determination may be made using a learned model for determining whether there is an incompatible region in the region of interest extracted from the medical image with the medical image as an input. The learned model in this case may be a model obtained by unsupervised learning, for example, a model learned to cluster medical images according to the presence or absence of an incompatible region. Also, it may be a model obtained by supervised learning, for example, a model learned using, as learning data, a pair of a medical image and information indicating whether there is an incompatible region in the region of interest extracted from the medical image.

[0045] FIG. 6 shows an example of a screen D1 presented on the display 24 by the presentation unit 15. As shown in FIG. 6, the presentation unit 15 presents the abnormal shadow S2 detected by the detection unit 14 by attaching an annotation M on the first image G1.

[0046] In addition, when the determination unit 13 determines that there is a non-conforming region N1 in the first region of interest A1, the presentation unit 15 identifies and presents the non-conforming region N1 in the first image G1. In the example of FIG. 6, the presentation unit 15 presents the non-conforming region N1 in the first image G1 by surrounding it with a thick line. Note that the method of presenting the non-conforming region N1 is not limited to this. For example, the non-conforming region N1 may be emphasized so as to be distinguishable by changing the line type (e.g., line thickness, color, solid line, dotted line, etc.) from other regions or by adding an annotation. Conversely, the non-conforming region N1 may be made distinguishable by emphasizing the regions other than the non-conforming region N1 in the first region of interest A1.

[0047] In addition, when the determination unit 13 determines that there is a non-conforming region N1 in the first region of interest A1, the presentation unit 15 may present the ratio of the non-conforming region N1 to the first region of interest A1. In the example of FIG. 6, the non-conforming region N1 is described as a "non-implementable" region where the detection process of abnormal shading by the detection unit 14 cannot be performed. In addition, the regions other than the non-conforming region N1 in the first region of interest A1 are described as "completed" regions where the detection of abnormal shading by the detection unit 14 has been completed.

[0048] Next, with reference to FIG. 7, the operation of the information processing apparatus 10 according to the present embodiment will be described. When the CPU 21 executes the information processing program 27, the determination process shown in FIG. 7 is executed. The determination process shown in FIG. 7 is executed, for example, when an instruction to start processing is received from the user via the input unit 25.

[0049] In step S10, the acquisition unit 11 acquires the first image G1 from the image acquisition device 2. In step S11, the extraction unit 12 extracts the first region of interest A1 from the first image G1 acquired in step S10. In step S12, the detection unit 14 detects abnormal shading included in the first region of interest A1 extracted in step S11. In step S13, the determination unit 13 determines whether or not there is a non-conforming region N1 that is inappropriate for extraction in the first region of interest A1 extracted in step S11.

[0050] When there is a non-conforming area N1 (i.e., when the determination in step S13 is affirmative), the process proceeds to step S14, and the presentation unit 15 identifies the non-conforming area N1 in the first image G1 and presents it together with the abnormal shadow detected in step S12. On the other hand, when there is no non-conforming area N1 (i.e., when the determination in step S13 is negative), the process proceeds to step S15, and the presentation unit 15 presents only the abnormal shadow detected in step S12. When step S14 or S15 is completed, this determination process ends. Note that after step S14, image synthesis processing according to the second embodiment described later and / or detection result synthesis processing according to the third embodiment may be performed.

[0051] As described above, the information processing apparatus 10 according to the first embodiment includes at least one processor. The processor acquires a first image G1 obtained by photographing a subject, extracts a first region of interest A1 from the first image G1, and determines whether there is a non-conforming area N1 that is not suitable for the extraction in the extracted first region of interest A1. That is, the information processing apparatus 10 determines the presence or absence of an unclear area in the first region of interest A1 in which a structure of a target for detecting and diagnosing an abnormal shadow or the like may be included. Therefore, even for an unclear first image G1, it can be utilized for diagnosis after recognizing the presence of an unclear area in the first image G1.

[0052] In the first embodiment described above, the detection unit 14 may detect only abnormal shadows included in areas other than the non-conforming area N1 in the first region of interest A1 and not detect abnormal shadows for the non-conforming area N1. This is because for an unclear non-conforming area N1, the possibility of non-detection and false detection of abnormal shadows is higher than that of other areas, and the reliability of the detection result is low.

[0053] On the other hand, in the above-described first embodiment, when the determination unit 13 determines that there is an incompatible region N1 in the first region of interest A1, the detection unit 14 may detect a structure such as an abnormal shadow included in the incompatible region N1 while reducing the detection accuracy. "Detecting while reducing the detection accuracy" means allowing the possibility of non-detection and false detection of the abnormal shadow and performing the detection of the abnormal shadow even with low reliability. In this case, it is preferable that the presentation unit 15 presents that the detection accuracy of the abnormal shadow is low for the incompatible region N1. In particular, when the ratio of the incompatible region N1 to the first region of interest A1 is equal to or greater than a predetermined threshold (for example, 20% or more), it is preferable to do so. This is because if the detection of the abnormal shadow is not performed in the incompatible region N1 when the ratio of the incompatible region N1 is high, the user has to visually confirm the abnormal shadow for many parts of the first image G1, and the advantages of CAD are lost. According to such a form, after recognizing that the detection accuracy is low for the unclear region, the detection result of the abnormal shadow for the entire first image G1 can be utilized for diagnosis.

[0054] Further, in the above-described first embodiment, the form in which the detection unit 14 detects the abnormal shadow has been described. However, the information processing apparatus 10 according to the present embodiment may not include the function of the detection unit 14 (that is, the CAD function), and the user may visually confirm the abnormal shadow. According to such a form, by making the user recognize that there is an unclear region in the first image G1, it is possible to suppress the user from overlooking the abnormal shadow, so that even the unclear first image G1 can be utilized for diagnosis by the user.

[0055] Also, in the above-described first embodiment, when the determination unit 13 determines that there is a non-conforming region N1 in the first region of interest A1, the extraction unit 12 may re-extract the first region of interest A1 while reducing the extraction accuracy. "Re-extracting while reducing the extraction accuracy" means allowing other regions (for example, the background portion in FIG. 5, etc.) to be extracted as the first region of interest A1, and changing the conditions for extracting the first region of interest A1 so that the non-conforming region N1 decreases. This condition is determined by, for example, the luminance value of pixels.

[0056] Also, in this case, it is preferable that the presentation unit 15 presents that the extraction accuracy of the first region of interest A1 is low. In particular, when the ratio of the non-conforming region N1 to the first region of interest A1 is equal to or greater than a predetermined threshold (for example, 20% or more), it is preferable to do so. This is because if many parts of the first image G1 are determined as the non-conforming region N1, it becomes difficult to complement the non-conforming region N1 (details will be described later). According to such a form, by making the user recognize that the extraction accuracy of the first region of interest A1 is low, it is possible to suppress the user from overlooking abnormal shadows, so that even an unclear first image G1 can be utilized for diagnosis by the user.

[0057] [Second Embodiment] In the above-described first embodiment, the presence or absence of the non-conforming region N1 in the first image G1 was determined. The information processing apparatus 10 according to the present embodiment has a function of complementing the non-conforming region N1 using a medical image different from the first image G1 when it is determined that the non-conforming region N1 exists in the first image G1. Hereinafter, an example of the configuration of the information processing apparatus 10 according to the present embodiment will be described, but redundant descriptions of the same configuration and operation as those in the first embodiment will be omitted.

[0058] Referring to FIG. 8, an example of the functional configuration of the information processing apparatus 10 according to the present embodiment will be described. As shown in FIG. 8, the information processing apparatus 10 according to the present embodiment includes an acquisition unit 11, an extraction unit 12, a determination unit 13, a detection unit 14, a presentation unit 15, and a synthesis unit 16, which are the same as those in the first embodiment. When the CPU 21 executes the information processing program 27, it functions as the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, the presentation unit 15, and the synthesis unit 16.

[0059] FIG. 9 shows an example of the screen D2 presented on the display 24 by the presentation unit 15. As shown in FIG. 9, when the determination unit 13 determines that there is an incompatible region N1 in the first region of interest A1 of the first image G1, the presentation unit 15 requests a second image G2 including a region corresponding to at least a part of the incompatible region N1.

[0060] The acquisition unit 11 acquires the second image G2 from the image acquisition device 2. FIG. 10 shows the second image G2. The second image G2 is an image of the fundus of the eye similar to the medical image G0, and the region N12 (illustrated by a broken line) corresponding to the incompatible region N1 of the first image G1 is clearly shown. On the other hand, a part of the shooting range is unclear, and there is also an incompatible region N2 that is not suitable for extracting the original second region of interest A2 (that is, the entire shooting range). The incompatible region N2 originally contains an abnormal shadow S2, but it is difficult to detect due to its unclear state.

[0061] The synthesis unit 16 generates a third image G3 by synthesizing the first image G1 acquired by the acquisition unit 11 and the second image G2. Specifically, the synthesis unit 16 synthesizes a part of the other image with one image so that the incompatible region in either the first image G1 or the second image G2 is complemented by the corresponding region in the other image, thereby generating the third image G3. In the third image G3 shown in FIG. 11, a part of the second image G2 is synthesized with the first image G1 so that the incompatible region N1 in the first image G1 is complemented by the region N12 corresponding to the incompatible region N1 in the second image G2.

[0062] The extraction unit 12 extracts the third region of interest A3 from the third image G3 synthesized by the synthesis unit 16 by the same method as extracting the first region of interest A1 from the first image G1.

[0063] The determination unit 13 determines whether there is an incompatible region in the third region of interest A3 extracted by the extraction unit 12. For example, when a part of the region N12 of the second image G2 corresponding to the incompatible region N1 of the first image G1 is unclear, the first image G1 cannot be completely complemented only by the second image G2. Therefore, the CPU 21 repeatedly performs a request and acquisition of a new image including a region corresponding to at least a part of the incompatible region, and a re-synthesis of the new image and the third image G3, until the determination unit 13 determines that there is no incompatible region in the third region of interest A3. The request, acquisition, and synthesis of the new image are performed in the same manner as the request, acquisition, and synthesis of the second image G2 described above.

[0064] When the determination unit 13 determines that there is no incompatible region in the third region of interest A3, the detection unit 14 detects abnormal shadows included in the third region of interest A3 extracted by the extraction unit 12 by the same method as detecting the abnormal shadows included in the first region of interest A1. In the example of the third image G3 in FIG. 11, both the abnormal shadows S1 and S2 can be detected by the detection unit 14.

[0065] FIG. 12 shows an example of a screen D3 presented on the display 24 by the presentation unit 15. As shown in FIG. 12, the presentation unit 15 presents the abnormal shadows S1 and S2 detected by the detection unit 14 by attaching an annotation M on the third image G3.

[0066] Next, with reference to FIG. 13, the operation of the information processing apparatus 10 according to the present embodiment will be described. By executing the information processing program 27 by the CPU 21, the image synthesis process shown in FIG. 13 is executed. The image synthesis process shown in FIG. 13 is executed after step S14 in the flowchart of FIG. 7. That is, in the determination process of the first embodiment, it is executed when it is determined that there is an incompatible region N1 in the first image G1.

[0067] In step S31, the presentation unit 15 requests a second image G2 including a region corresponding to at least a part of the non-conforming region N1 specified in step S14. In step S32, the acquisition unit 11 acquires the second image G2 from the image acquisition device 2. In step S33, the synthesis unit 16 synthesizes the first image G1 acquired in step S10 and the second image G2 acquired in step S32 to generate a third image G3.

[0068] In step S34, the extraction unit 12 extracts a third region of interest A3 from the third image G3 synthesized in step S33. In step S35, the determination unit 13 determines whether there is a non-conforming region that is inappropriate for the extraction with respect to the third region of interest A3 extracted in step S34.

[0069] If a non-conforming region exists (that is, if step S35 is an affirmative determination), the process proceeds to step S36, and the presentation unit 15 requests a new image including a region corresponding to at least a part of the non-conforming region. In step S37, the acquisition unit 11 acquires the new image from the image acquisition device 2. In step S38, the synthesis unit 16 re-synthesizes the new image acquired in step S37 and the third image G3 synthesized in step S33. When step S38 is completed, the process returns to step S34. That is, the processes of steps S34 to S38 are repeated until it is determined in step S35 that there is no non-conforming region in the third region of interest A3.

[0070] On the other hand, if no non-conforming region exists (that is, if step S35 is a negative determination), the process proceeds to step S39, and the detection unit 14 detects an abnormal shadow included in the third region of interest A3 extracted in step S34. In step S40, the presentation unit 15 presents the abnormal shadow detected in step S39, and ends this image synthesis process.

[0071] As described above, the information processing apparatus 10 according to the second embodiment includes at least one processor. When the processor determines that there is an incompatible region N1 in the first region of interest A1 of the first image G1, the processor requests a second image including a region corresponding to at least a part of the incompatible region N1, and synthesizes the first image G1 and the second image G2. That is, when it is determined that there is an incompatible region N1 in the first image G1, the information processing apparatus 10 uses a second image G2 different from the first image G1 to complement the incompatible region N1. Therefore, even if each of the first image G1 and the second image G2 is unclear, it can be used for diagnosis.

[0072] In the second embodiment described above, the form in which the synthesis unit 16 complements the incompatible region in either one of the first image G1 and the second image G2 with the other image has been described, but the present invention is not limited to this. For example, the synthesis unit 16 may generate a third image G3 by selecting and synthesizing the image with better image quality for each of a plurality of sections in the first image G1 and the second image G2. The "section" means, for example, a pixel and a block composed of a plurality of pixels. The quality of the image quality can be evaluated based on, for example, pixel values such as hue, saturation, luminance, and lightness indicated by each pixel. Further, by combining these forms, the synthesis unit 16 may complement the incompatible region with the other image, and select and synthesize the image with better image quality for each section in other regions.

[0073] Also, in the second embodiment described above, particularly when the synthesis unit 16 repeats the resynthesis based on the new image more than a predetermined number of times (for example, three times), the extraction unit 12 may re-extract the third region of interest A3 with reduced extraction accuracy. This is to terminate the process when the complementation of the incompatible region is not completed even after repeating the resynthesis. In this case, the presentation unit 15 preferably presents that the extraction accuracy of the third region of interest A3 is low.

[0074] In addition, in the above-described second embodiment, the form in which the acquisition and synthesis of a new image are repeated until there is no incompatible region in the third image G3 has been described, but the present invention is not limited to this. If the synthesis by the synthesis unit 16 is performed at least once, the synthesis process may be terminated even if there is an incompatible region. For example, a limit based on the number of syntheses (e.g., three times) may be provided, and when the number of syntheses exceeds the limit, the process may be terminated even if there is an incompatible region. Further, for example, when the ratio of the incompatible region to the third region of interest A3 in the third image G3 is equal to or less than a predetermined threshold (e.g., 5% or less), the process may be terminated even if there is an incompatible region. In these cases, the presentation unit 15 may present the ratio of the incompatible region to the third region of interest A3.

[0075] In addition, when the process is terminated with an incompatible region present, structures such as abnormal shadows included in the third region of interest A3 may be detected with reduced detection accuracy. In this case, it is preferable that the presentation unit 15 presents that the detection accuracy of the abnormal shadow is low for the incompatible region. In particular, when the ratio of the incompatible region to the third region of interest A3 is equal to or greater than a predetermined threshold (e.g., 20% or more), it is preferable to do so. This is because if the detection of abnormal shadows is not performed in the incompatible region when the ratio of the incompatible region is high, the user has to visually confirm the abnormal shadows for many parts of the third image G3, and the advantages of CAD are lost. According to such a form, after recognizing that the detection accuracy is low for unclear regions, the detection results of abnormal shadows for the entire third image G3 can be utilized for diagnosis.

[0076] In addition, in the above-described second embodiment, the form in which the detection unit 14 detects an abnormal shadow has been described, but the information processing apparatus 10 according to the present embodiment may not include the function of the detection unit 14 (i.e., the CAD function), and the user may visually confirm the abnormal shadow. Even in such a form, each of the unclear first image G1 and second image G2 can be utilized for diagnosis by the user.

[0077] Also, in the above-described second embodiment, in the determination process of the first embodiment, the case where the image synthesis process is performed when it is determined that the non-conforming region N1 exists in the first image G1 has been described, but it is not limited thereto. For example, in the first image G1, when the ratio of the non-conforming region N1 to the first region of interest A1 is equal to or greater than a predetermined threshold (for example, 20% or more), the image synthesis process according to this embodiment may be performed.

[0078] [Third Embodiment] In the above-described second embodiment, the detection of the abnormal shadow is performed based on the third image G3 obtained by synthesizing the first image G1 and the second image G2. The information processing apparatus 10 according to this embodiment has a function of detecting the abnormal shadow based on each of the first image G1 and the second image G2 and synthesizing the detection results. Hereinafter, an example of the configuration of the information processing apparatus 10 according to this embodiment will be described, but redundant descriptions of the same configuration and operation as those in the first and second embodiments will be omitted.

[0079] Similar to the second embodiment, with reference to FIG. 8, an example of the functional configuration of the information processing apparatus 10 according to this embodiment will be described. As shown in FIG. 8, the information processing apparatus 10 according to this embodiment includes a synthesis unit 16 in addition to the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, and the presentation unit 15 similar to those in the first embodiment. When the CPU 21 executes the information processing program 27, it functions as the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, the presentation unit 15, and the synthesis unit 16.

[0080] FIG. 9 shows an example of the screen D2 presented on the display 24 by the presentation unit 15. As shown in FIG. 9, when the determination unit 13 determines that the non-conforming region N1 exists in the first region of interest A1 of the first image G1, the presentation unit 15 requests the second image G2 including a region corresponding to at least a part of the non-conforming region N1. The acquisition unit 11 acquires the second image G2 (see FIG. 10) from the image acquisition device 2. The description of the second image G2 is the same as that in the second embodiment, so it is omitted.

[0081] The extraction unit 12 extracts the second region of interest A2 from the second image G2 acquired by the acquisition unit 11 by the same method as extracting the first region of interest A1 from the first image G1. The detection unit 14 detects abnormal shadows included in each of the first region of interest A1 and the second region of interest A2 extracted by the extraction unit 12 by the same method as detecting the abnormal shadow included in the first region of interest A1. In the examples of FIGS. 5 and 10, the detection unit 14 detects an abnormal shadow S2 from the first image G1 and an abnormal shadow S1 from the second image G2.

[0082] The combining unit 16 combines the detection results of the abnormal shadows detected from each of the first region of interest A1 and the second region of interest A2 by the detection unit 14. That is, the combining unit 16 summarizes the detection results of the abnormal shadows obtained from a plurality of different images as the detection results of the abnormal shadows for the same subject.

[0083] The determination unit 13 determines whether there is a common non-conforming region that is commonly unsuitable for extraction in the first region of interest A1 and the second region of interest A2 extracted by the extraction unit 12. If there is a common non-conforming region, there is a possibility that an abnormal shadow may be overlooked. Therefore, the CPU 21 repeats the request and acquisition of a new image including a region corresponding to at least a part of the common non-conforming region, the extraction of the region of interest from the new image, the detection of the abnormal shadow included in the region of interest, and the re-combination of the detection result of the abnormal shadow and the detection results of the abnormal shadows detected so far until the determination unit 13 determines that there is no common non-conforming region. The request, acquisition, extraction of the region of interest, and detection of the abnormal shadow of the new image are performed in the same manner as the request, acquisition, extraction of the region of interest, and detection of the abnormal shadow of the second image G2 described above.

[0084] FIG. 14 shows an example of the screen D4 presented on the display 24 by the presentation unit 15. As shown in FIG. 14, the presentation unit 15 presents the detection result of the abnormal shadow synthesized by the synthesis unit 16 by attaching the annotation M to each image. Note that, similar to the second embodiment, the synthesis unit 16 generates an image obtained by synthesizing the first image G1 and the second image G2, and the presentation unit 15 may present the detection result of the abnormal shadow synthesized by the synthesis unit 16 on the image (that is, one image) by attaching the annotation M.

[0085] Next, with reference to FIG. 15, the operation of the information processing apparatus 10 according to the present embodiment will be described. By the CPU 21 executing the information processing program 27, the detection result synthesis process shown in FIG. 15 is executed. The detection result synthesis process shown in FIG. 15 is executed after step S14 in the flowchart of FIG. 7. That is, it is executed when it is determined in the determination process of the first embodiment that the non-conforming region N1 exists in the first image G1.

[0086] In step S51, the presentation unit 15 requests the second image G2 including a region corresponding to at least a part of the non-conforming region N1 specified in step S14. In step S52, the acquisition unit 11 acquires the second image G2 from the image acquisition device 2. In step S53, the extraction unit 12 extracts the second region of interest A2 from the second image G2 acquired in step S52. In step S54, the detection unit 14 detects an abnormal shadow included in the second region of interest A2 extracted in step S53.

[0087] In step S55, the synthesis unit 16 synthesizes the detection result of the abnormal shadow included in the first region of interest A1 detected in step S12 and the detection result of the abnormal shadow included in the second region of interest A2 detected in step S54. In step S56, the determination unit 13 determines whether there is a common non-conforming region that is commonly extracted as inappropriate for extraction with respect to the first region of interest A1 extracted in step S11 and the second region of interest A2 extracted in step S53.

[0088] When there is a common non-conforming area (i.e., when the determination in step S56 is affirmative), for the new image including the area corresponding to at least a part of the common non-conforming area, the processes of steps S51 to S56 are performed. That is, the processes of steps S51 to S56 are repeated until it is determined in step S56 that there is no common non-conforming area.

[0089] On the other hand, when there is no common non-conforming area (i.e., when the determination in step S56 is negative), the process proceeds to step S57, and the presentation unit 15 presents the detection result of the abnormal shadow synthesized in step S55. When step S57 is completed, this detection result synthesis process ends.

[0090] As described above, the information processing apparatus 10 according to the third embodiment includes at least one processor. When the processor determines that there is a non-conforming area N1 in the first region of interest A1 of the first image G1, it requests a second image including an area corresponding to at least a part of the non-conforming area N1. Further, the second region of interest A2 is extracted from the second image G2, abnormal shadows included in each of the first region of interest A1 and the second region of interest A2 are detected, and the detection results of the abnormal shadows are synthesized. That is, when it is determined that there is a non-conforming area N1 in the first image G1, the information processing apparatus 10 complements the detection result of the abnormal shadow using a second image G2 different from the first image G1. Therefore, even if each of the first image G1 and the second image G2 is unclear, it can be utilized for diagnosis.

[0091] In the above third embodiment, particularly when the synthesis unit 16 repeats the resynthesis of the detection result of the abnormal shadow included in the new image more than a predetermined number of times (for example, three times), the extraction unit 12 may extract the region of interest of the new image with a reduced extraction accuracy. This is to terminate the process when the common non-conforming area does not disappear even after repeated resynthesis. In this case, it is preferable that the presentation unit 15 presents that the extraction accuracy of the region of interest is low.

[0092] In addition, in the above-described third embodiment, the form in which the synthesis of the detection results of abnormal shadows based on the new image is repeated until there is no common non-conforming region has been described, but the present invention is not limited to this. If the synthesis of the detection results by the synthesis unit 16 is performed at least once, the synthesis process may be terminated even if there is a common non-conforming region. For example, a limit based on the number of syntheses (e.g., three times) may be provided, and when the number of syntheses exceeds the limit, the process may be terminated even if there is a common non-conforming region. Further, for example, when the ratio of the common non-conforming region to the first region of interest A1 or the second region of interest A2 becomes equal to or less than a predetermined threshold (e.g., 5% or less), the process may be terminated even if there is a common non-conforming region.

[0093] Also, when the process is terminated while there is a common non-conforming region, structures such as abnormal shadows included in the common non-conforming region may be detected with reduced detection accuracy. In this case, the presentation unit 15 preferably presents that the detection accuracy of abnormal shadows is low for the common non-conforming region. In particular, when the ratio of the common non-conforming region to the first region of interest A1 or the second region of interest A2 is equal to or more than a predetermined threshold (e.g., 20% or more), it is preferable to do so. This is because if abnormal shadows are not detected in the common non-conforming region when the ratio of the common non-conforming region is high, the user has to visually confirm abnormal shadows for many parts of the first image G1 and the second image G2, and the advantages of CAD are lost. According to such a form, after recognizing that the detection accuracy is low for unclear regions, the detection results of abnormal shadows for the entire first image G1 and the second image G2 can be utilized for diagnosis.

[0094] In addition, in the above-described third embodiment, in the determination process of the first embodiment, the form in which the detection result synthesis process is performed when it is determined that there is a non-conforming region N1 in the first image G1 has been described, but the present invention is not limited to this. For example, in the first image G1, when the ratio of the non-conforming region N1 to the first region of interest A1 is equal to or more than a predetermined threshold (e.g., 20% or more), the detection result synthesis process according to the present embodiment may be performed.

[0095] [Fourth Embodiment] In the above-described first to third embodiments, the form in which the extraction unit 12 extracts the region of interest from the medical image has been described. As described above, as a method for extracting the region of interest by the extraction unit 12, a learned model that is learned to input a medical image and extract and output the region of interest can be used. In this case, it is required that the learned model can accurately extract the region of interest even when an unclear medical image as shown in FIGS. 5 and 10 is input.

[0096] As one method for improving the accuracy of this learned model, there is a method of performing learning using various patterns of unclear medical images as learning data. However, it has been difficult to acquire unclear medical images with a sufficient number and patterns by the image acquisition device 2. Therefore, in the present embodiment, the purpose is to improve the accuracy of the learning model by deliberately generating unclear medical images and using them as learning data.

[0097] As an example, a form in which the information processing apparatus 10 according to the present embodiment causes the learning model 4 used in the extraction unit 12 to be learned by unsupervised learning will be described. The learning model 4 is configured to include a deep learning model such as, for example, a CNN (Convolutional Neural Network), an FCN (Fully Convolutional Network), and a U-Net, and is a model that is learned to input a medical image and extract and output the region of interest. Also, for example, as such a model, the techniques described in Patent Document 1, Japanese Unexamined Patent Application Publication No. 2019-088458, and Japanese Unexamined Patent Application Publication No. 2020-114302 may be applied.

[0098] The CPU 21 acquires the original image obtained by photographing a subject from the image acquisition device 2. That is, the original image is an image obtained by at least one of an X-ray imaging device, a magnetic resonance imaging device, an ultrasonic device, a fundus imaging device, and an endoscope. Further, the original image includes a region of interest including at least one of a subject, a part of the tissues included in the subject, and a structure such as an abnormal part included in the subject or the tissue. Hereinafter, as an example of the original image, an example using the clear medical image G0 shown in FIG. 4 will be described. Since the medical image G0 has been described above, the description will be omitted.

[0099] The CPU 21 generates a pseudo image by changing at least a part of the pixel values of the medical image G0. The pixel value is a value indicating at least one of the hue, saturation, luminance, and brightness indicated by each pixel in the medical image G0. For example, by changing the brightness and contrast of the medical image G0, or by applying blur and noise, the pixel value of each pixel is changed. However, it is assumed that the CPU 21 does not change the resolution in generating the pseudo image.

[0100] Referring to FIG. 16, a specific example of the pseudo image will be described. A plurality of pseudo images P1 to P5 shown on the left side of FIG. 16 are pseudo images generated based on the medical image G0, respectively, and are input to the learning model 4 as learning data. The pseudo images P1 and P2 are images in which the brightness of the medical image G0 is made brighter and darker, respectively. The pseudo image P3 is an image in which the contrast of the medical image G0 is weakened. The pseudo images P4 and P5 are images in which a part of the region of the medical image G0 is made darker.

[0101] Further, on the right side of FIG. 16, for each of the input pseudo images, the region of interest extracted by the learning model 4 during learning is surrounded by a thick line and shown. For the pseudo images P1 and P2, the regions of interest are appropriately extracted. For the pseudo image P3, an error indicating that extraction is impossible is output. For the pseudo images P4 and P5, some unclear regions that should originally be extracted as the region of interest are not extracted as the region of interest.

[0102] As shown in FIG. 16, the pseudo-image may have pixel values changed for the entire medical image G0, or may have pixel values changed for a part of the medical image G0. Further, as shown in the pseudo-images P4 and P5, it is preferable to use, as learning data, a plurality of pseudo-images generated by changing pixel values in different regions for one medical image G0. According to such a form, the number of original images can be reduced, so that learning can be performed efficiently.

[0103] Further, as shown in the pseudo-images P1 to P5, it is preferable that the pseudo-image is an image generated by changing pixel values so that the image quality deteriorates in at least a part of the medical image G0. Specifically, "so that the image quality deteriorates" means processing such that the structures included in the medical image G0 become difficult to detect. For example, processing for weakening the contrast is mentioned. This is because, considering the operation phase of the learning model, when the input medical image is a fundus image, for example, the contrast may be weakened compared to a clear medical image due to insufficient light amount causing the image to become dark or due to stray light causing the image to become bright.

[0104] Further, as shown in FIG. 16, it is preferable that the pseudo-image is an image generated by changing pixel values in a region including at least the region of interest in the medical image G0. This is because for regions other than the region of interest, whether they are unclear is not particularly problematic.

[0105] Further, it is preferable that the pseudo-image is an image generated by changing at least a part of the pixel values of the medical image G0 while maintaining the presence or absence of the structures included in the medical image G0. In the present learning model 4, the priority is to accurately extract the region of interest even from an unclear medical image, and the priority of corresponding to changes in the presence or absence of the structures is low.

[0106] Alternatively, the pseudo-image may be an image generated based on the medical image G0 using an image generation model such as GAN (Generative Adversarial Networks) and VAE (Variational Autoencoder).

[0107] The CPU 21 trains the learning model 4 by inputting the pseudo-images P1 to P5 generated as described above into the learning model 4 as learning data. Also, as shown in the pseudo-images P3 to P5 in FIG. 16, when the learning model 4 fails to appropriately extract the region of interest from the input pseudo-image, the CPU 21 may re-train the learning model 4 by re-inputting the pseudo-image as learning data. According to such a form, the accuracy of the learning model 4 can be improved.

[0108] Note that during re-training, a correct label may be assigned to the pseudo-image. Specifically, the CPU 21 may input a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image into the learning model 4 as learning data to re-train the learning model 4. The information indicating the region of interest is, for example, information indicating the position of the region of interest in the pseudo-image.

[0109] Next, with reference to FIG. 17, the operation of the information processing apparatus 10 according to the present embodiment will be described. By the CPU 21 executing the information processing program 27, the learning process shown in FIG. 17 is executed. The learning process shown in FIG. 17 is executed, for example, when an instruction to start the process is received from the user via the input unit 25.

[0110] In step S71, the CPU 21 acquires the original image obtained by photographing the subject from the image acquisition device 2. In step S72, the CPU 21 changes at least some of the pixel values of the original image acquired in step S71 to generate a pseudo-image. In step S73, the CPU 21 uses the pseudo-image generated in step S72 as learning data to train the learning model 4. In step S74, the CPU 21 determines whether the learning model 4 can appropriately extract the region of interest from the input pseudo-image.

[0111] When the learning model 4 fails to appropriately extract the region of interest from the input pseudo-image (i.e., when the determination in step S74 is negative), the process returns to step S73, and the CPU 21 re-learns the learning model 4 by re-inputting the pseudo-image to the learning model 4. That is, re-learning using the same pseudo-image is repeated until the learning model 4 can appropriately extract the region of interest from the pseudo-image. The main learning process ends at the timing when the learning model 4 can appropriately extract the region of interest from the input pseudo-image (i.e., the timing when the determination in step S74 becomes positive).

[0112] As described above, the information processing apparatus 10 according to the fourth embodiment includes at least one processor. The processor learns a learning model for extracting a region of interest from an input image by inputting, as learning data, a pseudo-image generated by changing at least a part of the pixel values of the original image obtained by photographing a subject. Therefore, even an image that is not suitable for extracting the region of interest can be appropriately used for diagnosis by extracting the region of interest appropriately.

[0113] In the above-described fourth embodiment, the form in which the information processing apparatus 10 learns the learning model 4 used in the extraction unit 12 by unsupervised learning has been described, but the present invention is not limited to this. The information processing apparatus 10 may learn the learning model 4 by supervised learning and semi-supervised learning. Specifically, the CPU 21 may learn the learning model 4 by inputting, as learning data, a pair of a pseudo-image and information indicating the region of interest included in the pseudo-image. Also in this case, when the learning model 4 fails to extract the region of interest from the input pseudo-image, the CPU 21 may re-learn the learning model 4 by re-inputting the pseudo-image as learning data.

[0114] In addition, in each of the above embodiments, the description has been made using medical images, but the technology of the present disclosure is not applicable only to medical images. The technology of the present disclosure may be applied to images obtained by using a subject such as equipment, buildings, pipes, and welded parts in non-destructive inspections such as radiographic inspections and ultrasonic flaw detection inspections, for example.

[0115] Also, in each of the above embodiments, the form in which the information processing system 1 includes the information processing device 10 and the image acquisition device 2 has been described, but the present invention is not limited to this. For example, the information processing system 1 may include a single device having both the functions of the information processing device 10 and the functions of the image acquisition device 2. Also, for example, the information processing system 1 may include a plurality of image acquisition devices 2, and the information processing device 10 may acquire medical images from each of the plurality of image acquisition devices 2. Also, for example, the information processing device 10 may have a configuration including a plurality of different devices for each function such as the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, the presentation unit 15, and the synthesis unit 16.

[0116] Also, in each of the above embodiments, as the hardware structure of a processing unit that executes various processes such as, for example, the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, the presentation unit 15, and the synthesis unit 16, the following various processors can be used. In addition to the CPU, which is a general-purpose processor that executes software (program) and functions as various processing units as described above, the above various processors include a programmable logic device (PLD), such as an FPGA (Field Programmable Gate Array), which is a processor whose circuit configuration can be changed after manufacture, and a dedicated electric circuit, which is a processor having a circuit configuration designed specifically to execute specific processing, such as an ASIC (Application Specific Integrated Circuit).

[0117] One processing unit may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same type or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be composed of one processor.

[0118] As an example of configuring multiple processing units with one processor, first, as represented by computers such as clients and servers, one processor is configured by a combination of one or more CPUs and software, and this processor functions as multiple processing units. Second, as represented by a System on Chip (SoC), etc., there is a form in which a processor that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip is used. Thus, various processing units are configured using one or more of the above various processors as a hardware structure.

[0119] Furthermore, as a more specific hardware structure of these various processors, an electrical circuit (circuitry) combining circuit elements such as semiconductor elements can be used.

[0120] Also, in the above embodiments, the mode in which the information processing program 27 is pre-stored (installed) in the storage unit 22 has been described, but it is not limited to this. The information processing program 27 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and a USB (Universal Serial Bus) memory. Also, the information processing program 27 may be in a form downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores the information processing program in addition to the information processing program.

[0121] The technology of the present disclosure can also be appropriately combined with each of the above-mentioned embodiment examples. The above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is a description of one example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure. [Explanation of symbols]

[0122] 1. Information Processing Systems 2. Image acquisition device 4. Learning Model 10. Information processing device 11 Acquisition Department 12 Extraction part 13 Judgment section 14 Detection section 15 Presentation section 16 Synthesis section 21 CPU 22 Memory section 23 Memory 24 Display 25 Input section 26 Network Interface 27 Information Processing Programs 28 Bus A0 Area of ​​Interest A1 First area of ​​interest A2 Second Area of ​​Interest A3 Third Area of ​​Interest D1~D4 screen G0 Medical Imaging G1 1st image G2 2nd image G3 3rd image M annotation N1, N2 Nonconformity area N12 area P1~P5 Pseudo image S1, S2 abnormal shadow

Claims

1. Comprising at least one processor, The processor is, A pseudo-image generated by changing the pixel values of at least a part of the region including at least the region of interest in the original image obtained by photographing the subject, and the pixel values in the different regions for each of the one original images By inputting a plurality of the pseudo-images generated by changing as learning data, a learning model for extracting a region of interest from the input image is learned, The learning model is a model that takes as input an image including an unclear region in which the pixel value indicating at least one of the hue, saturation, brightness, and lightness shown by each pixel does not satisfy a predetermined reference value, and outputs the region of interest included in the input image An information processing apparatus.

2. The processor is, The learning model is learned by inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image The information processing apparatus according to claim 1.

3. The processor is, When the learning model fails to extract the region of interest from the input pseudo-image, the learning model is relearned by re-inputting the pseudo-image as learning data The information processing apparatus according to claim 1 or claim 2.

4. The processor is, When the learning model fails to extract the region of interest from the input pseudo-image, the learning model is relearned by re-inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image The information processing apparatus according to claim 3.

5. The pseudo-image is an image generated by changing pixel values so that the image quality deteriorates in at least a part of the original image The information processing apparatus according to any one of claims 1 to 4.

6. The pseudo-image is an image generated by changing at least a part of the pixel values of the original image while maintaining the presence or absence of the structure included in the original image The information processing apparatus according to any one of claims 1 to 5.

7. The pseudo-image is an image generated based on the original image using GAN (Generative Adversarial Networks) Is an image generated The information processing apparatus according to any one of claims 1 to 6.

8. The pixel value is a value indicating at least one of the hue, saturation, brightness, and lightness shown by each pixel The information processing apparatus according to any one of claims 1 to 7.

9. The region of interest is a region including at least one of the subject, a part of the tissues included in the subject, and an abnormal part included in the subject or the tissues. The information processing apparatus according to any one of claims 1 to 8.

10. The original image is an image obtained by at least one of a radiation imaging apparatus, a magnetic resonance imaging apparatus, an ultrasonic apparatus, a fundus imaging apparatus, and an endoscope. The information processing apparatus according to any one of claims 1 to 9.

11. The processor inputs a plurality of the pseudo-images as learning data to train the learning model by unsupervised learning, and when the learning model fails to extract a region of interest from the input pseudo-image, retrains the learning model by supervised learning that re-inputs a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image as learning data. The information processing apparatus according to any one of claims 1 to 10.

12. A pseudo-image generated by changing pixel values of at least a part of a region including at least a region of interest in an original image obtained by photographing a subject, and inputs a plurality of the pseudo-images generated by changing pixel values in different regions for one original image as learning data to train a learning model for extracting a region of interest from the input image, wherein the learning model takes, as an input, an image including an unclear region where at least one of the hue, saturation, brightness, and lightness indicated by each pixel does not satisfy a predetermined reference value, and outputs a region of interest included in the input image. An information processing method executed by a computer.

13. Trains the learning model by inputting a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image as learning data. The information processing method according to claim 12, executed by a computer.

14. When the learning model fails to extract a region of interest from the input pseudo-image, retrains the learning model by re-inputting the pseudo-image as learning data. The information processing method according to claim 12 or claim 13, executed by a computer.

15. When the learning model fails to extract the region of interest from the input pseudo-image, the learning model is re-learned by re-inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image. The information processing method according to claim 14, wherein a computer executes the process.

16. A pseudo-image generated by changing pixel values of at least a part of a region including at least the region of interest in the original image obtained by photographing a subject, and a plurality of the pseudo-images generated by changing pixel values in different regions for each of the one original image are input as learning data to learn a learning model for extracting the region of interest from the input image. The learning model is a model that takes, as input, an image including an unclear region where at least one of the hue, saturation, luminance, and lightness indicated by each pixel does not satisfy a predetermined reference value, and outputs the region of interest included in the input image. An information processing program for causing a computer to execute the process.

17. The learning model is learned by inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image. The information processing program according to claim 16, for causing a computer to execute the process.

18. When the learning model fails to extract the region of interest from the input pseudo-image, the learning model is re-learned by re-inputting the pseudo-image as learning data. The information processing program according to claim 16 or 17, for causing a computer to execute the process.

19. When the learning model fails to extract the region of interest from the input pseudo-image, the learning model is re-learned by re-inputting, as learning data, a pair of the pseudo-image and information indicating the region of interest included in the pseudo-image. The information processing program according to claim 18, for causing a computer to execute the process. ​

Images