Information processing device, information processing method, and information processing program

Abstract

To obtain an information processing apparatus, an information processing method, and an information processing program that can utilize, for diagnosis, even images that are unsuitable for extracting a region of interest.SOLUTION: An information processing apparatus 10 includes at least one processor. The processor is configured to: acquire a first image obtained by imaging a subject; extract a first region of interest from the first image; determine, for the extracted first region of interest, whether or not an unsuitable region that is unsuitable for extraction exists; and perform control to present the unsuitable region in the first image so as to be distinguishable from other regions.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, doctors generally make diagnoses based on medical images acquired by image acquisition devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices. Also known is a technology (so-called CAD (Computer Aided Detection / Diagnosis)) that uses a computer to assist in the detection and diagnosis of structures such as abnormal shadows and tissues contained in medical images. For example, Patent Document 1 describes analyzing medical images using a classifier trained by machine learning to identify the type of tissue or lesion contained in the medical image, i.e., the type of findings.

[0003] Furthermore, for example, Non-Patent Document 1 describes that CAD should not be performed if at least a part of a medical image is unclear in order to avoid non-detection and misdiagnosis of abnormal shadows. Blurred medical images can be obtained, for example, when images are taken in an inappropriate environment or when the photographer has poor imaging skills. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-032043 [Non-patent literature]

[0005] [Non-Patent Document 1] "A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy", Emma Beede et al., in CHI 2020 Paper 589 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in medical institutions in emerging countries, for example, it has been difficult to obtain clear medical images due to the inability to create an appropriate environment for taking medical images or the photographer's low imaging skills. Therefore, in recent years, there has been a demand for technology that allows even blurry images, i.e., images that are not suitable for extracting regions of interest that may contain abnormal shadows and structures such as tissue, to be used for diagnosis.

[0007] The present disclosure provides an information processing device, an information processing method, and an information processing program that can utilize images that are unsuitable for extracting a region of interest for diagnosis. [Means for solving the problem]

[0008] One aspect of the present disclosure is an information processing device comprising at least one processor, which acquires a first image obtained by photographing a subject, extracts a first region of interest from the first image, determines whether or not the extracted first region of interest contains any unsuitable regions that are unsuitable for extraction, and performs control to present the unsuitable regions in the first image in a manner that allows them to be distinguished from other regions.

[0009] In the above aspect, the processor may make the determination according to the degree of similarity between the shape of the extracted first region of interest and a reference shape that is predetermined for the first region of interest.

[0010] In the above aspect, the processor may input an image obtained by photographing the subject and make the determination using a trained model for determining whether or not an unqualified region exists in a region of interest extracted from the image. The trained model may be a trained model trained using a pair of an image obtained by photographing the subject and information indicating whether or not an unqualified region exists in a region of interest extracted from the image as training data.

[0011] In the above aspect, the processor may perform control to present a character string indicating whether or not an incompatible region exists on the screen on which the first image is presented.

[0012] In the above aspect, when it is determined that an unsuitable region exists, the processor may perform control to display the ratio of the unsuitable region to the first region of interest.

[0013] In the above aspect, when the processor determines that an unsuitable region exists, the processor may re-extract the first region of interest with reduced extraction accuracy.

[0014] In the above aspect, the processor may detect a structure included in the first region of interest, and if it determines that an unsuitable region exists, may detect the structure included in the unsuitable region by reducing the detection accuracy.

[0015] In the above-described embodiment, the processor may detect abnormal shadows from other regions, and may not detect abnormal shadows from the non-conforming regions.

[0016] In the above aspect, the processor may perform control to present abnormal shadows detected from other regions on the first image.

[0017] In the above aspect, when the processor determines that an unsuitable area exists, it may acquire a second image including an area corresponding to at least a portion of the unsuitable area, and generate a third image by combining the first image and the second image.

[0018] In the above aspect, the processor may generate a third image by combining a portion of one of the first and second images with the other image so that a non-matching area in the other image is complemented by a corresponding area in the other image.

[0019] In the above aspect, the processor may generate the third image by selecting one of the images with better image quality for each of a plurality of sections in the first image and the second image and combining the images.

[0020] In the above aspect, the processor may extract a third region of interest from the third image, determine whether or not an unsuitable region exists in the extracted third region of interest, and repeatedly acquire a new image including an area corresponding to at least a portion of the unsuitable region and recombine the new image with the third image until it is determined that no unsuitable region exists in the third region of interest.

[0021] In the above aspect, the processor may extract a third region of interest from the third image and detect a structure included in the third region of interest.

[0022] In the above aspect, when the processor determines that an unsuitable area exists, it may acquire a second image including an area corresponding to at least a portion of the unsuitable area, extract a second area of ​​interest from the second image, detect structures included in each of the first area of ​​interest and the second area of ​​interest, and combine the detection results of the structures detected from each of the first area of ​​interest and the second area of ​​interest.

[0023] In the above aspect, the processor may determine whether or not a common unsuitable area exists for extraction for the extracted first and second regions of interest, and may repeatedly acquire a new image including an area corresponding to at least a portion of the common unsuitable area, extract the region of interest from the new image, detect structures included in the region of interest, and recombine the detection results of the structures, until it is determined that no common unsuitable area exists.

[0024] In the above aspect, the first region of interest may be a region including at least one of the subject, a portion of tissue contained in the subject, and an abnormal portion contained in the subject or tissue.

[0025] In the above aspect, the first image may be an image obtained by at least one of a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope.

[0026] Another aspect of the present disclosure is an information processing method in which a computer executes a process of acquiring a first image obtained by photographing a subject, extracting a first region of interest from the first image, determining whether or not the extracted first region of interest contains any unsuitable regions that are unsuitable for extraction, and controlling the presentation of the unsuitable regions in the first image in a manner that makes them distinguishable from other regions.

[0027] Another aspect of the present disclosure is an information processing program that causes a computer to execute a process of acquiring a first image obtained by photographing a subject, extracting a first region of interest from the first image, determining whether or not the extracted first region of interest contains any unsuitable regions that are unsuitable for extraction, and controlling the presentation of the unsuitable regions in the first image in a manner that makes them distinguishable from other regions. [Effects of the Invention]

[0028] According to the above aspects, the information processing device, information processing method, and information processing program of the present disclosure can utilize even images that are unsuitable for extracting a region of interest for diagnosis. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a schematic configuration diagram of an information processing system. [Figure 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of an information processing device. [Figure 3] 1 is a block diagram showing an example of a functional configuration of an information processing device according to a first embodiment. [Figure 4] This is an example of a clear medical image. [Figure 5] 10 is an example of a blurred first image. [Figure 6] 10 is an example of a presented screen. [Figure 7] 10 is a flowchart illustrating an example of a determination process. [Figure 8] FIG. 2 is a block diagram showing an example of a functional configuration of an information processing device according to second and third embodiments. [Figure 9] 10 is an example of a presented screen. [Figure 10] 10 is an example of a blurred second image. [Figure 11] 10 is an example of a synthesized third image. [Figure 12] 10 is an example of a presented screen. [Figure 13] 10 is a flowchart illustrating an example of image synthesis processing. [Figure 14] 10 is an example of a presented screen. [Figure 15] 10 is a flowchart illustrating an example of a detection result synthesis process. [Figure 16] FIG. 10 is a diagram showing an example of a pseudo image input to a learning model. [Figure 17] 10 is a flowchart illustrating an example of a learning process. DETAILED DESCRIPTION OF THE INVENTION

[0030] Hereinafter, examples of embodiments for carrying out the technology of the present disclosure will be described in detail with reference to the drawings.

[0031] [First embodiment] An example of the configuration of an information processing system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the information processing system 1 includes an information processing device 10 and an image acquisition device 2. The information processing device 10 and the image acquisition device 2 are capable of communicating with each other via wired or wireless communication.

[0032] The image acquisition device 2 is a device (so-called modality) that acquires images obtained by photographing a subject. In this embodiment, a medical image is described as an example of a specific image acquired by the image acquisition device 2. As the image acquisition device 2, at least one of a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope can be applied, and an appropriate combination of these may also be applied.

[0033] For example, to clearly capture an image of the fundus, the image should be taken in a dark place. However, in medical institutions in emerging countries, it is sometimes difficult to create a dark place, resulting in unclear medical images. For example, to clearly capture a breast in a mammogram, the breast should be sufficiently compressed during the image capture. However, depending on the photographer's positioning technique and the shape of the breast, the breast may not be compressed sufficiently, resulting in unclear medical images. In recent years, there has been a demand for technology that allows even such unclear images to be used for diagnosis.

[0034] Therefore, the information processing device 10 according to this embodiment has a function for determining whether a medical image contains a blurred area. The term "blurred" refers to a situation in which the pixel values ​​of each pixel in the medical image, such as hue, saturation, luminance, and brightness, do not satisfy predetermined reference values. For example, a blurred area may occur when a pixel is darker than the reference value due to insufficient light or when a pixel is brighter than the reference value due to ambient light. An example of the configuration of the information processing device 10 according to this embodiment will be described below.

[0035] First, an example of the hardware configuration of an information processing device 10 according to this embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F 26 that performs wired or wireless communication with the image acquisition device 2 and an external network (not shown). The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected via a bus 28 such as a system bus and a control bus so as to be able to exchange various information with each other.

[0036] The storage unit 22 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), or a flash memory. The storage unit 22 stores an information processing program 27 according to this embodiment. The CPU 21 reads the information processing program 27 from the storage unit 22, loads it into the memory 23, and executes the loaded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. Various computers can be used as the information processing device 10, such as a console for the image acquisition device 2, a workstation, a server computer, or a personal computer.

[0037] Next, an example of the functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the information processing device 10 includes an acquisition unit 11, an extraction unit 12, a determination unit 13, a detection unit 14, and a presentation unit 15. The CPU 21 executes an information processing program 27, thereby functioning as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, and presentation unit 15.

[0038] The acquisition unit 11 acquires medical images obtained by photographing a subject from the image acquisition device 2. The medical images in this embodiment include regions of interest that include the subject, a portion of tissue contained in the subject, and at least one abnormality contained in the subject or tissue, as target structures to be detected and diagnosed. Examples of subjects include the human body and various organs of the human body, such as the fundus, lungs, breasts, stomach, liver, heart, and brain. Examples of tissues include elements that make up various organs, such as blood vessels, nerves, and muscles. Examples of abnormalities include lesions and abnormalities, such as tumors, damage, defects, nodules, and inflammation.

[0039] FIG. 4 shows medical image G0 as an example of a clear medical image. Medical image G0 is a clear image of the fundus obtained by a fundus photography device. In a fundus image, abnormal areas such as lesions may be included throughout the entire imaging range, including the fovea, macula, optic disc, and blood vessels, so the entire imaging range corresponds to the region of interest A0. In the example of FIG. 4, abnormal shadows S1 and S2 are included in the region of interest A0 as examples of abnormal areas.

[0040] 5 shows a first image G1 as an example of a blurred medical image. The first image G1 is an image of the fundus similar to the medical image G0, but a portion of the imaging range is blurred, resulting in the presence of an unsuitable region N1 that is unsuitable for extracting the original first region of interest A1 (i.e., the entire imaging range). The unsuitable region N1 should contain an abnormal shadow S1, but its blurring makes its detection difficult. The following description will be given assuming that the acquisition unit 11 has acquired the first image G1.

[0041] The extraction unit 12 extracts the first region of interest A1 from the first image G1 acquired by the acquisition unit 11. Methods for extracting the first region of interest A1 may include, as appropriate, methods using well-known image processing and methods using AI (Artificial Intelligence) technology. For example, the first region of interest A1 may be extracted by binarizing the first image G1, removing the background, and enhancing the edges of each structure, thereby identifying the contours of the imaging range. Alternatively, for example, the imaging range (i.e., the first region of interest A1) may be extracted from the first image G1 using a trained model that is trained to input an image of the fundus and extract and output the imaging range.

[0042] The detection unit 14 detects abnormal shadows included in the first region of interest A1 extracted by the extraction unit 12. In the example of the first image G1 in Fig. 5, the detection unit 14 detects an abnormal shadow S2, but does not detect an abnormal shadow S1 because it is included in the unclear unsuitable region N1. Note that known CAD technology can be appropriately applied as a method for detecting abnormal shadows.

[0043] The determination unit 13 determines whether or not an unsuitable region N1 that is unsuitable for extraction exists in the first region of interest A1 extracted by the extraction unit 12. Various methods can be used as the determination method. For example, the determination unit 13 may make the determination based on the degree of similarity between the shape of the first region of interest A1 extracted by the extraction unit 12 and a predetermined reference shape for the first region of interest A1. In the case of a fundus image, the region of interest is the entire imaging range, so the reference shape can be predetermined to be approximately circular. Therefore, if the degree of similarity between the outline shape of the first region of interest A1 extracted by the extraction unit 12 and the approximately circular reference shape is equal to or less than a predetermined threshold (i.e., there is a large deviation), the determination unit 13 may determine that an unsuitable region N1 exists. Note that, as a determination method based on the degree of similarity, known matching techniques such as feature-based matching and template matching can be appropriately applied.

[0044] For example, AI technology may be applied to the judgment, and the judgment may be made using a trained model that inputs a medical image and determines whether or not an unsuitable region exists in a region of interest extracted from the medical image. In this case, the trained model may be a model that has undergone unsupervised learning, for example, a model that has been trained to cluster medical images according to the presence or absence of an unsuitable region. Alternatively, the trained model may be a model that has undergone supervised learning, for example, a model that has been trained using pairs of medical images and information indicating whether or not an unsuitable region exists in a region of interest extracted from the medical image as training data.

[0045] Fig. 6 shows an example of a screen D1 presented on the display 24 by the presentation unit 15. As shown in Fig. 6, the presentation unit 15 presents the abnormal shadow S2 detected by the detection unit 14 by adding an annotation M on the first image G1.

[0046] Furthermore, when the determination unit 13 determines that the unqualified region N1 exists in the first region of interest A1, the presentation unit 15 identifies and presents the unqualified region N1 in the first image G1. In the example of FIG. 6, the presentation unit 15 presents the unqualified region N1 in the first image G1 by enclosing it with a thick line. Note that the manner in which the unqualified region N1 is presented is not limited to this, and the unqualified region N1 may be highlighted so as to be distinguishable, for example, by using a different line type (e.g., line thickness, color, solid line, dotted line, etc.) from other regions or by adding an annotation. Conversely, the unqualified region N1 may be made distinguishable by highlighting regions other than the unqualified region N1 in the first region of interest A1.

[0047] Furthermore, when the determination unit 13 determines that the first region of interest A1 includes an unqualified region N1, the presentation unit 15 may present the ratio of the unqualified region N1 to the first region of interest A1. In the example of Fig. 6, the unqualified region N1 is described as an "unfeasible" region where the detection unit 14 cannot perform the detection process for abnormal shadows. Furthermore, the region other than the unqualified region N1 in the first region of interest A1 is described as an "performed" region where the detection unit 14 has completed the detection of abnormal shadows.

[0048] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 7. The CPU 21 executes the information processing program 27, thereby executing the determination process shown in Fig. 7. The determination process shown in Fig. 7 is executed, for example, when a command to start the process is received from the user via the input unit 25.

[0049] In step S10, the acquisition unit 11 acquires a first image G1 from the image acquisition device 2. In step S11, the extraction unit 12 extracts a first region of interest A1 from the first image G1 acquired in step S10. In step S12, the detection unit 14 detects an abnormal shadow included in the first region of interest A1 extracted in step S11. In step S13, the determination unit 13 determines whether or not an unsuitable region N1 that is unsuitable for extraction exists in the first region of interest A1 extracted in step S11.

[0050] If an incompatible region N1 exists (i.e., if step S13 is judged as positive), the process proceeds to step S14, where the presenting unit 15 identifies the incompatible region N1 in the first image G1 and presents it together with the abnormal shadow detected in step S12. On the other hand, if an incompatible region N1 does not exist (i.e., if step S13 is judged as negative), the process proceeds to step S15, where the presenting unit 15 presents only the abnormal shadow detected in step S12. When step S14 or S15 is completed, this judgment process ends. Note that after step S14, an image synthesis process according to a second embodiment and / or a detection result synthesis process according to a third embodiment, which will be described later, may be performed.

[0051] As described above, the information processing device 10 according to the first embodiment includes at least one processor. The processor acquires a first image G1 obtained by imaging a subject, extracts a first region of interest A1 from the first image G1, and determines whether or not the extracted first region of interest A1 contains an unsuitable region N1 that is unsuitable for extraction. That is, the information processing device 10 determines whether or not there is a blurred region in the first region of interest A1, which may contain a target structure for which detection and diagnosis of an abnormal shadow or the like is desired. Therefore, even a blurred first image G1 can be used for diagnosis after recognizing that the first image G1 contains a blurred region.

[0052] In the first embodiment, the detector 14 may detect only abnormal shadows included in the first region of interest A1 other than the unqualified region N1, and may not detect abnormal shadows in the unqualified region N1. This is because the possibility of non-detection or erroneous detection of abnormal shadows is higher in the unclear unqualified region N1 than in the other regions, making the detection result less reliable.

[0053] On the other hand, in the first embodiment, when the determination unit 13 determines that the first region of interest A1 contains an unqualified region N1, the detection unit 14 may detect structures, such as abnormal shadows, contained in the unqualified region N1 with reduced detection accuracy. "Detecting with reduced detection accuracy" means detecting abnormal shadows even with low reliability, while accepting the possibility of non-detection or erroneous detection of abnormal shadows. In this case, the notification unit 15 preferably notifies the user that the detection accuracy of abnormal shadows is low for the unqualified region N1. This is particularly preferable when the ratio of the unqualified region N1 to the first region of interest A1 is equal to or greater than a predetermined threshold (e.g., 20% or greater). This is because if abnormal shadows were not detected in the unqualified region N1 when the ratio of the unqualified region N1 was high, the user would have to visually check for abnormal shadows in a large portion of the first image G1, thereby eliminating the advantage of CAD. According to this embodiment, it is possible to utilize the detection result of abnormal shadows for the entire first image G1 for diagnosis, while recognizing that the detection accuracy is low in unclear regions.

[0054] Furthermore, in the first embodiment described above, the detection unit 14 detects abnormal shadows, but the information processing device 10 according to this embodiment does not need to have the function of the detection unit 14 (i.e., the CAD function), and the user may visually check for abnormal shadows. According to this embodiment, by making the user aware of the presence of blurred areas in the first image G1, it is possible to prevent the user from overlooking abnormal shadows, and therefore even a blurred first image G1 can be used for diagnosis by the user.

[0055] Furthermore, in the first embodiment, when the determination unit 13 determines that the first region of interest A1 contains an unqualified region N1, the extraction unit 12 may re-extract the first region of interest A1 with reduced extraction accuracy. "Re-extracting the first region of interest A1 with reduced extraction accuracy" means that the extraction conditions for the first region of interest A1 are changed so that other regions (such as the background in FIG. 5) may be extracted as the first region of interest A1 and the unqualified region N1 is reduced. These conditions are determined, for example, by pixel brightness values, etc.

[0056] In this case, it is preferable that the presenting unit 15 presents that the accuracy of extraction of the first region of interest A1 is low. This is particularly preferable when the ratio of the unqualified region N1 to the first region of interest A1 is equal to or greater than a predetermined threshold (e.g., 20% or greater). This is because if a large portion of the first image G1 is determined to be the unqualified region N1, it becomes difficult to complement the unqualified region N1 (details will be described later). This configuration makes the user aware that the accuracy of extraction of the first region of interest A1 is low, thereby preventing the user from overlooking abnormal shadows. Therefore, even a blurred first image G1 can be used for diagnosis by the user.

[0057] [Second embodiment] In the first embodiment, the presence or absence of an unqualified region N1 in the first image G1 was determined. The information processing device 10 according to this embodiment has a function of complementing the unqualified region N1 using a medical image other than the first image G1 when it is determined that the unqualified region N1 exists in the first image G1. An example of the configuration of the information processing device 10 according to this embodiment will be described below, but redundant description of the configuration and operation similar to those of the first embodiment will be omitted.

[0058] An example of the functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 8. As shown in Fig. 8, the information processing device 10 according to this embodiment includes a synthesis unit 16 in addition to the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, and presentation unit 15 similar to those in the first embodiment. The CPU 21 executes an information processing program 27, thereby functioning as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, presentation unit 15, and synthesis unit 16.

[0059] 9 shows an example of a screen D2 presented on the display 24 by the presenting unit 15. As shown in Fig. 9, when the determining unit 13 determines that an incompatible region N1 exists in the first region of interest A1 of the first image G1, the presenting unit 15 requests a second image G2 that includes an area corresponding to at least a part of the incompatible region N1.

[0060] The acquisition unit 11 acquires a second image G2 from the image acquisition device 2. FIG. 10 shows the second image G2. The second image G2 is an image of the fundus similar to the medical image G0, and a region N12 (shown by a dashed line) corresponding to the unsuitable region N1 in the first image G1 is clearly visible. However, a portion of the imaging range is blurred, and an unsuitable region N2 exists that is unsuitable for extracting the actual second region of interest A2 (i.e., the entire imaging range). The unsuitable region N2 actually contains an abnormal shadow S2, but its blurring makes its detection difficult.

[0061] The composition unit 16 generates a third image G3 by combining the first image G1 and the second image G2 acquired by the acquisition unit 11. Specifically, the composition unit 16 generates the third image G3 by combining a portion of one of the first image G1 and the second image G2 with the other image so that an incompatible area in the other image is complemented by a corresponding area in the other image. In the third image G3 shown in Fig. 11, a portion of the second image G2 is combined with the first image G1 so that an incompatible area N1 in the first image G1 is complemented by an area N12 in the second image G2 that corresponds to the incompatible area N1.

[0062] The extraction unit 12 extracts the third region of interest A3 from the third image G3 synthesized by the synthesis unit 16 in the same manner as the extraction unit 12 extracted the first region of interest A1 from the first image G1.

[0063] The determination unit 13 determines whether or not an unqualified area exists in the third region of interest A3 extracted by the extraction unit 12. For example, if a portion of region N12 in the second image G2 corresponding to the unqualified region N1 in the first image G1 is unclear, the second image G2 alone cannot fully complement the first image G1. Therefore, the CPU 21 repeatedly requests and acquires a new image including an area corresponding to at least a portion of the unqualified region and recombines the new image with the third image G3 until the determination unit 13 determines that no unqualified area exists in the third region of interest A3. The request, acquisition, and composition of the new image are performed in the same manner as the request, acquisition, and composition of the second image G2 described above.

[0064] When the determination unit 13 determines that no unsuitable region exists in the third region of interest A3, the detection unit 14 detects abnormal shadows included in the third region of interest A3 extracted by the extraction unit 12 in the same manner as the detection of abnormal shadows included in the first region of interest A1. In the example of the third image G3 in Fig. 11, the detection unit 14 can detect both abnormal shadows S1 and S2.

[0065] 12 shows an example of a screen D3 presented on the display 24 by the presentation unit 15. As shown in Fig. 12, the presentation unit 15 presents the abnormal shadows S1 and S2 detected by the detection unit 14 by adding annotations M on the third image G3.

[0066] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 13. The CPU 21 executes the information processing program 27, thereby executing the image synthesis process shown in Fig. 13. The image synthesis process shown in Fig. 13 is executed after step S14 in the flowchart of Fig. 7. That is, it is executed when it is determined in the determination process of the first embodiment that an incompatible area N1 exists in the first image G1.

[0067] In step S31, the presenting unit 15 requests a second image G2 that includes an area corresponding to at least a portion of the incompatible area N1 identified in step S14. In step S32, the acquiring unit 11 acquires the second image G2 from the image acquiring device 2. In step S33, the combining unit 16 combines the first image G1 acquired in step S10 and the second image G2 acquired in step S32 to generate a third image G3.

[0068] In step S34, the extraction unit 12 extracts a third region of interest A3 from the third image G3 synthesized in step S33. In step S35, the determination unit 13 determines whether or not an inappropriate region that is inappropriate for extraction exists in the third region of interest A3 extracted in step S34.

[0069] If an unsuitable region exists (i.e., if step S35 is determined to be positive), the process proceeds to step S36, where the presenting unit 15 requests a new image including an area corresponding to at least a portion of the unsuitable region. In step S37, the acquiring unit 11 acquires the new image from the image acquiring device 2. In step S38, the combining unit 16 re-combines the new image acquired in step S37 with the third image G3 combined in step S33. When step S38 is completed, the process returns to step S34. That is, the processes of steps S34 to S38 are repeated until it is determined in step S35 that no unsuitable region exists in the third region of interest A3.

[0070] On the other hand, if no incompatible region exists (i.e., if step S35 returns a negative determination), the process proceeds to step S39, where the detection unit 14 detects an abnormal shadow included in the third region of interest A3 extracted in step S34. In step S40, the presentation unit 15 presents the abnormal shadow detected in step S39, and the image synthesis process ends.

[0071] As described above, the information processing device 10 according to the second embodiment includes at least one processor. When the processor determines that an unqualified region N1 exists in the first region of interest A1 of the first image G1, the processor requests a second image including an area corresponding to at least a portion of the unqualified region N1 and combines the first image G1 and the second image G2. That is, when the information processing device 10 determines that the unqualified region N1 exists in the first image G1, the information processing device 10 complements the unqualified region N1 using the second image G2, which is different from the first image G1. Therefore, even if the first image G1 and the second image G2 are unclear, they can be used for diagnosis.

[0072] In the second embodiment, the composition unit 16 complements the incompatible region in either the first image G1 or the second image G2 with the other image, but this is not limiting. For example, the composition unit 16 may generate the third image G3 by selecting and combining an image with better image quality for each of multiple sections in the first image G1 and the second image G2. A "section" refers to, for example, a pixel or a block composed of multiple pixels. The quality of image quality can be evaluated based on pixel values ​​such as hue, saturation, luminance, and brightness of each pixel. Furthermore, these configurations may be combined so that the composition unit 16 complements the incompatible region with the other image, while selecting and combining an image with better image quality for each of the other sections.

[0073] Furthermore, in the second embodiment, particularly when the composition unit 16 repeats the recomposition based on the new image more than a predetermined number of times (e.g., three times), the extraction unit 12 may re-extract the third region of interest A3 with reduced extraction accuracy. This is to terminate the process if the completion of the incompatible region is not completed even after repeated recomposition. In this case, it is preferable that the presentation unit 15 presents a message that the extraction accuracy of the third region of interest A3 is low.

[0074] Furthermore, in the second embodiment, the acquisition and synthesis of new images are repeated until no unsuitable areas exist in the third image G3. However, this is not limiting. The synthesis process may be terminated even if an unsuitable area exists as long as synthesis is performed by synthesis unit 16 at least once. For example, a limit may be set on the number of synthesis attempts (e.g., three times), and if the number of synthesis attempts exceeds the limit, the process may be terminated even if an unsuitable area exists. Furthermore, for example, if the ratio of unsuitable areas to the third region of interest A3 in the third image G3 falls below a predetermined threshold (e.g., 5% or less), the process may be terminated even if an unsuitable area exists. In these cases, presentation unit 15 may present the ratio of unsuitable areas to the third region of interest A3.

[0075] Furthermore, if processing is terminated when an unqualified region is present, structures such as abnormal shadows in the third region of interest A3 may be detected with reduced detection accuracy. In this case, the presentation unit 15 preferably presents a message indicating that the detection accuracy of abnormal shadows in the unqualified region is low. This is particularly preferable when the ratio of unqualified regions to the third region of interest A3 is equal to or greater than a predetermined threshold (e.g., 20% or greater). This is because if abnormal shadows are not detected in the unqualified region when the ratio of unqualified regions is high, the user would have to visually check for abnormal shadows in much of the third image G3, thereby eliminating the benefits of CAD. In this manner, the user can utilize the abnormal shadow detection results for the entire third image G3 for diagnosis, while recognizing that detection accuracy is low in unclear regions.

[0076] Furthermore, in the second embodiment, the detection unit 14 detects abnormal shadows, but the information processing device 10 according to this embodiment does not need to have the function of the detection unit 14 (i.e., the CAD function), and the user may visually check for abnormal shadows. Even in this embodiment, the first image G1 and the second image G2, each of which is unclear, can be used for diagnosis by the user.

[0077] In the second embodiment, the image synthesis process is performed when it is determined in the determination process of the first embodiment that the unsuitable region N1 exists in the first image G1. However, the present invention is not limited to this. For example, the image synthesis process according to this embodiment may be performed when the ratio of the unsuitable region N1 to the first region of interest A1 in the first image G1 is equal to or greater than a predetermined threshold (e.g., 20% or greater).

[0078] [Third embodiment] In the second embodiment, abnormal shadows were detected based on a third image G3 obtained by combining a first image G1 and a second image G2. The information processing device 10 according to this embodiment has a function of detecting abnormal shadows based on each of the first image G1 and the second image G2 and combining the detection results. An example of the configuration of the information processing device 10 according to this embodiment will be described below, but redundant descriptions of configurations and operations similar to those of the first and second embodiments will be omitted.

[0079] As in the second embodiment, an example of the functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 8. As shown in Fig. 8, the information processing device 10 according to this embodiment includes a synthesis unit 16 in addition to the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, and presentation unit 15 similar to those in the first embodiment. The CPU 21 executes an information processing program 27, thereby functioning as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, presentation unit 15, and synthesis unit 16.

[0080] 9 shows an example of a screen D2 presented on the display 24 by the presenting unit 15. As shown in FIG. 9, when the determining unit 13 determines that an unsuitable region N1 exists in the first region of interest A1 of the first image G1, the presenting unit 15 requests a second image G2 including an area corresponding to at least a part of the unsuitable region N1. The acquiring unit 11 acquires the second image G2 (see FIG. 10) from the image acquiring device 2. The description of the second image G2 is omitted as it is the same as that in the second embodiment.

[0081] The extraction unit 12 extracts the second region of interest A2 from the second image G2 acquired by the acquisition unit 11 using a method similar to that used to extract the first region of interest A1 from the first image G1. The detection unit 14 detects abnormal shadows included in the first region of interest A1 and the second region of interest A2 extracted by the extraction unit 12 using a method similar to that used to detect abnormal shadows included in the first region of interest A1. In the examples of FIGS. 5 and 10 , the detection unit 14 detects an abnormal shadow S2 from the first image G1 and an abnormal shadow S1 from the second image G2.

[0082] The combining unit 16 combines the detection results of abnormal shadows detected from the first region of interest A1 and the second region of interest A2 by the detecting unit 14. That is, the combining unit 16 combines the detection results of abnormal shadows obtained from a plurality of different images into a detection result of abnormal shadows for the same subject.

[0083] The determination unit 13 determines whether the first region of interest A1 and the second region of interest A2 extracted by the extraction unit 12 have a common unsuitable region that is unsuitable for extraction. The presence of a common unsuitable region may lead to the overlooking of an abnormal shadow. Therefore, the CPU 21 repeatedly requests and acquires a new image containing an area corresponding to at least a portion of the common unsuitable region, extracts a region of interest from the new image, detects an abnormal shadow contained in the region of interest, and recombines the detection result of the abnormal shadow with the detection results of abnormal shadows detected up to that point, until the determination unit 13 determines that no common unsuitable region exists. The request and acquisition of the new image, extraction of the region of interest, and detection of the abnormal shadow are performed in the same manner as the request and acquisition of the second image G2, extraction of the region of interest, and detection of the abnormal shadow described above.

[0084] Fig. 14 shows an example of a screen D4 presented on the display 24 by the presentation unit 15. As shown in Fig. 14, the presentation unit 15 presents the detection results of the abnormal shadows combined by the composition unit 16 by attaching an annotation M to each image. Note that, as in the second embodiment, the composition unit 16 may generate an image by combining the first image G1 and the second image G2, and the presentation unit 15 may present the detection results of the abnormal shadows combined by the composition unit 16 by attaching an annotation M to the image (i.e., one image).

[0085] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 15. The CPU 21 executes the information processing program 27, thereby executing the detection result combination process shown in Fig. 15. The detection result combination process shown in Fig. 15 is executed after step S14 in the flowchart of Fig. 7. That is, it is executed when it is determined in the determination process of the first embodiment that an incompatible area N1 exists in the first image G1.

[0086] In step S51, the presenting unit 15 requests a second image G2 including an area corresponding to at least a portion of the unsuitable area N1 identified in step S14. In step S52, the acquiring unit 11 acquires the second image G2 from the image acquiring device 2. In step S53, the extracting unit 12 extracts a second area of ​​interest A2 from the second image G2 acquired in step S52. In step S54, the detecting unit 14 detects an abnormal shadow included in the second area of ​​interest A2 extracted in step S53.

[0087] In step S55, the combining unit 16 combines the detection result of the abnormal shadow included in the first region of interest A1 detected in step S12 with the detection result of the abnormal shadow included in the second region of interest A2 detected in step S54. In step S56, the determining unit 13 determines whether or not a common unsuitable region that is unsuitable for extraction exists in the first region of interest A1 extracted in step S11 and the second region of interest A2 extracted in step S53.

[0088] If a common unqualified region exists (i.e., if step S56 is judged as positive), the processes of steps S51 to S56 are performed on a new image that includes a region corresponding to at least a part of the common unqualified region. That is, the processes of steps S51 to S56 are repeated until it is judged in step S56 that no common unqualified region exists.

[0089] On the other hand, if there is no common incompatible region (i.e., if the determination in step S56 is negative), the process proceeds to step S57, where the presentation unit 15 presents the detection result of the abnormal shadows synthesized in step S55. When step S57 is completed, this detection result synthesis process ends.

[0090] As described above, the information processing device 10 according to the third embodiment includes at least one processor. When the processor determines that an unqualified region N1 exists in the first region of interest A1 of the first image G1, the processor requests a second image including a region corresponding to at least a portion of the unqualified region N1. The processor also extracts the second region of interest A2 from the second image G2, detects abnormal shadows contained in both the first region of interest A1 and the second region of interest A2, and combines the abnormal shadow detection results. That is, when the information processing device 10 determines that the unqualified region N1 exists in the first image G1, the information processing device 10 complements the abnormal shadow detection results using the second image G2, which is different from the first image G1. Therefore, even if the first image G1 and the second image G2 are unclear, they can be used for diagnosis.

[0091] In the third embodiment, particularly when the synthesis unit 16 repeats the resynthesis of the detection results of abnormal shadows contained in the new image more than a predetermined number of times (e.g., three times), the extraction unit 12 may extract the region of interest in the new image with reduced extraction accuracy. This is to terminate the process if the common unsuitable region remains even after repeated resynthesis. In this case, it is preferable that the presentation unit 15 presents a message that the accuracy of the extraction of the region of interest is low.

[0092] In the third embodiment, the synthesis of the detection results of abnormal shadows based on new images is repeated until no common unsuitable regions exist. However, the present invention is not limited to this. As long as the synthesis unit 16 synthesizes the detection results at least once, the synthesis process may be terminated even if a common unsuitable region exists. For example, a limit may be set on the number of synthesis attempts (e.g., three times), and if the number of synthesis attempts exceeds the limit, the process may be terminated even if a common unsuitable region exists. Furthermore, for example, if the ratio of common unsuitable regions to the first region of interest A1 or the second region of interest A2 is equal to or less than a predetermined threshold (e.g., 5% or less), the process may be terminated even if a common unsuitable region exists.

[0093] Furthermore, if processing is terminated when a common unsuitable region exists, structures such as abnormal shadows contained in the common unsuitable region may be detected with reduced detection accuracy. In this case, the notification unit 15 preferably notifies the user that the detection accuracy of abnormal shadows is low for the common unsuitable region. This is particularly preferable when the ratio of common unsuitable regions to the first region of interest A1 or the second region of interest A2 is equal to or greater than a predetermined threshold (e.g., 20% or greater). This is because if abnormal shadows are not detected in the common unsuitable region when the ratio of common unsuitable regions is high, the user would have to visually check for abnormal shadows in many parts of the first image G1 and the second image G2, thereby losing the advantage of CAD. In this manner, the user can utilize the abnormal shadow detection results for the entire first image G1 and the second image G2 for diagnosis, while recognizing that detection accuracy is low for blurred regions.

[0094] In the third embodiment, the detection result combining process is performed when it is determined that the unqualified region N1 exists in the first image G1 in the determination process of the first embodiment. However, the present invention is not limited to this. For example, the detection result combining process according to this embodiment may be performed when the ratio of the unqualified region N1 to the first region of interest A1 in the first image G1 is equal to or greater than a predetermined threshold (e.g., 20% or greater).

[0095] [Fourth embodiment] In the first to third embodiments, the extraction unit 12 extracts a region of interest from a medical image. As described above, the extraction unit 12 can extract a region of interest by using a trained model that has been trained to input a medical image, extract, and output a region of interest. In this case, the trained model is required to be able to extract a region of interest with high accuracy even when a blurred medical image such as that shown in FIGS. 5 and 10 is input.

[0096] One method for improving the accuracy of this trained model is to use various patterns of blurred medical images as training data for training. However, it has been difficult to acquire a sufficient number and patterns of blurred medical images using the image acquisition device 2. Therefore, in this embodiment, the purpose is to improve the accuracy of the trained model by intentionally generating blurred medical images and using them as training data.

[0097] As an example, a description will be given of a case in which the information processing device 10 according to this embodiment trains the learning model 4 used in the extraction unit 12 by unsupervised learning. The learning model 4 is configured to include deep learning models such as a convolutional neural network (CNN), a fully convolutional network (FCN), and a U-Net, and is trained to input a medical image, extract, and output a region of interest. Furthermore, for example, the techniques described in Patent Document 1, JP 2019-088458 A, and JP 2020-114302 A may be applied as such a model.

[0098] The CPU 21 acquires an original image obtained by capturing an image of the subject from the image capture device 2. That is, the original image is an image acquired by at least one of the image capture devices 2, a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope. The original image also includes a region of interest that includes at least one of the subject, a portion of tissue contained in the subject, and structures such as an abnormality contained in the subject or tissue. Below, an example of the original image will be described using a clear medical image G0 shown in FIG. 4. The medical image G0 has been described above, so a description thereof will be omitted.

[0099] The CPU 21 generates a pseudo image by changing the pixel values ​​of at least a portion of the medical image G0. The pixel value is a value indicating at least one of the hue, saturation, luminance, and brightness of each pixel in the medical image G0. For example, the pixel value of each pixel is changed by changing the brightness and contrast of the medical image G0 or by adding blurring and noise. However, the CPU 21 does not change the resolution when generating the pseudo image.

[0100] Specific examples of pseudo images will be described with reference to Fig. 16. Pseudo images P1 to P5 shown on the left side of Fig. 16 are pseudo images generated based on the medical image G0, and are input to the learning model 4 as learning data. Pseudo images P1 and P2 are images of the medical image G0 with the brightness increased and darkened, respectively. Pseudo image P3 is an image of the medical image G0 with the contrast decreased. Pseudo images P4 and P5 are images of the medical image G0 with some regions darkened.

[0101] Also, on the right side of Figure 16, the regions of interest extracted by learning model 4 during learning for each of the input pseudo images are shown surrounded by thick lines. For pseudo images P1 and P2, the regions of interest are properly extracted. For pseudo image P3, an error indicating extraction is not possible is output. For pseudo images P4 and P5, some blurry regions that should have been extracted as regions of interest are not extracted as regions of interest.

[0102] 16, the pseudo image may be one in which the pixel values ​​of the entire medical image G0 have been changed, or one in which the pixel values ​​of only a portion of the medical image G0 have been changed. Furthermore, as shown in pseudo images P4 and P5, it is preferable to use multiple pseudo images generated by changing the pixel values ​​of different regions of a single medical image G0 as training data. This configuration allows for a reduction in the number of original images, thereby enabling efficient training.

[0103] Furthermore, as shown in pseudo images P1 to P5, it is preferable that the pseudo images are images generated by changing pixel values ​​in at least a portion of the medical image G0 so as to degrade the image quality. Specifically, "degrading the image quality" refers to processing that makes structures contained in the medical image G0 difficult to detect, such as processing that weakens the contrast. This is because, considering the operational phase of the learning model, if the input medical image is a fundus image, the image may be darkened due to insufficient light or brightened due to ambient light, resulting in weaker contrast than a clear medical image.

[0104] 16, the pseudo image is preferably an image generated by changing pixel values ​​of a region in the medical image G0 that includes at least the region of interest, because whether or not the region other than the region of interest is blurred is not a particular issue.

[0105] Furthermore, it is preferable that the pseudo image is an image generated by changing at least some pixel values ​​of the medical image G0 while maintaining the presence or absence of structures contained in the medical image G0. This is because the learning model 4 prioritizes accurately extracting a region of interest even from a blurred medical image, and responding to changes in the presence or absence of structures is a low priority.

[0106] The pseudo image may also be an image generated based on the medical image G0 using an image generation model such as GAN (Generative Adversarial Networks) and VAE (Variational Autoencoder).

[0107] The CPU 21 trains the learning model 4 by inputting the pseudo images P1 to P5 generated as described above into the learning model 4 as learning data. Also, as shown in pseudo images P3 to P5 in FIG. 16, if the learning model 4 is unable to properly extract a region of interest from the input pseudo image, the CPU 21 may re-train the learning model 4 by inputting the pseudo image again as learning data. This configuration can improve the accuracy of the learning model 4.

[0108] Note that, when re-learning, a correct label may be assigned to the pseudo image. Specifically, the CPU 21 may input a pair of a pseudo image and information indicating a region of interest included in the pseudo image as learning data to the learning model 4, and re-learn the learning model 4. The information indicating the region of interest is, for example, information indicating the position of the region of interest in the pseudo image.

[0109] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 17. The CPU 21 executes the information processing program 27, thereby executing the learning process shown in Fig. 17. The learning process shown in Fig. 17 is executed, for example, when a command to start the process is received from the user via the input unit 25.

[0110] In step S71, CPU 21 acquires an original image obtained by capturing an image of a subject from image acquisition device 2. In step S72, CPU 21 generates a pseudo image by changing at least some pixel values ​​of the original image acquired in step S71. In step S73, CPU 21 uses the pseudo image generated in step S72 as learning data to train learning model 4. In step S74, CPU 21 determines whether learning model 4 was able to properly extract a region of interest from the input pseudo image.

[0111] If the learning model 4 is unable to properly extract the region of interest from the input pseudo image (i.e., if step S74 is judged negative), the process returns to step S73, and the CPU 21 re-inputs the pseudo image into the learning model 4, causing the learning model 4 to re-learn. That is, re-learning using the same pseudo image is repeated until the learning model 4 can properly extract the region of interest from the pseudo image. When the learning model 4 is able to properly extract the region of interest from the input pseudo image (i.e., when step S74 is judged positive), this learning process ends.

[0112] As described above, the information processing device 10 according to the fourth embodiment includes at least one processor, and the processor inputs, as training data, pseudo images generated by changing at least some pixel values ​​of an original image obtained by imaging a subject, and trains a learning model for extracting a region of interest from the input image. Therefore, even if an image is unsuitable for extracting a region of interest, the region of interest can be appropriately extracted and used for diagnosis.

[0113] In the fourth embodiment, the information processing device 10 trains the learning model 4 used in the extraction unit 12 by unsupervised learning, but the present invention is not limited to this. The information processing device 10 may train the learning model 4 by supervised learning and semi-supervised learning. Specifically, the CPU 21 may train the learning model 4 by inputting a pair of a pseudo image and information indicating a region of interest contained in the pseudo image as training data. In this case, if the learning model 4 fails to extract a region of interest from the input pseudo image, the CPU 21 may re-train the learning model 4 by inputting the pseudo image as training data again.

[0114] Although the above embodiments have been described using medical images, the technology of the present disclosure is not limited to medical images. For example, the technology of the present disclosure may be applied to images obtained from equipment, buildings, pipes, welds, and the like as test objects in non-destructive testing such as radiographic testing and ultrasonic flaw detection testing.

[0115] Furthermore, in the above-described embodiments, the information processing system 1 includes the information processing device 10 and the image acquisition device 2, but this is not limiting. For example, the information processing system 1 may include a single device that has both the functions of the information processing device 10 and the functions of the image acquisition device 2. Alternatively, for example, the information processing system 1 may include multiple image acquisition devices 2, and the information processing device 10 may acquire medical images from each of the multiple image acquisition devices 2. Alternatively, for example, the information processing device 10 may be configured to include multiple devices with different functions, such as an acquisition unit 11, an extraction unit 12, a determination unit 13, a detection unit 14, a presentation unit 15, and a synthesis unit 16.

[0116] Furthermore, in each of the above embodiments, the following various processors can be used as the hardware structure of processing units that perform various processes, such as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, presentation unit 15, and synthesis unit 16. As described above, the various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as dedicated electrical circuits that are processors having a circuit configuration specifically designed to perform specific processes, such as a programmable logic device (PLD), a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and an ASIC (Application Specific Integrated Circuit).

[0117] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0118] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, and this processor functions as multiple processing units, as typified by computers such as client and server. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0119] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0120] In addition, in each of the above embodiments, the information processing program 27 is described as being pre-stored (installed) in the storage unit 22, but this is not limiting. The information processing program 27 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 27 may also be downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to not only information processing programs but also storage media that non-temporarily store information processing programs.

[0121] The technology of the present disclosure can also be achieved by appropriately combining the above-described exemplary embodiments. The above-described description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure and are merely examples of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or new elements may be substituted from the above-described description and illustrations, within the scope of the gist of the technology of the present disclosure. [Explanation of symbols]

[0122] 1. Information Processing Systems 2. Image acquisition device 4. Learning Model 10. Information processing equipment 11 Acquisition Department 12 Extraction part 13 Judgment section 14 Detector 15 Presentation section 16 Synthesis section 21 CPU 22 Memory section 23 Memory 24 displays 25 Input section 26 Network I / F 27 Information Processing Program 28 Bus A0 Area of ​​Interest A1 First area of ​​interest A2 Second Area of ​​Interest A3 Third Area of ​​Interest D1~D4 screen G0 Medical Imaging G1 1st image G2 2nd image G3 3rd image M annotation N1, N2 Nonconformity area N12 area P1~P5 Pseudo image S1, S2 abnormal shadow

Claims

1. at least one processor; The processor: acquiring a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not there is an unsuitable region that is unsuitable for extraction in the extracted first region of interest; performing control to present the unsuitable region in the first image in a manner that the unsuitable region is distinguishable from other regions; If it is determined that the unsuitable region exists, the first region of interest is re-extracted with reduced extraction accuracy. Information processing device.

2. The processor: The determination is made according to the degree of similarity between the shape of the extracted first region of interest and a predetermined reference shape for the first region of interest. The information processing device according to claim 1 .

3. The processor: An image obtained by photographing a subject is input, and the determination is made using a trained model for determining whether the unsuitable region exists in a region of interest extracted from the image; The trained model is a training model trained using a pair of an image obtained by photographing a subject and information indicating whether the non-conforming region exists in a region of interest extracted from the image as training data.

3. The information processing device according to claim 1.

4. The processor: and controlling the display of a character string indicating whether or not the incompatible area exists on the screen on which the first image is displayed. The information processing device according to any one of claims 1 to 3.

5. The processor: When it is determined that the unsuitable region exists, control is performed to display the ratio of the unsuitable region to the first region of interest. The information processing device according to any one of claims 1 to 4.

6. The processor: Detecting a structure included in the first region of interest; When it is determined that the unsuitable area exists, the structure included in the unsuitable area is detected with reduced detection accuracy. The information processing device according to any one of claims 1 to 5.

7. The processor: Detecting abnormal shadows from the other regions, and not detecting abnormal shadows from the non-conforming regions. The information processing device according to any one of claims 1 to 6.

8. The processor: and controlling the display of the abnormal shadow detected from the other region on the first image. The information processing device according to claim 7 .

9. If the processor determines that the non-conforming region exists, acquiring a second image including an area corresponding to at least a portion of the non-conforming area; A third image is generated by combining the first image and the second image. The information processing device according to any one of claims 1 to 8.

10. The processor: generating the third image by combining a portion of one of the first image and the second image with a portion of the other image such that the incompatible region in the other image is complemented by a corresponding region in the other image; The information processing device according to claim 9 .

11. The processor: The third image is generated by selecting one of the images with better image quality for each of a plurality of sections in the first image and the second image and combining them.

11. The information processing device according to claim 9 or 10.

12. The processor: extracting a third region of interest from the third image; determining whether or not the unsuitable region exists in the extracted third region of interest; until it is determined that the unconforming region does not exist in the third region of interest. obtaining a new image that includes an area corresponding to at least a portion of the non-conforming area; and recombining the new image with the third image. The information processing device according to any one of claims 9 to 11.

13. The processor: extracting a third region of interest from the third image; Detecting a structure included in the third region of interest The information processing device according to any one of claims 9 to 12.

14. If the processor determines that the non-conforming region exists, acquiring a second image including an area corresponding to at least a portion of the non-conforming area; extracting a second region of interest from the second image; Detecting structures included in each of the first region of interest and the second region of interest; Combining the detection results of the structures detected from the first region of interest and the second region of interest. The information processing device according to any one of claims 1 to 13.

15. The processor: determining whether or not a common unsuitable region that is unsuitable for extraction exists in the extracted first region of interest and the extracted second region of interest; until it is determined that the common incompatible region does not exist. obtaining a new image that includes a region corresponding to at least a portion of the common unmatched region; Extracting a region of interest from the new image; detecting structures included in the region of interest; Repeatedly recombine the detection results of the structure. The information processing device according to claim 14.

16. The first region of interest is a region including at least one of the subject, a part of tissue contained in the subject, and an abnormal part contained in the subject or the tissue. The information processing device according to any one of claims 1 to 15.

17. The first image is an image obtained by at least one of a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope. The information processing device according to any one of claims 1 to 16.

18. acquiring a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not there is an unsuitable region that is unsuitable for extraction in the extracted first region of interest; performing control to present the unsuitable region in the first image in a manner that the unsuitable region is distinguishable from other regions; If it is determined that the unsuitable region exists, the first region of interest is re-extracted with reduced extraction accuracy. An information processing method in which processing is performed by a computer.

19. acquiring a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not there is an unsuitable region that is unsuitable for extraction in the extracted first region of interest; performing control to present the unsuitable region in the first image in a manner that the unsuitable region is distinguishable from other regions; If it is determined that the unsuitable region exists, the first region of interest is re-extracted with reduced extraction accuracy. An information processing program that causes a computer to execute a process.

Images