Information processing device, information processing method, and information processing program

Abstract

To provide a device, method and program for processing information, which allow for utilizing even an image not suitable for extraction of a region of interest for diagnosis.SOLUTION: An information processing device 10 disclosed herein comprises at least one processor configured to acquire a first image obtained by photographing an object, extract a first region of interest from the first image, and determine the presence or absence of an unsuitable region not suitable for extraction in the extracted first region of interest.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Conventionally, doctors generally make diagnoses based on medical images acquired by image acquisition devices such as CT (Computed Tomography) devices and MRI (Magnetic Resonance Imaging) devices. Also known is a technology (so-called CAD (Computer Aided Detection / Diagnosis)) that uses a computer to assist in the detection and diagnosis of structures such as abnormal shadows and tissues contained in medical images. For example, Patent Document 1 describes a method of analyzing medical images using a classifier trained by machine learning to identify the type of tissue or lesion contained in the medical image, i.e., the type of findings.

[0003] For example, Non-Patent Document 1 describes that CAD is not performed when at least a part of a medical image is unclear in order to avoid non-detection and misdiagnosis of abnormal shadows. A unclear medical image can be obtained, for example, when the image is not taken in an appropriate environment or when the photographer has low imaging skills. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-032043 A [Non-patent literature]

[0005] [Non-Patent Document 1] "A Human-Centered Evaluation of a Deep Learning System Deployed in Clinics for the Detection of Diabetic Retinopathy", Emma Beede et al., in CHI 2020 Paper 589 Summary of the Invention [Problem to be solved by the invention]

[0006] However, for example, in medical institutions in emerging countries, it is sometimes difficult to obtain clear medical images due to the inability to create an appropriate environment for taking medical images or the photographer's low imaging skills. In recent years, therefore, there has been a demand for a technology that allows even unclear images, that is, images that are not suitable for extracting regions of interest that may include abnormal shadows and structures such as tissues, to be used for diagnosis.

[0007] The present disclosure provides an information processing device, an information processing method, and an information processing program that enable even images that are unsuitable for extracting a region of interest to be utilized for diagnosis. [Means for solving the problem]

[0008] One aspect of the present disclosure is an information processing device including at least one processor, the processor acquiring a first image obtained by imaging a subject, extracting a first region of interest from the first image, and determining whether or not a region that is unsuitable for extraction is present in the extracted first region of interest. and when it is determined that a non-conforming region exists in the first region of interest, the first region of interest is re-extracted with reduced extraction accuracy. .

[0009] In the above aspect, the processor may make the determination depending on a degree of similarity between a shape of the extracted first region of interest and a reference shape that is predetermined for the first region of interest.

[0010] In the above aspect, the processor may input an image obtained by photographing a subject, and make a judgment using a trained model for determining whether or not an unsuitable region exists in a region of interest extracted from the image.

[0011] In the above aspect, the trained model may be a trained model trained using a pair of an image obtained by photographing a subject and information indicating whether or not an unsuitable region exists in a region of interest extracted from the image as training data.

[0012] In the above aspect, when the processor determines that an unsuitable region exists in the first region of interest, the processor may identify and present the unsuitable region in the first image.

[0013] In the above aspect, when the processor determines that an unsuitable region exists in the first region of interest, the processor may present a ratio of the unsuitable region to the first region of interest.

[0015] In the above aspect, the processor may detect a structure included in the first region of interest, and if it determines that an unsuitable region exists in the first region of interest, it may detect the structure included in the unsuitable region by reducing the detection accuracy.

[0016] In the above aspect, if the processor determines that an unsuitable region exists in the first region of interest, the processor may request a second image including an area corresponding to at least a portion of the unsuitable region.

[0017] In the above aspect, the processor may acquire a second image and generate a third image by combining the first image and the second image.

[0018] In the above aspect, the processor may generate a third image by combining a portion of one of the first and second images with a portion of the other image such that an incompatible area in one of the images is complemented by a corresponding area in the other image.

[0019] In the above aspect, the processor may generate the third image by selecting one of the images having better image quality for each of a plurality of sections in the first image and the second image and combining the images.

[0020] In the above aspect, the processor may extract a third region of interest from the third image, determine whether or not an unsuitable region exists in the extracted third region of interest, and repeatedly obtain a new image including an area corresponding to at least a portion of the unsuitable region and recombine the new image with the third image until it is determined that no unsuitable region exists in the third region of interest.

[0021] In the above embodiment, the processor may extract a third region of interest from the third image, and detect a structure included in the third region of interest.

[0022] In the above aspect, the processor may acquire a second image, extract a second region of interest from the second image, detect structures contained in each of the first region of interest and the second region of interest, and combine detection results of the structures detected from each of the first region of interest and the second region of interest.

[0023] In the above aspect, the processor may determine whether or not a common unsuitable region that is unsuitable for extraction exists for the extracted first and second regions of interest, and may repeatedly perform the following steps until it is determined that no common unsuitable region exists: acquiring a new image including an area corresponding to at least a portion of the common unsuitable region; extracting the region of interest from the new image; detecting a structure included in the region of interest; and recombining the detection results of the structure.

[0024] In the above aspect, the first region of interest may be a region including at least one of the subject, a portion of tissue contained in the subject, and an abnormality contained in the subject or tissue.

[0025] In the above aspect, the first image may be an image obtained by at least one of a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope. In the above aspect, when the first region of interest is re-extracted with reduced extraction accuracy, the processor may present a message to the effect that the extraction accuracy of the first region of interest is low. Another aspect of the present disclosure is an information processing device including at least one processor, which acquires a first image obtained by photographing a subject, extracts a first region of interest from the first image, determines whether or not an unsuitable region for extraction exists in the extracted first region of interest, detects a structure included in the first region of interest, and, if it is determined that an unsuitable region exists in the first region of interest, detects the structure included in the unsuitable region by reducing detection accuracy.

[0026] Another aspect of the present disclosure is an information processing method, comprising: acquiring a first image obtained by imaging a subject; extracting a first region of interest from the first image; and determining whether or not a region that is unsuitable for extraction is present in the extracted first region of interest. and when it is determined that a non-conforming region exists in the first region of interest, the first region of interest is re-extracted with reduced extraction accuracy. The processing is executed by a computer. Another aspect of the present disclosure is an information processing method in which a computer executes a process of acquiring a first image obtained by photographing a subject, extracting a first region of interest from the first image, determining whether or not an unsuitable region for extraction exists in the extracted first region of interest, detecting a structure included in the first region of interest, and, if it is determined that an unsuitable region exists in the first region of interest, detecting the structure included in the unsuitable region by reducing detection accuracy.

[0027] Another aspect of the present disclosure is an information processing program, comprising: acquiring a first image obtained by imaging a subject; extracting a first region of interest from the first image; and determining whether or not a region that is unsuitable for extraction is present in the extracted first region of interest. and when it is determined that a non-conforming region exists in the first region of interest, the first region of interest is re-extracted with reduced extraction accuracy. It is for causing a computer to execute the process. Effect of the Invention

[0028] According to the above aspects, the information processing device, the information processing method, and the information processing program disclosed herein can utilize even an image that is not suitable for extracting a region of interest for diagnosis. [Brief description of the drawings]

[0029] [Figure 1] FIG. 1 is a schematic configuration diagram of an information processing system. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device. [Diagram 3] 1 is a block diagram showing an example of a functional configuration of an information processing device according to a first embodiment. [Figure 4] This is an example of a clear medical image. [Diagram 5] 1 is an example of a blurred first image. [Figure 6] 13 is an example of a presented screen. [Figure 7] 13 is a flowchart illustrating an example of a determination process. [Figure 8]FIG. 11 is a block diagram showing an example of a functional configuration of an information processing device according to second and third embodiments. [Figure 9] 13 is an example of a presented screen. [Figure 10] 13 is an example of a blurred second image. [Figure 11] 13 is an example of a synthesized third image. [Figure 12] 13 is an example of a presented screen. [Figure 13] 13 is a flowchart illustrating an example of an image synthesis process. [Figure 14] 13 is an example of a presented screen. [Figure 15] 13 is a flowchart illustrating an example of a detection result synthesis process. [Figure 16] FIG. 13 is a diagram showing an example of a pseudo image input to a learning model. [Figure 17] 13 is a flowchart illustrating an example of a learning process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0030] Hereinafter, examples of embodiments for carrying out the technology of the present disclosure will be described in detail with reference to the drawings.

[0031] [First embodiment] An example of the configuration of an information processing system 1 according to this embodiment will be described with reference to Fig. 1. As shown in Fig. 1, the information processing system 1 includes an information processing device 10 and an image acquisition device 2. The information processing device 10 and the image acquisition device 2 are capable of communicating with each other via wired or wireless communication.

[0032] The image acquisition device 2 is a device (so-called modality) that acquires an image obtained by photographing a subject. In this embodiment, a medical image is acquired by the image acquisition device 2 as an example of a specific image. As the image acquisition device 2, at least one of a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope can be applied, and these may be applied in appropriate combination.

[0033] Incidentally, for example, in order to take a clear image of the fundus, the image should be taken in a dark place, but in medical institutions in emerging countries, it is sometimes difficult to create a sufficiently dark place, resulting in unclear medical images. For example, in order to take a clear image of the breast in a mammogram, the breast should be sufficiently compressed, but depending on the positioning technique of the photographer and the shape of the breast, the breast may not be sufficiently compressed, resulting in unclear medical images. In recent years, there has been a demand for technology that allows such unclear images to be used for diagnosis.

[0034] Therefore, the information processing device 10 according to this embodiment has a function of determining whether or not a medical image includes a blurred area. Note that "blurred" means that the pixel values, such as hue, saturation, luminance, and brightness, of each pixel in the medical image do not satisfy a predetermined reference value. For example, a blurred area may occur when a pixel is darker than the reference value due to insufficient light, or when a pixel is brighter than the reference value due to disturbance light. An example of the configuration of the information processing device 10 according to this embodiment will be described below.

[0035] First, an example of a hardware configuration of the information processing device 10 according to the present embodiment will be described with reference to Fig. 2. As shown in Fig. 2, the information processing device 10 includes a CPU (Central Processing Unit) 21, a non-volatile storage unit 22, and a memory 23 as a temporary storage area. The information processing device 10 also includes a display 24 such as a liquid crystal display, an input unit 25 such as a keyboard and a mouse, and a network I / F 26 that performs wired or wireless communication with the image acquisition device 2 and an external network (not shown). The CPU 21, the storage unit 22, the memory 23, the display 24, the input unit 25, and the network I / F 26 are connected to each other via a bus 28 such as a system bus and a control bus so that various information can be exchanged between them.

[0036] The storage unit 22 is realized by a storage medium such as a hard disk drive (HDD), a solid state drive (SSD), and a flash memory. An information processing program 27 according to this embodiment is stored in the storage unit 22. The CPU 21 reads out the information processing program 27 from the storage unit 22, expands it in the memory 23, and executes the expanded information processing program 27. The CPU 21 is an example of a processor of the present disclosure. As the information processing device 10, various computers such as a console of the image acquisition device 2, a workstation, a server computer, and a personal computer can be applied.

[0037] Next, an example of a functional configuration of the information processing device 10 according to the present embodiment will be described with reference to Fig. 3. As shown in Fig. 3, the information processing device 10 includes an acquisition unit 11, an extraction unit 12, a determination unit 13, a detection unit 14, and a presentation unit 15. The CPU 21 executes an information processing program 27 to function as the acquisition unit 11, the extraction unit 12, the determination unit 13, the detection unit 14, and the presentation unit 15.

[0038] The acquisition unit 11 acquires a medical image obtained by photographing a subject from the image acquisition device 2. The medical image in this embodiment includes a region of interest including the subject, a part of tissue contained in the subject, and at least one abnormal part contained in the subject or tissue as a target structure to be detected and diagnosed. Examples of the subject include the human body and various organs of the human body such as the fundus, lungs, breasts, stomach, liver, heart, and brain. Examples of tissues include elements that constitute various organs such as blood vessels, nerves, and muscles. Examples of abnormal parts include lesions and abnormalities such as tumors, damage, defects, nodules, and inflammation.

[0039] Fig. 4 shows medical image G0 as an example of a clear medical image. Medical image G0 is a clear image of the fundus obtained by a fundus photography device. In an image of the fundus, abnormal areas such as lesions may be included in the entire photographing range including the fovea, macula, optic disc, and blood vessels, so the entire photographing range corresponds to the region of interest A0. In the example of Fig. 4, abnormal shadows S1 and S2 are included in the region of interest A0 as examples of abnormal areas.

[0040] 5 shows a first image G1 as an example of a blurred medical image. The first image G1 is an image of the fundus similar to the medical image G0, but a part of the photographed range is blurred, and an unsuitable area N1 that is unsuitable for extracting the original first region of interest A1 (i.e., the entire photographed range) is present. The unsuitable area N1 actually includes an abnormal shadow S1, but it is difficult to detect it due to the blurred area. The following description will be given on the assumption that the acquisition unit 11 has acquired the first image G1.

[0041] The extraction unit 12 extracts the first region of interest A1 from the first image G1 acquired by the acquisition unit 11. As a method for extracting the first region of interest A1, a method using known image processing, a method using AI (Artificial Intelligence) technology, etc. can be appropriately applied. For example, the first region of interest A1 may be extracted by performing binarization processing on the first image G1, removing the background, and emphasizing the edges of each structure, and identifying the contour of the shooting range. In addition, for example, the shooting range (i.e., the first region of interest A1) may be extracted from the first image G1 using a trained model that is trained to input an image of the fundus and extract and output the shooting range.

[0042] The detection unit 14 detects abnormal shadows included in the first region of interest A1 extracted by the extraction unit 12. In the example of the first image G1 in Fig. 5, the detection unit 14 detects an abnormal shadow S2, but does not detect an abnormal shadow S1 because it is included in an unclear unsuitable region N1. Note that a known CAD technique can be appropriately applied as a method for detecting abnormal shadows.

[0043] The determination unit 13 determines whether or not the first region of interest A1 extracted by the extraction unit 12 includes an unsuitable region N1 that is unsuitable for extraction. Various methods can be applied as a method of determination. For example, the determination unit 13 may perform the determination according to the degree of similarity between the shape of the first region of interest A1 extracted by the extraction unit 12 and a reference shape that is predetermined for the first region of interest A1. In the case of an image of a fundus, since the region of interest is the entire photographing range, the reference shape can be predetermined to be substantially circular. Therefore, when the degree of similarity between the shape of the outline of the first region of interest A1 extracted by the extraction unit 12 and the substantially circular reference shape is equal to or less than a predetermined threshold (i.e., there is a large deviation), it may be determined that the unsuitable region N1 exists. As a method of determination according to the degree of similarity, known matching techniques such as matching based on feature amounts and template matching can be appropriately applied.

[0044] For example, AI technology may be applied to the judgment, and the judgment may be made using a trained model for inputting a medical image and judging whether or not an unsuitable region exists in a region of interest extracted from the medical image. In this case, the trained model may be a model that has undergone unsupervised learning, for example, a model that has been trained to cluster medical images according to the presence or absence of an unsuitable region. Alternatively, the trained model may be a model that has undergone supervised learning, for example, a model that has been trained using a pair of a medical image and information indicating whether or not an unsuitable region exists in a region of interest extracted from the medical image as training data.

[0045] Fig. 6 shows an example of a screen D1 presented on the display 24 by the presentation unit 15. As shown in Fig. 6, the presentation unit 15 presents the abnormal shadow S2 detected by the detection unit 14 by attaching an annotation M to the first image G1.

[0046] Furthermore, when the determination unit 13 determines that the unsuitable region N1 exists in the first region of interest A1, the presentation unit 15 identifies and presents the unsuitable region N1 in the first image G1. In the example of FIG. 6, the presentation unit 15 presents the unsuitable region N1 in the first image G1 by enclosing it with a thick line. Note that the manner of presenting the unsuitable region N1 is not limited to this, and the unsuitable region N1 may be highlighted so as to be distinguishable, for example, by changing the line type (e.g., line thickness, color, solid line, dotted line, etc.) from other regions or by adding an annotation. Conversely, the unsuitable region N1 may be made distinguishable by highlighting regions other than the unsuitable region N1 in the first region of interest A1.

[0047] Furthermore, when the determination unit 13 determines that the unqualified region N1 exists in the first region of interest A1, the presentation unit 15 may present the ratio of the unqualified region N1 to the first region of interest A1. In the example of Fig. 6, the unqualified region N1 is described as an "unfeasible" region in which the detection unit 14 cannot perform detection processing for an abnormal shadow. Moreover, the region other than the unqualified region N1 in the first region of interest A1 is described as an "performed" region in which the detection unit 14 has completed detection of an abnormal shadow.

[0048] Next, the operation of the information processing device 10 according to the present embodiment will be described with reference to Fig. 7. The CPU 21 executes the information processing program 27 to execute the determination process shown in Fig. 7. The determination process shown in Fig. 7 is executed, for example, when a command to start the process is given by the user via the input unit 25.

[0049] In step S10, the acquisition unit 11 acquires a first image G1 from the image acquisition device 2. In step S11, the extraction unit 12 extracts a first region of interest A1 from the first image G1 acquired in step S10. In step S12, the detection unit 14 detects an abnormal shadow included in the first region of interest A1 extracted in step S11. In step S13, the determination unit 13 determines whether or not an unsuitable region N1 for extraction is present in the first region of interest A1 extracted in step S11.

[0050] If an incongruent region N1 exists (i.e., if step S13 is judged as positive), the process proceeds to step S14, where the presentation unit 15 identifies the incongruent region N1 in the first image G1 and presents it together with the abnormal shadow detected in step S12. On the other hand, if an incongruent region N1 does not exist (i.e., if step S13 is judged as negative), the process proceeds to step S15, where the presentation unit 15 presents only the abnormal shadow detected in step S12. When step S14 or S15 is completed, this judgment process ends. Note that after step S14, an image synthesis process according to a second embodiment and / or a detection result synthesis process according to a third embodiment, which will be described later, may be performed.

[0051] As described above, the information processing device 10 according to the first embodiment includes at least one processor, which acquires a first image G1 obtained by imaging a subject, extracts a first region of interest A1 from the first image G1, and determines whether or not an unsuitable region N1 for extraction exists in the extracted first region of interest A1. That is, the information processing device 10 determines whether or not there is a blurred region in the first region of interest A1, which may include a target structure for which detection and diagnosis of an abnormal shadow or the like is desired. Therefore, even a blurred first image G1 can be used for diagnosis after recognizing that the first image G1 has a blurred region.

[0052] In the first embodiment, the detection unit 14 may detect only abnormal shadows included in the first region of interest A1 other than the unqualified region N1, and may not detect abnormal shadows in the unqualified region N1. This is because the possibility of non-detection or erroneous detection of abnormal shadows is higher in the unclear unqualified region N1 than in the other regions, and the reliability of the detection result is lower.

[0053] On the other hand, in the above-mentioned first embodiment, when the determination unit 13 determines that the unqualified region N1 exists in the first region of interest A1, the detection unit 14 may detect structures such as abnormal shadows included in the unqualified region N1 by lowering the detection accuracy. "Detecting by lowering the detection accuracy" means that the possibility of non-detection or erroneous detection of an abnormal shadow is accepted, and the abnormal shadow is detected even if the reliability is low. In this case, it is preferable that the presentation unit 15 presents that the detection accuracy of the abnormal shadow is low for the unqualified region N1. In particular, it is preferable to do so when the ratio of the unqualified region N1 to the first region of interest A1 is equal to or higher than a predetermined threshold (e.g., 20% or more). This is because, if the detection of an abnormal shadow is not performed in the unqualified region N1 when the ratio of the unqualified region N1 is high, the user must visually check for abnormal shadows in many parts of the first image G1, and the advantage of CAD is lost. According to this embodiment, it is possible to utilize the detection result of the abnormal shadow for the entire first image G1 for diagnosis, while recognizing that the detection accuracy is low in the unclear region.

[0054] In the above first embodiment, the detection unit 14 detects abnormal shadows, but the information processing device 10 according to this embodiment does not need to have the function of the detection unit 14 (i.e., the CAD function), and the user may visually check for abnormal shadows. According to this embodiment, by making the user aware of the presence of a blurred area in the first image G1, it is possible to prevent the user from overlooking abnormal shadows, and therefore even a blurred first image G1 can be used for the user's diagnosis.

[0055] Furthermore, in the first embodiment, when the determination unit 13 determines that the unqualified region N1 exists in the first region of interest A1, the extraction unit 12 may re-extract the first region of interest A1 with reduced extraction accuracy. "Re-extracting with reduced extraction accuracy" means that it is permitted that other regions (such as the background part in FIG. 5) may be extracted as the first region of interest A1, and the conditions for extracting the first region of interest A1 are changed so that the unqualified region N1 is reduced. The conditions are determined, for example, by the luminance value of the pixel.

[0056] In this case, it is preferable that the presenting unit 15 presents that the accuracy of extraction of the first region of interest A1 is low. This is particularly preferable when the ratio of the unqualified region N1 to the first region of interest A1 is equal to or greater than a predetermined threshold (e.g., 20% or more). This is because if a large portion of the first image G1 is determined to be the unqualified region N1, it becomes difficult to complement the unqualified region N1 (described in detail later). According to this embodiment, by making the user aware that the accuracy of extraction of the first region of interest A1 is low, it is possible to prevent the user from overlooking an abnormal shadow, and therefore the user can utilize even a blurred first image G1 for diagnosis.

[0057] [Second embodiment] In the first embodiment, the presence or absence of an unqualified region N1 in the first image G1 was determined. The information processing device 10 according to this embodiment has a function of complementing the unqualified region N1 by using a medical image other than the first image G1 when it is determined that the unqualified region N1 exists in the first image G1. An example of the configuration of the information processing device 10 according to this embodiment will be described below, but redundant descriptions of the configuration and operation similar to those of the first embodiment will be omitted.

[0058] An example of a functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 8. As shown in Fig. 8, the information processing device 10 according to this embodiment includes a synthesis unit 16 in addition to the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, and presentation unit 15 similar to those in the first embodiment. The CPU 21 executes an information processing program 27 to function as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, presentation unit 15, and synthesis unit 16.

[0059] Fig. 9 shows an example of a screen D2 presented on the display 24 by the presenting unit 15. As shown in Fig. 9, when the determining unit 13 determines that an unqualified region N1 exists in the first region of interest A1 of the first image G1, the presenting unit 15 requests a second image G2 including a region corresponding to at least a part of the unqualified region N1.

[0060] The acquisition unit 11 acquires the second image G2 from the image acquisition device 2. The second image G2 is shown in FIG. 10. The second image G2 is an image of the fundus similar to the medical image G0, and a region N12 (indicated by a dashed line) corresponding to the unsuitable region N1 in the first image G1 is clearly captured. On the other hand, a part of the photographed range is unclear, and there is also an unsuitable region N2 that is unsuitable for extraction of the actual second region of interest A2 (i.e., the entire photographed range). The unsuitable region N2 actually includes an abnormal shadow S2, but it is difficult to detect it due to its unclearness.

[0061] The synthesis unit 16 generates a third image G3 by synthesizing the first image G1 and the second image G2 acquired by the acquisition unit 11. Specifically, the synthesis unit 16 generates the third image G3 by synthesizing a part of one of the first image G1 and the second image G2 with the other image such that an unsuitable area in one of the images G1 and G2 is complemented by a corresponding area in the other image. In the third image G3 shown in Fig. 11, a part of the second image G2 is synthesized with the first image G1 such that an unsuitable area N1 in the first image G1 is complemented by an area N12 in the second image G2 corresponding to the unsuitable area N1.

[0062] The extraction unit 12 extracts the third region of interest A3 from the third image G3 combined by the combination unit 16 in the same manner as the extraction of the first region of interest A1 from the first image G1.

[0063] The determination unit 13 determines whether or not an unqualified area exists in the third region of interest A3 extracted by the extraction unit 12. For example, if a part of the region N12 of the second image G2 corresponding to the unqualified region N1 of the first image G1 is unclear, the second image G2 alone cannot fully complement the first image G1. Therefore, the CPU 21 repeatedly requests and obtains a new image including an area corresponding to at least a part of the unqualified region, and recombines the new image with the third image G3, until the determination unit 13 determines that there is no unqualified region in the third region of interest A3. The request, acquisition, and composition of the new image are performed in the same manner as the request, acquisition, and composition of the second image G2 described above.

[0064] When the determination unit 13 determines that no unsuitable region exists in the third region of interest A3, the detection unit 14 detects an abnormal shadow included in the third region of interest A3 extracted by the extraction unit 12 in the same manner as the detection of the abnormal shadow included in the first region of interest A1. In the example of the third image G3 in Fig. 11, the detection unit 14 can detect both abnormal shadows S1 and S2.

[0065] Fig. 12 shows an example of a screen D3 presented on the display 24 by the presentation unit 15. As shown in Fig. 12, the presentation unit 15 presents the abnormal shadows S1 and S2 detected by the detection unit 14 by attaching an annotation M to the third image G3.

[0066] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 13. The CPU 21 executes the information processing program 27 to execute the image synthesis process shown in Fig. 13. The image synthesis process shown in Fig. 13 is executed after step S14 in the flowchart of Fig. 7. That is, it is executed when it is determined in the determination process of the first embodiment that an unsuitable area N1 exists in the first image G1.

[0067] In step S31, the presenting unit 15 requests a second image G2 including an area corresponding to at least a part of the unsuitable area N1 identified in step S14. In step S32, the acquiring unit 11 acquires the second image G2 from the image acquiring device 2. In step S33, the combining unit 16 combines the first image G1 acquired in step S10 and the second image G2 acquired in step S32 to generate a third image G3.

[0068] In step S34, the extraction unit 12 extracts a third region of interest A3 from the third image G3 synthesized in step S33. In step S35, the determination unit 13 determines whether or not an unsuitable region for extraction exists in the third region of interest A3 extracted in step S34.

[0069] If an unqualified region exists (i.e., step S35 is judged as positive), the process proceeds to step S36, where the presenter 15 requests a new image including an area corresponding to at least a part of the unqualified region. In step S37, the acquirer 11 acquires a new image from the image acquisition device 2. In step S38, the synthesizer 16 re-synthesizes the new image acquired in step S37 with the third image G3 synthesized in step S33. When step S38 is completed, the process returns to step S34. That is, the processes of steps S34 to S38 are repeated until it is judged in step S35 that no unqualified region exists in the third region of interest A3.

[0070] On the other hand, if there is no incompatible region (i.e., if step S35 is judged as negative), the process proceeds to step S39, where the detection unit 14 detects an abnormal shadow included in the third region of interest A3 extracted in step S34. In step S40, the presentation unit 15 presents the abnormal shadow detected in step S39, and the image synthesis process ends.

[0071] As described above, the information processing device 10 according to the second embodiment includes at least one processor, and when the processor determines that the unqualified region N1 exists in the first region of interest A1 of the first image G1, it requests a second image including a region corresponding to at least a part of the unqualified region N1, and synthesizes the first image G1 and the second image G2. That is, when the information processing device 10 determines that the unqualified region N1 exists in the first image G1, it complements the unqualified region N1 using the second image G2 different from the first image G1. Therefore, even the first image G1 and the second image G2, each of which is unclear, can be used for diagnosis.

[0072] In the second embodiment, the synthesis unit 16 complements the incompatible region in either the first image G1 or the second image G2 with the other image, but the present invention is not limited to this. For example, the synthesis unit 16 may generate the third image G3 by selecting and synthesizing one image having better image quality for each of a plurality of sections in the first image G1 and the second image G2. A "section" means, for example, a pixel and a block composed of a plurality of pixels. The quality of the image quality can be evaluated based on, for example, pixel values ​​such as hue, saturation, luminance, and brightness of each pixel. Furthermore, by combining these forms, the synthesis unit 16 may complement the incompatible region with the other image, while selecting and synthesizing one image having better image quality for each section for the other regions.

[0073] Furthermore, in the second embodiment, particularly when the synthesis unit 16 repeats resynthesis based on a new image more than a predetermined number of times (e.g., three times), the extraction unit 12 may re-extract the third region of interest A3 with a lower extraction accuracy. This is to terminate the process when the complementation of the unsuitable region is not completed even after repeated resynthesis. In this case, it is preferable that the presentation unit 15 presents that the extraction accuracy of the third region of interest A3 is low.

[0074] In the second embodiment, the acquisition and synthesis of a new image are repeated until there is no unsuitable area in the third image G3, but the present invention is not limited to this. As long as the synthesis unit 16 performs synthesis at least once, the synthesis process may be terminated even if there is an unsuitable area. For example, a limit is set based on the number of synthesis times (e.g., three times), and when the number of synthesis times exceeds the limit, the process may be terminated even if there is an unsuitable area. Also, for example, when the ratio of the unsuitable area to the third region of interest A3 in the third image G3 becomes equal to or less than a predetermined threshold (e.g., 5% or less), the process may be terminated even if there is an unsuitable area. In these cases, the presentation unit 15 may present the ratio of the unsuitable area to the third region of interest A3.

[0075] In addition, when the process is terminated in a state where an unqualified region exists, structures such as abnormal shadows included in the third region of interest A3 may be detected with a lowered detection accuracy. In this case, it is preferable for the presenting unit 15 to present that the detection accuracy of abnormal shadows is low for the unqualified region. This is particularly preferable when the ratio of the unqualified region to the third region of interest A3 is equal to or higher than a predetermined threshold (e.g., 20% or more). This is because if abnormal shadows are not detected in the unqualified region when the ratio of the unqualified region is high, the user must visually check for abnormal shadows in many parts of the third image G3, which would eliminate the advantage of CAD. According to this embodiment, the detection result of abnormal shadows in the entire third image G3 can be used for diagnosis after recognizing that the detection accuracy is low for unclear regions.

[0076] In the above second embodiment, the detection unit 14 detects abnormal shadows, but the information processing device 10 according to this embodiment does not need to have the function of the detection unit 14 (i.e., the CAD function), and the user may visually confirm abnormal shadows. Even in this embodiment, the first image G1 and the second image G2, each of which is unclear, can be used for diagnosis by the user.

[0077] In the second embodiment, the image synthesis process is performed when it is determined in the determination process of the first embodiment that the unqualified region N1 exists in the first image G1, but the present invention is not limited to this. For example, the image synthesis process according to the present embodiment may be performed when the ratio of the unqualified region N1 to the first region of interest A1 in the first image G1 is equal to or greater than a predetermined threshold (e.g., 20% or more).

[0078] [Third embodiment] In the second embodiment, abnormal shadows were detected based on the third image G3 obtained by combining the first image G1 and the second image G2. The information processing device 10 according to this embodiment has a function of detecting abnormal shadows based on the first image G1 and the second image G2, respectively, and combining the detection results. An example of the configuration of the information processing device 10 according to this embodiment will be described below, but redundant descriptions of configurations and functions similar to those of the first and second embodiments will be omitted.

[0079] As in the second embodiment, an example of a functional configuration of the information processing device 10 according to this embodiment will be described with reference to Fig. 8. As shown in Fig. 8, the information processing device 10 according to this embodiment includes a synthesis unit 16 in addition to the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, and presentation unit 15 similar to those in the first embodiment. The CPU 21 executes an information processing program 27 to function as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, presentation unit 15, and synthesis unit 16.

[0080] Fig. 9 shows an example of a screen D2 presented on the display 24 by the presenting unit 15. As shown in Fig. 9, when the determining unit 13 determines that an unqualified region N1 exists in the first region of interest A1 of the first image G1, the presenting unit 15 requests a second image G2 including an area corresponding to at least a part of the unqualified region N1. The acquiring unit 11 acquires the second image G2 (see Fig. 10) from the image acquiring device 2. The description of the second image G2 is omitted since it is similar to that of the second embodiment.

[0081] The extraction unit 12 extracts the second region of interest A2 from the second image G2 acquired by the acquisition unit 11 in a similar manner to the extraction of the first region of interest A1 from the first image G1. The detection unit 14 detects abnormal shadows included in the first region of interest A1 and the second region of interest A2 extracted by the extraction unit 12 in a similar manner to the detection of abnormal shadows included in the first region of interest A1. In the examples of Figures 5 and 10, the detection unit 14 detects abnormal shadows S2 from the first image G1 and abnormal shadows S1 from the second image G2.

[0082] The synthesis unit 16 synthesizes the detection results of abnormal shadows detected from the first region of interest A1 and the second region of interest A2 by the detection unit 14. That is, the synthesis unit 16 combines the detection results of abnormal shadows obtained from a plurality of different images into a detection result of abnormal shadows for the same subject.

[0083] The determination unit 13 determines whether or not a common unsuitable region that is unsuitable for extraction exists for the first region of interest A1 and the second region of interest A2 extracted by the extraction unit 12. If a common unsuitable region exists, it may lead to overlooking an abnormal shadow. Therefore, the CPU 21 repeats the following steps until the determination unit 13 determines that no common unsuitable region exists: requesting and acquiring a new image including a region corresponding to at least a part of the common unsuitable region, extracting the region of interest from the new image, detecting an abnormal shadow included in the region of interest, and recombining the detection result of the abnormal shadow and the detection results of abnormal shadows detected up to that point. The requesting and acquiring of the new image, the extraction of the region of interest, and the detection of the abnormal shadow are performed in the same manner as the requesting and acquiring of the second image G2, the extraction of the region of interest, and the detection of the abnormal shadow.

[0084] Fig. 14 shows an example of a screen D4 presented on the display 24 by the presentation unit 15. As shown in Fig. 14, the presentation unit 15 presents the detection result of the abnormal shadow combined by the composition unit 16 by attaching an annotation M to each image. Note that, similarly to the second embodiment, the composition unit 16 may generate an image by combining the first image G1 and the second image G2, and the presentation unit 15 may present the detection result of the abnormal shadow combined by the composition unit 16 by attaching an annotation M to the image (i.e., one image).

[0085] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 15. The CPU 21 executes the information processing program 27 to execute the detection result combination process shown in Fig. 15. The detection result combination process shown in Fig. 15 is executed after step S14 in the flowchart of Fig. 7. That is, it is executed when it is determined in the determination process of the first embodiment that an unsuitable area N1 exists in the first image G1.

[0086] In step S51, the presenting unit 15 requests a second image G2 including an area corresponding to at least a part of the unqualified area N1 identified in step S14. In step S52, the acquiring unit 11 acquires the second image G2 from the image acquiring device 2. In step S53, the extracting unit 12 extracts a second area of ​​interest A2 from the second image G2 acquired in step S52. In step S54, the detecting unit 14 detects an abnormal shadow included in the second area of ​​interest A2 extracted in step S53.

[0087] In step S55, the combining unit 16 combines the detection result of the abnormal shadow included in the first region of interest A1 detected in step S12 with the detection result of the abnormal shadow included in the second region of interest A2 detected in step S54. In step S56, the determination unit 13 determines whether or not a common unsuitable region that is unsuitable for extraction exists for the first region of interest A1 extracted in step S11 and the second region of interest A2 extracted in step S53.

[0088] If a common unqualified region exists (i.e., if step S56 is judged as positive), the processes of steps S51 to S56 are performed on a new image including a region corresponding to at least a part of the common unqualified region. That is, the processes of steps S51 to S56 are repeated until it is judged in step S56 that no common unqualified region exists.

[0089] On the other hand, if there is no common incompatible region (i.e., if step S56 is judged as negative), the process proceeds to step S57, where the presenting unit 15 presents the detection result of the abnormal shadows synthesized in step S55. When step S57 is completed, this detection result synthesis process ends.

[0090] As described above, the information processing device 10 according to the third embodiment includes at least one processor, and when the processor determines that the unqualified region N1 exists in the first region of interest A1 of the first image G1, it requests a second image including a region corresponding to at least a part of the unqualified region N1. In addition, it extracts the second region of interest A2 from the second image G2, detects abnormal shadows included in each of the first region of interest A1 and the second region of interest A2, and synthesizes the detection results of the abnormal shadow. That is, when it is determined that the unqualified region N1 exists in the first image G1, the information processing device 10 complements the detection result of the abnormal shadow using the second image G2 different from the first image G1. Therefore, even the first image G1 and the second image G2, each of which is unclear, can be used for diagnosis.

[0091] In the third embodiment, particularly when the synthesis unit 16 repeats the resynthesis of the detection results of the abnormal shadow contained in the new image more than a predetermined number of times (e.g., three times), the extraction unit 12 may extract the region of interest in the new image with lowered extraction accuracy. This is to terminate the process if the common unsuitable region does not disappear even after repeated resynthesis. In this case, it is preferable for the presentation unit 15 to present a message that the accuracy of the extraction of the region of interest is low.

[0092] In the above third embodiment, the synthesis of the detection results of abnormal shadows based on new images is repeated until no common unsuitable regions exist, but the present invention is not limited to this. As long as the synthesis unit 16 synthesizes the detection results at least once, the synthesis process may be terminated even if a common unsuitable region exists. For example, a limit is set on the number of synthesis times (e.g., three times), and when the number of synthesis times exceeds the limit, the process may be terminated even if a common unsuitable region exists. Also, for example, when the ratio of the common unsuitable region to the first region of interest A1 or the second region of interest A2 becomes equal to or less than a predetermined threshold (e.g., 5% or less), the process may be terminated even if a common unsuitable region exists.

[0093] In addition, when the process is terminated in a state where a common unsuitable region exists, structures such as abnormal shadows included in the common unsuitable region may be detected with a lowered detection accuracy. In this case, it is preferable for the presenting unit 15 to present that the detection accuracy of abnormal shadows is low for the common unsuitable region. In particular, it is preferable to do so when the ratio of the common unsuitable region to the first region of interest A1 or the second region of interest A2 is equal to or higher than a predetermined threshold (e.g., 20% or more). This is because, if abnormal shadows are not detected in the common unsuitable region when the ratio of the common unsuitable region is high, the user must visually check for abnormal shadows in many parts of the first image G1 and the second image G2, which would eliminate the advantage of CAD. According to this embodiment, the detection result of abnormal shadows for the entire first image G1 and the second image G2 can be used for diagnosis after recognizing that the detection accuracy is low for unclear regions.

[0094] In the above third embodiment, the detection result synthesis process is performed when it is determined in the determination process of the first embodiment that the unqualified region N1 exists in the first image G1, but the present invention is not limited to this. For example, the detection result synthesis process according to this embodiment may be performed when the ratio of the unqualified region N1 to the first region of interest A1 in the first image G1 is equal to or greater than a predetermined threshold (e.g., 20% or more).

[0095] [Fourth embodiment] In the above first to third embodiments, the extraction unit 12 extracts a region of interest from a medical image. As described above, a trained model that is trained to input a medical image, extract and output a region of interest can be used as a method for extracting a region of interest by the extraction unit 12. In this case, the trained model is required to be able to extract a region of interest with high accuracy even when a blurred medical image such as that shown in Fig. 5 and Fig. 10 is input.

[0096] One method for improving the accuracy of this trained model is to use various patterns of blurred medical images as training data to perform training. However, it is difficult to acquire a sufficient number and patterns of blurred medical images using the image acquisition device 2. Therefore, in this embodiment, the purpose is to improve the accuracy of the training model by intentionally generating blurred medical images and using them as training data.

[0097] As an example, a form will be described in which the information processing device 10 according to this embodiment learns the learning model 4 used in the extraction unit 12 by unsupervised learning. The learning model 4 is configured to include deep learning models such as CNN (Convolutional Neural Network), FCN (Fully Convolutional Network), and U-Net, and is a model that is trained to input a medical image, extract and output a region of interest. In addition, for example, the technologies described in the above Patent Document 1, JP 2019-088458 A, and JP 2020-114302 A may be applied as such a model.

[0098] The CPU 21 acquires an original image obtained by photographing the subject from the image acquisition device 2. That is, the original image is an image obtained by at least one of the image acquisition devices 2, which are a radiographic imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope. The original image also includes a region of interest including at least one of the subject, a part of tissue contained in the subject, and structures such as an abnormal part contained in the subject or tissue. Hereinafter, an example of the original image will be described using a clear medical image G0 shown in FIG. 4. The medical image G0 has been described above, and therefore will not be described.

[0099] The CPU 21 changes at least some pixel values ​​of the medical image G0 to generate a pseudo image. The pixel value is a value indicating at least one of the hue, saturation, luminance, and brightness indicated by each pixel in the medical image G0. For example, the pixel value of each pixel is changed by changing the brightness and contrast of the medical image G0, or by adding blurring and noise. However, the CPU 21 does not change the resolution when generating the pseudo image.

[0100] Specific examples of pseudo images will be described with reference to Fig. 16. A plurality of pseudo images P1 to P5 shown on the left side of Fig. 16 are pseudo images generated based on the medical image G0, and are input to the learning model 4 as learning data. The pseudo images P1 and P2 are images of the medical image G0 with the brightness increased and decreased, respectively. The pseudo image P3 is an image of the medical image G0 with the contrast decreased. The pseudo images P4 and P5 are images of the medical image G0 with some areas darkened.

[0101] Also, on the right side of Fig. 16, the region of interest extracted by learning model 4 during learning for each of the input pseudo images is illustrated surrounded by a thick line. The region of interest is properly extracted for pseudo images P1 and P2. For pseudo image P3, an error is output indicating that extraction is impossible. For pseudo images P4 and P5, some blurry regions that should have been extracted as regions of interest have not been extracted as regions of interest.

[0102] As shown in Fig. 16, the pseudo image may be one in which the pixel values ​​of the entire medical image G0 have been changed, or one in which the pixel values ​​of a part of the medical image G0 have been changed. Also, as shown in pseudo images P4 and P5, it is preferable to use multiple pseudo images generated by changing pixel values ​​in different regions of one medical image G0 as learning data. According to this embodiment, the number of original images can be reduced, and learning can be performed efficiently.

[0103] As shown in the pseudo images P1 to P5, the pseudo images are preferably images generated by changing pixel values ​​in at least a part of the medical image G0 so as to deteriorate the image quality. Specifically, "deteriorating the image quality" means processing that makes it difficult to detect structures included in the medical image G0, for example, processing that weakens the contrast. This is because, considering the operation phase of the learning model, when the input medical image is a fundus image, the image may become dark due to insufficient light or bright due to disturbance light, resulting in a weaker contrast than a clear medical image.

[0104] 16, the pseudo image is preferably an image generated by changing pixel values ​​of a region including at least the region of interest in the medical image G0, because it does not matter whether the regions other than the region of interest are blurred or not.

[0105] In addition, it is preferable that the pseudo image is an image generated by changing at least a part of the pixel values ​​of the medical image G0 while maintaining the presence or absence of structures contained in the medical image G0. This is because in the present learning model 4, the priority is to extract the region of interest with high accuracy even from a blurred medical image, and responding to changes in the presence or absence of structures is a low priority.

[0106] The pseudo image may be an image generated based on the medical image G0 using an image generation model such as Generative Adversarial Networks (GAN) and Variational Autoencoder (VAE).

[0107] The CPU 21 trains the learning model 4 by inputting the pseudo images P1 to P5 generated as described above to the learning model 4 as learning data. Also, as shown in pseudo images P3 to P5 in Fig. 16, when the learning model 4 is unable to properly extract a region of interest from an input pseudo image, the CPU 21 may re-train the learning model 4 by inputting the pseudo image again as learning data. According to this embodiment, the accuracy of the learning model 4 can be improved.

[0108] In addition, when re-learning, a correct label may be assigned to the pseudo image. Specifically, the CPU 21 may input a pair of a pseudo image and information indicating a region of interest included in the pseudo image to the learning model 4 as learning data, and re-learn the learning model 4. The information indicating the region of interest is, for example, information indicating the position of the region of interest in the pseudo image.

[0109] Next, the operation of the information processing device 10 according to this embodiment will be described with reference to Fig. 17. The CPU 21 executes the information processing program 27 to execute the learning process shown in Fig. 17. The learning process shown in Fig. 17 is executed, for example, when a command to start the process is given by the user via the input unit 25.

[0110] In step S71, CPU 21 acquires an original image obtained by photographing a subject from image acquisition device 2. In step S72, CPU 21 generates a pseudo image by changing at least a part of pixel values ​​of the original image acquired in step S71. In step S73, CPU 21 uses the pseudo image generated in step S72 as learning data to train learning model 4. In step S74, CPU 21 determines whether learning model 4 has properly extracted a region of interest from the input pseudo image.

[0111] If the learning model 4 cannot properly extract the region of interest from the input pseudo image (i.e., if step S74 is judged as negative), the process returns to step S73, and the CPU 21 re-inputs the pseudo image into the learning model 4 to cause the learning model 4 to re-learn. That is, re-learning using the same pseudo image is repeated until the learning model 4 can properly extract the region of interest from the pseudo image. When the learning model 4 can properly extract the region of interest from the input pseudo image (i.e., when step S74 is judged as positive), this learning process ends.

[0112] As described above, the information processing device 10 according to the fourth embodiment includes at least one processor, and the processor inputs a pseudo image generated by changing at least some pixel values ​​of an original image obtained by photographing a subject as learning data, thereby training a learning model for extracting a region of interest from the input image. Therefore, even if an image is not suitable for extracting a region of interest, the region of interest can be appropriately extracted and used for diagnosis.

[0113] In the fourth embodiment, the information processing device 10 learns the learning model 4 used in the extraction unit 12 by unsupervised learning, but the present invention is not limited to this. The information processing device 10 may learn the learning model 4 by supervised learning and semi-supervised learning. Specifically, the CPU 21 may train the learning model 4 by inputting a pair of a pseudo image and information indicating a region of interest contained in the pseudo image as learning data. In this case, if the learning model 4 cannot extract a region of interest from the input pseudo image, the CPU 21 may re-train the learning model 4 by inputting the pseudo image as learning data again.

[0114] In the above embodiments, the description has been given using medical images, but the technology of the present disclosure is not limited to medical images. For example, the technology of the present disclosure may be applied to images obtained from equipment, buildings, pipes, welds, and the like as specimens in non-destructive inspections such as radiographic inspections and ultrasonic flaw detection inspections.

[0115] In addition, in each of the above embodiments, the information processing system 1 includes the information processing device 10 and the image acquisition device 2, but the present invention is not limited to this. For example, the information processing system 1 may include one device having both the functions of the information processing device 10 and the functions of the image acquisition device 2. For example, the information processing system 1 may include a plurality of image acquisition devices 2, and the information processing device 10 may acquire medical images from each of the plurality of image acquisition devices 2. For example, the information processing device 10 may be configured to include a plurality of devices having different functions, such as an acquisition unit 11, an extraction unit 12, a determination unit 13, a detection unit 14, a presentation unit 15, and a synthesis unit 16.

[0116] In each of the above embodiments, the following various processors can be used as the hardware structure of the processing unit that executes various processes, such as the acquisition unit 11, extraction unit 12, determination unit 13, detection unit 14, presentation unit 15, and synthesis unit 16. As described above, the above various processors include a CPU, which is a general-purpose processor that executes software (programs) and functions as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing specific processes, such as an ASIC (Application Specific Integrated Circuit), etc.

[0117] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0118] As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with one IC (Integrated Circuit) chip, as typified by System on Chip (SoC), etc. In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

[0119] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0120] In addition, in each of the above embodiments, the information processing program 27 is pre-stored (installed) in the storage unit 22, but the present invention is not limited to this. The information processing program 27 may be provided in a form recorded on a recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), or a USB (Universal Serial Bus) memory. The information processing program 27 may also be downloaded from an external device via a network. Furthermore, the technology of the present disclosure extends to a storage medium that non-temporarily stores an information processing program, in addition to the information processing program.

[0121] The technology of the present disclosure can also be appropriately combined with each of the above-mentioned embodiment examples. The above-mentioned description and illustrated contents are detailed descriptions of the parts related to the technology of the present disclosure, and are merely one example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is a description of one example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the description and illustrated contents shown above, within the scope of the gist of the technology of the present disclosure. [Explanation of symbols]

[0122] 1. Information Processing Systems 2. Image acquisition device 4. Learning Model 10. Information processing device 11 Acquisition Department 12 Extraction part 13 Judgment section 14 Detection section 15 Presentation section 16 Synthesis section 21 CPU 22 Memory section 23 Memory 24 Display 25 Input section 26 Network Interface 27 Information Processing Programs 28 Bus A0 Area of ​​Interest A1 First area of ​​interest A2 Second Area of ​​Interest A3 Third Area of ​​Interest D1~D4 screen G0 Medical Imaging G1 1st image G2 2nd image G3 3rd image M annotation N1, N2 Nonconformity area N12 area P1~P5 Pseudo image S1, S2 abnormal shadows

Claims

1. At least one processor; The processor, Obtaining a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not a region unsuitable for extraction exists for the extracted first region of interest; When it is determined that the unsuitable region exists in the first region of interest, the first region of interest is re-extracted with a lower extraction accuracy. Information processing device.

2. The processor, The determination is made according to a degree of similarity between a shape of the extracted first region of interest and a reference shape that is predetermined for the first region of interest. The information processing device according to claim 1 .

3. The processor, An image obtained by photographing a subject is input, and the judgment is made using a trained model for judging whether the unsuitable region exists in a region of interest extracted from the input image.

3. The information processing device according to claim 1 or 2.

4. The trained model is a trained model trained using a pair of an image obtained by photographing a subject and information indicating whether the unconformity region exists in a region of interest extracted from the image as training data. The information processing device according to claim 3 .

5. The processor, When it is determined that the unqualified region exists in the first region of interest, the unqualified region in the first image is identified and presented. The information processing device according to any one of claims 1 to 4.

6. The processor, When it is determined that the unsuitable region exists in the first region of interest, a ratio of the unsuitable region to the first region of interest is displayed. The information processing device according to any one of claims 1 to 5.

7. The processor, Detecting a structure included in the first region of interest; When it is determined that the unconformity region exists in the first region of interest, a structure included in the unconformity region is detected by lowering detection accuracy. The information processing device according to any one of claims 1 to 6.

8. The processor, requesting a second image including an area corresponding to at least a portion of the unqualified area when determining that the unqualified area exists in the first area of ​​interest; The information processing device according to any one of claims 1 to 7.

9. The processor, acquiring the second image; A third image is generated by synthesizing the first image and the second image. The information processing device according to claim 8.

10. The processor, The third image is generated by combining one of the first image and the second image with a part of the other image such that the incompatible area in the other image is complemented by a corresponding area in the other image. The information processing device according to claim 9.

11. The processor, The third image is generated by selecting one of the first and second images having a better image quality for each of a plurality of sections in the first and second images and combining the images. The information processing device according to claim 9 or 10.

12. The processor, Extracting a third region of interest from the third image; determining whether the unsuitable region exists in the extracted third region of interest; until it is determined that the unsuitable region does not exist in the third region of interest. obtaining a new image including an area corresponding to at least a portion of the unconforming area; and repeatedly combining the new image with the third image. The information processing device according to any one of claims 9 to 11.

13. The processor, Extracting a third region of interest from the third image; Detecting a structure included in the third region of interest The information processing device according to any one of claims 9 to 12.

14. The processor, acquiring the second image; Extracting a second region of interest from the second image; Detecting structures included in each of the first region of interest and the second region of interest; Combining the detection results of the structures detected from the first region of interest and the second region of interest. The information processing device according to any one of claims 9 to 13.

15. The processor, determining whether or not a common unsuitable region that is unsuitable for extraction exists for the extracted first region of interest and the extracted second region of interest; until it is determined that the common incompatible region does not exist. obtaining a new image that includes a region corresponding to at least a portion of the common unmatched region; Extracting a region of interest from the new image; Detecting structures within the region of interest; Repeatedly recombine the detection results for the structure in question. The information processing device according to claim 14.

16. The first region of interest is a region including at least one of the subject, a part of tissue included in the subject, and an abnormal part included in the subject or the tissue. The information processing device according to any one of claims 1 to 15.

17. The first image is an image obtained by at least one of a radiation imaging device, a magnetic resonance imaging device, an ultrasound device, a fundus imaging device, and an endoscope. The information processing device according to any one of claims 1 to 16.

18. The processor, When the first region of interest is re-extracted with reduced extraction accuracy, a message is displayed indicating that the extraction accuracy of the first region of interest is low. The information processing device according to any one of claims 1 to 17.

19. A method for detecting a signal from a computer comprising: The processor, Obtaining a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not a region unsuitable for extraction exists for the extracted first region of interest; Detecting a structure included in the first region of interest; When it is determined that the unconformity region exists in the first region of interest, a structure included in the unconformity region is detected by lowering detection accuracy. Information processing device.

20. Obtaining a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not a region unsuitable for extraction exists for the extracted first region of interest; When it is determined that the unsuitable region exists in the first region of interest, the first region of interest is re-extracted with a lower extraction accuracy. An information processing method in which processing is performed by a computer.

21. Obtaining a first image obtained by photographing a subject; Extracting a first region of interest from the first image; determining whether or not a region unsuitable for extraction exists for the extracted first region of interest; Detecting a structure included in the first region of interest; When it is determined that the unconformity region exists in the first region of interest, a structure included in the unconformity region is detected by lowering detection accuracy. An information processing method in which processing is performed by a computer.

22. Obtaining a first image obtained by imaging the subject; Extracting a first region of interest from the first image; determining whether or not a region unsuitable for extraction exists for the extracted first region of interest; When it is determined that the unsuitable region exists in the first region of interest, the first region of interest is re-extracted with a lower extraction accuracy. An information processing program for causing a computer to execute processing.

Images