Image diagnosis assistance device, image diagnosis assistance system, and image diagnosis assistance method

Abstract

An image diagnosis assistance device (5) is provided with: an image selection unit (511) that selects, as a diagnosis image, an arbitrary image from among a plurality of images that include the same subject and have been subjected to different processes; and an estimation unit (512) that estimates a diagnosis candidate region that is a diagnosis candidate in the diagnosis image by performing estimation processing on the diagnosis image using the learned model, and outputs the reliability of the diagnosis candidate region. The image selection unit (511) selects, as a diagnostic image, any one of the plurality of images on the basis of the reliability of the diagnostic candidate region.

Description

TECHNICAL FIELD

[0001] The present application relates to an image diagnosis assistance apparatus, an image diagnosis assistance system, and an image diagnosis assistance method. BACKGROUND

[0002] In recent years, in the medical field, an image recognition technique based on AI (Artificial Intelligence) has been proposed (for example, refer to Patent Literature 1).

[0003] In the technique described in Patent Literature 1, an estimation process is performed on a captured image photographed by an endoscope using a learned model, thereby estimating a diagnosis candidate region such as a lesion in the captured image.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent No. 6952214 SUMMARY

[0007] PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] However, in the image recognition technique based on AI, depending on the quality of an input captured image, sometimes sufficient performance cannot be exerted, and a diagnosis candidate region such as a lesion cannot be estimated with high accuracy. That is, sometimes an image suitable for diagnosis cannot be provided.

[0009] The present application has been achieved in view of the above-described circumstances, and an object thereof is to provide an image diagnosis assistance apparatus, an image diagnosis assistance system, and an image diagnosis assistance method capable of providing an image suitable for diagnosis.

[0010] SOLUTION TO PROBLEM

[0011] In order to solve the above-described problems and achieve the object, an image diagnosis assistance apparatus according to the present application includes an image selection section that selects an arbitrary image of a plurality of images that contain the same subject and on which mutually different processes have been performed as a diagnosis image, and an estimation section that performs an estimation process on the diagnosis image using a learned model, thereby estimating a diagnosis candidate region that becomes a diagnosis candidate in the diagnosis image, and outputs a reliability of the diagnosis candidate region, wherein the image selection section selects an arbitrary image of the plurality of images as the diagnosis image based on the reliability of the diagnosis candidate region.

[0012] The image diagnostic assistance system of the present invention includes: a camera device that generates a camera image by capturing a subject; and an image diagnostic assistance device that processes the camera image, wherein the image diagnostic assistance device includes: an image selection unit that selects any image from a plurality of images containing the same subject and having undergone different processing on the camera image as a diagnostic image; and an estimation unit that performs estimation processing on the diagnostic image using a learned model, thereby estimating diagnostic candidate regions in the diagnostic image that become diagnostic candidates, and outputting the reliability of the diagnostic candidate regions, wherein the image selection unit selects any image from the plurality of images as the diagnostic image based on the reliability of the diagnostic candidate regions.

[0013] The image diagnostic assistance method involved in this invention is an image diagnostic assistance method executed by an image diagnostic assistance device, comprising the following steps: selecting any image from a plurality of images containing the same subject and subjected to different processing as a diagnostic image; and performing estimation processing on the diagnostic image using a learned model, thereby estimating diagnostic candidate regions in the diagnostic image that become diagnostic candidates, and outputting the reliability of the diagnostic candidate regions, wherein, in the step of selecting the diagnostic image, based on the reliability of the diagnostic candidate regions, any image from the plurality of images is selected as the diagnostic image.

[0014] The effects of the invention

[0015] The image diagnostic aid device, image diagnostic aid system, and image diagnostic aid method according to the present invention can provide images suitable for diagnosis. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the structure of the endoscope system involved in the implementation method.

[0017] Figure 2 This is a diagram illustrating the structure of the endoscope system involved in the implementation method.

[0018] Figure 3 It is a diagram that conceptually represents the functions of the control unit.

[0019] Figure 4 This is a flowchart illustrating an image-assisted diagnostic method.

[0020] Figure 5 This is a diagram representing a specific example of an image being displayed.

[0021] Figure 6 This is a diagram representing a specific example of an image being displayed.

[0022] Figure 7 This is a diagram representing a specific example of an image being displayed.

[0023] Figure 8 This is a diagram representing a specific example of an image being displayed.

[0024] Figure 9 This is a diagram illustrating a variation of the implementation method, Example 1.

[0025] Figure 10 This is a diagram illustrating a variation of the implementation method, example 2.

[0026] Figure 11 This is a diagram illustrating variation 3 of the implementation method. Detailed Implementation

[0027] Hereinafter, embodiments for carrying out the present invention (hereinafter referred to as embodiments) will be described with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below. Furthermore, in the accompanying drawings, the same reference numerals are used to label the same parts.

[0028] [Structure of the endoscopic system]

[0029] Figure 1 and Figure 2 This is a diagram illustrating the structure of the endoscope system 1 according to the embodiment.

[0030] Endoscopic system 1 corresponds to the image diagnostic auxiliary system involved in this invention. This endoscopic system 1 is used in the medical field for observing a BD (Browser Diagnostic and Endoscopic) bed. Figure 1 The patient on the PA (examined body) Figure 1 The system within the body (in this embodiment, the large intestine). For example... Figure 1 and Figure 2 As shown, the endoscope system 1 has an endoscope 2 and a processing device 3.

[0031] Endoscope 2 corresponds to the imaging device involved in this invention. In this embodiment, endoscope 2 is a so-called flexible endoscope. A portion of the endoscope 2 is inserted into the body of the subject PA, taking pictures of the body and outputting the image signal generated by the pictures. Moreover, as Figure 1 and Figure 2 As shown, the endoscope 2 includes an insertion section 21, an operation section 22, a universal cable 23, and a connector section 24. Furthermore, in Figure 2 For ease of explanation, the illustrations of the operation section 22, the universal cable 23, and the connector section 24 are omitted.

[0032] The insertion portion 21 is at least a portion that is flexible and is inserted into the body of the subject PA. For example... Figure 2 As shown, a light guide 25, an illumination lens 26, and a camera unit 27 are provided inside the insertion part 21.

[0033] The light guide 25 passes through the insertion part 21, the operation part 22, and the universal cable 23, and is wound to the connector part 24. One end of the light guide 25 is located in the front end portion within the insertion part 21. Furthermore, when the endoscope 2 is connected to the processing device 3, the other end of the light guide 25 is located within the processing device 3. The light guide 25 transmits light supplied from the light source device 4 within the processing device 3 from one end to the other.

[0034] The illumination lens 26 faces one end of the light guide 25 within the insertion portion 21. Furthermore, the illumination lens 26 illuminates the body of the subject PA with light transmitted through the light guide 25.

[0035] The camera unit 27 is located at the front end of the insertion unit 21. The camera unit 27 then captures images of the subject PA inside the body and outputs the image signal generated by the capture. Figure 2 As shown, the camera unit 27 includes a lens unit 271 and an image sensor 272.

[0036] The lens unit 271 receives the reflected light (subject image) from the light irradiated into the body of the subject PA by the illumination lens 26, and images the subject image onto the light-receiving surface of the imaging element 272.

[0037] The imaging element 272 is composed of a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) that receives the image of the subject and converts it into an electrical signal, and generates an image signal by capturing the image of the subject. Furthermore, the image signal generated by the imaging unit 27 will be described below as an image.

[0038] The operating unit 22 is connected to the base portion of the insertion unit 21. Furthermore, the operating unit 22 handles various operations related to the endoscope 2.

[0039] The general cable 23 extends from the operation section 22 in a direction different from the extension direction of the insertion section 21, and is equipped with a signal line and a light guide 25, etc., that electrically connect the camera section 27 to the control device 5 in the processing device 3.

[0040] The connector part 24 is provided at the end of the universal cable 23 and can be detachably connected to the processing device 3.

[0041] like Figure 2 As shown, the processing device 3 includes a light source device 4 and a control device 5.

[0042] Under the control of the control device 5, the light source device 4 supplies light to the other end of the light guide 25. In this embodiment, the light source device 4 emits white light as light in the first wavelength range. In addition, the light source device 4 may also be configured to emit excitation light that excites fluorescent agents such as indigo green, narrowband light used in NBI (Narrow Band Imaging), etc., as light in a second wavelength range different from the first wavelength range.

[0043] The control device 5 is equivalent to the image diagnostic auxiliary device involved in this invention. For example... Figure 2 As shown, the control device 5 includes a control unit 51, a display unit 52, an input unit 53, a storage unit 54, and a communication unit 55.

[0044] The control unit 51 is configured to include controllers such as CPUs (Central Processing Units), MPUs (Microprocessing Units), or integrated circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays), controlling the overall operation of the endoscope system 1. For example... Figure 2 As shown, the control unit 51 has the functions of an image selection unit 511, an estimation unit 512, a cropped image generation unit 513, a high image quality processing unit 514, a display control unit 515, and a communication control unit 516.

[0045] Figure 3 This is a diagram that conceptually represents the function of the control unit 51.

[0046] exist Figure 3 In the figure, the "input image processing" of the input camera image (endoscopic image), indicated by reference numeral 51B1, includes an image selection unit 511, an image cropping generation unit 513, and a high image quality enhancement processing unit 514. Furthermore, in Figure 3 In the figure, the "estimation process" indicated by reference numeral 51B2 includes an estimation unit 512. Furthermore, in Figure 3 In the figure, “display image generation”, indicated by reference numeral 51B3, includes a display control unit 515.

[0047] Furthermore, the functions of the image selection unit 511, estimation unit 512, cropped image generation unit 513, high image quality processing unit 514, display control unit 515, and communication control unit 516 in the control unit 51 will be explained in the "Image Diagnosis Assistance Method" described later.

[0048] The display unit 52 corresponds to the notification unit according to the present invention. The display unit 52 is an LCD (Liquid Crystal Display) or EL (Electro Luminescence) display, etc., and displays a display image generated by the control unit 51 under the control of the control unit 51.

[0049] The input unit 53 corresponds to the operation receiving unit according to the present invention. This input unit 53 is constructed using a keyboard, mouse, switch, touch panel, etc., and receives user operations performed by users such as surgical operators. Furthermore, the input unit 53 outputs operation signals corresponding to the user operation to the control unit 51.

[0050] The storage unit 54 stores various programs executed by the storage control unit 51, as well as information required for processing by the control unit 51.

[0051] The communication unit 55 is connected to an external device in a communicative manner. Furthermore, under the control of the control unit 51, the communication unit 55 sends specified information (data) to the external device.

[0052] [Image-assisted diagnostic methods]

[0053] Next, refer to Figure 3 and Figure 4 The image diagnostic assistance method performed by the control device 5 described above will be explained.

[0054] Figure 4 This is a flowchart illustrating an image-assisted diagnostic method.

[0055] First, the image selection unit 511 acquires an image generated by the imaging unit 27 when white light of a first wavelength range is irradiated into the body of the subject PA from the light source device 4, capturing the reflected light (subject image) from the body of that white light (subject image) (step S1). This image corresponds to the first image according to the present invention. Then, the image selection unit 511 selects this image as a diagnostic image and inputs it into the estimation unit 512.

[0056] After step S1, the estimation unit 512 performs estimation processing on the diagnostic image using the learned model for estimation processing, thereby estimating a diagnostic candidate region for each specified region in the diagnostic image as a diagnostic candidate, and outputting the reliability of the diagnostic candidate region (step S2).

[0057] In addition, the reliability of the diagnostic candidate region is a value that indicates the level of reliability.

[0058] Specifically, reliability is a value representing the accuracy of image recognition within a diagnostic candidate region; it can also be described as an indicator of the probability that an object predicted to be in the image belongs to a specific category. The reliability of the diagnostic candidate region can be used to determine whether an object has been accurately identified within that region.

[0059] Here, the learned model for estimation processing corresponds to the learned model involved in this invention. This learned model for estimation processing is pre-stored in storage unit 54. Specifically, the learned model for estimation processing is a learned model that has undergone repeated learning processing using multiple sets of training images and teaching data, each set consisting of training images and teaching data. The training images are photographic images taken inside an organism. The teaching data is data annotated with the classification category, correct location, and size of lesions, etc., in the training images. The learning model used for this learning processing is, for example, a CNN (Convolutional Neural Network). Furthermore, the learned model for estimation processing includes a weight file (learning parameters) containing the weight values ​​and bias values ​​of each layer of the CNN.

[0060] Furthermore, the neural network used in the learning process during the generation and estimation of the learned model is not limited to CNNs; other neural networks can also be used. For example, DNNs (Deep Neural Networks), Transformers, and GANs (Generative Adversarial Networks) can be appropriately employed. Additionally, various well-known learning algorithms can be used as the machine learning algorithm within the neural network. For instance, supervised learning algorithms that utilize backpropagation can be employed.

[0061] After step S2, the image selection unit 511 determines whether the predetermined conditions have been met within a predetermined number of times or within a predetermined time based on the reliability of the diagnostic candidate region output from the estimation unit 512 in step S2 (step S3).

[0062] The following conditions can be exemplified as the conditions specified for use in step S3.

[0063] The condition for this regulation is that a region in the diagnostic image contains a "potentially abnormal area".

[0064] Here, "a region that may be an anomaly" refers to a region whose reliability is less than a first threshold and is at least a second threshold lower than the first threshold. Furthermore, "a region that is normal" as explained below refers to a region whose reliability is less than the second threshold. And, "a region that is an anomaly" as explained below refers to a region whose reliability is at least the first threshold.

[0065] Typical estimation processes determine whether a region is normal or abnormal based on a threshold. Specifically, if the reliability of a region is above a first threshold, it is determined to be an "abnormal region," and if the reliability of a region is below the first threshold, it is determined to be a "normal region." In other words, regions with reliability values ​​below the first threshold are uniformly determined to be "normal regions" regardless of their reliability values. On the other hand, in the estimation process, regions with reliability values ​​close to the first threshold, even if below it, are sometimes overlooked—that is, regions that are "determined to be normal but may be abnormal." For example, if the quality of the image input to the estimation process is low, accurate estimation may not be possible. In such cases, regions that are actually abnormal may be overlooked. In this invention, a second threshold, smaller than the first threshold, is set to avoid such a risk. The configuration is such that regions with reliability values ​​below the first threshold but above the second threshold are determined to be "potentially abnormal regions," and the characteristics of the image input to the estimation process are altered to perform the estimation process.

[0066] If the conditions are met (step S3: Yes), the communication control unit 516 sends a diagnostic image containing "areas that may be abnormal" and the reliability of each specified area in the diagnostic image from the communication unit 55 to an external device (step S4).

[0067] After step S4, the control unit 51 (cropped image generation unit 513 and high image quality processing unit 514) generates a cropped image and an image quality correction image (step S5). Here, the camera image obtained in step S1 and the cropped image and image quality correction image generated in step S5 are equivalent to the "multiple images containing the same subject and subjected to different processing" involved in the present invention.

[0068] Specifically, in step S5, the cropped image generation unit 513 generates a cropped image that magnifies the region in the diagnostic image containing "potentially abnormal areas". Furthermore, the high-image-quality processing unit 514 performs high-image-quality processing on the cropped image using a learned high-image-quality processing model, thereby generating a high-image-quality corrected image, similar to that generated by an endoscope (hereinafter referred to as a high-image-quality endoscope) that produces high-image-quality camera images. In addition to performing high-image-quality processing through the aforementioned AI processing, high-image-quality processing can also be performed through, for example, super-resolution processing or conventional image processing (grayscale processing, edge enhancement processing, frequency filtering processing, etc.).

[0069] Here, the learned model for high image quality enhancement is pre-stored in storage unit 54. Specifically, the learned model for high image quality enhancement is a learned model that has undergone repeated learning processing using multiple sets of training images and teaching data, one set for each training image and one set for teaching data. The training images are images whose image quality is reduced from camera images generated by a high image quality endoscope (hereinafter referred to as high image quality images) to an image quality corresponding to the cropped image. The teaching data is the high image quality images. The learning model used for this learning processing is, for example, a CNN. Furthermore, the learned model for high image quality enhancement includes a weight file (learning parameters) containing the weight values ​​and bias values ​​of each layer of the CNN.

[0070] Furthermore, the neural network used in the learning process of generating the learned model for high-quality image processing is not limited to CNNs; other neural networks can also be used. Additionally, various well-known learning algorithms can be employed as the machine learning algorithm within the neural network. For example, supervised learning algorithms using backpropagation can be used.

[0071] After step S5, the image selection unit 511 switches the diagnostic image to one of the cropped image and the image quality correction image generated in step S5 (step S6). For example, the image selection unit 511 selects the image that was preset in the user operation on the input unit 53 from the cropped image and the image quality correction image generated in step S5 as the diagnostic image. Then, the image selection unit 511 inputs the switched diagnostic image (cropped image or image quality correction image) to the estimation unit 512. That is, the control unit 51 returns to step S2. Furthermore, for ease of explanation, let's assume... Figure 4 As explained above, while the display image generation needs to always run as a moving image, the image input to the estimation process can also be different from the display image. That is, the image processing used for estimation only needs to run in the background and is not actually required to be as described above. Figure 4That kind of branching approach.

[0072] If the conditions are not met (step S3: No), the display control unit 515 generates a display image for the display unit 52 to display (step S7).

[0073] Furthermore, details regarding the display of images will be explained in the "Specific Examples of Displaying Images" section described later.

[0074] [Specific examples of displaying images]

[0075] Next, a specific example of the display image displayed on the display unit 52 will be described.

[0076] Figures 5 to 8 This is a diagram representing a specific example of an image being displayed.

[0077] For example, the display control unit 515 generates the image diagnostic assistance method described above. Figure 5 The display unit 515 then causes the display unit 52 to display the display image F1.

[0078] like Figure 5 As shown, image F1 contains observation position image F11 and diagnostic image F12.

[0079] The observation position image F11 is an image that overlays the observation position (the front end position of the insertion part 21) OP at the current time point on an image representing the shape of the observed object (the large intestine in this embodiment).

[0080] The diagnostic image F12 is the image (camera image, cropped image, or image quality corrected image) selected as the diagnostic image by the image selection unit 511. That is, when the image selection unit 511 switches the diagnostic image (step S6), the display control unit 515 switches the diagnostic image F12 on the displayed image F1 to the switched diagnostic image. Furthermore, if the estimation process determines that there is an "abnormal region" in the diagnostic image F12, the display control unit 515 will display identification information F13 (…). Figure 5 The identification information F13 is superimposed on the diagnostic image F12, where the identification information F13 is information that distinguishes the region that matches the "abnormal region" from other regions. That is, the display control unit 515 is equivalent to the notification control unit involved in the present invention.

[0081] For example, Figure 6The diagnostic image F121 shown is the image that was determined to meet the specified conditions in step S3. In this diagnostic image F121, region Ar1 is a region that "may be an abnormality". In the case of this diagnostic image F121, it was determined to meet the specified conditions in step S3, therefore a cropped image (or image quality corrected image) F122, which enlarges the region Ar2 containing region Ar1, is generated. Figure 7 (Step S5). Then, the cropped image (or image quality correction image) F122 is selected as the diagnostic image (step S6) and input to the estimation unit 512. In addition, the diagnostic image F12 displaying image F1 is switched to the cropped image (or image quality correction image) F122.

[0082] Additionally, for example, the display control unit 515 generates... Figure 8 The display unit 515 then causes the display unit 52 to display the display image F2.

[0083] Here, if the estimation process determines that there is an "abnormal region" in the diagnostic image F12, the control unit 51 sequentially stores the diagnostic image F12 in the storage unit 54.

[0084] Furthermore, based on user operations on the input unit 53, the display control unit 515 generates a display image F2 that shows a list of thumbnail images FT1 to FT9 of multiple diagnostic images F12 stored in the storage unit 54.

[0085] According to the above-described implementation method, the following effects are achieved.

[0086] In the control device 5 according to this embodiment, the image selection unit 511 selects any one of the following images as a diagnostic image based on the reliability of the diagnostic candidate region: the camera image acquired in step S1, the cropped image generated in step S5, and the image quality correction image. Then, the image selection unit 511 inputs the diagnostic image to the estimation unit 512.

[0087] Therefore, according to the control device 5 of this embodiment, a high-quality image can be input to the estimation unit 512, and an image suitable for diagnosis can be provided that estimates the diagnostic candidate region such as lesions with high accuracy.

[0088] Furthermore, to improve the accuracy of estimation processing, it is usually necessary to prepare appropriate teaching data and perform repeated learning. In contrast, in this structure, by appropriately and adaptively switching the image input to the estimation processing based on the reliability of the output in the estimation processing, the accuracy of the estimation processing can be improved even if the input image changes to an image that is unsuitable for the estimation processing, without the need for further learning.

[0089] In the control device 5 of this embodiment, the display control unit 515 overlays identification information F13 onto the diagnostic image F12. The identification information F13 is information that distinguishes the region that meets the criteria of "an abnormal region" from other regions.

[0090] Therefore, users such as surgeons can make appropriate diagnoses based on the image with identification information F13 superimposed on the diagnostic image F12.

[0091] In the control device 5 according to this embodiment, the communication control unit 516 transmits a diagnostic image containing "areas that may be abnormal" and the reliability of each specified area in the diagnostic image from the communication unit 55 to an external device (step S4).

[0092] Therefore, by relearning the diagnostic image in this external device, a learned model that can accurately estimate lesions can be regenerated.

[0093] In the control device 5 according to this embodiment, the image selection unit 511 determines whether the specified conditions have been met within a specified number of times or within a specified time based on the reliability of the diagnostic candidate region output from the estimation unit 512 in step S2.

[0094] Therefore, if a diagnosis is incorrectly made only once and the specified conditions are met, the diagnostic image will not be switched immediately, thus suppressing false detections.

[0095] (Other implementation methods)

[0096] The methods for implementing the present invention have been described above, but the present invention is not limited to the embodiments described above.

[0097] In the above embodiment, the image diagnostic aid device according to the present invention is mounted in the endoscope system 1 in which the insertion part 21 is composed of a flexible endoscope, but it is not limited thereto. For example, the image diagnostic aid device according to the present invention may also be mounted in an endoscope system in which the insertion part 21 is composed of a rigid endoscope. In addition, the image diagnostic aid device according to the present invention may also be mounted in medical observation systems such as surgical microscopes (for example, see Japanese Patent Application Laid-Open No. 2016-42981) that magnify a predetermined field of view of the subject (inside a biological body) or on the surface of the subject (on a biological body) for observation.

[0098] In the above embodiments, the "multiple images containing the same subject and subjected to different processing" involved in the present invention can be any two images, including the camera image obtained in step S1, the cropped image and image quality correction image generated in step S5, and the second camera image shown below.

[0099] The second image is generated by the camera unit 27 capturing the reflected light (fluorescence, etc.) from the subject PA in a state where light (excitation light, narrowband light) of the second wavelength range is irradiated into the body of the subject PA from the light source device 4.

[0100] In the above embodiment, a display unit 52 is used as the notification unit according to the present invention, but it is not limited thereto. A sound output unit such as a speaker that outputs sound may also be used as the notification unit according to the present invention.

[0101] In the above embodiments, the condition specified by the present invention is that a portion of the diagnostic image contains a region that is "potentially an abnormality," but it is not limited to this. Alternatively, the condition specified by the present invention may be that a portion of the diagnostic image is a "potentially an abnormality," and that there is no "abnormality region" in the diagnostic image.

[0102] Alternatively, in the above embodiments, variations 1 to 3 shown below may also be used.

[0103] (Variation Example 1)

[0104] Figure 9 This is a diagram illustrating a variation of embodiment 1. Specifically, Figure 9 Is with Figure 2 The corresponding diagram.

[0105] In the control unit 51 involved in this modified example 1, such as Figure 9 As shown, a permission setting unit 517 is added to the control unit 51 described in the above embodiment.

[0106] In this variation 1, the image selection unit 511 selects the image selected by the user operation as a diagnostic image based on the user operation performed on the input unit 53 to select a camera image or a cropped image (or an image quality correction image).

[0107] Then, the permission setting unit 517 sets the state of the image selection unit 511 to the permitted state or the prohibited state as shown below, based on the user operation on the input unit 53.

[0108] The permission state is a state in which the image selection unit 511 is permitted to select a diagnostic image corresponding to the aforementioned user operation. That is, in this permission state, the image selection unit 511 selects the image selected by the user operation as the diagnostic image.

[0109] The prohibited state is a state in which the image selection unit 511 is prohibited from selecting a diagnostic image corresponding to the aforementioned user operation. That is, in this prohibited state, even if a camera image or a cropped image (or an image quality correction image) is selected through user operation, the image selection unit 511 will not select the image selected through user operation as the diagnostic image.

[0110] Even when using the structure of this modified example 1 described above, it achieves the same effect as the above-described embodiment.

[0111] (Variation Example 2)

[0112] Figure 10 This is a diagram illustrating a variation of the implementation method, example 2. Specifically, Figure 10 Is with Figure 4 The corresponding diagram.

[0113] The storage unit 54 involved in this variation 2 stores the first to third learning parameters as shown below.

[0114] The first learning parameter is the learning parameter of the model that has been learned and is used when performing estimation processing on the camera image obtained in step S1.

[0115] The second learning parameter is the learning parameter of the learned model used for estimation processing, which corresponds to the cropped image generated in step S5 and is used when performing estimation processing on the cropped image.

[0116] The third learning parameter is the learning parameter of the learned model used for estimation processing when performing estimation processing on the image quality correction image, which corresponds to the image quality correction image generated in step S5.

[0117] Furthermore, in this modified example 2, in step S2, the estimation unit 512 uses the learning parameters corresponding to the input diagnostic image stored in the first to third learning parameters of the storage unit 54 to perform estimation processing on the diagnostic image. That is, when the input diagnostic image is the camera image acquired in step S1, the estimation unit 512 sets the learning parameters of the learned model for estimation processing as the first learning parameter to perform estimation processing on the camera image. In addition, when the diagnostic image is switched to a cropped image or an image quality correction image in step S6, the estimation unit 512 switches the learning parameters from the first learning parameter to the second or third learning parameter (step S8). Then, the estimation unit 512 sets the learning parameters of the learned model for estimation processing as the second or third learning parameter to perform estimation processing on the cropped image or the image quality correction image.

[0118] According to the above description, this modified example 2, in addition to having the same effects as the above-described embodiments, also has the following effects.

[0119] In this modified example 2, the estimation unit 512 uses the learning parameters corresponding to the input diagnostic image stored in the first to third learning parameters of the storage unit 54 to perform estimation processing on the diagnostic image. Therefore, by using the learning parameters corresponding to the input diagnostic image, lesions, etc., can be estimated with higher accuracy.

[0120] (Variation Example 3)

[0121] Figure 11 This is a diagram illustrating a variation of the implementation method, example 3. Specifically, Figure 11 Is with Figure 4 The corresponding diagram.

[0122] The storage unit 54 involved in this variation 3 stores the first to third completed learning models as shown below.

[0123] The first learned model is the learned model used for estimation processing that corresponds to the camera image obtained in step S1 and is used when performing estimation processing on the camera image.

[0124] The second learned model is the learned model used for estimation processing that corresponds to the cropped image generated in step S5 and is used when performing estimation processing on the cropped image.

[0125] The third learned model is the learned model used for estimation processing, which corresponds to the image quality correction image generated in step S5 and is used when performing estimation processing on the image quality correction image.

[0126] Furthermore, in this modified example 3, in step S2, the estimation unit 512 uses the estimation processing learning completion model corresponding to the input diagnostic image stored in the first to third learning completion models in the storage unit 54 to perform estimation processing on the diagnostic image. That is, when the input diagnostic image is the camera image acquired in step S1, the estimation unit 512 uses the first learning completion model to perform estimation processing on the camera image. In addition, when the diagnostic image is switched to a cropped image or an image quality correction image in step S6, the estimation unit 512 switches the estimation processing learning completion model from the first learning completion model to the second learning completion model or the third learning completion model (step S9). Then, the estimation unit 512 uses the second learning completion model or the third learning completion model to perform estimation processing on the cropped image or the image quality correction image.

[0127] According to the above description, this modified example 3, in addition to having the same effects as the above-described embodiments, also has the following effects.

[0128] In this modified example 3, the estimation unit 512 uses the estimation processing learning models corresponding to the input diagnostic image stored in the first to third learning models in the storage unit 54 to perform estimation processing on the diagnostic image. Therefore, by using the estimation processing learning models corresponding to the input diagnostic image, lesions, etc., can be estimated with higher accuracy.

[0129] Explanation of reference numerals in the attached figures

[0130] 1. Endoscopic system

[0131] 2. Endoscope

[0132] 3. Processing device

[0133] 4. Light source device

[0134] 5. Control Device

[0135] 21 Insertion section

[0136] 22 Operations Department

[0137] 23 General-purpose cables

[0138] 24 Connector Section

[0139] 25 Optical guide

[0140] 26 Illumination Lenses

[0141] 27. Camera Department

[0142] 51 Control Department

[0143] 52 Display Section

[0144] 53 Input Section

[0145] 54 Storage Department

[0146] 55 Ministry of Communications

[0147] 271 Lens Unit

[0148] 272 camera elements

[0149] 511 Image Selection Unit

[0150] 512 Estimation Department

[0151] 513 Image cropping generation unit

[0152] 514 High Image Quality Processing Unit

[0153] 515 Display Control Department

[0154] 516 Communications Control Department

[0155] 517 Permission / No Setting Department

[0156] Ar1 and Ar2 regions

[0157] BD bed

[0158] F1 and F2 display images.

[0159] F11 View Position Image

[0160] F12, F121 Diagnostic Images

[0161] F122 Crop Image

[0162] F13 Identification Information

[0163] Thumbnail images of FT1~FT9

[0164] OP observation position

[0165] PA Subject

Claims

1. An image diagnostic auxiliary device, comprising: An image selection unit selects any one of multiple images containing the same subject that have undergone different processing as a diagnostic image; and The estimation unit performs estimation processing on the diagnostic image using the learned model, thereby estimating the diagnostic candidate regions in the diagnostic image that will become diagnostic candidates, and outputs the reliability of the diagnostic candidate regions. in, The image selection unit selects any one of the plurality of images as the diagnostic image based on the reliability of the diagnostic candidate regions.

2. The image diagnostic auxiliary device according to claim 1, wherein, It also has: The notification department, and the information specified in its notifications; and The notification control unit then notifies the diagnostic candidate region.

3. The image diagnostic auxiliary device according to claim 1, wherein, The plurality of images includes at least two of a first camera image, a cropped image, an image quality corrected image, and a second camera image, wherein the first camera image is an image captured of the reflected light from the subject illuminated by light of the first wavelength range; the cropped image is an image in which a portion of the first camera image has been magnified; the image quality corrected image is an image in which the image quality of the cropped image has been corrected; and the second camera image is an image captured of the reflected light from the subject illuminated by light of the second wavelength range, which is different from the first wavelength range.

4. The image diagnostic auxiliary device according to claim 3, wherein, The cropped image is an image in which the region containing the diagnostic candidate region in the first camera image is magnified.

5. The image diagnostic auxiliary device according to claim 1, wherein, It also includes an operation processing unit that processes user operations for selecting any one of the plurality of images. The image selection unit selects any one of the plurality of images as the diagnostic image based on the user's operation.

6. The image diagnostic auxiliary device according to claim 5, wherein, It also includes a permission setting unit, which is set to either a permission state that allows the image selection unit to select any image from the plurality of images corresponding to the user operation, or a prohibition state that prohibits the image selection unit from making such selection.

7. The image diagnostic auxiliary device according to claim 1, wherein, The reliability of the diagnostic candidate region is a value representing the accuracy of identification within the diagnostic candidate region. If the reliability of the diagnostic candidate region is less than a first threshold and greater than a second threshold that is lower than the first threshold, the image selection unit will switch the diagnostic image selected at the current time point to another image among the plurality of images.

8. The image diagnostic auxiliary device according to claim 1, wherein, It also has: A display unit that displays a specified image; and The display control unit causes the display unit to display the diagnostic image selected by the image selection unit.

9. The image diagnostic auxiliary device according to claim 1, wherein, If the reliability of the diagnostic candidate region meets the specified conditions a specified number of times or within a specified time, the image selection unit will switch the diagnostic image selected at the current time point to another image among the plurality of images.

10. The image diagnostic auxiliary device according to claim 1, wherein, It also has: The communication unit, which is connected to external devices in a communicative manner; and A communication control unit that transmits the diagnostic image and the reliability of the diagnostic candidate region from the communication unit to the external device.

11. The image diagnostic auxiliary device according to claim 1, wherein, When the diagnostic image is switched by the image selection unit, the estimation unit switches the learning parameters of the learned model used in the estimation process to learning parameters corresponding to the switched diagnostic image.

12. The image diagnostic auxiliary device according to claim 1, wherein, When the diagnostic image is switched by the image selection unit, the estimation unit switches the learned model used for the estimation process to the learned model corresponding to the switched diagnostic image.

13. An image diagnostic assistance system, comprising: A camera device that generates a camera image by photographing a subject; and An image diagnostic aid device that processes the captured images. in, The image diagnostic auxiliary device includes: The image selection unit selects any one of a plurality of images containing the same subject and having undergone different processing on the captured images as a diagnostic image; as well as The estimation unit performs estimation processing on the diagnostic image using the learned model, thereby estimating the diagnostic candidate regions in the diagnostic image that will become diagnostic candidates, and outputs the reliability of the diagnostic candidate regions. The image selection unit selects any one of the plurality of images as the diagnostic image based on the reliability of the diagnostic candidate region.

14. An image diagnostic assistance method, executed by an image diagnostic assistance device, comprising the following steps: Select any one of multiple images containing the same subject that have undergone different processing as the diagnostic image; as well as The learned model is used to perform estimation processing on the diagnostic image, thereby estimating the diagnostic candidate regions in the diagnostic image that become diagnostic candidates, and outputting the reliability of the diagnostic candidate regions. In the step of selecting the diagnostic image, based on the reliability of the diagnostic candidate region, any image from the plurality of images is selected as the diagnostic image.

Images