Diagnostic imaging assistance device, diagnostic imaging assistance system, and diagnostic imaging assistance method

Abstract

This diagnostic imaging assistance device 5 comprises: an image selection unit 511 that selects, as a diagnostic image, one of a plurality of images including the same subject and in which different processes are executed; and an estimation unit 512 that executes an estimation process for the diagnostic image by using a trained model to estimate a diagnostic candidate region, which is a diagnostic candidate region in the diagnostic image, and outputs a degree of reliability of the diagnostic candidate region. The image selection unit 511 selects one of the plurality of images as a diagnostic image on the basis of the degree of reliability of the diagnostic candidate region.

Description

Image diagnosis support device, image diagnosis support system, and image diagnosis support method

[0001] The present invention relates to an image diagnosis support device, an image diagnosis support system, and an image diagnosis support method.

[0002] In recent years, image recognition technologies based on AI (Artificial Intelligence) have been proposed in the medical field (see, for example, Patent Document 1). The technology described in Patent Document 1 estimates a diagnostic candidate region such as a lesion in an image captured by an endoscope by performing an estimation process using a trained model on the image.

[0003] Patent No. 6952214

[0004] However, depending on the quality of the captured image input, AI-based image recognition technology may not be able to provide sufficient performance and may not be able to accurately estimate a diagnostic candidate region such as a lesion. In other words, it may not be possible to provide an image suitable for diagnosis.

[0005] The present invention has been made in view of the above, and aims to provide an image diagnosis support device, an image diagnosis support system, and an image diagnosis support method that can provide images suitable for diagnosis.

[0006] In order to solve the above-mentioned problems and achieve the object, the image diagnosis support device of the present invention comprises an image selection unit that selects one of multiple images that include the same subject and have been subjected to different processing as a diagnostic image, and an estimation unit that performs an estimation process on the diagnostic image using a trained model to estimate a diagnostic candidate region in the diagnostic image that will become a diagnostic candidate and output the reliability of the diagnostic candidate region, and the image selection unit selects one of the multiple images as the diagnostic image based on the reliability of the diagnostic candidate region.

[0007] The image diagnosis support system of the present invention comprises an imaging device that generates an imaged image by imaging a subject, and an image diagnosis support device that processes the imaged image. The image diagnosis support device comprises an image selection unit that selects, as a diagnostic image, one of a plurality of images that include the same subject and have been subjected to different processes on the imaged image, and an estimation unit that performs an estimation process on the diagnostic image using a trained model to estimate a diagnostic candidate region in the diagnostic image that will become a diagnostic candidate, and outputs the reliability of the diagnostic candidate region. The image selection unit selects, as the diagnostic image, one of the plurality of images based on the reliability of the diagnostic candidate region.

[0008] The image diagnosis support method according to the present invention is an image diagnosis support method executed by an image diagnosis support device, and includes the steps of: selecting one of a plurality of images containing the same subject and having been subjected to different processing as a diagnostic image; and performing an estimation process on the diagnostic image using a trained model to estimate a diagnostic candidate region in the diagnostic image that will serve as a diagnostic candidate, and outputting the reliability of the diagnostic candidate region. In the step of selecting the diagnostic image, one of the plurality of images is selected as the diagnostic image based on the reliability of the diagnostic candidate region.

[0009] According to the image diagnosis support device, image diagnosis support system, and image diagnosis support method of the present invention, it is possible to provide images suitable for diagnosis.

[0010] FIG. 1 is a diagram illustrating the configuration of an endoscope system according to an embodiment. FIG. 2 is a diagram illustrating the configuration of an endoscope system according to an embodiment. FIG. 3 is a diagram conceptually illustrating the functions of a control unit. FIG. 4 is a flowchart illustrating an image diagnosis support method. FIG. 5 is a diagram illustrating a specific example of a display image. FIG. 6 is a diagram illustrating a specific example of a display image. FIG. 7 is a diagram illustrating a specific example of a display image. FIG. 8 is a diagram illustrating a specific example of a display image. FIG. 9 is a diagram illustrating a first modification of the embodiment. FIG. 10 is a diagram illustrating a second modification of the embodiment. FIG. 11 is a diagram illustrating a third modification of the embodiment.

[0011] Hereinafter, a mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the description of the drawings, the same parts are given the same reference numerals.

[0012] 1 and 2 are diagrams illustrating the configuration of an endoscope system 1 according to an embodiment. The endoscope system 1 corresponds to an image diagnosis support system according to the present invention. This endoscope system 1 is used in the medical field and is a system for observing the inside of the body (the large intestine in this embodiment) of a subject PA (FIG. 1) who is a patient on a bed BD (FIG. 1). As shown in FIGS. 1 and 2, this endoscope system 1 includes an endoscope 2 and a processing device 3.

[0013] The endoscope 2 corresponds to an imaging device according to the present invention. In this embodiment, the endoscope 2 is a so-called flexible endoscope. A portion of the endoscope 2 is inserted into the body of a subject PA, images the interior of the body, and outputs image signals generated by the images. As shown in FIGS. 1 and 2 , the endoscope 2 includes an insertion section 21, an operation section 22, a universal cord 23, and a connector section 24. Note that, for ease of explanation, the operation section 22, the universal cord 23, and the connector section 24 are not shown in FIG. 2 .

[0014] The insertion section 21 has at least a portion that is flexible and is inserted into the body of the subject PA. As shown in FIG. 2 , a light guide 25, an illumination lens 26, and an imaging section 27 are provided within the insertion section 21.

[0015] The light guide 25 is routed from the insertion section 21, through the operation section 22 and the universal cord 23, to the connector section 24. One end of the light guide 25 is located at the tip portion within the insertion section 21. When the endoscope 2 is connected to the processing device 3, the other end of the light guide 25 is located within the processing device 3. The light guide 25 transmits light supplied from the light source device 4 within the processing device 3 from the other end to one end. The illumination lens 26 faces one end of the light guide 25 within the insertion section 21. The illumination lens 26 irradiates the light transmitted by the light guide 25 into the body of the subject PA.

[0016] The imaging unit 27 is provided at the tip portion of the insertion unit 21. The imaging unit 27 captures images of the inside of the subject PA and outputs image signals generated by the image capture. As shown in FIG. 2 , the imaging unit 27 includes a lens unit 271 and an imaging element 272.

[0017] The lens unit 271 takes in the returning light (subject image) of the light irradiated into the body of the subject PA from the illumination lens 26 , and forms the subject image on the light receiving surface of the image sensor 272 .

[0018] The image sensor 272 is configured with a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) that receives light from a subject image and converts it into an electrical signal, and generates an image signal by capturing the subject image. Note that, hereinafter, the image signal generated by the image capturing unit 27 will be referred to as a captured image.

[0019] The operation section 22 is connected to the proximal end portion of the insertion section 21. The operation section 22 receives various operations on the endoscope 2.

[0020] The universal cord 23 extends from the operating section 22 in a direction different from the direction in which the insertion section 21 extends, and is a cord on which signal lines that electrically connect the imaging section 27 and the control device 5 in the processing device 3, a light guide 25, etc. are arranged.

[0021] The connector portion 24 is provided at the end of the universal cord 23 and is detachably connected to the processing device 3 .

[0022] As shown in FIG. 2 , the processing device 3 includes a light source device 4 and a control device 5 .

[0023] The light source device 4 supplies light to the other end of the light guide 25 under the control of the control device 5. In the present embodiment, the light source device 4 emits white light as light of the first wavelength band. Note that the light source device 4 may be configured to emit excitation light for exciting a fluorescent agent such as indocyanine green, narrow band light used in NBI (Narrow Band Imaging), or the like as light of a second wavelength band different from the first wavelength band.

[0024] The control device 5 corresponds to the image diagnosis support device according to the present invention. As shown in FIG. 2 , the control device 5 includes a control unit 51, a display unit 52, an input unit 53, a storage unit 54, and a communication unit 55.

[0025] The control unit 51 includes a controller such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array), and controls the overall operation of the endoscope system 1. As shown in FIG. 2 , the control unit 51 has functions as an image selection unit 511, an estimation unit 512, a trimmed image generation unit 513, an image quality improvement processing unit 514, a display control unit 515, and a communication control unit 516.

[0026] FIG. 3 is a diagram conceptually illustrating the functions of the control unit 51. In FIG. 3, "input image processing" indicated by reference numeral 51B1, to which a captured image (endoscopic image) is input, includes an image selection unit 511, a trimmed image generation unit 513, and an image quality improvement processing unit 514. Also in FIG. 3, "estimation processing" indicated by reference numeral 51B2 includes an estimation unit 512. Furthermore, in FIG. 3, "display image generation" indicated by reference numeral 51B3 includes a display control unit 515. Note that the functions of the image selection unit 511, the estimation unit 512, the trimmed image generation unit 513, the image quality improvement processing unit 514, the display control unit 515, and the communication control unit 516 in the control unit 51 will be described in the "Image diagnosis support method" described later.

[0027] The display unit 52 corresponds to a notification unit according to the present invention. The display unit 52 is an LCD (Liquid Crystal Display) or an EL (Electro Luminescence) display, and displays a display image generated by the control unit 51 under the control of the control unit 51.

[0028] The input unit 53 corresponds to an operation receiving unit according to the present invention. The input unit 53 is configured using a keyboard, a mouse, a switch, a touch panel, etc., and receives user operations by a user such as a surgeon. The input unit 53 then outputs an operation signal corresponding to the user operation to the control unit 51.

[0029] The storage unit 54 stores various programs executed by the control unit 51, information necessary for the processing of the control unit 51, and the like.

[0030] The communication unit 55 is connected to an external device so as to be able to communicate with the external device, and under the control of the control unit 51, the communication unit 55 transmits predetermined information (data) to the external device.

[0031] [Image diagnosis support method] Next, the image diagnosis support method executed by the control device 5 described above will be described with reference to Figures 3 and 4. Figure 4 is a flowchart showing the image diagnosis support method. First, the image selection unit 511 acquires a captured image generated by the imaging unit 27 capturing return light (subject image) of the white light from inside the subject PA while the light source device 4 is irradiating the inside of the subject PA (step S1). This captured image corresponds to the first captured image according to the present invention. Then, the image selection unit 511 selects this captured image as a diagnostic image and inputs this diagnostic image to the estimation unit 512.

[0032] After step S1, the estimation unit 512 performs estimation processing on the diagnostic image using a trained model for estimation processing, thereby estimating diagnostic candidate regions that will become diagnostic candidates for each specified region in the diagnostic image, and outputs the reliability of the diagnostic candidate regions (step S2).

[0033] The reliability of a diagnostic candidate region is a value indicating the level of reliability. Specifically, the reliability is a value indicating the accuracy of image recognition in the diagnostic candidate region, and can also be considered an index indicating the probability that an object in the image is predicted to belong to a specific class. The reliability of a diagnostic candidate region can be used to determine whether an object has been accurately recognized within the region.

[0034] Here, the trained model for estimation processing corresponds to the trained model according to the present invention. This trained model for estimation processing is pre-stored in the storage unit 54. Specifically, the trained model for estimation processing is a trained model obtained by repeatedly performing a training process on the trained model using multiple sets of training images and teacher data, each set consisting of a training image and teacher data. The training images are captured images of an in-vivo image. The teacher data is data annotated with the classification class, correct position, and size of lesions and the like in the training images. The trained model used in the training process is, for example, a convolutional neural network (CNN). The trained model for estimation processing includes a weight file (learning parameters) having weight values ​​and bias values ​​for each layer of the CNN.

[0035] The neural network used in the learning process for generating the trained model for estimation is not limited to a CNN, and other neural networks may be used. For example, neural networks such as a deep neural network (DNN), a transformer, or a generative adversarial network (GAN) may be used as appropriate. Furthermore, various well-known learning algorithms may be used as machine learning algorithms in the neural network. For example, a supervised learning algorithm using backpropagation may be used.

[0036] After step S2, the image selection unit 511 determines whether a predetermined condition has been satisfied a predetermined number of times or for a predetermined period of time based on the reliability of the diagnostic candidate region output from the estimation unit 512 in step S2 (step S3).

[0037] The predetermined condition used in step S3 can be exemplified by the following condition. The predetermined condition is that a "region that may be an abnormal region" exists in a part of the diagnostic image. Here, a "region that may be an abnormal region" refers to a region whose reliability is less than a first threshold and equal to or greater than a second threshold that is lower than the first threshold. Furthermore, a "region that is a normal region" described below refers to a region whose reliability is less than the second threshold. Furthermore, a "region that is an abnormal region" described below refers to a region whose reliability is equal to or greater than the first threshold.

[0038] A typical estimation process uses a single threshold to determine whether a region is normal or abnormal. Specifically, if the reliability of the region is equal to or greater than a first threshold, it is determined to be an "abnormal region." If the reliability of the region is less than the first threshold, it is determined to be a "normal region." In other words, regions that are less than the first threshold are determined to be "normal regions" regardless of the value of the reliability. On the other hand, the estimation process may overlook regions that are less than the first threshold but have a reliability close to the first threshold, i.e., "regions that were determined to be normal but may be abnormal." For example, the quality of the image input to the estimation process may be low, preventing accurate estimation. In such cases, abnormal regions may be overlooked. To avoid this risk, the present invention provides a second threshold that is smaller than the first threshold. If the reliability of the region is less than the first threshold but greater than or equal to the second threshold, it is determined to be a "possibly abnormal region," and the characteristics of the image input to the estimation process are changed to perform the estimation process.

[0039] If it is determined that the specified conditions are met (step S3: Yes), the communication control unit 516 causes the communication unit 55 to transmit a diagnostic image including ``areas that may be abnormal'' and the reliability of each specified area in the diagnostic image to an external device (step S4).

[0040] After step S4, the control unit 51 (the trimmed image generating unit 513 and the image quality improving processing unit 514) generates a trimmed image and an image quality corrected image (step S5). Here, the captured image acquired in step S1 and the trimmed image and the image quality corrected image generated in step S5 correspond to "multiple images including the same subject and subjected to different processes" according to the present invention.

[0041] Specifically, in step S5, the trimmed image generation unit 513 generates a trimmed image by enlarging an area in the diagnostic image that includes a "potentially abnormal area." The image quality improvement processing unit 514 performs image quality improvement processing on the trimmed image using the trained model for image quality improvement processing, thereby generating a high-quality, image-corrected image that appears as if it were generated by an endoscope that generates high-quality captured images (hereinafter referred to as a "high-quality endoscope"). Note that the image quality improvement processing may be performed using, in addition to the AI ​​processing described above, super-resolution processing or classical image processing (gradation processing, edge enhancement processing, frequency filtering, etc.), for example.

[0042] Here, the trained model for image quality improvement processing is pre-stored in the storage unit 54. Specifically, the trained model for image quality improvement processing is a trained model obtained by repeatedly executing a training process on the trained model using multiple sets of training images and teacher data, each set consisting of a training image and teacher data. The training images are images obtained by lowering the image quality of captured images (hereinafter referred to as high-quality images) generated by a high-quality endoscope to an image quality corresponding to the trimmed image. The teacher data is the high-quality images. The trained model used in the training process is, for example, a CNN. The trained model for image quality improvement processing includes a weight file (learning parameters) having weight values ​​and bias values ​​for each layer of the CNN.

[0043] The neural network used in the learning process for generating the trained model for image quality improvement processing is not limited to a CNN, and other neural networks may be used. Furthermore, various well-known learning algorithms may be used as the machine learning algorithm for the neural network. For example, a supervised learning algorithm using backpropagation may be used.

[0044] After step S5, the image selection unit 511 switches the diagnostic image to either the cropped image or the image quality correction image generated in step S5 (step S6). For example, the image selection unit 511 selects, as the diagnostic image, one of the cropped images or the image quality correction images generated in step S5 that has been preset by a user operation on the input unit 53. The image selection unit 511 then inputs the switched diagnostic image (cropped image or image quality correction image) to the estimation unit 512. That is, the control unit 51 returns to step S2. Note that, for convenience of explanation, the display image generation needs to be constantly running as a moving image, while the image input to the estimation processing can be a different image from the display image. That is, the image processing for the estimation processing only needs to run in the background, and in reality, there is no need to branch and process as shown in FIG. 4.

[0045] If it is determined that the predetermined condition is not satisfied (step S3: No), the display control unit 515 generates a display image to be displayed on the display unit 52 (step S7). Details of the display image will be described later in "Specific examples of display images."

[0046] [Specific Examples of Display Images] Next, specific examples of display images displayed on the display unit 52 will be described. Figures 5 to 8 are diagrams showing specific examples of display images. For example, the display control unit 515 generates the display image F1 shown in Figure 5 in the image diagnosis support method described above. Then, the display control unit 515 causes the display image F1 to be displayed on the display unit 52.

[0047] As shown in FIG. 5, the display image F1 includes an observation position image F11 and a diagnostic image F12.

[0048] The observation position image F11 is an image in which the current observation position (the tip position of the insertion portion 21) OP is superimposed on an image showing the shape of the observation target (the large intestine in this embodiment).

[0049] The diagnostic image F12 is an image (a captured image, a cropped image, or an image quality correction image) selected as a diagnostic image by the image selection unit 511. That is, when the diagnostic image is switched by the image selection unit 511 (step S6), the display control unit 515 switches the diagnostic image F12 on the display image F1 to the switched diagnostic image. Furthermore, if the estimation process determines that there is an "abnormal area" in the diagnostic image F12, the display control unit 515 superimposes identification information F13 ( FIG. 5 ) that distinguishes the area corresponding to the "abnormal area" from other areas on the diagnostic image F12. That is, the display control unit 515 corresponds to a notification control unit according to the present invention.

[0050] For example, the diagnostic image F121 shown in FIG. 6 is a captured image determined to satisfy the predetermined conditions in step S3. In this diagnostic image F121, area Ar1 is a "region that may be an abnormal area." In the case of this diagnostic image F121, since it is determined to satisfy the predetermined conditions in step S3, a trimmed image (or image quality correction image) F122 ( FIG. 7 ) is generated by enlarging area Ar2 including area Ar1 (step S5). Then, the trimmed image (or image quality correction image) F122 is selected as the diagnostic image (step S6) and input to the estimation unit 512. Furthermore, the diagnostic image F12 in the display image F1 is switched to the trimmed image (or image quality correction image) F122.

[0051] 8 , for example. Then, the display control unit 515 displays the display image F2 on the display unit 52. Here, if the estimation process determines that there is an "abnormal area" in the diagnostic image F12, the control unit 51 sequentially stores the diagnostic images F12 in the storage unit 54. Then, in response to a user operation on the input unit 53, the display control unit 515 generates a display image F2 that displays a list of thumbnail images FT1 to FT9 of the multiple diagnostic images F12 stored in the storage unit 54.

[0052] The present embodiment described above provides the following advantages. In the control device 5 according to the present embodiment, the image selection unit 511 selects, as a diagnostic image, one of the captured image acquired in step S1 and the cropped image and image quality correction image generated in step S5, based on the reliability of the diagnostic candidate region. The image selection unit 511 then inputs the selected diagnostic image to the estimation unit 512. Therefore, the control device 5 according to the present embodiment can input a high-quality image to the estimation unit 512, accurately estimate a diagnostic candidate region such as a lesion, and provide an image suitable for diagnosis.

[0053] In general, to improve the accuracy of estimation processing, it is necessary to prepare appropriate training data and repeat learning. In contrast, with this configuration, by appropriately and adaptively switching the image input to the estimation processing according to the reliability output by the estimation processing, it is possible to improve the accuracy of the estimation processing without re-learning, even in situations where the input image changes to an image unsuitable for the estimation processing.

[0054] In the control device 5 according to this embodiment, the display control unit 515 superimposes identification information F13, which distinguishes the area corresponding to the "area that is abnormal" from other areas, on the diagnostic image F12. This allows a user such as an operator to make an appropriate diagnosis based on the image in which the identification information F13 is superimposed on the diagnostic image F12.

[0055] In the control device 5 according to this embodiment, the communication control unit 516 transmits the diagnostic image including the "region that may be an abnormal region" and the reliability of each predetermined region in the diagnostic image from the communication unit 55 to an external device (step S4). Therefore, the external device can perform a new learning process using the diagnostic image, thereby generating a new trained model for estimation process that can accurately estimate lesions, etc.

[0056] In the control device 5 according to this embodiment, the image selection unit 511 determines whether a predetermined condition has been satisfied a predetermined number of times or for a predetermined period of time based on the reliability of the diagnostic candidate region output from the estimation unit 512 in step S2. Therefore, even if it is erroneously determined that the predetermined condition has been satisfied only once, the diagnostic image is not immediately switched, and erroneous detection can be suppressed.

[0057] Other Embodiments Up to this point, the embodiments for carrying out the present invention have been described, but the present invention should not be limited to only the above-described embodiments. In the above-described embodiments, the image diagnosis support device according to the present invention is mounted on an endoscope system 1 having an insertion section 21 configured by a flexible endoscope, but this is not limiting. For example, the image diagnosis support device according to the present invention may be mounted on an endoscope system having an insertion section 21 configured by a rigid endoscope. Furthermore, the image diagnosis support device according to the present invention may be mounted on a medical observation system such as a surgical microscope (see, for example, JP 2016-42981 A) that magnifies and observes a predetermined field of view inside a subject (inside a living body) or on the surface of a subject (surface of a living body).

[0058] In the above-described embodiment, the "multiple images including the same subject and on which different processes have been performed" according to the present invention may include at least two of the captured image obtained in step S1, the cropped image and image quality correction image generated in step S5, and the second captured image shown below.

[0059] The second captured image is an image generated by the imaging unit 27 capturing the return light (fluorescence, etc.) of the light in the second wavelength band from inside the subject PA while the light source device 4 is irradiating the body of the subject PA with light in the second wavelength band (excitation light or narrowband light).

[0060] In the above-described embodiment, the display unit 52 is used as the notification unit according to the present invention, but this is not limited to this, and an audio output unit such as a speaker that outputs audio may also be used as the notification unit according to the present invention.

[0061] In the above-described embodiment, the predetermined condition according to the present invention is that a part of the diagnostic image has a "region that may be an abnormal region," but this is not limiting. The predetermined condition according to the present invention may be that a part of the diagnostic image has a "region that may be an abnormal region," and the diagnostic image does not have an "region that is an abnormal region."

[0062] In addition, in the above-described embodiment, the following modified examples 1 to 3 may be adopted.

[0063] (Modification 1) Fig. 9 is a diagram illustrating Modification 1 of the embodiment. Specifically, Fig. 9 is a diagram corresponding to Fig. 2. In the control unit 51 according to Modification 1, as shown in Fig. 9, a permission / denial setting unit 517 is added to the control unit 51 described in the embodiment above.

[0064] In response to a user operation to select a captured image or a trimmed image (or an image quality corrected image) on the input unit 53, the image selection unit 511 according to the present first modification selects the image selected by the user operation as a diagnostic image.

[0065] The permission setting unit 517 sets the state of the image selection unit 511 to either a permission state or a prohibition state, as shown below, in response to a user operation on the input unit 53. The permission state is a state in which the image selection unit 511 is permitted to select a diagnostic image in response to the user operation described above. That is, in the permission state, the image selection unit 511 selects an image selected by the user operation as a diagnostic image. The prohibition state is a state in which the image selection unit 511 is prohibited from selecting a diagnostic image in response to the user operation described above. That is, in the prohibition state, even if a captured image or a cropped image (or an image quality corrected image) is selected by the user operation, the image selection unit 511 does not select the image selected by the user operation as a diagnostic image.

[0066] Even when the configuration of the present modified example 1 described above is adopted, the same effects as those of the above-described embodiment are achieved.

[0067] (Variation 2) Fig. 10 is a diagram illustrating Variation 2 of the embodiment. Specifically, Fig. 10 is a diagram corresponding to Fig. 4. The storage unit 54 according to Variation 2 stores the following first to third learning parameters. The first learning parameter corresponds to the captured image acquired in step S1, and is a learning parameter of a trained model for estimation processing used when performing estimation processing on the captured image.

[0068] The second learning parameters correspond to the trimmed image generated in step S5 and are learning parameters of a trained model for estimation processing that is used when performing estimation processing on the trimmed image.

[0069] The third learning parameters correspond to the image quality corrected image generated in step S5, and are learning parameters of a trained model for estimation processing that is used when performing estimation processing on the image quality corrected image.

[0070] In this second modification, in step S2, the estimation unit 512 performs an estimation process on the input diagnostic image using a learning parameter corresponding to the input diagnostic image from among the first to third learning parameters stored in the storage unit 54. That is, if the input diagnostic image is a captured image acquired in step S1, the estimation unit 512 performs an estimation process on the captured image using the learning parameter of the trained model for estimation process as the first learning parameter. Furthermore, if the diagnostic image is switched to a cropped image or an image quality corrected image in step S6, the estimation unit 512 switches the learning parameter from the first learning parameter to the second learning parameter or the third learning parameter (step S8). Then, the estimation unit 512 performs an estimation process on the cropped image or the image quality corrected image using the learning parameter of the trained model for estimation process as the second learning parameter or the third learning parameter.

[0071] The above-described second modification provides the same effects as the above-described embodiment, as well as the following effects: In the second modification, the estimation unit 512 executes estimation processing on an input diagnostic image using a learning parameter corresponding to the input diagnostic image, among the first to third learning parameters stored in the storage unit 54. Therefore, by using the learning parameter corresponding to the input diagnostic image, it is possible to estimate a lesion or the like with even greater accuracy.

[0072] (Variation 3) Fig. 11 is a diagram illustrating Variation 3 of the embodiment. Specifically, Fig. 11 is a diagram corresponding to Fig. 4. The storage unit 54 according to Variation 3 stores first to third trained models described below. The first trained model corresponds to the captured image acquired in step S1 and is a trained model for estimation processing used when performing estimation processing on the captured image.

[0073] The second trained model corresponds to the trimmed image generated in step S5 and is a trained model for estimation processing used when performing estimation processing on the trimmed image.

[0074] The third trained model corresponds to the image quality corrected image generated in step S5 and is a trained model for estimation processing used when performing estimation processing on the image quality corrected image.

[0075] In the third modification, in step S2, the estimation unit 512 performs an estimation process on the input diagnostic image using a trained model for estimation processing that corresponds to the input diagnostic image, among the first to third trained models stored in the storage unit 54. That is, if the input diagnostic image is a captured image acquired in step S1, the estimation unit 512 performs an estimation process on the captured image using the first trained model. Furthermore, if the diagnostic image is switched to a cropped image or an image with corrected image quality in step S6, the estimation unit 512 switches the trained model for estimation processing from the first trained model to the second trained model or the third trained model (step S9). The estimation unit 512 then performs an estimation process on the cropped image or the image with corrected image quality using the second trained model or the third trained model.

[0076] The third modification described above provides the same effects as the above-described embodiment, as well as the following effects: In the third modification, the estimation unit 512 performs estimation processing on an input diagnostic image using a trained model for estimation processing that corresponds to the input diagnostic image, among the first to third trained models stored in the storage unit 54. Therefore, by using the trained model for estimation processing that corresponds to the input diagnostic image, it is possible to estimate lesions and the like with even greater accuracy.

[0077] REFERENCE SIGNS LIST 1 Endoscope system 2 Endoscope 3 Processing device 4 Light source device 5 Control device 21 Insertion section 22 Operation section 23 Universal cord 24 Connector section 25 Light guide 26 Illumination lens 27 Imaging section 51 Control section 52 Display section 53 Input section 54 Storage section 55 Communication section 271 Lens unit 272 Imaging element 511 Image selection section 512 Estimation section 513 Trimmed image generation section 514 Image quality improvement processing section 515 Display control section 516 Communication control section 517 Permit / reject setting section Ar1, Ar2 Area BD Bed F1, F2 Display image F11 Observation position image F12, F121 Diagnostic image F122 Trimmed image F13 Identification information FT1 to FT9 Thumbnail image OP Observation position PA Subject

Claims

1. An imaging diagnostic device equipped with a processor, The processor: Selecting one of a plurality of images including the same subject and having been subjected to different processing as a diagnostic image; performing an estimation process on the diagnostic image using the trained model to estimate a diagnostic candidate region in the diagnostic image that will be a diagnostic candidate, and outputting the reliability of the diagnostic candidate region; When the reliability of the diagnostic candidate region satisfies a predetermined condition, the image diagnosis support device switches the currently selected diagnostic image to another image from the plurality of images and executes the estimation process again.

2. Further comprising a notification unit that notifies predetermined information, The processor: The image diagnosis support device according to claim 1 , wherein the diagnostic candidate region is notified from the notifying unit.

3. The plurality of images are 2. The image diagnosis support device according to claim 1, comprising at least two images: a first captured image obtained by capturing return light of light in a first wavelength band from the subject irradiated with light in the first wavelength band; a cropped image obtained by enlarging a partial area of ​​the first captured image; an image quality corrected image obtained by correcting the image quality of the cropped image; and a second captured image obtained by capturing return light of light in the second wavelength band from the subject irradiated with light in a second wavelength band different from the first wavelength band.

4. The trimmed image is 4. The image diagnosis support device according to claim 3, wherein the image is an enlarged image of a region including the diagnostic candidate region in the first captured image.

5. an operation receiving unit that receives a user operation to select one of the plurality of images; The processor: The image diagnosis support device according to claim 1 , wherein one of the plurality of images is selected as the diagnostic image in response to the user operation.

6. The processor: The image diagnosis support device according to claim 5 , wherein the selection of any one of the plurality of images in response to the user operation is set to an allowed state, or to a prohibited state.

7. The reliability of the diagnostic candidate region is a value indicating the accuracy of recognition in the diagnostic candidate region; The processor:

2. The image diagnosis support device according to claim 1, wherein, when the reliability of the diagnostic candidate region is less than a first threshold and is equal to or greater than a second threshold that is lower than the first threshold, the currently selected diagnostic image is switched to another image among the plurality of images.

8. Further comprising a display unit for displaying a predetermined image, The processor: The image diagnosis support device according to claim 1 , wherein the selected diagnostic image is displayed on the display unit.

9. The processor:

2. The image diagnosis support device according to claim 1, wherein the currently selected diagnostic image is switched to another image from among the plurality of images when the reliability of the diagnostic candidate region satisfies the predetermined condition a predetermined number of times or for a predetermined period of time.

10. Further, a communication unit is provided for communication with an external device, The processor: The image diagnosis support device according to claim 1 , wherein the diagnostic image and the reliability of the diagnostic candidate region are transmitted from the communication unit to the external device.

11. The processor: The image diagnosis support device according to claim 1 , wherein when the diagnostic image is switched, the learning parameters of the trained model used in the estimation process are switched to learning parameters corresponding to the switched diagnostic image.

12. The processor: The image diagnosis support device according to claim 1 , wherein when the diagnostic image is switched, the trained model used in the estimation process is switched to a trained model corresponding to the switched diagnostic image.

13. an imaging device that captures an image of a subject to generate a captured image; An image diagnosis support system comprising the image diagnosis support device according to claim 1 that processes the captured image.

14. An image diagnosis support method executed by a processor of an image diagnosis support device, comprising: The processor: selecting one of a plurality of images containing the same subject and having been subjected to different processing as a diagnostic image; performing an estimation process on the diagnostic image using the trained model to estimate a diagnostic candidate region in the diagnostic image that will be a diagnostic candidate, and outputting the reliability of the diagnostic candidate region; An image diagnosis support method in which, if the reliability of the diagnostic candidate region satisfies a predetermined condition, the currently selected diagnostic image is switched to another image from the plurality of images and the estimation process is performed again.