Image processing device, endoscope, image processing method, and program

Abstract

This image processing device comprises a processor. The processor recognizes the position of a region under observation in a medical image on the basis of the medical image including the region under observation, determines whether to output the size on the basis of the position, and measures the size on the basis of the medical image when the processor determines that the size is to be output.

Description

Image processing device, endoscope, image processing method, and program

[0001] The technology disclosed herein relates to an image processing device, an endoscope, an image processing method, and a program.

[0002] JP 2022-535873 A discloses a method for processing colon images and videos, which generates instructions for presenting a graphical user interface (GUI) for dynamically tracking at least one polyp in multiple endoscopic images of a patient's colon.

[0003] The method described in JP 2022-535873 A includes tracking the position of a region depicting a polyp, and if the position of the region is outside each of the endoscopic images, calculating a vector from within each of the endoscopic images to the position of the region outside each of the endoscopic images, augmenting each of the endoscopic images with a representation of the vector to create an augmented endoscopic image, and generating an instruction for displaying the augmented endoscopic image in a GUI, repeating the steps for a plurality of endoscopic images.

[0004] Japanese Patent Application Laid-Open Publication No. 2020-093076 discloses a medical image processing device that includes an acquisition unit that acquires a tomographic image of a test eye, and a first processing unit that executes a first detection process to detect at least one retinal layer out of multiple retinal layers in the acquired tomographic image using a trained model obtained by learning data showing at least one retinal layer out of multiple retinal layers in the tomographic image of the test eye.

[0005] International Publication No. 2020 / 110214 discloses an endoscopic system that includes an image input unit that sequentially inputs multiple observation images obtained by capturing an image of a subject with an endoscope, a lesion detection unit that detects a lesion that is the target of endoscopic observation from the observation image, an oversight risk analysis unit that determines the degree of oversight risk, which is the risk that an operator will overlook a lesion, based on the observation image, a notification control unit that controls a notification means and notification method for the detection of a lesion based on the degree of oversight risk, and a notification unit that notifies the operator of the detection of a lesion based on the control of the notification control unit.

[0006] In the endoscope system described in WO 2020 / 110214, the oversight risk analysis unit includes a lesion analysis unit that analyzes the oversight risk based on the state of the lesion, a lesion size analysis unit that estimates the size of the lesion itself, and a lesion position analysis unit that analyzes the position of the lesion in the observation image.

[0007] One embodiment of the technology disclosed herein provides an image processing device, an endoscope, an image processing method, and a program that enable a user or the like to accurately grasp the size of an observation area shown in a medical image.

[0008] A first aspect of the technology of the present disclosure is an image processing device that includes a processor, which recognizes the position of an observation area within a medical image based on the medical image in which the observation area appears, determines whether to output the size of the observation area based on the position, and outputs the size if it is determined that the size should be output.

[0009] A second aspect of the technology disclosed herein is an image processing device according to the first aspect, in which the medical image is a plurality of frames arranged in a chronological order, and the processor recognizes the position of each of the plurality of frames and determines whether to output using the amount of change in position between the plurality of frames.

[0010] A third aspect of the technology of the present disclosure is the image processing device according to the second aspect, in which the amount of change in position between multiple frames is defined based on the distance between positions between the multiple frames.

[0011] A fourth aspect of the technology of the present disclosure is an image processing device according to the second or third aspect, in which the amount of change in position between multiple frames is determined based on the degree of overlap of the observation target area between the multiple frames.

[0012] A fifth aspect of the technology of the present disclosure is an image processing device according to any one of the first to fourth aspects, in which a processor measures size by performing AI-based processing on a medical image.

[0013] A sixth aspect of the technology of the present disclosure is an image processing device according to any one of the first to fourth aspects, in which a processor derives the distance from the observation position to the observation target area by performing AI-based processing on the medical image, and measures the size based on the distance and the number of pixels in the range to be measured within the observation target area.

[0014] A seventh aspect of the technology of the present disclosure is an image processing device according to any one of the first to sixth aspects, in which the processor determines to perform output when the position is in a first region within the medical image, and determines not to perform output when the position is in a second region outside the first region within the medical image.

[0015] An eighth aspect of the technology of the present disclosure is an image processing device according to any one of the first to sixth aspects, in which the medical image is a plurality of frames in chronological order, and the processor recognizes a position in each of the plurality of frames and determines whether to output based on the amount of change in position between the plurality of frames and whether the position is located in a first region within the medical image or a second region outside the first region within the medical image.

[0016] A ninth aspect of the technology of the present disclosure is an image processing device according to any one of the first to eighth aspects, in which the medical images are multiple frames in chronological order, and the processor recognizes the position of each of the multiple frames using an AI bounding box method, and determines whether to output using the amount of change in the bounding box.

[0017] A tenth aspect of the technology of the present disclosure is an image processing device according to any one of the first to eighth aspects, in which the medical images are multiple frames in chronological order, and the processor recognizes the position of each of the multiple frames using an AI segmentation method and determines whether or not to perform a measurement using the amount of change in the segmentation area.

[0018] An eleventh aspect of the technology of the present disclosure is an image processing device according to any one of the first to tenth aspects, in which the processor determines whether to output based on whether the position is at the edge of the medical image.

[0019] A twelfth aspect of the technology of the present disclosure is an image processing device according to any one of the first to seventh aspects, in which the medical image is a plurality of frames in chronological order, and the processor determines whether to output based on a position within a first frame selected from the plurality of frames in accordance with given instructions, and a position within at least one second frame from the plurality of frames that was obtained earlier than the first frame.

[0020] A thirteenth aspect of the technique of the present disclosure is the image processing device according to any one of the first to twelfth aspects, in which the medical image is a moving image.

[0021] A fourteenth aspect of the technology of the present disclosure is an image processing device according to any one of the first to twelfth aspects, in which the processor outputs the size when it determines to perform output.

[0022] A fifteenth aspect of the technique of the present disclosure is the image processing device according to the fourteenth aspect, in which the output of the size is realized by displaying the size on the first screen.

[0023] A sixteenth aspect of the technology of the present disclosure is an image processing device according to any one of the first to fifteenth aspects, in which the processor outputs past results in which the size was measured when it determines not to perform output.

[0024] A seventeenth aspect of the technique of the present disclosure is the image processing device according to the sixteenth aspect, in which the output of the past result is realized by displaying the past result on the second screen.

[0025] An 18th aspect of the technology of the present disclosure is an image processing device according to the 17th aspect, in which, when the processor determines that output should be performed, the current results of the measured size are displayed on the second screen, and the past results and the current results are displayed on the second screen in a manner that allows them to be distinguished depending on whether the processor determines that output should not be performed or that output should be performed.

[0026] A 19th aspect of the technology of the present disclosure is an image processing device according to any one of the first to eighteenth aspects, in which, when the processor determines that output will not be performed, it outputs non-output specification information that can identify that output will not be performed.

[0027] A twentieth aspect of the technique of the present disclosure is the image processing device according to the nineteenth aspect, in which the output of the non-output specific information is realized by displaying the non-output specific information on the third screen.

[0028] A 21st aspect of the technology of the present disclosure is an image processing device according to any one of the first to 20th aspects, in which the medical image is an endoscopic image obtained by capturing an image using an endoscope.

[0029] A twenty-second aspect of the technique of the present disclosure is the image processing device according to any one of the first to twenty-first aspects, in which the observation target region is a lesion.

[0030] A 23rd aspect of the technology of the present disclosure is an endoscope comprising an image processing device according to any one of the first to 22nd aspects and a module that is inserted into a body including an observation target area and acquires a medical image by imaging the observation target area.

[0031] A 24th aspect of the technology of the present disclosure is an image processing method that includes recognizing the position of an observation area within a medical image based on the medical image in which the observation area appears, determining whether or not to output the size of the observation area based on the position, and outputting the size if it is determined that output should be performed.

[0032] A 25th aspect of the technology of the present disclosure is a program for causing a computer to execute processing including recognizing the position of an observation area within a medical image based on the medical image in which the observation area appears, determining whether or not to output the size of the observation area based on the position, and outputting the size if it is determined that output should be performed.

[0033] 1 is a conceptual diagram showing an example of a manner in which an endoscope is used. FIG. 1 is a conceptual diagram showing an example of the overall configuration of an endoscope. FIG. 2 is a block diagram showing an example of the hardware configuration of the electrical system of an endoscope. FIG. 3 is a block diagram showing an example of the main functions of a processor included in an endoscope, and an example of information stored in an NVM. FIG. 1 is a conceptual diagram showing an example of the processing content of a recognition unit and a control unit. FIG. 1 is a conceptual diagram showing an example of the processing content of a recognition unit and a determination unit. FIG. 2 is a conceptual diagram showing an example of the processing content of a determination unit when a lesion is included in a peripheral region. FIG. 3 is a conceptual diagram showing an example of the processing content of a measurement unit. FIG. 4 is a conceptual diagram showing an example of a manner in which an endoscopic image is displayed on a first screen, and a size is displayed on a second screen. FIG. 4 is a conceptual diagram showing an example of a manner in which an endoscopic image is displayed on a first screen, and non-output specific information is displayed on a second screen. FIG. 5 is a flowchart showing an example of the flow of medical support processing. FIG. 6 is a conceptual diagram showing a first modified example of the processing content of a recognition unit and a determination unit.

[0034] Hereinafter, exemplary embodiments of an image processing device, an endoscope, an image processing method, and a program according to the techniques of the present disclosure will be described with reference to the accompanying drawings.

[0035] First, the terms used in the following description will be explained.

[0036] CPU is an abbreviation for "Central Processing Unit." GPU is an abbreviation for "Graphics Processing Unit." RAM is an abbreviation for "Random Access Memory." NVM is an abbreviation for "Non-volatile memory." EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory." ASIC is an abbreviation for "Application Specific Integrated Circuit." PLD is an abbreviation for "Programmable Logic Device." FPGA is an abbreviation for "Field-Programmable Gate Array." SoC is an abbreviation for "System-on-a-chip." SSD is an abbreviation for "Solid State Drive." USB is an abbreviation for "Universal Serial Bus." HDD is an abbreviation for "Hard Disk Drive." EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". AI is an abbreviation for "Artificial Intelligence". BLI is an abbreviation for "Blue Light Imaging". LCI is an abbreviation for "Linked Color Imaging". I / F is an abbreviation for "Interface". IoU is an abbreviation for "Intersection over Union". FIFO is an abbreviation for "First In First Out".

[0037] As an example, as shown in Fig. 1 , an endoscopic system 10 includes an endoscope 12 and a display device 14. The endoscope 12 is used by a doctor 16 in an endoscopic examination. The endoscopic examination is assisted by staff such as a nurse 17. In this embodiment, the endoscope 12 is an example of an "endoscope" according to the technology of the present disclosure.

[0038] The endoscope 12 is communicatively connected to a communication device (not shown), and information obtained by the endoscope 12 is transmitted to the communication device. An example of the communication device is a server and / or a client terminal (e.g., a personal computer and / or a tablet terminal) that manages various information such as electronic medical records. The communication device receives the information transmitted from the endoscope 12 and executes processing using the received information (e.g., processing to store the information in an electronic medical record, etc.).

[0039] The endoscope 12 includes an endoscope body 18. The endoscope 12 is a device for performing medical examinations on a large intestine 22 contained in the body of a subject 20 (e.g., a patient) using the endoscope body 18. In this embodiment, the large intestine 22 is an object to be observed by a doctor 16.

[0040] The endoscope body 18 is inserted into the large intestine 22 of the subject 20. The endoscope 12 causes the endoscope body 18 inserted into the large intestine 22 of the subject 20 to capture images of the inside of the large intestine 22 inside the body of the subject 20, and also performs various medical procedures on the large intestine 22 as necessary.

[0041] The endoscope 12 acquires and outputs images showing the state inside the body by imaging the inside of the large intestine 22 of the subject 20. In this embodiment, the endoscope 12 is an endoscope having an optical imaging function that irradiates the inside of the large intestine 22 with light 26 and captures an image of the reflected light obtained by reflection on the intestinal wall 24 of the large intestine 22.

[0042] Although an endoscopic examination of the large intestine 22 is illustrated here, this is merely one example, and the technology disclosed herein can also be applied to endoscopic examination of hollow organs such as the esophagus, stomach, duodenum, or trachea.

[0043] The endoscope 12 is equipped with a control device 28, a light source device 30, and an image processing device 32. The control device 28, the light source device 30, and the image processing device 32 are installed on a wagon 34. The wagon 34 has a plurality of stands arranged vertically, and the image processing device 32, the control device 28, and the light source device 30 are installed from the lower stand to the upper stand. In addition, the display device 14 is installed on the top stand of the wagon 34.

[0044] The control device 28 controls the entire endoscope 12. The image processing device 32, under the control of the control device 28, performs various image processing on the images obtained by the endoscope body 18 capturing images of the intestinal wall 24.

[0045] The display device 14 displays various information including images. Examples of the display device 14 include a liquid crystal display and an EL display. Alternatively, a tablet terminal with a display may be used instead of the display device 14 or together with the display device 14.

[0046] A plurality of screens are displayed side by side on the display device 14. In the example shown in Fig. 1, a first screen 36 and a second screen 38 are shown as examples of the plurality of screens.

[0047] An endoscopic image 40 is displayed on the first screen 36. The endoscopic image 40 is a circular image. That is, the endoscopic image 40 is an image acquired by imaging the intestinal wall 24 inside the large intestine 22 of the subject 20 using the endoscope main body 18. In the example shown in FIG. 1 , an image of the intestinal wall 24 is shown as an example of the endoscopic image 40. The intestinal wall 24 shown in the endoscopic image 40 also includes a lesion 42. In the example shown in FIG. 1 , the lesion 42, which is the observation area that is being closely watched by the physician 16, is also shown in the endoscopic image 40. There are various types of lesions 42, and examples of the types of lesions 42 include neoplastic polyps and non-neoplastic polyps.

[0048] In this embodiment, the endoscopic image 40 is an example of a "medical image," "frame," and "endoscopic image" according to the technology of the present disclosure. Furthermore, in this embodiment, the lesion 42 is an example of an "observation target region" and a "lesion" according to the technology of the present disclosure. While the lesion 42 is illustrated here as an example, the technology of the present disclosure is not limited thereto. The observation target region may be an organ (e.g., the duodenal papilla), a marked region, a treated region (e.g., a region where traces remain after removal of a polyp, etc.), or the like.

[0049] A moving image is displayed on the first screen 36. The endoscopic image 40 displayed on the first screen 36 is one frame included in a moving image that is configured to include multiple frames in chronological order. In other words, the first screen 36 displays multiple frames of the endoscopic image 40 at a default frame rate (e.g., 30 frames / second or 60 frames / second).

[0050] An example of a moving image displayed on the first screen 36 is a moving image in a live view format. The live view format is merely one example, and a moving image that is temporarily stored in a memory or the like and then displayed, such as a moving image in a post-view format, may also be used. Furthermore, each frame included in a moving image for recording that is stored in a memory or the like may be reproduced and displayed on the first screen 36 as the endoscopic image 40.

[0051] The second screen 38 is a rectangular screen smaller than the first screen 36. In the example shown in FIG. 1 , the second screen 38 is superimposed on the lower right of the first screen 36 when viewed from the front. While a superimposed display is illustrated here, this is merely an example, and an embedded display may also be used. The display position of the second screen 38 may be anywhere within the screen of the display device 14, but it is preferable that the second screen 38 be displayed in a position that allows comparison with the endoscopic image 40. A position identification image 44 is displayed on the second screen 38. The position identification image 44 is an image corresponding to the endoscopic image 40 and is an image that a user (e.g., the doctor 16) refers to in order to identify the position of the lesion 42 within the endoscopic image 40.

[0052] The position identification image 44 has an outer frame 44A, a target mark 44B, and a lesion image 44C. The outer frame 44A is a frame in the shape of a circular frame obtained by reducing the circular outline of the endoscopic image 40, with upper and lower portions cut out by the upper and lower sides of the second screen 38.

[0053] The target mark 44B is a cross-shaped mark that intersects with the center of the display area of ​​the position identification image 44. The intersection of the target marks 44B corresponds to the center point of the endoscopic image 40.

[0054] The lesion image 44C is an image corresponding to the lesion 42 in the endoscopic image 40, and is displayed in a display mode according to the size, shape, and type of the lesion 42. An example of the lesion image 44C is the segmentation area itself indicating the lesion 42 recognized by the AI ​​segmentation method for each endoscopic image 40, or an image that is similar to the segmentation area.

[0055] 2, the endoscope body 18 includes an operating section 46 and an insertion section 48. The insertion section 48 is partially curved by operating the operating section 46. The insertion section 48 is inserted into the large intestine 22 (see FIG. 1) while curving in accordance with the shape of the large intestine 22 (see FIG. 1) in accordance with the operation of the operating section 46 by the physician 16 (see FIG. 1).

[0056] The distal end portion 50 of the insertion section 48 is provided with a camera 52, an illumination device 54, and a treatment tool opening 56. The camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end portion 50. Note that, although an example in which the camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end portion 50 is given here, this is merely one example, and the camera 52 and the illumination device 54 may be provided on the side surface of the distal end portion 50, so that the endoscope 12 is configured as a side-viewing endoscope.

[0057] The camera 52 is a device that captures an image of the inside of the subject 20 (for example, the inside of the large intestine 22) to obtain an endoscopic image 40 as a medical image. An example of the camera 52 is a CMOS camera. However, this is merely an example, and other types of cameras such as a CCD camera may also be used. The camera 52 is an example of a "module" according to the technology of the present disclosure.

[0058] The illumination device 54 has illumination windows 54A and 54B. The illumination device 54 emits light 26 (see FIG. 1 ) through the illumination windows 54A and 54B. Examples of the light 26 emitted from the illumination device 54 include visible light (e.g., white light) and invisible light (e.g., near-infrared light). The illumination device 54 also emits special light through the illumination windows 54A and 54B. Examples of the special light include light for BLI and / or light for LCI. The camera 52 captures images of the inside of the large intestine 22 by an optical method while the light 26 is being emitted from the illumination device 54 inside the large intestine 22.

[0059] The treatment tool opening 56 is an opening for allowing a treatment tool 58 to protrude from the distal end portion 50. The treatment tool opening 56 is also used as a suction port for sucking blood, internal waste, etc., and as a delivery port for delivering fluid.

[0060] A treatment tool insertion port 60 is formed in the operation section 46, and the treatment tool 58 is inserted into the insertion section 48 from the treatment tool insertion port 60. The treatment tool 58 passes through the insertion section 48 and protrudes to the outside from the treatment tool opening 56. In the example shown in Fig. 2, a puncture needle is shown as the treatment tool 58 protruding from the treatment tool opening 56. Here, a puncture needle is shown as the treatment tool 58, but this is merely one example, and the treatment tool 58 may also be a grasping forceps, a papillotomy knife, a snare, a catheter, a guidewire, a cannula, and / or a puncture needle with a guide sheath, etc.

[0061] The endoscope body 18 is connected to the control device 28 and the light source device 30 via a universal cord 62. The control device 28 is connected to an image processing device 32 and a reception device 64. The image processing device 32 is also connected to the display device 14. That is, the control device 28 is connected to the display device 14 via the image processing device 32.

[0062] Note that, because the image processing device 32 is exemplified here as an external device for expanding the functions performed by the control device 28, an example in which the control device 28 and the display device 14 are indirectly connected via the image processing device 32 is given, but this is merely one example. For example, the display device 14 may be directly connected to the control device 28. In this case, for example, the functions of the image processing device 32 may be incorporated in the control device 28, or the control device 28 may be equipped with a function for causing a server (not shown) to execute the same processing as that executed by the image processing device 32 (for example, the medical support processing described below) and receiving and using the processing results from the server.

[0063] The reception device 64 receives instructions from the doctor 16 and outputs the received instructions as an electrical signal to the control device 28. Examples of the reception device 64 include a keyboard, a mouse, a touch panel, a foot switch, a microphone, and / or a remote control device.

[0064] The control device 28 controls the light source device 30 , exchanges various signals with the camera 52 , and exchanges various signals with the image processing device 32 .

[0065] The light source device 30 emits light under the control of the control device 28 and supplies the light to the illumination device 54. The illumination device 54 has a built-in light guide, and the light supplied from the light source device 30 passes through the light guide and is irradiated from illumination windows 54A and 54B. The control device 28 causes the camera 52 to capture an image, acquires an endoscopic image 40 (see FIG. 1 ) from the camera 52, and outputs the image to a predetermined output destination (for example, the image processing device 32).

[0066] The image processing device 32 performs various types of image processing on the endoscopic image 40 input from the control device 28. The image processing device 32 outputs the endoscopic image 40 that has been subjected to various types of image processing to a predetermined output destination (for example, the display device 14).

[0067] Although the embodiment has been described above with reference to an example in which the endoscopic image 40 output from the control device 28 is output to the display device 14 via the image processing device 32, this is merely one example. For example, the control device 28 and the display device 14 may be connected, and the endoscopic image 40 that has been subjected to image processing by the image processing device 32 may be displayed on the display device 14 via the control device 28.

[0068] 3 , the control device 28 includes a computer 66, a bus 68, and an external I / F 70. The computer 66 includes a processor 72, a RAM 74, and an NVM 76. The processor 72, the RAM 74, the NVM 76, and the external I / F 70 are connected to the bus 68.

[0069] For example, the processor 72 has at least one CPU and at least one GPU, and controls the entire control device 28. The GPU operates under the control of the CPU and is responsible for executing various graphic processing operations and performing calculations using neural networks. The processor 72 may be one or more CPUs that have integrated GPU functionality, or one or more CPUs that do not have integrated GPU functionality. In the example shown in FIG. 3 , the computer 66 is equipped with one processor 72, but this is merely an example, and the computer 66 may be equipped with multiple processors 72.

[0070] The RAM 74 is a memory that temporarily stores information and is used as a work memory by the processor 72. The NVM 76 is a nonvolatile storage device that stores various programs, various parameters, and the like. An example of the NVM 76 is a flash memory (for example, an EEPROM and / or an SSD). Note that the flash memory is merely one example, and the NVM 76 may be another nonvolatile storage device such as an HDD, or may be a combination of two or more types of nonvolatile storage devices.

[0071] The external I / F 70 controls the exchange of various information between one or more devices (hereinafter also referred to as “first external devices”) existing outside the control device 28 and the processor 72. An example of the external I / F 70 is a USB interface.

[0072] The external I / F 70 is connected to the camera 52 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the camera 52 and the processor 72. The processor 72 controls the camera 52 via the external I / F 70. The processor 72 also acquires, via the external I / F 70, endoscopic images 40 (see FIG. 1 ) obtained by the camera 52 capturing an image of the inside of the large intestine 22 (see FIG. 1 ).

[0073] The light source device 30 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the light source device 30 and the processor 72. The light source device 30 supplies light to the illumination device 54 under the control of the processor 72. The illumination device 54 irradiates the light supplied from the light source device 30.

[0074] A reception device 64 is connected to the external I / F 70 as one of the first external devices, and the processor 72 acquires instructions accepted by the reception device 64 via the external I / F 70 and executes processing according to the acquired instructions.

[0075] The image processing device 32 includes a computer 78 and an external I / F 80. The computer 78 includes a processor 82, a RAM 84, and an NVM 86. The processor 82, the RAM 84, the NVM 86, and the external I / F 80 are connected to a bus 88. In the present embodiment, the image processing device 32 is an example of an "image processing device" according to the technology of the present disclosure, the computer 78 is an example of a "computer" according to the technology of the present disclosure, and the processor 82 is an example of a "processor" according to the technology of the present disclosure.

[0076] The hardware configuration of the computer 78 (i.e., the processor 82, RAM 84, and NVM 86) is basically the same as the hardware configuration of the computer 66, so a description of the hardware configuration of the computer 78 will be omitted here.

[0077] The external I / F 80 controls the exchange of various information between the processor 82 and one or more devices (hereinafter also referred to as "second external devices") that exist outside the image processing device 32. An example of the external I / F 80 is a USB interface.

[0078] The control device 28 is connected to the external I / F 80 as one of the second external devices. In the example shown in Fig. 3, the external I / F 70 of the control device 28 is connected to the external I / F 80. The external I / F 80 controls the exchange of various information between the processor 82 of the image processing device 32 and the processor 72 of the control device 28. For example, the processor 82 acquires the endoscopic image 40 (see Fig. 1) from the processor 72 of the control device 28 via the external I / Fs 70 and 80, and performs various image processing on the acquired endoscopic image 40.

[0079] The display device 14, which serves as one of the second external devices, is connected to the external I / F 80. The processor 82 controls the display device 14 via the external I / F 80, thereby causing the display device 14 to display various information (e.g., an endoscopic image 40 that has been subjected to various image processing).

[0080] During an endoscopic examination, the doctor 16 checks the endoscopic image 40 via the display device 14 to determine whether or not medical treatment is required for the lesion 42, and if necessary, performs medical treatment on the lesion 42. The size of the lesion 42 is an important factor in determining whether or not medical treatment is required.

[0081] In recent years, advances in machine learning have made it possible to use AI to detect and differentiate lesions 42 based on endoscopic images 40. By applying this technology, it is possible to measure the size of lesions 42 from endoscopic images 40.

[0082] However, even if the size of the lesion 42 is measured, the measured size may vary significantly depending on the imaging conditions of the camera 52. For example, if the camera 52 moves vigorously or if there is significant body movement, the endoscopic image 40 may become blurred, making it difficult to accurately measure the size of the lesion 42 using an AI-based image processing method. Furthermore, optical effects (e.g., aberrations) of the objective lens of the camera 52 may distort the peripheral portion of the endoscopic image 40. Therefore, if the lesion 42 is located at the peripheral portion of the endoscopic image 40, the lesion 42 may be erroneously measured. If the physician 16 determines whether medical treatment is necessary based on the erroneously measured size, there is a risk that medical treatment may be performed even when it is not actually necessary, or that medical treatment may not be performed even when it is actually necessary.

[0083] In view of the above circumstances, in this embodiment, as shown in FIG. 4 as an example, medical support processing is performed by a processor 82 of the image processing device 32.

[0084] A medical support program 90 is stored in the NVM 86. The medical support program 90 is an example of a "program" according to the technology of the present disclosure. The processor 82 performs medical support processing by reading the medical support program 90 from the NVM 86 and executing the read medical support program 90 on the RAM 84. The medical support processing is realized by the processor 82 operating as a recognition unit 82A, a determination unit 82B, a measurement unit 82C, and a control unit 82D in accordance with the medical support program 90 executed on the RAM 84.

[0085] The NVM 86 stores a recognition model 92 and a distance derivation model 94. The recognition model 92 and the distance derivation model 94 are examples of "AI" according to the technology of the present disclosure. As will be described in detail later, the recognition model 92 is used by the recognition unit 82A, and the distance derivation model 94 is used by the measurement unit 82C.

[0086] As an example, as shown in FIG. 5, the recognition unit 82A and the control unit 82D acquire the endoscopic image 40 generated by the camera 52 capturing images at an imaging frame rate (e.g., several tens of frames per second) from the camera 52 on a frame-by-frame basis.

[0087] The control unit 82D displays the endoscopic image 40 as a live view image on the first screen 36. That is, every time the control unit 82D acquires an endoscopic image 40 frame by frame from the camera 52, it displays the acquired endoscopic image 40 in sequence on the first screen 36 at a display frame rate (e.g., several tens of frames per second).

[0088] The recognition unit 82A recognizes the position of the lesion 42 in the endoscopic image 40 (i.e., the position of the lesion 42 shown in the endoscopic image 40) by performing a recognition process 96 on the endoscopic image 40 acquired from the camera 52. The recognition process 96 is performed by the recognition unit 82A on the acquired endoscopic image 40 every time the endoscopic image 40 is acquired.

[0089] The recognition process 96 is an image recognition process using an AI segmentation method. Here, the recognition process 96 uses a recognition model 92.

[0090] The recognition model 92 is a trained model for object detection using an AI segmentation method, and is optimized by performing machine learning on a neural network using first training data. The first training data is a data set including a plurality of data (i.e., data for a plurality of frames) in which first example data and first ground truth data are associated with each other.

[0091] The first example data is an image corresponding to the endoscopic image 40. The first supervised data is supervised data (i.e., annotation) for the first example data. Here, an annotation that identifies a lesion shown in the image used as the first example data is used as an example of the first supervised data.

[0092] The recognition unit 82A acquires an endoscopic image 40 from the camera 52 and inputs the acquired endoscopic image 40 to the recognition model 92. As a result, each time an endoscopic image 40 is input, the recognition model 92 identifies the position of a segmentation region 100 identified by the segmentation method as the position of a lesion 42 shown in the input endoscopic image 40, and outputs position identification information 98 that can identify the position of the segmentation region 100. An example of the position identification information 98 is coordinates that identify the segmentation region 100 within the endoscopic image 40.

[0093] As an example, as shown in FIG. 6 , the determination unit 82B acquires position identification information 98 from the recognition unit 82A each time the recognition unit 82A performs recognition processing 96 (see FIG. 5 ) for each endoscopic image 40. The determination unit 82B then determines whether or not to output the size of the lesion 42 based on the position identification information 98. In this embodiment, the size of the lesion 42 is output when the size of the lesion 42 is measured, and the size of the lesion 42 is not output when the size of the lesion 42 is not measured. Therefore, in the example shown in FIG. 6 , the determination unit 82B determines whether or not to output the size of the lesion 42 by determining whether or not to measure the size of the lesion 42. In other words, measuring the size of the lesion 42 means outputting the size of the lesion 42, and not measuring the size of the lesion 42 means not outputting the size of the lesion 42.

[0094] The determination unit 92B determines whether the location of the lesion 42 is in the peripheral region 40A of the endoscopic image 40 or in a region other than the peripheral region 40A. The peripheral region 40A refers to a circular region whose outer periphery is the outer edge of the endoscopic image 40 and whose inner periphery is a circle offset from the outer edge of the endoscopic image 40 toward the center of the endoscopic image 40 by a length α. The length α may be a fixed value determined in advance as a length defining the circular region in which accurate measurement of the size of the lesion 42 becomes impossible due to the influence of aberrations of the objective lens of the camera 52. Alternatively, the length α may be a variable value that is changed in accordance with instructions and / or imaging conditions received by the reception device 64 from a user or the like. Here, the region other than the peripheral region 40A is an example of a "first region" according to the technology of the present disclosure, and the peripheral region 40A is an example of a "second region" and "periphery" according to the technology of the present disclosure.

[0095] The determination unit 82B determines whether the entire segmentation region 100 is included in the peripheral portion 40A based on the position identification information 98, thereby determining whether the position of the lesion 42 is in the peripheral portion 40A of the endoscopic image 40.

[0096] Here, if the peripheral portion 40A does not include the entire segmentation region 100, it is determined that the location of the lesion 42 is not in the peripheral portion 40A of the endoscopic image 40, and if the peripheral portion 40A includes the entire segmentation region 100, it is determined that the location of the lesion 42 is in the peripheral portion 40A of the endoscopic image 40.

[0097] Note that, here, the determination criterion is whether or not the entire segmentation region 100 is included in the peripheral portion 40A, but this is merely one example, and the determination criterion may also be whether or not a specified proportion (e.g., 80%) of the segmentation region 100 is included in the peripheral portion 40A. Furthermore, the proportion may be a fixed value, or may be a variable value that is changed according to instructions and / or imaging conditions accepted by the user or the like via the acceptance device 64.

[0098] The determination unit 82B also calculates the amount of change in the position of the lesion 42 between adjacent endoscopic images 40 in time series (hereinafter simply referred to as "lesion position change amount"). The determination unit 82B then determines whether the amount of change in the lesion position is equal to or greater than a threshold value. The threshold value may be a fixed value or a variable value that is changed according to instructions and / or imaging conditions received by the user or the like via the reception device 64.

[0099] The determination unit 82B calculates the amount of change in the segmentation region 100 (hereinafter also referred to as the "segmentation region change amount") as the amount of change in the lesion position. Then, it is determined whether the segmentation region change amount is equal to or greater than a threshold. The segmentation region change amount is defined based on the degree of overlap between one segmentation region 100 and another segmentation region 100 obtained from endoscopic images 40 adjacent in time series. For example, the segmentation region change amount may be defined based on IoU, or may simply be defined based on the number of pixels in the area where one segmentation region 100 and another segmentation region 100 overlap.

[0100] Regardless of whether the position of the lesion 42 is in the peripheral region 40A or not, if the amount of change in the segmentation region is equal to or greater than the threshold, the determination unit 82B determines not to measure the size of the lesion 42 (in other words, not to output the size of the lesion 42).Furthermore, provided that the position of the lesion 42 is not in the peripheral region 40A, if the amount of change in the segmentation region is less than the threshold for two consecutive frames, the determination unit 82B determines to measure the size of the lesion 42 (in other words, to output the size of the lesion 42).

[0101] Here, an example is given in which it is determined that the size of the lesion 42 will be measured if the amount of change in the segmentation area is less than the threshold value for two consecutive frames. However, this is merely one example, and it may be determined that the size of the lesion 42 will be measured if the amount of change in the segmentation area is less than the threshold value for three or more consecutive frames, or it may be determined that the size of the lesion 42 will be measured if the amount of change in the segmentation area is less than the threshold value for a single frame.

[0102] As an example, as shown in Figure 7, if the position of the lesion 42 is in the peripheral region 40A, the judgment unit 82B judges not to measure the size of the lesion 42 (in other words, not to output the size of the lesion 42) regardless of whether the amount of change in lesion position is greater than or equal to the threshold value, and if the position of the lesion 42 is not in the peripheral region 40A, the judgment unit 82B judges to measure the size of the lesion 42 (in other words, to output the size of the lesion 42) on the condition that the amount of change in lesion position is less than the threshold value.

[0103] In the following, for the sake of convenience, the result of the determination made by the determining unit 82B as to whether or not to measure the size of the lesion 42 will also be referred to as the "determination result."

[0104] 8, when the determination unit 82B determines that the size of the lesion 42 is to be measured, the measurement unit 82C measures the size 112 of the lesion 42 based on the endoscopic image 40. When the determination unit 82B determines that the size of the lesion 42 is not to be measured, the measurement unit 82C does not measure the size 112.

[0105] The measurement unit 82C acquires the endoscopic image 40 used in the determination by the determination unit 82B from the recognition unit 82A, and derives distance information 102 based on the acquired endoscopic image 40. The distance information 102 is information indicating the distance from the camera 52 to the intestinal wall 24 (see FIG. 1 ), including the lesion 42. The distance information 102 is derived for each of all pixels constituting the endoscopic image 40. Note that the distance information 102 may also be derived for each block of the endoscopic image 40 that is larger than a pixel (for example, a pixel group composed of several to several hundred pixels).

[0106] The distance information 102 is derived by an AI method. In this embodiment, a distance derivation model 94 is used to derive the distance information 102.

[0107] The distance derivation model 94 is optimized by performing machine learning on the neural network using second training data. The second training data is a data set including a plurality of data (i.e., a plurality of frames of data) in which second example data and second answer data are associated with each other.

[0108] The second example data is an image corresponding to the endoscopic image 40. The second supervised data is supervised data (i.e., annotations) for the second example data. Here, an annotation that specifies the distance corresponding to each pixel in the image used as the second example data is used as an example of the second supervised data.

[0109] The measurement unit 82C acquires the endoscopic image 40 used in the determination by the determination unit 82B from the recognition unit 82A and inputs the acquired endoscopic image 40 to the distance derivation model 94. As a result, the distance derivation model 94 outputs distance information 102 for each pixel of the input endoscopic image 40. That is, in the measurement unit 82C, information indicating the distance from the position of the camera 52 (e.g., the position of an image sensor or objective lens mounted on the camera 52) to the intestinal wall 24 shown in the endoscopic image 40 is output as distance information 102 from the distance derivation model 94 for each pixel of the endoscopic image 40. The position of the camera 52 is an example of an "observation position" according to the technology of the present disclosure.

[0110] The measurement unit 82C generates a distance image 104 based on the distance information 102 output from the distance derivation model 94. The distance image 104 is an image in which the distance information 102 is distributed for each pixel included in the endoscopic image 40.

[0111] The measurement unit 82C refers to the position identification information 98 obtained based on the endoscopic image 40 input to the distance derivation model 94, and extracts, from the distance image 104, distance information 102 corresponding to the position identified from the position identification information 98. The distance information 102 extracted from the distance image 104 may be distance information 102 corresponding to a specific position (e.g., the center of gravity) of the lesion 42, or a statistical value (e.g., median, average, or mode) of the distance information 102 for multiple pixels (e.g., all pixels) included in the lesion 42.

[0112] The measurement unit 82C extracts a pixel count 106 from the endoscopic image 40. The pixel count 106 is the number of pixels on a line segment 108 within an image region (i.e., an image region showing the lesion 42) at a position identified from the position identification information 98 within the entire image region of the endoscopic image 40 input to the distance derivation model 94. An example of the line segment 108 is the longest line segment parallel to the long side of the circumscribing rectangular frame 110 in the image region showing the lesion 42. Note that the line segment 108 is merely an example, and instead of the line segment 108, the longest line segment parallel to the short side of the circumscribing rectangular frame 110 in the image region showing the lesion 42 may be used. In this embodiment, the pixel count 106 is an example of the "pixel count" according to the technology of the present disclosure. Furthermore, in this embodiment, the line segment 108 is an example of the "range to be measured within the observation region" according to the technology of the present disclosure.

[0113] The measurement unit 82C calculates a size 112 of the lesion 42 in real space based on the distance information 102 extracted from the distance image 104 and the number of pixels 106 extracted from the endoscopic image 40. The size 112 refers to, for example, the length of the lesion 42 in real space.

[0114] A calculation formula 114 is used to calculate the size 112. The measurement unit 82C inputs the distance information 102 extracted from the distance image 104 and the number of pixels 106 extracted from the endoscopic image 40 into the calculation formula 114. The calculation formula 114 is an calculation formula in which the distance information 102 and the number of pixels 106 are independent variables and the size 112 is a dependent variable. The calculation formula 114 outputs the size 112 corresponding to the input distance information 102 and number of pixels 106.

[0115] Note that, although the length of the lesion 42 in real space is exemplified as the size 112 here, the technology of the present disclosure is not limited to this, and the size 112 may be the surface area or volume of the lesion 42 in real space. In this case, for example, an arithmetic expression 114 is used in which the number of pixels in the entire image region representing the lesion 42 and the distance information 102 are independent variables, and the surface area or volume of the lesion 42 in real space is a dependent variable.

[0116] 9 and 10 , the control unit 82D changes the display content displayed on the second screen 38 depending on the determination result. As shown in FIG. 9 , when the determination unit 82B determines that the size 112 should be measured, the control unit 82D acquires the endoscopic image 40 used in the determination by the determination unit 82B from the camera 52, and displays the endoscopic image 40 acquired from the camera 52 on the first screen 36.

[0117] The control unit 82D acquires from the measurement unit 82C the size 112 measured by the measurement unit 82C based on the endoscopic image 40 displayed on the first screen 36. The control unit 82D also acquires from the recognition unit 82A the segmentation region 100 and the position identification information 98 corresponding to the endoscopic image 40 displayed on the first screen 36.

[0118] The control unit 82D displays the segmentation region 100 acquired from the recognition unit 82A as the lesion image 44C (see FIG. 1 ) on the second screen 38. At this time, the segmentation region 100 is displayed on the second screen 38 at a position identified from the position identification information 98 acquired by the control unit 82D from the recognition unit 82A. The control unit 82D also displays the size 112 acquired from the measurement unit 82C on the second screen 38. The control unit 82D also displays dimension lines 115 on the second screen 38 so that it is possible to identify which part of the segmentation region 100 the size 112 corresponds to. The dimension lines 115 are created and displayed by the control unit 82D, for example, based on the position identification information 98 acquired from the recognition unit 82A. The dimension lines 115 may be created, for example, in a manner similar to that used for creating the line segment 108 (i.e., in a manner similar to that used for creating the circumscribed rectangular frame 110).

[0119] On the other hand, when the determination unit 82B determines that the size 112 will not be measured, as shown in FIG. 10 as an example, the control unit 82D displays the endoscopic image 40 on the first screen 36 and the segmentation region 100 on the second screen 38 in the same manner as the example shown in FIG. 9 . The control unit 82D also does not display the size 112 on the second screen 38, and instead displays no-output specification information 116 on the second screen 38. The no-output specification information 116 is information that can specify that the size 112 will not be output (in other words, that measurement of the size 112 was not performed) (here, as an example, information that can specify that the determination unit 82B has determined that measurement of the size 112 will not be performed). In the example shown in FIG. 10 , the text "Measurement Unavailable" is displayed on the second screen 38. The text "Cannot measure" is merely an example, and it may be text such as "Cannot output," or any other information (e.g., a mark or symbol) that can identify that size 112 will not be output.

[0120] In this embodiment, the non-output specific information 116 is an example of the "non-output specific information" according to the technology of the present disclosure. Also, in this embodiment, the second screen 38 is an example of the "first screen," "second screen," and "third screen" according to the technology of the present disclosure.

[0121] Next, the operation of the portion of the endoscope system 10 related to the technique of the present disclosure will be described with reference to FIG.

[0122] Fig. 11 shows an example of the flow of medical support processing performed by the processor 82. The flow of medical support processing shown in Fig. 11 is an example of an "image processing method" according to the technique of the present disclosure.

[0123] 11 , first, in step ST10, the recognition unit 82A determines whether or not one frame of image data has been captured by the camera 52 inside the large intestine 22. If one frame of image data has not been captured by the camera 52 inside the large intestine 22 in step ST10, the determination is negative, and the determination in step ST10 is made again. If one frame of image data has been captured by the camera 52 inside the large intestine 22 in step ST10, the determination is positive, and the medical support process proceeds to step ST12.

[0124] In step ST12, the recognition unit 82A and the control unit 82D acquire one frame of the endoscopic image 40 obtained by capturing an image of the large intestine 22 with the camera 52 (see FIG. 5). For ease of explanation, the following description will be given on the assumption that the endoscopic image 40 shows a lesion 42. After the processing of step ST12 is executed, the medical support processing proceeds to step ST14.

[0125] In step ST14, the control unit 82D displays the endoscopic image 40 acquired in step ST12 on the first screen 36 (see FIGS. 5, 9, and 10). After the processing of step ST14 is executed, the medical support processing proceeds to step ST16.

[0126] In step ST16, the recognition unit 82A performs a recognition process 96 using the endoscopic image 40 acquired in step ST12 to recognize the position of the lesion 42 in the endoscopic image 40 and acquires position identification information 98 (see FIG. 5 ). After the process of step ST16 is executed, the medical support process proceeds to step ST18.

[0127] In step ST18, the determination unit 82B determines whether or not to measure the size 112 of the lesion 42 shown in the endoscopic image 40 acquired in step ST12, based on the position identification information 98 acquired by the recognition unit 82A in step ST16 (see FIGS. 6 and 7 ). If it is determined in step ST18 that the size 112 of the lesion 42 shown in the endoscopic image 40 should be measured, the determination is affirmative, and the medical support processing proceeds to step ST20. If it is determined in step ST18 that the size 112 of the lesion 42 shown in the endoscopic image 40 should not be measured, the determination is negative, and the medical support processing proceeds to step ST24.

[0128] In step ST20, the measurement unit 82C measures the size 112 of the lesion 42 shown in the endoscopic image 40 acquired in step ST12 (see FIG. 8). After the processing of step ST20 is executed, the medical support processing proceeds to step ST22.

[0129] In step ST22, the control unit 82D displays the size 112 measured by the measurement unit 82C in step ST20 on the second screen 38 (see FIG. 9). After the process of step ST22 is executed, the medical support process proceeds to step ST26.

[0130] In step ST24, the control unit 82D displays the non-output specifying information 116 on the second screen 38 (see FIG. 10). After the process of step ST24 is executed, the medical support process proceeds to step ST26.

[0131] In step ST26, the control unit 82D determines whether a condition for terminating the medical support process is satisfied. One example of the condition for terminating the medical support process is that an instruction to terminate the medical support process has been given to the endoscope system 10 (for example, that an instruction to terminate the medical support process has been accepted by the acceptance device 64).

[0132] In step ST26, if the condition for terminating the medical support process is not satisfied, the determination is negative and the medical support process proceeds to step ST 10. In step ST26, if the condition for terminating the medical support process is satisfied, the determination is positive and the medical support process ends.

[0133] As described above, in the endoscopic system 10 according to the present embodiment, the recognition unit 82A recognizes the position of the lesion 42 in the endoscopic image 40 based on the endoscopic image 40 in which the lesion 42 is captured (see FIG. 5 ). When measuring the size 112 of the lesion 42 whose position in the endoscopic image 40 has been recognized by the recognition unit 82A, if the camera 52 moves violently or if the patient's body moves violently, the endoscopic image 40 becomes blurred, making it difficult to accurately measure the size 112 of the lesion 42 using the AI ​​method with the endoscopic image 40. Furthermore, if the peripheral portion 40A of the endoscopic image 40 is distorted due to the optical effect of the objective lens of the camera 52, it becomes difficult to accurately measure the size 112 of the lesion 42 using the AI ​​method with the endoscopic image 40. In other words, if the size 112 is measured using a measurement method that does not take into account the blurring of the endoscopic image 40 and / or distortion of the peripheral portion 40A (for example, a measurement method that assumes the use of a distance derivation model 94 that was created without taking into account the blurring of the endoscopic image 40 and / or distortion of the peripheral portion 40A), there is a risk that an inaccurate size 112 will be measured.

[0134] Therefore, in the endoscopic system 10 according to the present embodiment, the determination unit 82B determines whether or not to measure the size 112 of the lesion 42 based on the position of the lesion 42 in the endoscopic image 40 used by the recognition unit 82A (see FIGS. 6 and 7 ). If the determination unit 82B determines that the size 112 should be measured, the measurement unit 82C measures the size 112 of the lesion 42 shown in the endoscopic image 40 based on the endoscopic image 40 used by the recognition unit 82A and the determination unit 82B (see FIG. 8 ).

[0135] Therefore, the doctor 16 can accurately grasp the size 112 of the lesion 42 shown in the endoscopic image 40. As a result, it is possible to prevent the doctor 16 from performing a medical treatment when it is not actually necessary, or from not performing a medical treatment when it is actually necessary.

[0136] Furthermore, in the endoscopic system 10 according to this embodiment, the recognition unit 82A recognizes the position of the lesion 42 within each endoscopic image 40 (see FIG. 5 ). Then, the amount of change in the position of the lesion 42 between adjacent endoscopic images 40 in time series (e.g., the amount of change in the segmentation region defined by IoU) is used to determine whether to measure the size 112 of the lesion 42 (see FIG. 6 ). Therefore, even if the sharpness of the endoscopic images 40 changes or body movement occurs between adjacent endoscopic images 40 in time series, the physician 16 can accurately determine the size 112 of the lesion 42 captured in adjacent endoscopic images 40 in time series.

[0137] In the endoscopic system 10 according to the present embodiment, the recognition unit 82A recognizes the position of the lesion 42 within each endoscopic image 40 using an AI segmentation method (see FIG. 5 ). Then, the determination unit 82B uses the amount of change in the segmentation region to determine whether or not to measure the size 112 of the lesion 42 (see FIG. 6 ).

[0138] Furthermore, in the endoscopic system 10 according to this embodiment, whether or not to measure the size 112 of the lesion 42 is determined based on whether the position of the lesion 42 shown in the endoscopic image 40 is in the peripheral portion 40A of the endoscopic image 40 (see FIGS. 6 and 7 ). Therefore, it is possible to prevent the size 112 of the lesion 42 from being inaccurately measured due to optical effects such as distortion affecting the peripheral portion 40A of the endoscopic image 40.

[0139] Furthermore, in the endoscopic system 10 according to this embodiment, when the size 112 of the lesion 42 is measured by the measuring unit 82C, the measured size 112 is displayed on the second screen 38 (see FIG. 9 ). Therefore, the doctor 16 can visually recognize the size 112 of the lesion 42 shown in the endoscopic image 40.

[0140] Furthermore, in the endoscope system 10 according to this embodiment, when it is determined that the size 112 of the lesion 42 will not be measured, no-output specification information 116 is displayed on the second screen 38 (see FIG. 10 ). The no-output specification information 116 is information that can specify that the size 112 will not be measured. Therefore, the doctor 16 can visually understand that the size 112 of the lesion 42 will not be measured.

[0141] In the above embodiment, an example in which the size 112 is displayed when the size 112 is measured has been described, but the technology of the present disclosure is not limited to this. For example, if the determination unit 82B determines not to output the size 112, the size 112 may be measured but not displayed, and if the determination unit 82B determines to output the size 112, the size 112 may be measured and displayed in the same manner as in the above embodiment.

[0142] In the above embodiment, the determination unit 82B makes a determination each time the recognition unit 82A obtains the position identification information 98. However, the technology of the present disclosure is not limited to this. For example, as shown in FIG. 12 , the determination unit 82B may determine whether to measure the size 112 of the lesion 42 based on the position of the lesion 42 in an endoscopic image 40 (hereinafter referred to as a “first frame FL1”) selected from among a plurality of endoscopic images 40 in time series in accordance with a given instruction (in the example shown in FIG. 12 , an instruction 118 received by the reception device 64) and the position of the lesion 42 in at least one endoscopic image 40 (hereinafter referred to as a “second frame FL2”) obtained before the first frame FL1 from among the plurality of endoscopic images 40 in time series. Here, the first frame FL1 is an example of a “first frame” according to the technology of the present disclosure, and the second frame FL2 is an example of a “second frame” according to the technology of the present disclosure.

[0143] 12 shows an example in which a plurality of pieces of position identification information 98 corresponding to a plurality of second frames FL2 obtained before a first frame FL1 are stored in a storage area 120. The storage area 120 is, for example, an area provided in the RAM 74. The plurality of pieces of position identification information 98 corresponding to a specific number of second frames FL2 (e.g., several frames to several hundred frames) are stored in the storage area 120 in a FIFO manner. In the example shown in FIG. 12, the determination unit 82B determines whether to measure the size 112 of the lesion 42 based on a representative piece of position identification information 98 selected from the plurality of pieces of position identification information 98 stored in the storage area 120 and the position identification information 98 corresponding to the first frame FL1.

[0144] A first example of the position specifying information 98 selected as the representative is, for example, the position specifying information 98 corresponding to the second frame FL2 adjacent to the first frame FL1 in the time series. A second example of the position specifying information 98 selected as the representative is, for example, a statistical value (e.g., an average, a median, or a mode) obtained from the plurality of position specifying information 98 stored in the storage area 120. A third example of the position specifying information 98 selected as the representative is, for example, the position specifying information 98 randomly selected from the plurality of position specifying information 98 stored in the storage area 120. A fourth example of the position specifying information 98 selected as the representative is, for example, the position specifying information 98 located in the center of the plurality of position specifying information 98 stored in the storage area 120 in the time series. A fifth example of the position specifying information 98 selected as the representative is, for example, the position specifying information 98 selected from the plurality of position specifying information 98 stored in the storage area 120 in accordance with an instruction received by the reception device 64.

[0145] 12 , whether or not to measure the size 112 of the lesion 42 is determined based on the position of the lesion 42 in a first frame FL1 selected in accordance with an instruction 118 received by the reception device 64 from among the plurality of endoscopic images 40 in time series, and the position of the lesion 42 in at least one second frame FL2 obtained earlier than the first frame FL1 from among the plurality of endoscopic images 40 in time series. Therefore, the determination of whether or not to measure the size 112 of the lesion 42 captured in the plurality of endoscopic images 40 in time series can be performed at a timing intended by the physician 16.

[0146] In the above embodiment, an example was given in which non-output specification information 116 is displayed on second screen 38 when determination unit 82B determines that measurement of size 112 of lesion 42 will not be performed, but the technology of the present disclosure is not limited to this. For example, when determination unit 82B determines that measurement of size 112 of lesion 42 will not be performed, previous results of measurements by measurement unit 82C (i.e., size 112 previously measured by measurement unit 82C) may be displayed on second screen 38.

[0147] In this case, for example, as shown in FIG. 13, in the medical support process, the process of step ST24A is executed instead of the process of step ST24 shown in FIG.

[0148] In step ST24A, the control unit 82D displays past measurement results by the measurement unit 82C (i.e., sizes 112 previously measured by the measurement unit 82C) on the second screen 38. A first example of the sizes 112 previously measured by the measurement unit 82C is the size 112 previously measured by the measurement unit 82C. A second example of the sizes 112 previously measured by the measurement unit 82C is a statistical value of the sizes 112 previously measured by the measurement unit 82C (e.g., the median, mean, mode, maximum, or minimum value of the sizes 112 of the lesions 42 captured in the past few to several hundred frames). A third example of the sizes 112 previously measured by the measurement unit 82C is the size 112 measured in the previous endoscopic examination (e.g., the size 112 of the lesion 42 at the same position as the lesion 42 captured in the endoscopic image 40 currently displayed on the first screen 36).

[0149] In the example shown in Fig. 14 , a previously measured size 112 is displayed on the second screen 38. In the example shown in Fig. 14 , the size 112 displayed on the second screen 38 is displayed in a manner that is distinguishable from the current measurement result by the measurement unit 82C (e.g., the size 112 displayed on the second screen 38 shown in Fig. 9 ). For example, the size 112 is displayed in a thick line on the second screen 38 shown in Fig. 9 , whereas the size 112 is displayed in a thin line on the second screen 38 shown in Fig. 14 . The thin line display is merely an example, and the size 112 may be displayed semi-transparently, or in a different color, font, or brightness from the current measurement result by the measurement unit 82C. The size 112 displayed on the second screen 38 shown in Fig. 14 may be displayed in any manner that is distinguishable from the size 112 displayed on the second screen 38 shown in Fig. 9 .

[0150] 13 and 14, if it is determined that the size 112 will not be measured, the previous measurement results by the measurement unit 82C are displayed on the second screen 38. Therefore, if it is determined that the size 112 will not be measured, the doctor 16 can visually recognize the size 112 that was measured in the past.

[0151] 14, past results of measurement by the measuring unit 82C (for example, size 112 displayed on the second screen 38 shown in FIG. 14) are displayed on the second screen 38 in a manner that allows them to be distinguished from current results of measurement by the measuring unit 82C (for example, size 112 displayed on the second screen 38 shown in FIG. 9). This allows the doctor 16 to visually distinguish between past results of measurement by the measuring unit 82C and current results of measurement by the measuring unit 82C.

[0152] In the above embodiment, an example was described in which the amount of change in lesion position is calculated based on the degree of overlap of the segmentation regions 100 between adjacent endoscopic images 40 in time series. However, the technology of the present disclosure is not limited to this. For example, the amount of change in lesion position may be calculated based on the distance between the positions of the lesions 42 between adjacent endoscopic images 40 in time series. In this case, for example, the distance between the centers of the segmentation regions 100 between adjacent endoscopic images 40 in time series (i.e., the amount of deviation of the center positions) may be calculated as the amount of change in lesion position. In this case, the same effects as those of the above embodiment can be expected.

[0153] In the above embodiment, an example was described in which the position of the lesion 42 was recognized for each endoscopic image 40 using an AI segmentation method, but the technology of the present disclosure is not limited to this. For example, the position of the lesion 42 may be recognized for each endoscopic image 40 using an AI bounding box method. In this case, as shown in FIG. 15 as an example, the amount of change in the bounding box 122 is calculated by the determination unit 82B, and a determination as to whether or not to measure the size 112 of the lesion 42 is made based on the amount of change in the bounding box 122 in the same manner as in the above embodiment. In this case, the same effects as in the above embodiment can be expected.

[0154] In the above embodiment, an AI-based image recognition process was exemplified as the recognition process 96, but the technology disclosed herein is not limited to this, and the position of the lesion 42 shown in the endoscopic image 40 may be recognized by the recognition unit 82A by performing a non-AI-based image recognition process (e.g., template matching, etc.).

[0155] In the above embodiment, an example was described in which a determination as to whether to measure the size 112 is made using the amount of change in the position of the lesion 42 between adjacent endoscopic images 40 in a time series. However, the technology of the present disclosure is not limited to this. A determination as to whether to measure the size 112 may be made using the amount of change in the position of the lesion 42 between three or more frames of endoscopic images 40 in a time series. In this case, the amount of change in the position of the lesion 42 between three or more frames of endoscopic images 40 in a time series may be a statistical value such as the average, median, mode, or maximum value of the amount of change between the three or more frames of endoscopic images 40 in a time series. Alternatively, a determination as to whether to measure the size 112 may be made using the amount of change in the position of the lesion 42 between multiple frames in a time series spaced apart by one or more frames.

[0156] In the above embodiment, an example in which the size 112 is displayed on the second screen 38 has been described, but the technology of the present disclosure is not limited to this. For example, the size 112 may be displayed on the first screen 36. In this case, for example, the size 112 may be displayed near the lesion 42 in the endoscopic image 40, or may be displayed outside the endoscopic image 40. Furthermore, the size 112 may be displayed on a display device different from the display device 14.

[0157] In the above embodiment, the display device 14 is exemplified as an output destination of the size 112, but the technology of the present disclosure is not limited to this, and the output destination of the size 112 may be other than the display device 14. As an example, as shown in Fig. 16, output destinations of the size 112 include an audio playback device 124, a printer 126, and / or an electronic medical record management device 128.

[0158] The size 112 may be output as sound by an audio playback device 124. The size 112 may also be printed as text on a medium (e.g., paper) by a printer 126. The size 112 may also be stored in an electronic medical record 130 managed by an electronic medical record management device 128 together with the display content of the first screen 36 and / or the second screen 38.

[0159] In the above embodiment, an example has been described in which the size 112 is displayed when the determination unit 82B determines that the size 112 should be measured (in other words, when the determination unit 82B determines that the size 112 should be output), and the size 112 is not displayed when the determination unit 82B determines that the size 112 should not be measured (in other words, when the determination unit 82B determines that the size 112 should not be output), but the technology of the present disclosure is not limited to this. For example, even when the determination unit 82B determines that the size 112 should not be measured, the size 112 may be measured and the measured size 112 may be displayed.

[0160] In this case, the concept of not displaying the size 112 also includes the concept of lowering the display level of the size 112. For example, the concept of not displaying the size 112 also includes the concept of displaying the size 112 in a display manner that is not visually perceived by a user or the like (e.g., the doctor 16). In this case, the display manner may include, for example, reducing the font size of the size 112, thinning the size 112, dotting the size 112, blinking the size 112, displaying the size 112 for an imperceptible display time, or making the size 112 transparent.

[0161] The concept of not displaying the size 112 also includes the concept of displaying the size 112 in a display mode that is visually perceived by a user or the like (e.g., the doctor 16), but in a second display mode that is different from the first display mode shown in FIG. 9 . The first display mode refers to, for example, a display mode (e.g., a display mode defined by a display position, a font type, a font size, a font color, and / or brightness) that can identify that the determination unit 82B has determined that the size 112 will be measured (in other words, that the determination unit 82B has determined that the size 112 will be output). The second display mode refers to, for example, a display mode (e.g., a display mode defined by a display position, a font type, a font size, a font color, and / or brightness) that can identify that the determination unit 82B has determined that the size 112 will not be measured (in other words, that the determination unit 82B has determined that the size 112 will not be output). The first display mode and the second display mode are different from each other as long as the determination result can be identified by a user, etc. The same can be said for the various outputs such as the audio output, printing, and saving described above.

[0162] In the above embodiment, an example of the form in which the non-output specific information 116 is displayed on the second screen 38 has been described, but the form in which the non-output specific information 116 is output is not limited to this. For example, the non-output specific information 116 may be output as audio by the audio playback device 124, may be recorded on a medium (e.g., paper) by the printer 126, or may be stored in a memory and / or the electronic medical record 130, etc.

[0163] 13 and 14 show an example in which past measurement results by the measurement unit 82C are displayed on the second screen 38, but the technology of the present disclosure is not limited to this. For example, past measurement results by the measurement unit 82C may be output as audio by the audio playback device 124, or may be recorded on a medium (e.g., paper) by the printer 126.

[0164] In the above embodiment, an example was described in which arithmetic formula 114 was used to calculate size 112, but the technology of the present disclosure is not limited to this, and size 112 may be measured by performing AI processing on endoscopic image 40. In this case, for example, a trained model may be used that outputs size 112 of lesion 42 when endoscopic image 40 including lesion 42 is input. When creating the trained model, deep learning may be performed on a neural network using training data in which annotations indicating the size of the lesion are added as ground truth data for lesions shown in images used as example data.

[0165] In the above embodiment, an example of deriving the distance information 102 using the distance derivation model 94 has been described, but the technology of the present disclosure is not limited to this. For example, other methods for deriving the distance information 102 using an AI method include a method that combines segmentation and depth estimation (for example, regression learning that provides distance information 102 for the entire image (for example, all pixels that make up the image), or unsupervised learning that learns the distance for the entire image in an unsupervised manner).

[0166] In the above embodiment, an example was given in which the distance from the camera 52 to the intestinal wall 24 was derived using an AI system, but the distance from the camera 52 to the intestinal wall 24 may also be measured. In this case, for example, a distance measuring sensor may be provided at the tip portion 50 (see FIG. 2 ) so that the distance from the camera 52 to the intestinal wall 24 is measured by the distance measuring sensor.

[0167] In the above embodiment, an endoscopic image 40 is exemplified, but the technology of the present disclosure is not limited to this, and the technology of the present disclosure can also be applied to medical images other than the endoscopic image 40 (for example, images obtained by a modality other than the endoscope 12, such as a radiological image or an ultrasound image).

[0168] In the above embodiment, an example of measuring the size 112 of a lesion 42 shown in a moving image is given, but this is merely one example, and the technology disclosed herein can also be applied to frame-by-frame or still images showing the lesion 42.

[0169] In the above embodiment, an example was given in which the distance information 102 extracted from the distance image 104 was input to the arithmetic expression 114, but the technology of the present disclosure is not limited to this. For example, without generating the distance image 104, distance information 102 corresponding to the position identified from the position identification information 98 may be extracted from all the distance information 102 output from the distance derivation model 94, and the extracted distance information 102 may be input to the arithmetic expression 114.

[0170] In the above embodiment, an example was described in which medical support processing was performed by the processor 82 of the computer 78 included in the endoscope 12, but the technology of the present disclosure is not limited to this, and the device that performs medical support processing may be provided external to the endoscope 12. Examples of devices that may be provided external to the endoscope 12 include at least one server and / or at least one personal computer that are communicably connected to the endoscope 12. Furthermore, medical support processing may be performed in a distributed manner by multiple devices.

[0171] In the above embodiment, an example has been described in which the medical support program 90 is stored in the NVM 86, but the technology of the present disclosure is not limited to this. For example, the medical support program 90 may be stored in a portable, computer-readable, non-transitory storage medium such as an SSD or USB memory. The medical support program 90 stored in the non-transitory storage medium is installed in the computer 78 of the endoscope 12. The processor 82 executes medical support processing in accordance with the medical support program 90.

[0172] Alternatively, the medical support program 90 may be stored in a storage device such as another computer or server connected to the endoscope 12 via a network, and the medical support program 90 may be downloaded and installed on the computer 78 in response to a request from the endoscope 12.

[0173] It is not necessary to store the entire medical support program 90 in a storage device such as another computer or server device connected to the endoscope 12, or to store the entire medical support program 90 in the NVM 86; only a portion of the medical support program 90 may be stored.

[0174] The hardware resources that execute the medical support processing can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource that executes medical support processing by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with a circuit configuration specifically designed to execute specific processing. Each processor has built-in or connected memory, and executes medical support processing by using the memory.

[0175] The hardware resource for executing the medical support processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource for executing the medical support processing may be a single processor.

[0176] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes medical support processing. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute medical support processing, on a single IC chip, as typified by SoC. In this way, medical support processing is realized using one or more of the various processors described above as hardware resources.

[0177] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The above medical support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0178] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0179] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0180] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0181] The following additional notes are provided regarding the above-described embodiments.

[0182] (Supplementary Note 1) An image processing device comprising a processor, which recognizes the position of an observation area in a medical image based on the medical image in which the observation area is captured, determines whether or not to measure the size of the observation area based on the position, and measures the size based on the medical image if it is determined that the measurement is to be performed.

[0183] (Supplementary Note 2) The medical image is a plurality of frames in a time series, and the processor recognizes the position for each of the frames and determines whether or not to perform the measurement using the amount of change in the position between the plurality of frames.

[0184] (Supplementary Note 3) The image processing device according to Supplementary Note 2, wherein the amount of change in the position between the plurality of frames is defined based on a distance between the position between the plurality of frames.

[0185] (Supplementary Note 4) The image processing device according to Supplementary Note 2, wherein the amount of change in the position between the plurality of frames is defined based on the degree of overlap of the observation target regions between the plurality of frames.

[0186] (Supplementary Note 5) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the medical images are a plurality of frames in a time series, and the processor recognizes the position for each frame using an AI bounding box method, and determines whether or not to perform the measurement using a change in the bounding box.

[0187] (Supplementary Note 6) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 4, wherein the medical image is a plurality of frames in a time series, and the processor recognizes the position for each frame using an AI segmentation method, and determines whether or not to perform the measurement using the amount of change in the segmentation area.

[0188] (Supplementary Note 7) The image processing device according to any one of Supplementary Notes 1 to 6, wherein the processor determines whether or not to perform the measurement based on whether or not the position is at a peripheral portion of the medical image.

[0189] (Supplementary Note 8) The image processing device according to any one of Supplementary Note 1 to Supplementary Note 7, wherein the medical images are a plurality of frames in a time series, and the processor determines whether to perform the measurement based on the position in a first frame selected from the plurality of frames in accordance with given instructions, and the position in at least one second frame from the plurality of frames that was obtained earlier than the first frame.

[0190] (Supplementary Note 9) The image processing device according to any one of Supplementary Notes 1 to 8, wherein the medical image is a moving image.

[0191] (Supplementary Note 10) The image processing device according to any one of Supplementary Notes 1 to 9, wherein the processor outputs the size when it is determined that the measurement is to be performed.

[0192] (Supplementary Note 11) The image processing device according to Supplementary Note 10, wherein the output of the size is realized by displaying the size on the first screen.

[0193] (Supplementary Note 12) The image processing device according to any one of Supplementary Notes 1 to 11, wherein the processor outputs a past result of the measurement when it is determined not to perform the measurement.

[0194] (Supplementary Note 13) The image processing device according to Supplementary Note 12, wherein the output of the past result is realized by displaying the past result on a second screen.

[0195] (Supplementary Note 14) The image processing device described in Supplementary Note 13, wherein the processor, when determining to perform the measurement, displays the current result of the measurement on the second screen, and displays the past result and the current result on the second screen in a distinguishable manner depending on whether it is determined not to perform the measurement or to perform the measurement.

[0196] (Supplementary Note 15) The image processing device according to any one of Supplementary Notes 1 to 14, wherein the processor outputs non-measurement specification information capable of specifying that the measurement will not be performed when it is determined that the measurement will not be performed.

[0197] (Supplementary Note 16) The image processing device according to Supplementary Note 15, wherein the output of the non-measurement specification information is realized by displaying the non-measurement specification information on a third screen.

[0198] (Supplementary Note 17) The image processing device according to any one of Supplementary Notes 1 to 16, wherein the medical image is an endoscopic image obtained by capturing an image using an endoscope.

[0199] (Supplementary Note 18) The image processing device according to any one of Supplementary Notes 1 to 17, wherein the observation target region is a lesion.

[0200] (Supplementary Note 19) An endoscope comprising: the image processing device according to any one of Supplementary Note 1 to Supplementary Note 18; and a module that is inserted into a body including the observation target area and captures an image of the observation target area to obtain the medical image.

[0201] (Supplementary Note 20) An image processing method comprising: recognizing a position of an observation target area in a medical image based on the medical image in which the observation target area appears; determining whether or not to measure the size of the observation target area based on the position; and measuring the size based on the medical image if it is determined that the measurement is to be performed.

[0202] (Supplementary Note 21) A program for causing a computer to execute a process including: recognizing the position of an observation area within a medical image based on the medical image in which the observation area appears; determining whether or not to measure the size of the observation area based on the position; and, if it is determined that the measurement is to be performed, measuring the size based on the medical image.

Claims

1. a processor; The processor: Recognizing the position of the observation target area in a medical image based on the medical image in which the observation target area is captured; determining whether or not to output the size of the observation area based on the position; When it is determined that the output is to be performed, the size is output. Image processing device.

2. the medical images are a plurality of frames in a time series, The processor: Recognizing the position in each of the plurality of frames; The amount of change in the position between the plurality of frames is used to determine whether or not to perform the output. The image processing device according to claim 1 .

3. The amount of change in the position between the plurality of frames is defined based on the distance between the positions between the plurality of frames. The image processing device according to claim 2 .

4. The amount of change in the position between the plurality of frames is defined based on the degree of overlap of the observation target regions between the plurality of frames. The image processing device according to claim 2 .

5. The processor measures the size by performing AI processing on the medical image. The image processing device according to claim 1 .

6. The processor: deriving a distance from an observation position to the observation target region by performing AI processing on the medical image; The size is measured based on the distance and the number of pixels in the range to be measured within the observation area. The image processing device according to claim 1 .

7. The processor: determining that the output is to be performed when the position exists in a first region within the medical image; If the position is in a second area outside the first area in the medical image, it is determined that the output is not to be performed. The image processing device according to claim 1 .

8. the medical images are a plurality of frames in a time series, The processor: Recognizing the position in each of the plurality of frames; Whether or not to output is determined based on the amount of change in the position between the plurality of frames and whether the position exists in a first region within the medical image or a second region outside the first region within the medical image. The image processing device according to claim 1 .

9. the medical images are a plurality of frames in a time series, The processor: Recognizing the position in each of the plurality of frames using an AI bounding box method; Determine whether to output the data using the amount of change in the bounding box. The image processing device according to claim 1 .

10. the medical images are a plurality of frames in a time series, The processor: Recognizing the position in each of the plurality of frames using an AI segmentation method; The amount of change in the segmentation region is used to determine whether or not to perform the output. The image processing device according to claim 1 .

11. The processor determines whether to perform the output based on whether the position is at the edge of the medical image. The image processing device according to claim 1 .

12. the medical images are a plurality of frames in a time series, The processor determines whether to perform the output based on the position in a first frame selected from the plurality of frames in accordance with a given instruction and the position in at least one second frame obtained earlier than the first frame from the plurality of frames. The image processing device according to claim 1 .

13. When the processor determines that the output is to be performed, the processor outputs the size. The image processing device according to claim 1 .

14. The size is output by displaying the size on the first screen. The image processing device according to claim 13 .

15. When the processor determines not to perform the output, it outputs the past result in which the size was measured. The image processing device according to claim 1 .

16. The output of the past results is realized by displaying the past results on the second screen. The image processing device according to claim 15.

17. The processor: When it is determined that the output is to be performed, the current result of the size measurement is displayed on the second screen; When it is determined that the output is not to be performed and when it is determined that the output is to be performed, the past result and the current result are displayed on the second screen in a distinguishable manner. The image processing device according to claim 16.

18. When determining that the output will not be performed, the processor outputs non-output specifying information that can specify that the output will not be performed. The image processing device according to claim 1 .

19. The output of the non-output specific information is realized by displaying the non-output specific information on the third screen. The image processing device according to claim 18.

20. The medical image is an endoscopic image obtained by imaging using an endoscope. The image processing device according to claim 1 .

21. The observation target area is a lesion. The image processing device according to claim 1 .

22. An image processing device according to any one of claims 1 to 21; a module that is inserted into a body including the observation target region and captures the observation target region to obtain the medical image. Endoscope.

23. Recognizing the position of the observation target area in a medical image based on the medical image in which the observation target area is captured; determining whether to output the size of the observation area based on the position; and outputting the size when it is determined that the output is to be performed. Image processing methods.

24. Recognizing the position of the observation target area in a medical image based on the medical image in which the observation target area is captured; determining whether to output the size of the observation area based on the position; and A program for causing a computer to execute a process including outputting the size when it is determined that the output should be performed.

Images