Medical support device, endoscope, medical support method, and program

Abstract

A medical support device includes a processor. The processor recognizes the positions of the plurality of observation target regions in the medical image on the basis of the medical image reflecting the plurality of observation target regions, determines the priority order of the plurality of observation target regions on the basis of the positions, measures the sizes of the plurality of observation target regions, and outputs the sizes on the basis of the priority order.

Description

Technical Field

[0001] The technology of the present invention relates to a medical support device, an endoscope, a medical support method, and a program. Background Art

[0002] Japanese Patent Application Laid-Open No. 2021-178110 discloses an information processing device located between a radiography control device and an image management device. The information processing device described in Japanese Patent Application Laid-Open No. 2021-178110 includes an abnormality detection unit that detects abnormalities in medical images received from the radiography control device, and a control unit that determines the content or destination of the medical images or detection results based on the detection results of the abnormality detection unit.

[0003] In the information processing device described in Japanese Patent Application Laid-Open No. 2021-178110, an abnormality detection unit generates a lesion name, lesion location, lesion size, lesion type, malignancy, or a malignancy map based on a medical image. Furthermore, in the information processing device described in Japanese Patent Application Laid-Open No. 2021-178110, a control unit assigns a priority to the medical image when displaying the medical image based on the detection result, and transmits the priority to the image management device in association with the medical image.

[0004] International Publication No. 2020 / 188682 discloses a diagnosis support device. The diagnosis support device described in International Publication No. 2020 / 188682 includes an abnormal symptom identification unit, a lesion extraction function unit, and a function control unit.

[0005] The abnormal symptom identification unit is configured to perform processing for identifying an abnormal symptom occurring in the diagnostic target organ based on at least one of body information including one or more information capable of estimating the condition of the diagnostic target organ of the subject and an endoscopic image obtained by imaging the diagnostic target organ. The lesion extraction functional unit is configured to include a plurality of different lesion extraction units specialized for each abnormal symptom that may occur in the diagnostic target organ as lesion extraction processing for extracting a lesion candidate area from the endoscopic image. The lesion extraction functional control unit is configured to perform processing for selecting a lesion extraction unit corresponding to the abnormal symptom identified by the abnormal symptom identification unit from the lesion extraction units of the plurality of lesions, and to control the lesion extraction functional unit to perform lesion extraction processing on the single lesion extraction unit.

[0006] Furthermore, the diagnosis support device described in International Publication No. 2020 / 188682 includes a display control unit configured to perform processing for displaying an endoscopic image and information indicating the position of a lesion candidate region extracted by a single lesion extraction unit on a display device. Summary of the Invention

[0007] Technical issues to be solved by the invention

[0008] One embodiment of the technology involved in the present invention provides a medical support device, an endoscope, a medical support method, and a program that enable a user or the like to grasp the size of an observation target area that is expected to be of high interest to the user or the like when multiple observation target areas are projected on a medical image.

[0009] Means for solving technical problems

[0010] A first embodiment of the technology of the present invention provides a medical support device including a processor that identifies positions of multiple observation target areas within a medical image based on the medical image showing the multiple observation target areas, determines priorities of the multiple observation target areas based on the positions, measures sizes of the multiple observation target areas, and outputs the sizes based on the priorities.

[0011] A second aspect according to the technology of the present invention is the medical support apparatus according to the first aspect, wherein the processor outputs the size of each observation target region in order of priority.

[0012] A third aspect according to the technology of the present invention is the medical support device according to the second aspect, wherein the output of the size of each observation target region is performed every time an instruction is given.

[0013] A fourth aspect according to the present invention is the medical support device according to any one of the first to third aspects, wherein the output of the size is achieved by displaying the size on a screen.

[0014] A fifth aspect according to the technology of the present invention is the medical support device according to the fourth aspect, wherein the size is displayed on the screen in a display format corresponding to the priority order.

[0015] A sixth aspect according to the technology of the present invention is the medical support apparatus according to the fourth aspect or the fifth aspect, wherein the medical image is displayed on the screen, and the size is displayed within the medical image.

[0016] A seventh aspect of the present invention is the medical support device according to any one of the fourth to sixth aspects, wherein a medical image is displayed on a screen, and region specifying information capable of specifying an observation target region corresponding to the output size is displayed within the medical image.

[0017] An eighth aspect of the technology of the present invention is a medical support device according to any one of the fourth to seventh aspects, wherein the screen includes a first display area and a second display area, wherein the medical image is displayed in the first display area, and a graph indicating the distribution of the position of each observation target area is displayed in the second display area, and region-specific information capable of specifying the observation target area corresponding to the output size is displayed in the graph.

[0018] A ninth aspect according to the technology of the present invention is the medical support device according to any one of the fourth to eighth aspects, wherein the size displayed on the screen is switched according to a priority order.

[0019] A tenth aspect according to the technology of the present invention is the medical support device according to the ninth aspect, wherein the size of the image displayed on the screen is switched every time an instruction is given.

[0020] An eleventh aspect according to the technology of the present invention is the medical support device according to any one of the first to tenth aspects, wherein the position is identified using AI, and the priority is determined based on a confidence level obtained from the AI.

[0021] A twelfth aspect according to the technology of the present invention is the medical support apparatus according to any one of the first to eleventh aspects, wherein the closer the position is to the center of the medical image, the higher the priority.

[0022] A thirteenth aspect according to the technology of the present invention is the medical support apparatus according to any one of the first to twelfth aspects, wherein the processor acquires depths of the plurality of observation target regions, and the priority is determined based on the positions and the depths.

[0023] A fourteenth aspect according to the present invention is the medical support device according to any one of the first to thirteenth aspects, wherein the processor recognizes the type of the observation target region based on the medical image, and the priority is determined based on the position and the type.

[0024] A fifteenth aspect according to the technology of the present invention is the medical support device according to any one of the first to fourteenth aspects, wherein the processor measures the size in a priority order.

[0025] A sixteenth aspect according to the technology of the present invention is the medical support device according to any one of the first to fifteenth aspects, wherein the medical image is an endoscopic image obtained by imaging with an endoscope.

[0026] A seventeenth aspect according to the technology of the present invention is the medical support device according to any one of the first to sixteenth aspects, wherein the observation target region is a lesion.

[0027] An eighteenth aspect of the present invention is an endoscope comprising: the medical support device according to any one of the first to seventeenth aspects; and a module that acquires medical images by being inserted into a body including an observation target area and imaging the observation target area.

[0028] The nineteenth method involved in the technology of the present invention is a medical support method, which includes the following steps: identifying the positions of multiple observation object areas within a medical image based on a medical image in which multiple observation object areas are mapped; determining the priority of the multiple observation object areas based on the positions; measuring the sizes of the multiple observation object areas; and outputting the sizes based on the priority.

[0029] A twentieth embodiment of the technology of the present invention is a program for causing a computer to execute medical support processing, the medical support processing comprising the following steps: identifying positions of multiple observation object areas within a medical image based on a medical image showing multiple observation object areas; determining priorities of the multiple observation object areas based on the positions; measuring sizes of the multiple observation object areas; and outputting the sizes based on the priorities. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a conceptual diagram showing an example of how to use the endoscope system.

[0031] Figure 2 This is a conceptual diagram showing an example of the overall structure of an endoscope.

[0032] Figure 3 This is a block diagram showing an example of the hardware configuration of the electrical system of the endoscope.

[0033] Figure 4 This is a block diagram showing an example of the main functions of a processor included in an endoscope and an example of information stored in an NVM.

[0034] Figure 5 This is a conceptual diagram showing an example of the processing contents of the recognition unit and the control unit.

[0035] Figure 6 This is a conceptual diagram showing an example of the processing content of the determination unit.

[0036] Figure 7 This is a conceptual diagram showing an example of the processing content of the measurement unit.

[0037] Figure 8 This is a conceptual diagram showing an example of a mode in which an endoscopic image is displayed in a first display area and dimensions are displayed within the image in a second display area.

[0038] Figure 9This is a flowchart showing an example of the flow of medical support processing.

[0039] Figure 10 This is a conceptual diagram showing an example of how the display content in the diagram is switched according to an instruction received by the receiving device 64 .

[0040] Figure 11 This is a conceptual diagram showing a first modified example of the processing content of the determination unit.

[0041] Figure 12 This is a conceptual diagram showing a second modified example of the processing content of the determination unit.

[0042] Figure 13 This is a conceptual diagram showing an example of how the segmented images displayed in the figure are surrounded by a circumscribed rectangular frame.

[0043] Figure 14 This is a conceptual diagram showing an example of a method in which size and text information are displayed in a pop-up format from inside the figure to outside the figure, and the size and text information are displayed on the screen in a display size corresponding to the priority order.

[0044] Figure 15 This is a conceptual diagram showing an example of a method in which size and text information are displayed in a pop-up format from inside an endoscopic image to outside the endoscopic image, and the size and text information are displayed on the screen in a display size corresponding to a priority order.

[0045] Figure 16 This is a conceptual diagram showing an example of how a lesion appearing in an endoscopic image is surrounded by a circumscribed rectangular frame.

[0046] Figure 17 This is a conceptual diagram showing an example of an output address of a size. DETAILED DESCRIPTION

[0047] Hereinafter, an example of an embodiment of a medical support device, an endoscope, a medical support method, and a program according to the technology of the present invention will be described with reference to the drawings.

[0048] First, the terms used in the following description are explained.

[0049] CPU stands for Central Processing Unit. GPU stands for Graphics Processing Unit. RAM stands for Random Access Memory. NVM stands for Non-volatile Memory. EEPROM stands for Electrically Erasable Programmable Read-Only Memory. ASIC stands for Application Specific Integrated Circuit. PLD stands for Programmable Logic Device. FPGA stands for Field-Programmable Gate Array. SoC stands for System-on-a-Chip. SSD stands for Solid State Drive. USB stands for Universal Serial Bus. HDD stands for Hard Disk Drive. EL stands for Electro-Luminescence. CMOS stands for "Complementary Metal Oxide Semiconductor." CCD stands for "Charge Coupled Device." AI stands for "Artificial Intelligence." BLI stands for "Blue Light Imaging." LCI stands for "Linked Color Imaging." I / F stands for "Interface." SSL stands for "Sessile Serrated Lesion." NP stands for "Neoplastic Polyp." HP stands for "Hyperplastic Polyp."

[0050] As an example, Figure 1As shown, the endoscope system 10 includes an endoscope 12 and a display device 14. The endoscope 12 is used by a doctor 16 during endoscopic examination. The endoscopic examination is assisted by staff such as a nurse 17. In this embodiment, the endoscope 12 is an example of an "endoscope" involved in the technology of the present invention.

[0051] The endoscope 12 is communicatively connected to a communication device (not shown), and information obtained by the endoscope 12 is transmitted to the communication device. Examples of the communication device include a server and / or a client terminal (e.g., a personal computer and / or a tablet terminal) that manages various information such as electronic medical records. The communication device receives information transmitted from the endoscope 12 and executes processing to use the received information (e.g., processing to store the information in an electronic medical record, etc.).

[0052] The endoscope 12 includes an endoscope body 18. The endoscope 12 is a device for diagnosing and treating a large intestine 22 included in the body of a subject 20 (eg, a patient) using the endoscope body 18. In this embodiment, the large intestine 22 is an object to be observed by the doctor 16.

[0053] The endoscope body 18 is inserted into the large intestine 22 of the subject 20. The endoscope 12 images the interior of the large intestine 22 of the subject 20 with the endoscope body 18 inserted therein, and performs various medical procedures on the large intestine 22 as needed.

[0054] The endoscope 12 captures and outputs images of the interior of the large intestine 22 of the subject 20. In this embodiment, the endoscope 12 is an endoscope having an optical imaging function for capturing light reflected by the intestinal wall 24 of the large intestine 22 by irradiating light 26 into the large intestine 22.

[0055] In addition, although the endoscope examination of the large intestine 22 is illustrated here, this is merely an example, and the technology of the present invention is also applicable to the endoscope examination of luminal organs such as the esophagus, stomach, duodenum, or trachea.

[0056] The endoscope 12 includes a control device 28, a light source device 30, and an image processing device 32. The control device 28, light source device 30, and image processing device 32 are mounted on a cart 34. A plurality of stages are mounted vertically on the cart 34, with the image processing device 32, control device 28, and light source device 30 mounted on stages from the bottom to the top. Furthermore, the display device 14 is mounted on the topmost stage of the cart 34.

[0057] The control device 28 controls the entire endoscope 12. The image processing device 32 performs various image processing on an image obtained by imaging the intestinal wall 24 using the endoscope body 18 under the control of the control device 28.

[0058] The display device 14 displays various information including images. Examples of the display device 14 include a liquid crystal display and an EL display. Alternatively, a tablet terminal with a display may be used in place of or in addition to the display device 14.

[0059] The screen 35 is displayed on the display device 14. The screen 35 includes a plurality of display areas. The plurality of display areas are arranged in the screen 35. Figure 1 In the example shown, a first display area 36 and a second display area 38 are shown as examples of multiple display areas. The size of the first display area 36 is larger than that of the second display area 38. The first display area 36 is used as the main display area, and the second display area 38 is used as the sub-display area. In this embodiment, the screen 35 is an example of a "screen" involved in the technology of the present invention, the first display area 36 is an example of a "first display area" involved in the technology of the present invention, and the second display area 38 is an example of a "second display area" involved in the technology of the present invention.

[0060] An endoscopic image 40 is displayed in the first display area 36. The endoscopic image 40 is an image acquired by the endoscope body 18 by imaging the intestinal wall 24 in the large intestine 22 of the subject 20. Figure 1 In the example shown, an image showing the intestinal wall 24 is shown as an example of the endoscopic image 40. In addition, the intestinal wall 24 shown in the endoscopic image 40 includes a plurality of lesions 42 (e.g., Figure 1 In the example shown, there are three lesions 42. The doctor 16 can visually recognize the morphology of the intestinal wall 24 including the multiple lesions 42 through the endoscopic image 40. There are many types of lesions 42, and examples of the types of lesions 42 include neoplastic polyps (e.g., NP or SSL belonging to NP) and non-neoplastic polyps (e.g., HP).

[0061] In this embodiment, endoscopic image 40 is an example of a "medical image" and "endoscopic image" within the scope of the present invention. Furthermore, in this embodiment, lesion 42 is an example of an "observation target region" and "lesion" within the scope of the present invention. While lesion 42 is illustrated here, the present invention is not limited thereto. The multiple regions of interest (i.e., multiple observation target regions) observed by doctor 16 may also include multiple organs (e.g., the bile duct opening and pancreatic duct opening included in the duodenal papilla), multiple marked areas, artificial treatment instruments (e.g., artificial clips), or multiple treated areas (e.g., multiple areas with traces of polyp removal). Furthermore, the multiple regions of interest (i.e., multiple observation target regions) observed by doctor 16 may also include multiple combinations of at least one lesion 42, at least one organ, at least one marked area, at least one artificial treatment instrument, and at least one treated area.

[0062] A moving image is displayed in the first display area 36. The endoscopic image 40 displayed in the first display area 36 is one frame included in a moving image composed of multiple frames along a time series. That is, the first display area 36 displays multiple frames of the endoscopic image 40 at a predetermined frame rate (e.g., 30 frames / second or 60 frames / second).

[0063] An example of a moving image displayed in the first display area 36 is a live view moving image. The live view method is merely an example; a moving image may be temporarily stored in a memory or the like and then displayed, as in a post-capture view method. Furthermore, each frame included in a recording moving image stored in a memory or the like may be reproduced and displayed as the endoscopic image 40 in the first display area 36.

[0064] Within the screen 35, a second display area 38 is adjacent to the first display area 36 and is displayed in the lower right corner of the main view within the screen 35. The second display area 38 can be displayed anywhere within the screen 35 of the display device 14, but is preferably displayed at a location where it can be compared with the endoscopic image 40. A plurality of segmented images 44 are displayed within the second display area 38. The segmented images 44 are image areas that specify the locations within the endoscopic image 40 of lesions 42 identified by performing segmented object recognition processing on the endoscopic image 40 using AI.

[0065] The plurality of segmented images 44 displayed in the second display area 38 are images corresponding to the endoscopic image 40 and are referred to by the doctor 16 to identify the position of the lesion 42 within the endoscopic image 40 .

[0066] While multiple segmented images 44 are shown here as an example, if lesions 42 are identified by performing bounding box-based object recognition processing on endoscopic image 40 using AI, multiple bounding boxes may be displayed instead of the multiple segmented images 44. Alternatively, multiple segmented images 44 and multiple bounding boxes may be used together. The segmented images 44 and bounding boxes are merely examples; any image may be used as long as it can clearly identify the positional relationship between multiple lesions 42 within endoscopic image 40.

[0067] As an example, Figure 2 As shown, the endoscope body 18 includes an operating portion 46 and an insertion portion 48. The insertion portion 48 is partially bent by the operation of the operating portion 46. The insertion portion 48 is adjusted according to the doctor 16 (refer to FIG. Figure 1 ) to operate the operating unit 46, while according to the large intestine 22 (refer to Figure 1 ) is bent into the shape and inserted into the large intestine 22.

[0068] The distal end portion 50 of the insertion portion 48 is provided with an imaging device 52, an illumination device 54, and an opening 56 for a treatment instrument. The imaging device 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end portion 50. Here, the imaging device 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end portion 50. However, this is merely an example, and the imaging device 52 and the illumination device 54 may also be provided on the side surface of the distal end portion 50, thereby configuring the endoscope 12 as a side-viewing mirror.

[0069] The imaging device 52 is a device that captures the interior of the subject 20 (e.g., the large intestine 22) to obtain an endoscopic image 40 as a medical image. A CMOS imaging device is an example of the imaging device 52. However, this is merely an example, and other types of imaging devices, such as a CCD imaging device, may also be used. The imaging device 52 is an example of a "module" within the technology of the present invention.

[0070] The lighting device 54 has lighting windows 54A and 54B. The lighting device 54 irradiates the light 26 (see Figure 1 ). Examples of the type of light 26 emitted from the lighting device 54 include visible light (e.g., white light) and non-visible light (e.g., near-infrared light). Furthermore, the lighting device 54 emits special light through the lighting windows 54A and 54B. Examples of special light include light for BLI and / or light for LCI. The imaging device 52 optically captures the interior of the large intestine 22 while the lighting device 54 is irradiating the large intestine 22 with light 26.

[0071] The treatment instrument opening 56 is an opening for allowing a treatment instrument 58 to protrude from the distal end portion 50. The treatment instrument opening 56 is also used as a suction port for suctioning blood and body waste, and a delivery port for delivering fluid.

[0072] The operation portion 46 is formed with a treatment instrument insertion port 60, and the treatment instrument 58 is inserted into the insertion portion 48 through the treatment instrument insertion port 60. The treatment instrument 58 passes through the insertion portion 48 and projects from the treatment instrument opening 56 to the outside. Figure 2 In the illustrated example, a puncture needle is shown as the treatment instrument 58 protruding from the treatment instrument opening 56. While a puncture needle is shown as the treatment instrument 58, this is merely an example, and the treatment instrument 58 may also be a grasper, a papillotomy device, a snare, a catheter, a guide wire, a cannula, and / or a puncture needle with a guide sheath.

[0073] The endoscope body 18 is connected to the control device 28 and the light source device 30 via a universal cord 62. The control device 28 is connected to an image processing device 32 and a receiving device 64. The image processing device 32 is also connected to the display device 14. That is, the control device 28 is connected to the display device 14 via the image processing device 32.

[0074] In addition, since the image processing device 32 is illustrated here as an external device used to expand the functions performed by the control device 28, and thus an indirect connection between the control device 28 and the display device 14 via the image processing device 32 is shown, this is merely an example. For example, the display device 14 may also be directly connected to the control device 28. In this case, for example, the functions of the image processing device 32 may be installed on the control device 28, or a server (not shown) may be configured to perform the same processing as that performed by the image processing device 32 (for example, the medical support processing described below), and the function of receiving and using the processing results implemented by the server may be installed on the control device 28.

[0075] The receiving device 64 receives instructions from the doctor 16 and outputs the received instructions as electrical signals to the control device 28. Examples of the receiving device 64 include a keyboard, a mouse, a touch panel, a foot switch, a microphone, and / or a remote control device.

[0076] The control device 28 controls the light source device 30 , or exchanges various signals with the imaging device 52 , or exchanges various signals with the image processing device 32 .

[0077] The light source device 30 emits light under the control of the control device 28 and supplies light to the lighting device 54. The lighting device 54 has a built-in light guide, and the light supplied from the light source device 30 is irradiated from the lighting windows 54A and 54B via the light guide. The control device 28 causes the camera 52 to capture an image, and the endoscopic image 40 (see FIG. 4 ) is obtained from the camera 52. Figure 1 ) and output to a predetermined output address (for example, the image processing device 32).

[0078] The image processing device 32 performs various image processing on the endoscopic image 40 input from the control device 28. The image processing device 32 outputs the endoscopic image 40 subjected to various image processing to a predetermined output destination (for example, the display device 14).

[0079] In addition, although the example described herein is of an embodiment in which the endoscopic image 40 output from the control device 28 is output to the display device 14 via the image processing device 32, this is merely an example. For example, the control device 28 and the display device 14 may be connected, and the endoscopic image 40, which has been image-processed by the image processing device 32, may be displayed on the display device 14 via the control device 28.

[0080] As an example, Figure 3 As shown, the control device 28 includes a computer 66, a bus 68, and an external I / F 70. The computer 66 includes a processor 72, a RAM 74, and an NVM 76. The processor 72, RAM 74, NVM 76, and the external I / F 70 are connected to the bus 68.

[0081] For example, the processor 72 includes at least one CPU and at least one GPU, and controls the entire control device 28. The GPU operates under the control of the CPU and is responsible for executing various processes of the graphics system and performing calculations using neural networks. In addition, the processor 72 may be one or more CPUs with integrated GPU functions, or one or more CPUs without integrated GPU functions. Figure 3 In the illustrated example, a single processor 72 is mounted on the computer 66 . However, this is merely an example, and a plurality of processors 72 may be mounted on the computer 66 .

[0082] RAM 74 is a memory for temporarily storing information and is used as working memory by processor 72. NVM 76 is a nonvolatile storage device that stores various programs and parameters. An example of NVM 76 is a flash memory (e.g., EEPROM and / or SSD). Flash memory is merely an example; other nonvolatile storage devices such as HDDs may also be used, or a combination of two or more nonvolatile storage devices may be used.

[0083] The external I / F 70 is responsible for exchanging various information between one or more devices (hereinafter also referred to as "first external devices") existing outside the control device 28 and the processor 72. An example of the external I / F 70 is a USB interface.

[0084] The external I / F 70 is connected to the imaging device 52 as one of the first external devices. The external I / F 70 is responsible for the exchange of various information between the imaging device 52 and the processor 72. The processor 72 controls the imaging device 52 via the external I / F 70. In addition, the processor 72 obtains the image of the large intestine 22 (see FIG. 2 ) by the imaging device 52 through the external I / F 70. Figure 1 ) and the endoscopic image 40 (refer to Figure 1 ).

[0085] The external I / F 70 is connected to the light source device 30 as one of the first external devices. The external I / F 70 is responsible for exchanging various information between the light source device 30 and the processor 72. Under the control of the processor 72, the light source device 30 supplies light to the lighting device 54. The lighting device 54 irradiates the light supplied by the light source device 30.

[0086] The receiving device 64 is connected to the external I / F 70 as one of the first external devices. The processor 72 acquires an instruction received by the receiving device 64 via the external I / F 70 and executes a process corresponding to the acquired instruction.

[0087] The image processing device 32 includes a computer 78 and an external I / F 80. The computer 78 includes a processor 82, a RAM 84, and an NVM 86. The processor 82, the RAM 84, the NVM 86, and the external I / F 80 are connected to a bus 88. In this embodiment, the image processing device 32 is an example of a "medical support device" according to the technology of the present invention, the computer 78 is an example of a "computer" according to the technology of the present invention, and the processor 82 is an example of a "processor" according to the technology of the present invention.

[0088] Furthermore, since the hardware structure of the computer 78 (ie, the processor 82 , the RAM 84 , and the NVM 86 ) is substantially the same as that of the computer 66 , the description of the hardware structure of the computer 78 will be omitted herein.

[0089] The external I / F 80 is responsible for exchanging various information between one or more devices (hereinafter also referred to as "second external devices") existing outside the image processing device 32 and the processor 82. An example of the external I / F 80 is a USB interface.

[0090] The control device 28 is connected to the external I / F 80 as one of the second external devices. Figure 3In the example shown, the external I / F 80 is connected to the external I / F 70 of the control device 28. The external I / F 80 is responsible for the exchange of various information between the processor 82 of the image processing device 32 and the processor 72 of the control device 28. For example, the processor 82 obtains the endoscopic image 40 (see FIG. 4 ) from the processor 72 of the control device 28 via the external I / Fs 70 and 80. Figure 1 ) and perform various image processing on the acquired endoscopic image 40.

[0091] The display device 14 is connected as one of the second external devices to the external I / F 80. The processor 82 controls the display device 14 via the external I / F 80 to display various information (eg, the endoscopic image 40 subjected to various image processing).

[0092] However, during an endoscopic examination, the doctor 16 determines whether medical treatment is required for the multiple lesions 42 shown in the endoscopic image 40 while viewing the endoscopic image 40 via the display device 14. If necessary, the doctor performs medical treatment on the multiple lesions 42. The size of the multiple lesions 42 is an important factor in determining whether medical treatment is required.

[0093] In recent years, the development of machine learning has made it possible to detect and identify multiple lesions 42 based on endoscopic images 40 using AI. By applying this technology, the sizes of multiple lesions 42 can be measured from endoscopic images 40.

[0094] However, when multiple lesions 42 are shown in the endoscopic image 40, the doctor 16 may be concerned about different lesions 42 depending on their locations within the endoscopic image 40. Prioritizing the size of a lesion 42 of lesser interest to the doctor 16, or presenting the sizes of both a lesion 42 of greater interest and a lesion 42 of lesser interest to the doctor 16 in a state where they cannot be distinguished, may lead to inefficiency or confusion in the endoscopic examination. In contrast, prioritizing the size of a lesion 42 of greater interest to the doctor 16 and presenting it to the doctor 16, or presenting the sizes of both a lesion 42 of greater interest and a lesion 42 of lesser interest to the doctor 16 in a state where they can be distinguished, can be useful in the endoscopic examination.

[0095] Therefore, in view of such a situation, in this embodiment, as an example, Figure 4 As shown, medical support processing is performed by the processor 82 of the image processing device 32.

[0096] NVM 86 stores a medical support program 90. Medical support program 90 is an example of a "program" within the scope of the present invention. Processor 82 reads medical support program 90 from NVM 86 and executes it on RAM 84, thereby performing medical support processing. Medical support processing is achieved by the recognition unit 82A, determination unit 82B, measurement unit 82C, and control unit 82D operating in accordance with medical support program 90 executed by processor 82 on RAM 84.

[0097] NVM 86 stores a recognition model 92 and a distance derivation model 94. These models are examples of "AI" within the present invention. Details will be described later. Recognition model 92 is used by recognition unit 82A, while distance derivation model 94 is used by measurement unit 82C.

[0098] As an example, Figure 5 As shown, the recognition unit 82A and the control unit 82D acquire, from the imaging device 52 , the endoscopic image 40 generated by imaging at an imaging frame rate (eg, several tens of frames / second) using the imaging device 52 in units of one frame.

[0099] The control unit 82D displays the endoscopic image 40 as a live view image in the first display area 36. That is, each time the control unit 82D acquires the endoscopic image 40 from the imaging device 52 in units of one frame, it sequentially displays the acquired endoscopic image 40 in the first display area 36 at a display frame rate (e.g., several tens of frames per second).

[0100] The recognition unit 82A performs recognition processing 96 on the endoscopic image 40 acquired from the imaging device 52 to recognize the position and type of the lesion 42 within the endoscopic image 40 (i.e., the position of the lesion 42 reflected in the endoscopic image 40). The recognition processing 96 is performed by the recognition unit 82A on the acquired endoscopic image 40 each time the endoscopic image 40 is acquired.

[0101] Recognition processing 96 is an object recognition processing using a segmentation method based on AI. Here, as recognition processing 96, processing using the recognition model 92 is performed.

[0102] Recognition model 92 is a learned model for object recognition using a segmented AI approach, optimized by machine learning of a neural network using first supervisory data. The first supervisory data is a dataset including a plurality of data (i.e., multiple frames of data) associated with first example question data and first correct answer data.

[0103] The first example data is an image corresponding to the endoscopic image 40. The first correct answer data is correct answer data (i.e., annotation) for the first example data. Here, as an example of the first correct answer data, an annotation that specifies the location and type of a lesion appearing in the image used as the first example data is used.

[0104] The recognition unit 82A acquires an endoscopic image 40 from the imaging device 52 and inputs the acquired endoscopic image 40 into the recognition model 92. Thus, each time an endoscopic image 40 is input, the recognition model 92 identifies the position of the segmented image 44 identified in the segmented manner as the position of the lesion 42 reflected in the input endoscopic image 40, and outputs position identification information 98 that can identify the position of the segmented image 44. An example of the position identification information 98 is the coordinates of the segmented image 44 within the endoscopic image 40. Furthermore, each time an endoscopic image 40 is input, the recognition model 92 identifies the type of the lesion 42 reflected in the input endoscopic image 40 (e.g., the name of the lesion (e.g., NP, SSL, HP, etc.)) and outputs type information 100 indicating the identified type. The segmented image 44 is associated with the position identification information 98 and the type information 100.

[0105] Based on the position-identifying information 98 and the plurality of segmented images 44, the control unit 82D displays a graph 102 in the second display area 38 showing the distribution of the positions of the plurality of lesions 42 for each endoscopic image 40. Graph 102 is generated by the recognition unit 82A. The distribution of the positions of the plurality of lesions 42 for each endoscopic image 40 in graph 102 is represented by the plurality of segmented images 44 generated by the recognition unit 82A for each endoscopic image 40. For example, graph 102 displayed in the second display area 38 is updated at the same display frame rate as that used for the first display area 36. That is, the display of the plurality of segmented images 44 in the second display area 38 is synchronized with the display timing of the endoscopic image 40 displayed in the first display area 36. Thus, the doctor 16 can refer to graph 102 displayed in the second display area 38 while observing the endoscopic image 40 displayed in the first display area 36, ​​thereby being able to grasp the approximate positions of the plurality of lesions 42 within the endoscopic image 40 displayed in the first display area 36. In this embodiment, FIG102 is an example of a “graph” involved in the technology of the present invention.

[0106] As an example, Figure 6 As shown, the determination unit 82B performs recognition processing 96 (see FIG. 1 ) on the endoscopic image 40 each time the recognition unit 82A performs recognition processing 96 on the endoscopic image 40 as a unit. Figure 5), the identification unit 82B obtains an image 102 including the position identification information 98, the type information 100, and the plurality of segmented images 44 from the identification unit 82A. Then, the determination unit 82B determines a priority order 104 for the plurality of lesions 42 based on the position identification information 98 obtained from the identification unit 82A. The priority order 104 is a ranking assigned to each of the plurality of lesions 42 according to the degree of priority (in other words, according to the degree of importance), and is determined such that a lesion 42 expected to receive high attention from the doctor 16 is assigned a high priority, while a lesion 42 expected to receive low attention from the doctor 16 is assigned a low priority.

[0107] exist Figure 6 In the illustrated example, the priority 104 is determined by the determination unit 82B based on the position identification information 98 and the category information 100 .

[0108] To achieve this, first, the determination unit 82B derives the position weight 106 based on the position specifying information 98 and derives the category weight 108 based on the category information 100 .

[0109] The position weight 106 is the weight (i.e., priority (in other words, importance)) of the position of the lesion 42 within the endoscopic image 40. For example, the closer to the center of the endoscopic image 40, the larger the position weight 106. Here, for example, when the value of the position weight 106 is set to "x", the position weight 106 is a value determined within the range of "0 ≤ x ≤ 0.5". As a first example of a method for deriving the position weight 106 from the position-specific information 98, a method using an expression with the position-specific information 98 as the dependent variable and the position weight 106 as the independent variable can be cited. As a second example of a method for deriving the position weight 106 from the position-specific information 98, a method using a table with the position-specific information 98 as input and the position weight 106 as output can be cited.

[0110] The category weight 108 is the category (i.e., priority (in other words, importance)) of the lesion 42 represented by the category information 100. For example, the more severe the category of the lesion 42, the larger the category weight 108. For example, the category weight 108 of a tumorous polyp is larger than the category weight 108 of a non-tumorous polyp. Here, for example, when the value of the category weight 108 is set to "y", the category weight 108 is a value determined within the range of "0≤y≤0.5". As a first example of a method of deriving the category weight 108 from the category information 100, a method of using a table that takes the category information 100 as input and the category weight 108 as output can be cited. As a second example of a method of deriving the category weight 108 from the category information 100, a method of using an expression that takes the category information 100 as a dependent variable and the category weight 108 as an independent variable, on the premise that the category information 100 is represented by a variable that can specify the category of the lesion 42, can be cited.

[0111] The determination unit 82B calculates the total weight 110 based on the position weight 106 and the category weight 108. Figure 6 In the illustrated example, the sum of the position weight 106 and the category weight 108 is shown as the total weight 110 .

[0112] The sum of the position weight 106 and the category weight 108 is merely an example; the product of the position weight 106 and the category weight 108 may also be used. Furthermore, a coefficient may be multiplied by at least one of the position weight 106 and the category weight 108. The coefficient may be a fixed value or a variable value.

[0113] When the coefficient is set as a variable value, the coefficient can be determined according to various conditions (for example, the type of endoscopic examination, the specifications of the endoscope 12, and / or the user of the endoscope 12), or can be determined according to instructions given by the doctor 16 or the like via the receiving device 64. In addition, the position weight 106 itself and / or the type weight 108 itself can also be changed in the same manner.

[0114] The determination unit 82B determines the priority 104 for each of the lesions 42 based on the total weight 110 calculated for each of the lesions 42. The greater the total weight 110, the higher the priority 104. For example, when the sum of the position weight 106 and the category weight 108 is used as the total weight 110, focusing on the position weight 106, the greater the total weight 110, and thus the higher the priority 104. In this embodiment, the closer the lesion 42 is to the center of the endoscopic image 40, the greater the position weight 106. Therefore, the closer the lesion 42 is to the center of the endoscopic image 40, the higher the priority 104. Furthermore, focusing on the category weight 108, the greater the category weight 108, the greater the total weight 110, and thus the higher the priority 104. In this embodiment, the higher the severity of the lesion 42, the greater the category weight 108. Therefore, the higher the severity of the lesion 42, the higher the priority 104.

[0115] The specifying unit 82B assigns the specified priorities 104 to the plurality of lesions 42. Figure 6 In the illustrated example, the assignment of the priorities 104 to the plurality of lesions 42 is achieved by assigning the priorities 104 determined by the determination unit 82B to each of the plurality of segmented images 44 corresponding to the plurality of lesions 42 .

[0116] When the priority order 104 of a plurality of lesions 42 is determined by the determination unit 82B, as an example, Figure 7As shown, the measuring unit 82C measures the size 112 of the lesion 42 based on the endoscopic image 40 acquired from the imaging device 52 (for example, the endoscopic image 40 used by the identification unit 82A to obtain the plurality of segmented images 44, the position identification information 98, and the type information 100 used by the determination unit 82B). The measuring unit 82C measures the size 112 of the plurality of lesions 42 according to the priority 104 determined by the determination unit 82B. In other words, the size 112 of the lesions 42 is measured in order from the lesion 42 with the highest priority 104 to the lesion 42 with the lowest priority 104.

[0117] The measuring unit 82C acquires distance information 114 of a plurality of lesions 42 based on the endoscopic image 40 acquired from the imaging device 52. The distance information 114 indicates the distance from the imaging device 52 (i.e., the observation position) to the intestinal wall 24 (refer to Figure 1 ). The distance from the imaging device 52 to the intestinal wall 24 including the lesion 42 is an example of the "depth" involved in the technology of the present invention. In addition, here, the distance from the imaging device 52 to the intestinal wall 24 including the lesion 42 is shown as an example, but this is just an example, and the depth from the imaging device 52 to the intestinal wall 24 including the lesion 42 may be displayed in a numerical value (for example, multiple numerical values ​​that specify the depth in stages (for example, numerical values ​​of several stages to dozens of stages)) instead of the distance.

[0118] The distance information 114 is acquired for each of all pixels constituting the endoscopic image 40. Alternatively, the distance information 114 may be acquired for each block larger than the pixels of the endoscopic image 40 (eg, a pixel group consisting of several to several hundred pixels).

[0119] The acquisition of the distance information 114 by the measurement unit 82C is achieved by, for example, deriving the distance information 114 using an AI system. In the present embodiment, a distance derivation model 94 is used to derive the distance information 114.

[0120] The distance derivation model 94 is optimized by performing machine learning on the neural network using the second supervised data.

[0121] The second supervisory data is a data set including a plurality of data (ie, a plurality of frames of data) associated with the second example question data and the second correct answer data.

[0122] The second example data is an image corresponding to the endoscopic image 40. The second correct answer data is correct answer data (i.e., annotation) for the second example data. Here, as an example of the second correct answer data, an annotation is used that specifies the distance corresponding to each pixel shown in the image used as the second example data.

[0123] The measurement unit 82C acquires the endoscopic image 40 from the imaging device 52 and inputs the acquired endoscopic image 40 into the distance derivation model 94. Consequently, the distance derivation model 94 outputs distance information 114 in units of pixels of the input endoscopic image 40. Specifically, in the measurement unit 82C, information indicating the distance from the position of the imaging device 52 (e.g., the position of the image sensor or objective lens mounted on the imaging device 52) to the intestinal wall 24 reflected in the endoscopic image 40 is output from the distance derivation model 94 as distance information 114 in units of pixels of the endoscopic image 40.

[0124] The measurement unit 82C generates a distance image 116 based on the distance information 114 output from the distance derivation model 94 . The distance image 116 is an image in which the distance information 114 is distributed in units of pixels included in the endoscopic image 40 .

[0125] The measuring unit 82C acquires the position specifying information 98 from the determining unit 82B according to the priority 104 determined by the determining unit 82B. That is, the measuring unit 82C acquires the position specifying information 98 assigned to the segmented image 44 in descending order of priority 104. For example, Figure 6 In the illustrated example, since the three segment images 44 are assigned the first to third priorities 104 , the measurement unit 82C sequentially acquires the position specifying information 98 from the position specifying information 98 assigned to the segment image 44 with the first priority 104 to the position specifying information 98 assigned to the segment image 44 with the third priority 104 .

[0126] The measurement unit 82C sequentially acquires the position-specific information 98 from the plurality of segmented images 44 in descending order of priority 104, and extracts distance information 114 corresponding to the position specified by the position-specific information 98 from the distance image 116 with reference to the acquired position-specific information 98. Examples of the distance information 114 extracted from the distance image 116 include distance information 114 corresponding to a specific position (e.g., the center of gravity) of the lesion 42, or statistical values ​​(e.g., the median, average, or mode) of the distance information 114 for a plurality of pixels (e.g., all pixels) included in the lesion 42.

[0127] The measuring unit 82C extracts a pixel count 118 from the endoscopic image 40. The pixel count 118 is the number of pixels on a line segment 120 that intersects the image region at the position specified by the position specifying information 98 (i.e., the image region representing the lesion 42) within the entire image region of the endoscopic image 40 input to the distance derivation model 94. An example of the line segment 120 is the longest line segment parallel to the long side of a circumscribed rectangular frame 122 of the image region representing the lesion 42. The line segment 120 is merely an example; instead of the line segment 120, the longest line segment parallel to the short side of the circumscribed rectangular frame 122 of the image region representing the lesion 42 may be used.

[0128] The measuring unit 82C calculates the size 112 of the lesion 42 in the real space based on the distance information 114 extracted from the distance image 116 and the number of pixels 118 extracted from the endoscopic image 40. The size 112 refers to, for example, the length of the lesion 42 in the real space.

[0129] Calculation of size 112 uses equation 124. Measurement unit 82C inputs distance information 114 extracted from distance image 116 and number of pixels 118 extracted from endoscopic image 40 into equation 124. Equation 124 uses distance information 114 and number of pixels 118 as independent variables and size 112 as a dependent variable. Equation 124 outputs size 112 corresponding to the input distance information 114 and number of pixels 118.

[0130] In addition, Figure 7 In the example shown, the size 112 of the lesion 42 corresponding to the segmented image 44 assigned the first priority 104 is measured by the measuring unit 82C. Figure 6 The sizes 112 of the lesions 42 corresponding to the segmented images 44 assigned the second and third priorities 104 are also measured by the measuring unit 82C in descending order of priority 104 .

[0131] In addition, here, an example is given in which the sizes 112 of the plurality of lesions 42 are measured sequentially according to the priority 104 . However, the technology of the present invention is not limited thereto, and the sizes 112 of the plurality of lesions 42 may be measured simultaneously.

[0132] While the length of the lesion 42 in real space is exemplified here as the dimension 112, the technology of the present invention is not limited thereto, and the dimension 112 may also be the surface area or volume of the lesion 42 in real space. In this case, for example, a computational expression 124 is used that uses the number of pixels in the entire image region representing the lesion 42 and the distance information 114 as independent variables, and uses the surface area or volume of the lesion 42 in real space as a dependent variable.

[0133] As an example, Figure 8 As shown, the control unit 82D displays the image 102 in the second display area 38. Furthermore, the control unit 82D displays a size 112 within the image 102 based on the priority 104 assigned to the plurality of segment images 44. For example, the size 112 is displayed superimposed on the image 102. The superimposed display is merely an example, and an embedded display may also be used.

[0134] In FIG. 102 , a plurality of segmented images 44 are shown, and are arranged in descending order of priority 104 (in Figure 8 In the example shown, the dimensions 112 measured by the measuring unit 82C are displayed sequentially (from the first to the third digit). Specifically, the dimensions 112 displayed in FIG. 102 are switched according to the priority 104. The time interval for switching the dimensions 112, that is, the time during which the dimensions 112 of each priority 104 are continuously displayed, can be a fixed time of several seconds to several tens of seconds, or a variable time that can be changed according to instructions given by the doctor 16 or the like via the receiving device 64.

[0135] The control unit 82D displays a dimension line 126 in FIG102 as information that can specifically display which lesion 42 among the multiple lesions 42 the dimension 112 in FIG102 is the dimension 112 of. The dimension line 126 is a mark that can specify which part of the segmented image 44 the dimension 112 corresponds to. The dimension line 126 is produced and displayed by the control unit 82D, for example, based on the position-specific information 98 obtained from the recognition unit 82A. The production of the dimension line 126 can be carried out, for example, according to the same method as the production of the line segment 120 (that is, the same method as the case of using the circumscribed rectangular frame 122). In this embodiment, the dimension line 126 is an example of "region-specific information" involved in the technology of the present invention.

[0136] Next, refer to Figure 9 The operation of the portion of the endoscope system 10 according to the technology of the present invention will be described. Figure 9 The illustrated flow of medical support processing is an example of a “medical support method” according to the technology of the present invention.

[0137] exist Figure 9 In the medical support process shown, first, in step ST10, the recognition unit 82A determines whether one frame of imaging has been performed by the imaging device 52 within the large intestine 22. If, in step ST10, one frame of imaging has not been performed by the imaging device 52 within the large intestine 22, the determination is negative, and the determination of step ST10 is repeated. If, in step ST10, one frame of imaging has been performed by the imaging device 52 within the large intestine 22, the determination is positive, and the medical support process proceeds to step ST12.

[0138] In step ST12, the recognition unit 82A and the control unit 82D acquire a single frame of the endoscopic image 40 (see FIG. 1 ) obtained by imaging the large intestine 22 using the imaging device 52. Figure 5 ) In addition, for the sake of convenience, the description here is based on the premise that a plurality of lesions 42 are reflected in the endoscopic image 40. After the processing of step ST12 is executed, the medical support process proceeds to step ST14.

[0139] In step ST14, the control unit 82D displays the endoscopic image 40 acquired in step ST12 in the first display area 36 (see FIG. Figure 1 、 Figure 5 and Figure 8 ). After executing the process of step ST14, the medical support process proceeds to step ST16.

[0140] In step ST16, the recognition unit 82A performs recognition processing 96 using the endoscopic image 40 acquired in step ST12 to recognize the positions and types of the plurality of lesions 42 in the endoscopic image 40, and acquires position identification information 98 and type information 100 (see FIG. Figure 5 ). After executing the process of step ST16, the medical support process proceeds to step ST18.

[0141] In step ST18, the determination unit 82B determines the priority 104 (see FIG. 1 ) for the plurality of lesions 42 reflected in the endoscopic image 40 acquired in step ST12 based on the position identification information 98 and the type information 100 acquired by the recognition unit 82A in step ST16. Figure 6 ). After executing the process of step ST18, the medical support process proceeds to step ST20.

[0142] In step ST20, the measuring unit 82C measures the sizes 112 (see FIG. 112 ) of the plurality of lesions 42 that appear in the endoscopic image 40 acquired in step ST12. Figure 7 After executing the process of step ST20, the medical support process proceeds to step ST22.

[0143] In step ST22, the control unit 82D displays the plurality of sizes 112 (see FIG. 112 ) measured by the measuring unit 82C in step ST20 for the plurality of lesions 42 in the second display area 38 based on the priority 104 determined in step ST18. Figure 8 ). After executing the process of step ST22, the medical support process proceeds to step ST24.

[0144] In step ST24, the control unit 82D determines whether a condition for terminating the medical support process is satisfied. An example of a condition for terminating the medical support process is a condition in which an instruction to terminate the medical support process is given to the endoscope system 10 (for example, a condition in which the instruction to terminate the medical support process is received by the receiving device 64).

[0145] In step ST24, if the conditions for ending the medical support process are not met, the determination is negative, and the medical support process proceeds to step ST10. In step ST24, if the conditions for ending the medical support process are met, the determination is positive, and the medical support process ends.

[0146] As described above, in the endoscope system 10 of this embodiment, the recognition unit 82A recognizes the positions of multiple lesions 42 within the endoscopic image 40 based on the endoscopic image 40 showing multiple lesions 42. Furthermore, the determination unit 82B determines the priority 104 of the multiple lesions 42 based on the positions of the multiple lesions 42 within the endoscopic image 40. Furthermore, the measurement unit 82C measures the sizes 112 of the multiple lesions 42. Furthermore, the control unit 82D displays the sizes 112 on the screen 35 based on the priority 104. In this embodiment, as an example, the closer the position of the lesion 42 is to the center of the endoscopic image 40, the higher the priority 104. Therefore, if the position of the lesion 42 within the endoscopic image 40 is closer to the center of the endoscopic image 40 and the doctor 16 is paying more attention to it, the doctor 16 can understand the size 112 of the lesion 42 expected to be of high interest to the doctor 16 among the multiple lesions 42 shown in the endoscopic image 40.

[0147] Furthermore, in the endoscope system 10 of the present embodiment, the size 112 of each lesion 42 is sequentially displayed on the screen 35 according to the priority 104. Therefore, the doctor 16 can sequentially grasp the size 112 of the lesion 42 from the size 112 of the lesion 42 expected to be of high interest to the doctor 16, to the size 112 of the lesion 42 expected to be of low interest to the doctor 16, among the plurality of lesions 42 shown in the endoscopic image 40.

[0148] Furthermore, in the endoscope system 10 of this embodiment, the size 112 is displayed on the screen 35. Therefore, when a plurality of lesions 42 appear in the endoscopic image 40, the doctor 16 can visually recognize the size 112 of the lesion 42 expected to be of high interest to the doctor 16.

[0149] Furthermore, in the endoscope system 10 of the present embodiment, an endoscopic image 40 is displayed in the first display area 36 on the screen 35. Furthermore, a diagram 102 is displayed in the second display area 38 on the screen 35, and a dimension line 126 is displayed on the diagram 102 as information that can identify the lesion 42 corresponding to the dimension 112 displayed in the second display area 38. Therefore, the doctor 16 can visually recognize from the diagram 102 which lesion 42 among the multiple lesions 42 reflected in the endoscopic image 40 the dimension 112 displayed on the screen 35 represents.

[0150] Furthermore, in the endoscope system 10 of the present embodiment, the sizes 112 displayed on the screen 35 are switched according to the priority order 104. Therefore, when a plurality of lesions 42 are shown in the endoscopic image 40, the doctor 16 can visually recognize the sizes 112 of the lesions 42 in order from the sizes 112 of the lesions 42 expected to be of high interest to the doctor 16 to the sizes 112 of the lesions 42 expected to be of low interest to the doctor 16.

[0151] Furthermore, in the endoscope system 10 of this embodiment, the closer the position of the lesion 42 is to the center of the endoscopic image 40, the higher the priority 104. Therefore, if the doctor 16 is paying close attention to the central portion of the endoscopic image 40, the doctor 16 can understand the size 112 of the lesion 42 that the doctor 16 is paying close attention to.

[0152] Furthermore, in the endoscope system 10 of this embodiment, the recognition unit 82A recognizes the positions and types of the multiple lesions 42 within the endoscopic image 40 based on the endoscopic image 40 showing the multiple lesions 42. Furthermore, the determination unit 82B determines the priority 104 of the multiple lesions 42 based on the positions and types of the multiple lesions 42 within the endoscopic image 40. Furthermore, the control unit 82D displays the size 112 on the screen 35 based on the priority 104. In this embodiment, as an example, the closer the position of the lesion 42 is to the center of the endoscopic image 40, the higher the priority 104, and the more severe the type of lesion 42, the higher the priority 104. Therefore, when the types of lesions 42 that are of high concern to the doctor 16 and the types of lesions 42 that are of low concern to the doctor 16 are mixed among the multiple lesions 42 reflected in the endoscopic image 40, the doctor 16 can grasp the size 112 of the types of lesions 42 that the doctor 16 expresses high concern and the size 112 of the types of lesions 42 that the doctor 16 does not express high concern.

[0153] Furthermore, in the endoscope system 10 of this embodiment, the measurement unit 82C measures the sizes 112 of the plurality of lesions 42 according to the priority order 104. Therefore, the sizes 112 of the lesions 42 that are of high interest to the doctor 16 among the plurality of lesions 42 appearing in the endoscopic image 40 can be measured with priority. Consequently, the sizes 112 of the lesions 42 that are of high interest to the doctor 16 can be quickly presented to the doctor 16.

[0154] Furthermore, in the above-described embodiment, a case where the position weight 106 increases as the position is closer to the center of the endoscopic image 40 is described, but the technology of the present invention is not limited thereto. For example, the position weight 106 may be increased as the position is closer to a designated position within the endoscopic image 40 (for example, an area designated by the doctor 16 as the area the doctor 16 is looking at). In addition, the position weight 106 may be decreased as the position is closer to a designated position within the endoscopic image 40 (for example, an area within the endoscopic image 40 that is determined to be affected by the optical influence (for example, deformation) of the lens of the imaging device 52 (for example, the edge of the endoscopic image 40)). The designated position within the endoscopic image 40 may be a position that is predetermined and fixed based on various conditions, or a position that is changed based on various conditions and / or given instructions.

[0155] In the above embodiment, the size 112 of a plurality of lesions 42 is sequentially displayed on the screen 35 according to the priority 104, but the technology of the present invention is not limited thereto. Figure 10 As shown, the size 112 of each of the plurality of lesions 42 is shown on the screen 35 (in Figure 10 In the example shown, the display (in the second display area 38 within the screen 35) may also be performed each time an instruction 128 is given to the endoscope 12. An example of the instruction 128 is an instruction given by the doctor 16. For example, the instruction 128 is received by the receiving device 64, and the control unit 82D switches the display of the sizes 112 of the plurality of lesions 42 according to the priority 104 each time the receiving device 64 receives the instruction 128. Thus, when a plurality of lesions 42 are shown in the endoscopic image 40, the doctor 16 can grasp the sizes 112 of the lesions 42 in order from lesions 42 that are expected to be of high interest to the doctor 16 to lesions 42 that are expected to be of low interest, at the timing intended by the doctor 16.

[0156] In the above embodiment, the priority 104 is determined based on the location and type of the lesion 42, but the technology of the present invention is not limited thereto. For example, the priority 104 may be determined based on the location of the lesion 42 without considering the type of the lesion 42. In addition, as an example, Figure 11 and Figure 12As shown, the priority 104 may be determined based on information other than the position identification information 98 and the category information 100 .

[0157] exist Figure 11 In the example shown, the determination unit 82B determines the priority 104 based on the position identification information 98 and the confidence 130. The confidence 130 is an index indicating the likelihood of the position and type of each of the plurality of lesions 42 and is used for the recognition model 92 to recognize the position and type of the lesion 42. The confidence 130 is used in the recognition process 96 (see Figure 5 ) When identifying the positions and types of multiple lesions 42, the multiple lesions 42 are obtained from the identification model 92. Here, the confidence level 130 is exemplified as an indicator indicating the likelihood of the positions and types of each of the multiple lesions 42. However, if the confidence level 130 is an indicator indicating the likelihood of the positions of each of the multiple lesions 42, the technology of the present invention is established.

[0158] Determination unit 82B derives confidence weight 132 based on confidence 130. The greater the confidence 130, the greater the confidence weight 132. Here, for example, when the value of confidence weight 132 is set to "z", confidence weight 132 is a value determined within the range of "0 ≤ z ≤ 0.5". As a first example of a method for deriving confidence weight 132 from confidence 130, a method using a calculation expression with confidence 130 as a dependent variable and confidence weight 132 as an independent variable can be cited. As a second example of a method for deriving confidence weight 132 from confidence 130, a method using a table with confidence 130 as input and confidence weight 132 as output can be cited.

[0159] The determination unit 82B calculates the total weight 134 based on the position weight 106 and the confidence weight 132 in the same manner as the total weight 110 in the above embodiment. The determination unit 82B also determines the priority 104 based on the total weight 134 in the same manner as the above embodiment, and assigns the determined priority 104 to the plurality of lesions 42. Figure 11 In the illustrated example, the confidence level 130 influences the determination of the priority 104 , thereby enabling the priority 104 to be determined with a high degree of accuracy.

[0160] exist Figure 11 In the example shown, confidence weight 132 is shown, but as an example, Figure 12 As shown, a depth weight 136 may be applied instead of the confidence weight 132 .

[0161] exist Figure 12In the example shown, the determination unit 82B determines the priority 104 based on the position identification information 98 and the depth weight 136. The depth weight 136 is a value determined based on the depth from the observation position in the depth direction (hereinafter also referred to as "depth"). For example, the depth weight 136 is based on the distance image 116 (refer to Figure 7 ) is derived from the distance information 114 extracted from the distance image 116. The depth weight 136 may be the distance itself represented by the distance information 114 extracted from the distance image 116, or may be a numerical value obtained by dividing the distance represented by the distance information 114 extracted from the distance image 116 into several to several hundred stages. The smaller the depth, the larger the depth weight 136. Furthermore, without limitation to this, the greater the depth, the greater the depth weight 136, the greater the depth weight 136 as the depth decreases, or the greater the depth weight 136 as the depth increases, etc., may be determined based on instructions given by the doctor 16 via the receiving device 64.

[0162] The determination unit 82B calculates the total weight 138 based on the position weight 106 and the depth weight 136 in the same manner as the total weight 110 in the above embodiment. The determination unit 82B also determines the priority 104 based on the total weight 138 and assigns the determined priority 104 to the plurality of lesions 42 in the same manner as the above embodiment. Figure 12 In the example shown, depth affects the determination of the priority 104. Therefore, among the multiple lesions 42 reflected in the endoscopic image 40, the lesion 42 located at a smaller depth is the lesion 42 that is of higher concern to the doctor 16, and the doctor 16 can prioritize the size 112 of the lesion 42 that the doctor 16 is highly concerned about.

[0163] In the above embodiment, the dimension line 126 is displayed corresponding to the segmented image 44 as an example of a method for specifying the information of the lesion 42 corresponding to the dimension 112 displayed in the image 102, but the technology of the present invention is not limited to this. Figure 13 As shown, a circumscribed rectangular frame 140 may be displayed in FIG102 relative to the segmented image 44 that can identify the position of the lesion 42 corresponding to the size 112 shown in FIG102 within the endoscopic image 40. In this case, the dimension line 126 may also be displayed in FIG102 along with the circumscribed rectangular frame 140. Furthermore, when the position of the lesion 42 is identified using AI using a bounding box method, a bounding box may be used as the circumscribed rectangular frame 140. Furthermore, the circumscribed rectangular frame 140 is an example of "region-specific information" involved in the technology of the present invention.

[0164] In the above embodiment, the example of displaying the size 112 in the image 102 in order of priority 104 is given, but this is only an example. Figure 14 As shown, the size 112 can also be displayed outside the image 102 (ie, outside the second display area 38). Figure 14 In the example shown, a dimension 112 is displayed in a pop-up manner from inside the diagram 102 to outside the diagram 102. Figure 14 In the example shown, balloons appear from each of the plurality of segmented images 44, and each balloon includes a size 112 and text information 142 that can identify the priority 104. Thus, the doctor 16 can simultaneously view the size 112 and priority 104 of each of the plurality of lesions 42 on the screen 35. Figure 14 In the illustrated example, text information 142 is shown, but this is merely an example, and information (for example, an image or a symbol) that allows the doctor 16 to visually recognize the priority 104 may also be used.

[0165] exist Figure 15 In the example shown, the size 112 and the text information 142 are displayed in a pop-up manner from each of the plurality of segment images 44, but this is only an example. Figure 15 As shown, the sizes 112 and text information 142 may be displayed as pop-ups from the plurality of lesions 42 appearing in the endoscopic image 40 in the same manner as the case where the sizes 112 and text information 142 are displayed as pop-ups from the plurality of segmented images 44 .

[0166] In addition, Figure 15 In the example shown, size 112 and text information 142 are displayed in a display size corresponding to priority 104. For example, the higher the priority 104, the larger the display size of size 112 and text information 142. This is merely an example; size 112 and text information 142 may be displayed in a manner that makes them more prominent as priority 104 increases.

[0167] In this way, by displaying the size 112 on the screen 35 in a display manner corresponding to the priority 104, the doctor 16 can visually identify the size 112 of the lesion 42 that is expected to be of high concern to the doctor 16 and the size 112 of the lesion 42 that is expected to be of low concern to the doctor 16 when multiple lesions 42 appear in the endoscopic image 40.

[0168] exist Figure 14 and Figure 15The example shown uses a pop-up display using a balloon, but this is merely an example. The size 112 may be displayed on the screen 35 in a manner that allows identification of the lesion 42 to which the size 112 relates (e.g., by connecting the segmented image 44 in the image 102 or the lesion 42 in the endoscopic image 40 with the size 112 by a line). Similarly, the size 112 may be displayed on the screen 35 in a manner that allows identification of the lesion 42 to which the text information 142 relates.

[0169] In the above embodiment, the size 112 of each of the plurality of lesions 42 is shown in the diagram 102, but the technology of the present invention is not limited thereto. Figure 16 As shown, the size 112 may be displayed in the endoscopic image 40. This allows the doctor 16 to visually recognize the endoscopic image 40 and the sizes 112 of the plurality of lesions 42 together.

[0170] The size 112 may be displayed in an alpha-blended display format. Alternatively, the size 112 may be displayed in a display format that allows the priority 104 to be specified (eg, font size, font color, and / or transparency).

[0171] In addition, as an example, Figure 16 As shown, a circumscribed rectangular frame 144 corresponding to the lesion 42 corresponding to the size 112 displayed in the endoscopic image 40 may also be displayed in the endoscopic image 40. Regarding the circumscribed rectangular frame 144, for example, when the position of the lesion 42 is identified by AI using a bounding box method, a bounding box may be used as the circumscribed rectangular frame 144.

[0172] In addition, Figure 16 In the example shown, each time the display of the size 112 is switched according to the priority 104, the display of the circumscribed rectangular frame 144 is switched to specifically display which lesion 42 corresponds to the size 112 within the endoscopic image 40. The circumscribed rectangular frame 144 is an example of "region-specific information" according to the present invention.

[0173] In this way, by displaying the circumscribed rectangular frame 144 surrounding the lesion 42 corresponding to the size 112 displayed in the endoscopic image 40 in the endoscopic image 40, the doctor 16 can easily visually identify which lesion 42 among the multiple lesions 42 projected on the endoscopic image 40 the size 112 displayed in the endoscopic image 40 is the size 112.

[0174] exist Figure 16In the example shown, the display of the size 112 and the circumscribed rectangular frame 144 in the endoscopic image 40 is switched according to the priority 104. However, the sizes 112 of each of the plurality of lesions 42 may be collectively displayed in the endoscopic image 40. In this case, by assigning dimension lines to each of the plurality of lesions 42, it is possible to identify which lesion 42 the displayed plurality of dimensions 112 are. In addition, the display method of the circumscribed rectangular frame 144, such as the line type, color, and / or brightness, may be changed according to the priority 104.

[0175] In the above examples, the size 112 is displayed on the screen 35. However, this is merely an example, and the size 112 may be displayed on the screen 35 and / or at least one screen other than the screen 35. Furthermore, correspondingly, information that can identify the priority 104 may be displayed on the screen 35 and / or at least one screen other than the screen 35, and information that can identify the lesion 42 corresponding to the displayed size 112 (e.g., a size line and / or a circumscribed rectangular frame) may also be displayed.

[0176] In the above examples, examples are given of methods of determining the priority 104 based on two pieces of information: location-specific information 98 and information other than location-specific information 98. However, the priority 104 may also be determined based on three or more pieces of information including the location-specific information 98 (for example, three or more pieces of information among location-specific information 98, category information 100, confidence 130, and depth).

[0177] Alternatively, the priority 104 may be determined based on one or more pieces of information other than the position specifying information 98 (for example, one or more pieces of information among the category information 100 , the confidence level 130 , and the depth).

[0178] In the above embodiment, an example of a method of measuring the dimension 112 in units of one frame is given, but this is just an example. The statistical value of the dimension 112 (for example, the average value, the median value or the mode, etc.) measured based on multiple frames of endoscopic images 40 along a time series can also be displayed in the same display method as the above embodiment.

[0179] For example, the size 112 can also be measured when the displacement of the position of the lesion 42 between multiple frames is lower than a threshold value, and the measured size 112 itself or the statistical value of the size 112 measured based on the endoscopic images 40 of multiple frames along the time series can be displayed on the screen 35.

[0180] In the above embodiment, the position of the lesion 42 is identified by AI-based segmentation for each endoscopic image 40. However, the technology of the present invention is not limited thereto. For example, the position of the lesion 42 may be identified by AI-based bounding box for each endoscopic image 40.

[0181] In this case, the amount of change in the bounding box is calculated by the processor 82 , and a determination of whether to measure the size 112 of the lesion 42 may be made based on the amount of change in the bounding box in the same manner as in the above embodiment.

[0182] For example, the amount of change in the bounding box refers to the amount of change in the position of the lesion 42. The amount of change in the position of the lesion 42 may be the amount of change in the position of the lesion 42 between adjacent endoscopic images 40 along the time series, or the amount of change in the position of the lesion 42 between three or more endoscopic images 40 along the time series (for example, a statistical value such as the average, median, mode, or maximum value of the amount of change between the three or more endoscopic images 40 along the time series). Alternatively, the amount of change in the position of the lesion 42 between multiple frames along the time series at intervals of one or more frames may be used.

[0183] In the above embodiment, AI-based object recognition processing is exemplified as the recognition processing 96, but the technology of the present invention is not limited to this. The recognition unit 82A can also recognize the lesion 42 reflected in the endoscopic image 40 by performing non-AI-based object recognition processing (for example, template matching, etc.).

[0184] In the above embodiment, the display device 14 is shown as an example of the output address of the size 112, but the technology of the present invention is not limited to this, and the output address of the size 112 may also be a device other than the display device 14. As an example, Figure 17 As shown, as the output address of the size 112, a voice playback device 146, a printer 148, and / or an electronic medical record management device 150 can be cited.

[0185] The dimensions 112 may be output as audio via the audio playback device 146. Alternatively, the dimensions 112 may be printed as text on a medium (e.g., paper) via the printer 148. Alternatively, the dimensions 112 may be stored in the electronic medical record 152 managed by the electronic medical record management device 150.

[0186] While the above embodiment illustrates an example of using equation 124 to calculate size 112, the technology of the present invention is not limited to this. Size 112 can also be measured by processing endoscopic image 40 using AI. In this case, for example, a learned model can be used that outputs size 112 of lesion 42 when inputting endoscopic image 40 containing lesion 42. To create the learned model, deep learning can be performed on a neural network using supervised data, which includes annotations indicating lesion size, as correct answer data for lesions appearing in the image used as example data.

[0187] In the above embodiment, an example of deriving distance information 114 using distance derivation model 94 is described, but the technology of the present invention is not limited thereto. For example, other methods of deriving distance information 114 using AI include methods that combine segmentation and depth estimation (e.g., regression learning that assigns distance information 114 to the entire image (e.g., all pixels constituting the image), or unsupervised learning that uses unsupervised learning to determine the distance of the entire image).

[0188] In the above embodiment, an example of deriving the distance from the imaging device 52 to the intestinal wall 24 by AI is given, but the distance from the imaging device 52 to the intestinal wall 24 may also be measured. In this case, for example, the distal end portion 50 (see Figure 2 ) A distance measuring sensor is set up to measure the distance from the camera device 52 to the intestinal wall 24.

[0189] In the above embodiment, an endoscopic image 40 is illustrated as an example, but the technology of the present invention is not limited to this. The technology of the present invention is also applicable to medical images other than the endoscopic image 40 (for example, images obtained through modalities other than the endoscope 12, such as radiographic images or ultrasonic images).

[0190] In the above embodiment, an example of measuring the size 112 of the lesion 42 shown in a moving image is given. However, this is merely an example, and the technology of the present invention is also applicable to frame-by-frame images or still images showing the lesion 42 .

[0191] In the above embodiment, an example is given in which the distance information 114 extracted from the distance image 116 is input into the calculation formula 124 , but the technology of the present invention is not limited to this.

[0192] For example, instead of generating the distance image 116 , the distance information 114 corresponding to the position specified by the position specifying information 98 may be extracted from all the distance information 114 output from the distance derivation model 94 , and the extracted distance information 114 may be input into the calculation formula 124 .

[0193] In the above embodiment, the medical support process is performed by the processor 82 of the computer 78 included in the endoscope 12. However, the technology of the present invention is not limited to this embodiment, and a device that performs the medical support process may be provided external to the endoscope 12. Examples of the device provided external to the endoscope 12 include at least one server and / or at least one personal computer that is communicatively connected to the endoscope 12. Furthermore, the medical support process may be distributed across multiple devices.

[0194] While the above embodiment illustrates an example in which the medical support program 90 is stored in the NVM 86, the present invention is not limited thereto. For example, the medical support program 90 may be stored in a portable computer-readable non-transitory storage medium such as an SSD or USB memory stick. The medical support program 90 stored in the non-transitory storage medium is installed in the computer 78 of the endoscope 12. The processor 82 executes medical support processing in accordance with the medical support program 90.

[0195] Alternatively, the medical support program 90 may be stored in advance in a storage device such as another computer or server connected to the endoscope 12 via a network, downloaded in response to a request from the endoscope 12 , and installed in the computer 78 .

[0196] Furthermore, it is not necessary to store all of the medical support program 90 in a storage device such as another computer or server device connected to the endoscope 12 , or it is not necessary to store all of the medical support program 90 in the NVM 86 , but a portion of the medical support program 90 may be stored.

[0197] The various processors listed below can be used as hardware resources for executing medical support processes. Examples of processors include general-purpose processors (CPUs), which function as hardware resources for executing medical support processes by executing software, i.e., programs. Other examples of processors include processors with circuit structures specifically designed to execute specific processes, such as FPGAs, PLDs, and ASICs (specialized circuits). All processors have built-in or connected memory, and all processors execute medical support processes using memory.

[0198] The hardware resource that performs medical support processing can be composed of one of these various processors, or a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs medical support processing can be a single processor.

[0199] As examples of systems composed of a single processor, one approach involves combining one or more CPUs and software to form a single processor, which functions as a hardware resource for executing medical support processing. Another approach involves using a processor, typically a SoC, that implements the overall functionality of a system, including multiple hardware resources for executing medical support processing, on a single IC chip. In this manner, medical support processing is implemented using one or more of these various processors as hardware resources.

[0200] Furthermore, as the hardware configuration of these various processors, more specifically, a circuit formed by combining circuit elements such as semiconductor elements can be used. The above-mentioned medical support processing is merely an example.

[0201] Therefore, it is of course possible to delete unnecessary steps, add new steps, or change the processing order without departing from the scope of the present invention.

[0202] The above-mentioned records and illustrated contents are detailed descriptions of the parts involved in the technology of the present invention, and are merely examples of the technology of the present invention. For example, the descriptions related to the above-mentioned structure, function, action and effect are descriptions related to an example of the structure, function, action and effect of the parts involved in the technology of the present invention. Therefore, it is of course possible to delete unnecessary parts, add new elements, or replace the above-mentioned records and illustrated contents without departing from the scope of the main purpose of the technology of the present invention. In addition, in order to avoid complexity and make the parts involved in the technology of the present invention easy to understand, in the above-mentioned records and illustrated contents, on the basis of being able to implement the technology of the present invention, descriptions related to technical common sense that does not need to be particularly explained have been omitted.

[0203] In this specification, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" can mean only A, only B, or a combination of A and B. Furthermore, in this specification, when three or more items are linked together using "and / or," the same concept as "A and / or B" applies.

[0204] All documents, patent applications, and technical specifications described in this specification are incorporated herein by reference to the same extent as if each individual document, patent application, or technical specification was specifically and individually indicated to be incorporated by reference.

[0205] The following supplementary notes are further disclosed with respect to the above embodiment.

[0206] [Note 1]

[0207] A medical support device, wherein:

[0208] It has a processor,

[0209] The above processors

[0210] identifying positions of the plurality of observation target regions within the medical image based on the medical image in which the plurality of observation target regions are reflected,

[0211] Determine the priority of the plurality of observation target areas based on the positions,

[0212] The sizes of the plurality of observation target areas are measured according to the priority order.

[0213] [Note 2]

[0214] An endoscope, comprising:

[0215] The medical support device described in Supplementary Note 1; and

[0216] The module is inserted into a body including the observation target area and photographs the observation target area to obtain the medical image.

[0217] [Note 3]

[0218] A medical support method, comprising the following steps:

[0219] identifying positions of the plurality of observation target regions within the medical image based on the medical image showing the plurality of observation target regions;

[0220] Determining the priority of the plurality of observation target areas based on the positions; and

[0221] The sizes of the plurality of observation target areas are measured according to the priority order.

[0222] [Note 4]

[0223] A program for causing a computer to execute a medical support process, the medical support process comprising the following steps:

[0224] identifying positions of the plurality of observation target regions within the medical image based on the medical image showing the plurality of observation target regions;

[0225] Determining the priority of the plurality of observation target areas based on the positions; and

[0226] The sizes of the plurality of observation target areas are measured according to the priority order.

Claims

1. A medical support device, wherein: It has a processor, The processor identifying positions of the plurality of observation target regions within the medical image based on the medical image in which the plurality of observation target regions are reflected; determining the priority of the plurality of observation target areas based on the positions, measuring the sizes of the plurality of observation target areas, The sizes are output based on the priority.

2. The medical support device according to claim 1, wherein The processor sequentially outputs the size of each of the observation target areas according to the priority order.

3. The medical support device according to claim 2, wherein: Outputting the size of each of the observation target areas is performed every time an instruction is given.

4. The medical support device according to claim 1, wherein Outputting the size is achieved by displaying the size on a screen.

5. The medical support device according to claim 4, wherein In the screen, the size is displayed in a display mode corresponding to the priority order.

6. The medical support device according to claim 4, wherein displaying the medical image on the screen, The size is displayed within the medical image.

7. The medical support device according to claim 4, wherein The medical image is displayed on the screen, and region specifying information capable of specifying the observation target region corresponding to the output size is displayed within the medical image.

8. The medical support device according to claim 4, wherein The screen includes a first display area and a second display area. In the first display area, the medical image is displayed. In the second display area, a graph showing the distribution of the positions of each of the observation target areas is displayed, and area specifying information capable of specifying the observation target area corresponding to the output size is displayed within the graph.

9. The medical support device according to claim 4, wherein The sizes displayed on the screen are switched according to the priority order.

10. The medical support device according to claim 9, wherein The size displayed in the screen is switched every time an instruction is given.

11. The medical support device according to claim 1, wherein The location is identified using AI, The priority is determined based on the confidence obtained from the AI.

12. The medical support device according to claim 1, wherein The closer the position is to the center of the medical image, the higher the priority.

13. The medical support device according to claim 1, wherein The processor obtains the depths of the plurality of observation object areas, The priority is determined based on the position and the depth.

14. The medical support device according to claim 1, wherein the processor identifying the type of the observation target region based on the medical image, The priority is determined based on the position and the category.

15. The medical support device according to claim 1, wherein The processor determines the size according to the priority order.

16. The medical support device according to claim 1, wherein The medical image is an endoscopic image obtained by taking an image using an endoscope.

17. The medical support device according to claim 1, wherein The observation target region is a lesion.

18. An endoscope, wherein: It has: The medical support device according to any one of claims 1 to 17; and The module is inserted into a body including the observation target area and photographs the observation target area to acquire the medical image.

19. A medical support method, wherein: It includes the following steps: identifying positions of the plurality of observation target regions within the medical image based on the medical image in which the plurality of observation target regions are reflected; determining priorities of the plurality of observation target areas based on the positions; measuring the sizes of the plurality of observation target areas; as well as The sizes are output based on the priority.

20. A program, wherein It is used to enable a computer to perform a medical support process, the medical support process comprising the following steps: identifying positions of the plurality of observation target regions within the medical image based on the medical image in which the plurality of observation target regions are reflected; determining priorities of the plurality of observation target areas based on the positions; measuring the sizes of the plurality of observation target areas; as well as The sizes are output based on the priority.

Images