Medical support device, endoscope apparatus, medical support method, and program

Abstract

To provide a medical support device, an endoscope apparatus, a medical support method, and a program capable of allowing a user or the like to accurately recognize the position of a lumen appearing in a medical image in the medical image.SOLUTION: A medical support method includes: causing a trained model to generate certainty information in which a certainty of a lumen being present in each of a plurality of divided regions obtained by dividing a medical image or an image corresponding to the medical image in a circumferential direction is given to the plurality of divided regions; outputting first information indicating that the lumen is present in any of the plurality of divided regions on the basis of the certainty information; and outputting second information indicating that the lumen is present in the central region of the medical image in a case where the certainty information is information in which a value exceeding a threshold value is given as the certainty to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions.SELECTED DRAWING: Figure 16

Description

Technical Field

[0001] The present disclosure relates to a medical support device, an endoscope device, a medical support method, and a program.

Background Art

[0002] Patent Document 1 discloses an endoscope insertion direction detection method including: a first step of inputting an endoscope image; a second step of detecting a direction of light and dark change in the endoscope image; and a third step of generating information regarding the insertion direction of the endoscope based on the detection result. Patent Document 1 also discloses an endoscope insertion direction detection method including: a first step of setting an insertion candidate direction as a candidate for the insertion direction of the endoscope; a second step of inputting an endoscope image; a third step of detecting a direction of light and dark change in the endoscope image; a fourth step of evaluating the similarity between a plurality of insertion candidate directions and the direction of light and dark change; and a fifth step of determining the insertion direction of the endoscope based on the evaluation result.

[0003] Patent Document 2 discloses a movement support system including: a plurality of operation information calculation units that calculate a plurality of operation information indicating a plurality of operations that are different in time and correspond to a plurality of operation target scenes that are scenes requiring a plurality of operations that are different in time based on a captured image acquired by an imaging unit disposed in an insertion unit; and a presentation information generation unit that generates presentation information for the insertion unit based on the plurality of operation information calculated by the plurality of operation information calculation units.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

[0005] One embodiment according to the present disclosure provides a medical support device, an endoscope device, a medical support method, and a program that can accurately grasp the position of a lumen shown in a medical image for a user or the like within the medical image.

Means for Solving the Problems

[0006] A first aspect according to the present disclosure includes a processor. By inputting a medical image generated by imaging the inside of a luminal organ including the lumen to a learned model, the processor generates confidence information in which a confidence level indicating the presence of a lumen is assigned to each of a plurality of divided regions obtained by dividing the medical image or an image corresponding to the medical image along the circumferential direction. The output process is performed to distinguish between a case where the confidence information is not information in which values exceeding a threshold are assigned as confidence levels to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, and a case where the confidence information is information in which values exceeding a threshold are assigned as confidence levels to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions. This is a medical support device.

[0007] A second aspect according to the present disclosure is such that when the confidence information is not information in which values exceeding a threshold are assigned as confidence levels to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, the output process outputs first information indicating the presence of a lumen in any of the plurality of divided regions based on the confidence information. When the confidence information is information in which values exceeding a threshold are assigned as confidence levels to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, this is a medical support device according to the first aspect, including a process of not outputting the first information.

[0008] A third aspect according to the present disclosure is such that when the confidence information is information in which values exceeding a threshold are assigned as confidence levels to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, a process is performed to output second information indicating the presence of a lumen in the central region of the medical image or not output the second information. This is a medical support device according to the first aspect or the second aspect.

[0009] A fourth aspect according to the present disclosure is a medical support device according to the third aspect, in which output processing performs processing that does not output second information when the confidence information is not information in which values exceeding a threshold are given as confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among a plurality of divided regions.

[0010] A fifth aspect according to the present disclosure is a medical support device according to the third aspect, in which output processing includes processing that outputs second information when the confidence information is information in which values are given to two or more divided regions that are equally spaced along the circumference around the medical image or the center of the image among a plurality of divided regions and have a positional relationship exceeding 90 degrees in the circumferential direction.

[0011] A sixth aspect according to the present disclosure is a medical support device according to the third aspect, in which output processing includes processing that outputs second information when the confidence information is information in which values are given to two or more divided regions having a positional relationship of 120 degrees or more in the circumferential direction among a plurality of divided regions.

[0012] A seventh aspect according to the present disclosure is a medical support device according to the third aspect, in which output processing includes processing that outputs second information when the confidence information is information in which values are given to two or more divided regions arranged at regular intervals over the entire circumference of the medical image or the image among a plurality of divided regions.

[0013] An eighth aspect according to the present disclosure is a medical support device according to the third aspect, in which output processing includes processing that outputs second information when the confidence information is information in which values are given to all of a plurality of divided regions.

[0014] A ninth aspect according to the present disclosure is a medical support device according to any one of the second to eighth aspects, in which the processor outputs first information when the confidence information is information in which a value is given to a single divided region among a plurality of divided regions, and when the confidence information is information in which values are given to two or more divided regions having a positional relationship of 90 degrees or less in the circumferential direction among a plurality of divided regions.

[0015] The tenth aspect according to the present disclosure is a medical support device according to any one of the first to ninth aspects, wherein each of a plurality of divided regions is a region obtained by radially dividing a medical image.

[0016] The eleventh aspect according to the present disclosure is a medical support device according to the tenth aspect, wherein there are eight divided regions radially.

[0017] The twelfth aspect according to the present disclosure is a medical support device according to any one of the first to eleventh aspects, wherein the learned model is an example image showing a sample of a medical image, the example image being divided into a plurality of regions corresponding to a plurality of divided regions, and the correct data associated with the example image. The correct data when a sample of the lumen appears outside the central region of the example image is an annotation capable of specifying the position of the region where the sample of the lumen appears among the plurality of regions, and the correct data when a sample of the lumen appears in the central region of the example image is an annotation capable of specifying all positions of the plurality of regions.

[0018] The thirteenth aspect according to the present disclosure is that the output process outputs the first information indicating that there is a lumen in any of the plurality of divided regions based on the confidence information when the confidence information is not information in which values exceeding a threshold are given as confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions. When the confidence information is information in which values exceeding a threshold are given as confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, the second information indicating that there is a lumen in the central region of the medical image is displayed on the screen. A medical support device according to any one of the first to twelfth aspects, including a process of outputting the second information.

[0019] A fourteenth aspect according to the present disclosure is the medical support device according to the thirteenth aspect, in which a processor displays a medical image on a screen, displays first information in the medical image in the state displayed on the screen, and displays second information in the medical image in the state displayed on the screen.

[0020] A fifteenth aspect according to the present disclosure is that when the output process is such that the confidence information is not information in which values exceeding a threshold are given as confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among a plurality of divided regions, output first information indicating that there is a lumen in any of the plurality of divided regions based on the confidence information; and when the confidence information is information in which values exceeding a threshold are given as confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, output second information indicating that there is a lumen in the central region of the medical image. The medical support device according to any one of the first to fourteenth aspects, wherein the first information and the second information are visible information capable of visually specifying the position of the lumen shown in the medical image.

[0021] A sixteenth aspect according to the present disclosure is the medical support device according to any one of the first to fifteenth aspects, wherein the medical image is an endoscopic image generated by imaging the inside of a luminal organ including the lumen with an endoscope.

[0022] A seventeenth aspect according to the present disclosure is an endoscope device including the medical support device according to any one of the first to sixteenth aspects and an endoscope, wherein the medical image is generated by imaging the inside of a luminal organ including the lumen with the endoscope.

[0023] The 18th aspect according to the present disclosure includes: inputting a medical image generated by imaging the inside of a luminal organ including the lumen to a learned model, thereby generating confidence information in which a confidence level indicating the presence of a lumen is assigned to each of a plurality of divided regions obtained by dividing the medical image or an image corresponding to the medical image along the circumferential direction; and performing output processing capable of distinguishing between a case where the confidence information is not information in which a value exceeding a threshold is assigned as the confidence level to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, and a case where the confidence information is information in which a value exceeding a threshold is assigned as the confidence level to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions. This is a medical support method.

[0024] The 19th aspect according to the present disclosure is a program for causing a computer to execute a process including: inputting a medical image generated by imaging the inside of a luminal organ including the lumen to a learned model, thereby generating confidence information in which a confidence level indicating the presence of a lumen is assigned to each of a plurality of divided regions obtained by dividing the medical image or an image corresponding to the medical image along the circumferential direction; and performing output processing capable of distinguishing between a case where the confidence information is not information in which a value exceeding a threshold is assigned as the confidence level to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, and a case where the confidence information is information in which a value exceeding a threshold is assigned as the confidence level to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions.

Brief Description of Drawings

[0025]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Embodiments for Carrying Out the Invention

[0026] Hereinafter, an example of an embodiment of a medical support device, an endoscope device, a medical support method, and a program according to the present disclosure will be described with reference to the accompanying drawings.

[0027] First, the language used in the following description will be explained.

[0028] CPU refers to the abbreviation of "Central Processing Unit". GPU refers to the abbreviation of "Graphics Processing Unit". GPGPU refers to the abbreviation of "General-Purpose computing on Graphics Processing Units". APU refers to the abbreviation of "Accelerated Processing Unit". TPU refers to the abbreviation of "Tensor Processing Unit". RAM refers to the abbreviation of "Random Access Memory". EEPROM refers to the abbreviation of "Electrically Erasable Programmable Read-Only Memory". ASIC refers to the abbreviation of "Application Specific Integrated Circuit". PLD refers to the abbreviation of "Programmable Logic Device". FPGA refers to the abbreviation of "Field-Programmable Gate Array". SoC refers to the abbreviation of "System-on-a-chip". SSD refers to the abbreviation of "Solid State Drive". USB refers to the abbreviation of "Universal Serial Bus". HDD refers to the abbreviation of "Hard Disk Drive". EL refers to the abbreviation of "Electro-Luminescence". CMOS refers to the abbreviation of "Complementary Metal Oxide Semiconductor". CCD refers to the abbreviation of "Charge Coupled Device". AI refers to the abbreviation of "Artificial Intelligence". BLI refers to the abbreviation of "Blue Light Imaging". LCI refers to the abbreviation of "Linked Color Imaging". I / F refers to the abbreviation of "Interface". SSL refers to the abbreviation of "Sessile Serrated Lesion". LAN refers to the abbreviation of "Local Area Network". WAN refers to the abbreviation of "Wide Area Network".5G refers to the abbreviation of "5th Generation Mobile Communication System".

[0029] In the following description, a tagged processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU, a GPU, a GPGPU, an APU, or a TPU, etc.

[0030] In the following description, a tagged memory is a memory such as a RAM in which information is temporarily stored and is used as a work memory by the processor.

[0031] In the following description, a tagged storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory, magnetic disks, or magnetic tapes, etc. Also, another example of storage includes cloud storage.

[0032] In the following embodiments, a tagged external I / F manages the exchange of various information between a plurality of interconnected devices. An example of an external I / F includes a USB interface. A communication I / F including a communication processor and an antenna, etc. may be applied to the external I / F. The communication I / F manages communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G, Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0033] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0034] FIG. 1 is a conceptual diagram showing an example of an aspect in which the endoscope apparatus 10 is being used. As shown in FIG. 1, the endoscope apparatus 10 is used by a doctor 12 in an endoscopy or the like. The endoscopy is assisted by staff such as a nurse 14.

[0035] The endoscope apparatus 10 is communicably connected to a communication device (not shown), and the information obtained by the endoscope apparatus 10 is transmitted to the communication device. Examples of the communication device include a server that manages various information such as an electronic medical record, a personal computer, and / or a tablet terminal. The communication device receives the information transmitted from the endoscope apparatus 10 and executes processing using the received information (for example, processing for storing in an electronic medical record or the like).

[0036] The endoscope apparatus 10 includes an endoscope scope 16, a display device 18, a light source device 20, a control device 22, and a medical support device 24. In the present embodiment, the endoscope apparatus 10 is an example of the "endoscope apparatus" according to the present disclosure, and the medical support device 24 is an example of the "medical support device" according to the present disclosure.

[0037] The endoscope apparatus 10 is a modality for performing medical treatment on the large intestine 28, which is a luminal organ contained in the body of a subject 26 (for example, a patient) using the endoscope scope 16. In the present embodiment, the large intestine 28 is an object to be observed by the doctor 12.

[0038] The endoscope 16 is used by the doctor 12 and inserted into the body of the subject 26. In this embodiment, the endoscope 16 is inserted into the large intestine 28 of the subject 26. In this embodiment, the endoscope 16 is an example of the "endoscope" according to the present disclosure, and the large intestine 28 is an example of the "lumen organ" according to the present disclosure.

[0039] The endoscope apparatus 10 causes the endoscope 16 inserted into the large intestine 28 of the subject 26 to image the inside of the large intestine 28 including the lumen 42, and performs various medical procedures on the large intestine 28 as necessary. The large intestine 28 has a lumen 42. The endoscope 16 is inserted into the lumen 42. The position of the lumen 42 in the large intestine 28 can be medically specified based on the morphological pattern (for example, the shape and orientation of the plurality of folds 43, etc.) of the plurality of folds 43 that are characteristic regions in the large intestine 28. Although it will be described in detail later, in this embodiment, the position of the lumen 42 is recognized by an AI that has been machine-learned with various information such as the morphological pattern of the plurality of folds 43, and the recognition result is provided to the doctor 12 as information that can be visually grasped. In this embodiment, the lumen 42 is an example of the "lumen" according to the present disclosure.

[0040] The endoscope apparatus 10 acquires and outputs an image showing the state including the lumen 42 in the large intestine 28 by imaging the inside of the large intestine 28 including the lumen 42. In this embodiment, the endoscope apparatus 10 is an endoscope apparatus having an optical imaging function that images the reflected light obtained by irradiating light 30 in the large intestine 28 and reflected by the intestinal wall 32 of the large intestine 28.

[0041] Here, although an endoscopic examination of the large intestine 28 is exemplified, this is only an example, and the present disclosure is also applicable to endoscopic examinations of lumen organs such as the esophagus, stomach, duodenum, or trachea.

[0042] The light source device 20, the control device 22, and the medical support device 24 are installed in the wagon 34. The wagon 34 is provided with a plurality of platforms along the vertical direction, and the medical support device 24, the control device 22, and the light source device 20 are installed from the lower platform to the upper platform. Also, a display device 18 is installed on the uppermost platform of the wagon 34.

[0043] The control device 22 controls the entire endoscope device 10. The medical support device 24 performs various image processes on the image obtained by imaging the intestinal wall 32 with the endoscope scope 16 under the control of the control device 22.

[0044] The display device 18 displays various information including images. Examples of the display device 18 include a liquid crystal display or an EL display. Also, instead of or together with the display device 18, a tablet terminal with a display may be used.

[0045] A screen 35 is displayed on the display device 18. The screen 35 includes a plurality of display areas. The plurality of display areas are arranged side by side within the screen 35. In the example shown in FIG. 1, as an example of the plurality of display areas, a first display area 35A and a second display area 35B are shown. The size of the first display area 35A is larger than the size of the second display area 35B. The first display area 35A is used as the main display area, and the second display area 35B is used as the sub display area. The size relationship between the first display area 35A and the second display area 35B is not limited to this, and any size relationship that fits within the screen 35 may be used.

[0046] An endoscopic moving image 39 is displayed in the first display area 35A. The endoscopic moving image 39 is a moving image generated by imaging the inside of the large intestine 28 of the subject 26 with the endoscope scope 16. The intestinal wall 32 shown in the endoscopic moving image 39 includes a lumen 42 as an area of interest (i.e., an observation target area) to be gazed at by the doctor 12, and the doctor 12 can visually recognize the state of the intestinal wall 32 including the lumen 42 through the endoscopic moving image 39.

[0047] The image displayed in the first display area 35A is one frame 40 included in a moving image composed of a plurality of frames 40 along a time series. That is, in the first display area 35A, a plurality of frames 40 along a time series are displayed at a predetermined frame rate (for example, a dozen or so frames per second or several dozen frames per second).

[0048] As an example of the moving image displayed in the first display area 35A, a live view type moving image can be mentioned. The live view type is only an example, and it may be a moving image that is temporarily stored in a memory or the like and then displayed, such as a moving image of a post view type. Also, each frame included in the recorded moving image stored in a memory or the like may be reproduced and displayed on the screen 35 (for example, the first display area 35A) as the endoscope moving image 39.

[0049] The display position of the second display area 35B is at the lower right in the front view within the screen 35. The display position of the second display area 35B may be anywhere within the screen 35 of the display device 18, but it is preferably displayed at a position where it can be compared with the endoscope moving image 39. Auxiliary information 44 for assisting medical judgments and the like by the doctor 12 in the endoscope examination is displayed in the second display area 35B. The auxiliary information 44 is information referred to by the doctor 12. As an example of the auxiliary information 44, various information regarding the subject 26 into which the endoscope scope 16 is inserted into the body, and / or various information obtained by performing the medical support process described later, etc. can be mentioned.

[0050] FIG. 2 is a conceptual diagram showing an example of the overall configuration of the endoscope apparatus 10. As shown in FIG. 2, the endoscope scope 16 includes an operation unit 46 and an insertion unit 48. The insertion unit 48 is partially curved when the operation unit 46 is operated. The insertion unit 48 is inserted into the large intestine 28 while curving according to the shape of the large intestine 28 in accordance with the operation of the operation unit 46 by the doctor 12 (see FIG. 1).

[0051] At the distal end 50 of the insertion portion 48, a camera 52, an illumination device 54, and an opening 56 for a treatment instrument are provided. The camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end 50. Here, a configuration example in which the camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end 50 is given, but this is merely an example. The camera 52 and the illumination device 54 may be provided on the side surface of the distal end 50, so that the endoscope scope 16 may be configured as a side view mirror.

[0052] The camera 52 is mounted on the endoscope scope 16, and generates a frame 40 by being inserted into the body cavity of the subject 26 and imaging an observation target area. In the present embodiment, the camera 52 generates an endoscopic moving image 39 including a plurality of frames 40 along a time series by imaging the inside of the large intestine 28 including the lumen 42. An example of the camera 52 is a CMOS camera. However, this is merely an example, and other types of cameras such as a CCD camera may be used. In the present embodiment, the frame 40 is an example of the "medical image" and the "endoscopic image" according to the present disclosure.

[0053] The illumination device 54 has illumination windows 54A and 54B. The illumination device 54 irradiates light 30 (see FIG. 1) through the illumination windows 54A and 54B. Examples of the type of light 30 irradiated from the illumination device 54 include visible light (for example, white light, etc.) and non-visible light (for example, near-infrared light, etc.). Further, the illumination device 54 irradiates special light through the illumination windows 54A and 54B. Examples of the special light include light for BLI and / or light for LCI. The camera 52 optically images the inside of the large intestine 28 in a state where the light 30 is irradiated by the illumination device 54 inside the large intestine 28.

[0054] The opening 56 for a treatment instrument is an opening for protruding the treatment instrument 58 from the distal end 50. Further, the opening 56 for a treatment instrument is also used as a suction port for sucking blood, body dirt, etc., and a delivery port for delivering a fluid.

[0055] The operation unit 46 is formed with a treatment instrument insertion port 60, and the treatment instrument 58 is inserted into the insertion unit 48 through the treatment instrument insertion port 60. The treatment instrument 58 passes through the insertion unit 48 and protrudes outside from the treatment instrument opening 56. In the example shown in FIG. 2, as the treatment instrument 58, a mode in which a puncture needle protrudes from the treatment instrument opening 56 is shown. Here, the puncture needle is exemplified as the treatment instrument 58, but this is merely an example, and the treatment instrument 58 may be a grasping forceps, a papillotome knife, a snare, a catheter, a guide wire, a cannula, and / or a puncture needle with a guide sheath, etc.

[0056] The endoscope scope 16 is connected to the light source device 20 and the control device 22 via the universal cord 62. The medical support device 24 and the reception device 64 are connected to the control device 22. Further, the display device 18 is connected to the medical support device 24. That is, the control device 22 is connected to the display device 18 via the medical support device 24.

[0057] Here, since the medical support device 24 is exemplified in the position of an external device for expanding the functions performed by the control device 22, an example of a form in which the control device 22 and the display device 18 are indirectly connected via the medical support device 24 is given, but this is merely an example. For example, the display device 18 may be directly connected to the control device 22. In this case, for example, the functions of the medical support device 24 may be mounted on the control device 22, or the same processing as the processing executed by the medical support device 24 (for example, the medical support processing described later) may be executed on a server (not shown), and as long as the control device 22 is equipped with a function of receiving and using the processing result by the server.

[0058] The reception device 64 receives an instruction from the doctor 12 and outputs the received instruction to the control device 22 as an electrical signal. Examples of the reception device 64 include a keyboard, a mouse, a touch panel, a foot switch, a microphone, and / or a remote control device, etc.

[0059] The control device 22 controls the light source device 20, exchanges various signals with the camera 52, and exchanges various signals with the medical support device 24.

[0060] The light source device 20 emits light under the control of the control device 22 and supplies the light to the illumination device 54. The illumination device 54 incorporates a light guide, and the light supplied from the light source device 20 is irradiated from the illumination windows 54A and 54B via the light guide. The control device 22 causes the camera 52 to perform imaging, acquires the endoscopic moving image 39 (see FIG. 1) from the camera 52, and outputs it to a predetermined output destination (for example, the medical support device 24).

[0061] The medical support device 24 supports medical treatment (here, as an example, endoscopic examination) by performing various image processes on the endoscopic moving image 39 input from the control device 22. The medical support device 24 outputs the endoscopic moving image 39 subjected to various image processes to a predetermined output destination (for example, the display device 18).

[0062] Here, a configuration example in which the endoscopic moving image 39 output from the control device 22 is output to the display device 18 via the medical support device 24 has been described, but this is merely an example. For example, the control device 22 and the display device 18 may be connected, and the endoscopic moving image 39 subjected to image processing by the medical support device 24 may be displayed on the display device 18 via the control device 22.

[0063] FIG. 3 is a block diagram showing an example of the hardware configuration of the electrical system of the endoscope apparatus 10. As shown in FIG. 3, the control device 22 includes a computer 66, a bus 68, and an external I / F 70. The computer 66 includes a processor 72, a memory 74, and a storage 76. The processor 72, the memory 74, the storage 76, and the external I / F 70 are connected to the bus 68. The processor 72 controls the entire control device 22. The memory 74 and the storage 76 are used by the processor 72.

[0064] The external I / F 70 controls the exchange of various information between one or more devices existing outside the control device 22 (hereinafter also referred to as "first external devices") and the processor 72.

[0065] A camera 52 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the camera 52 and the processor 72. The processor 72 controls the camera 52 via the external I / F 70. Further, the processor 72 acquires, via the external I / F 70, the endoscopic moving image 39 (see FIG. 1) obtained by imaging the inside of the large intestine 28 (see FIG. 1) by the camera 52.

[0066] A light source device 20 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the light source device 20 and the processor 72. The light source device 20 supplies light to the illumination device 54 under the control of the processor 72. The illumination device 54 irradiates the light supplied from the light source device 20.

[0067] A reception device 64 is connected to the external I / F 70 as one of the first external devices, and the processor 72 acquires, via the external I / F 70, the instruction received by the reception device 64 and executes processing according to the acquired instruction.

[0068] The medical support device 24 includes a computer 78 and an external I / F 80. The computer 78 includes a processor 82, a memory 84, and a storage 86. The processor 82, the memory 84, the storage 86, and the external I / F 80 are connected to a bus 88. In the present embodiment, the computer 78 is an example of the "computer" according to the present disclosure, and the processor 82 is an example of the "processor" according to the present disclosure.

[0069] Note that since the hardware configuration of the computer 78 (that is, the processor 82, the memory 84, and the storage 86) is basically the same as the hardware configuration of the computer 66, the description of the hardware configuration of the computer 78 is omitted here.

[0070] The external I / F 80 controls the exchange of various information between one or more devices existing outside the medical support device 24 (hereinafter also referred to as "second external devices") and the processor 82.

[0071] The control device 22 is connected to the external I / F 80 as one of the second external devices. In the example shown in FIG. 3, the external I / F 70 of the control device 22 is connected to the external I / F 80. The external I / F 80 controls the exchange of various information between the processor 82 of the medical support device 24 and the processor 72 of the control device 22. For example, the processor 82 acquires the endoscopic moving image 39 (see FIG. 1) from the processor 72 of the control device 22 via the external I / Fs 70 and 80, and performs various image processes on the acquired endoscopic moving image 39.

[0072] The display device 18 is connected to the external I / F 80 as one of the second external devices. The processor 82 controls the display device 18 via the external I / F 80, thereby causing the display device 18 to display various information (for example, the endoscopic moving image 39 on which various image processes have been performed, etc.).

[0073] FIG. 4 is a block diagram showing an example of the main functions of the processor 82 included in the medical support device 24 and an example of the information stored in the storage 86. As shown in FIG. 4, a medical support program 90 is stored in the storage 86. The medical support program 90 is an example of the "program" according to the present disclosure. The processor 82 reads the medical support program 90 from the storage 86 and executes the read medical support program 90 on the memory 84 to perform medical support processing. The medical support processing is realized by operating as the recognition unit 82A and the control unit 82B according to the medical support program 90 executed by the processor 82 on the memory 84. In the present embodiment, the medical support processing is an example of the "output processing" according to the present disclosure.

[0074] The storage 86 stores a lumen recognition model 92. Although details will be described later, the lumen recognition model 92 is a learned model used in AI-based processing and is used by the recognition unit 82A. In the present embodiment, the lumen recognition model 92 is an example of the "learned model" according to the present disclosure.

[0075] FIG. 5 is a block diagram showing an example of the hardware configuration of the electrical system of the information processing apparatus 100 used for generating the lumen recognition model 92. As shown in FIG. 5, the information processing apparatus 100 includes a computer 102 and an external I / F 104. The computer 102 includes a processor 106, a memory 108, and a storage 110. The processor 106, the memory 108, the storage 110, and the external I / F 104 are connected to a bus 112.

[0076] Note that the hardware configuration of the computer 102 (that is, the processor 106, the memory 108, and the storage 110) is basically the same as the hardware configuration of the computer 66, and thus the description of the hardware configuration of the computer 102 is omitted here.

[0077] The information processing apparatus 100 includes a reception device 116. The reception device 116 is a keyboard and / or a mouse, etc., and receives instructions from the user of the information processing apparatus 100, etc. The reception device 116 is connected to the bus 112. The processor 106 acquires the instructions received by the reception device 116 and operates according to the acquired instructions.

[0078] The display device 118 displays various information including images. Examples of the display device 118 include a liquid crystal display or an EL display, etc. The display device 118 is connected to the bus 112. The processor 106 causes the display device 118 to display the results obtained by executing various processes.

[0079] The external I / F 104 controls the exchange of various information between one or more devices existing outside the information processing apparatus 100 (hereinafter, also referred to as "third external devices") and the processor 106. A medical support device 24 is connected to the external I / F 104 as one of the third external devices. In the example shown in FIG. 5, the external I / F 80 of the medical support device 24 is connected to the external I / F 104. The external I / F 104 controls the exchange of various information between the processor 82 (see FIGS. 3 and 4) of the medical support device 24 and the processor 106 of the information processing apparatus 100. For example, the information processing apparatus 100 generates a lumen recognition model 92 and transmits the generated lumen recognition model 92 to the medical support device 24 via the external I / Fs 80 and 104 in response to a request from the medical support device 24.

[0080] A machine learning processing program 120 is stored in the storage 110. The processor 106 reads the machine learning processing program 120 from the storage 110 and performs machine learning processing by executing the read machine learning processing program 120 on the memory 84. The machine learning processing is realized by the processor 106 operating as a teacher data generation unit 106A and a learning execution unit 106B according to the machine learning processing program 120 executed on the memory 108.

[0081] An example image set 122 is stored in the storage 110. Although details will be described later, the example image set 122 is used by the teacher data generation unit 106A.

[0082] FIG. 6 is a conceptual diagram showing an example of the processing content of the teacher data generation unit 106A. As shown in FIG. 6, the information processing apparatus 100 is used by an annotator 124. The annotator 124 refers to an operator who gives machine learning annotations to given data (that is, an operator who performs labeling).

[0083] In the example shown in FIG. 6, a keyboard 116A and a mouse 116B are shown as an example of the reception device 116. The annotator 124 gives instructions to the computer 102 via the keyboard 116A and the mouse 116B.

[0084] The example image set 122 includes a plurality of example images 122A with different depicted contents. The example image 122A is an image predefined as a medical image used for object recognition processing (for example, the process in which the recognition unit 82A recognizes the lumen 42 based on the frame 40 and the lumen recognition model 92). The image predefined as a medical image used for object recognition processing is an image corresponding to the frame 40. In other words, the image corresponding to the frame 40 can also be said to be an image assuming the frame 40. Further in other words, the image assuming the frame 40 can also be said to be an image showing a sample of the frame 40. Here, as a first example of the image showing a sample of the frame 40, an image obtained by actually imaging the inside of the large intestine with a camera can be cited. As a second example of the image showing a sample of the frame 40, a virtual image (for example, an image generated by a generative AI such as Stable Diffusion or Midjourney) can be cited.

[0085] The teacher data generation unit 106A acquires the example image 122A from the example image set 122 according to the instruction received by the reception device 116. The teacher data generation unit 106A displays the example image 122A on the screen 118A of the display device 118. With the example image 122A displayed on the screen 118A, the annotator 124 instructs the teacher data generation unit 106A, via the reception device 116, of the lumen corresponding position, which is the position of the lumen depicted in the example image 122A within the example image 122A. The teacher data generation unit 106A generates the teacher data 128 by associating the correct answer data 126 with the example image 122A based on the lumen corresponding position instructed via the reception device 116. Associating the correct answer data 126 with the example image 122A is realized by assigning, as the correct answer data 126, an annotation capable of specifying the lumen corresponding position to the lumen corresponding position within the example image 122A. Although it will be described in detail later, as an example of the lumen corresponding position, the lumen corresponding positions 139 shown in FIGS. 8 and 9, and the lumen corresponding positions 149 shown in FIGS. 12 and 13 can be cited.

[0086] In this way, the teacher data generation unit 106A repeatedly performs a process of associating correct answer data 126 with each of all the example images 122A included in the example image set 122 in accordance with the instructions given from the annotator 124, thereby generating a plurality of teacher data 128.

[0087] The teacher data 128 is roughly classified into teacher data 128A as a comparative example for the present disclosure and teacher data 128B to which the present disclosure is applied. The teacher data 128A is data obtained by associating correct answer data 126 with an example image 122A1 which is the first example of the example image 122A, and the teacher data 128B is data obtained by associating correct answer data 126 with an example image 122A2 which is the second example of the example image 122A. Here, the teacher data 128B is an example of the "teacher data" according to the present disclosure, and the teacher data 128A is a comparative example for the teacher data 128B. Also, the example image 122A2 is an example of the "example image" according to the present disclosure, and the example image 122A1 is a comparative example for the example image 122A2. Also, the correct answer data 126 is an example of the "correct answer data" according to the present disclosure.

[0088] FIG. 7 is a conceptual diagram showing an example of the configuration of the example image 122A1. As shown in FIG. 7, the inside of the large intestine 132 is depicted in the example image 122A1. In the example shown in FIG. 7, an intestinal wall 136 with a plurality of folds 134 formed and a lumen 138 are depicted in the example image 122A1. Here, the lumen 138 is an example of the "lumen sample" according to the present disclosure.

[0089] The example image 122A1 is divided into a plurality of divided regions 130A. The plurality of divided regions 130A includes a central region 130A1 and eight radial regions 130A2 to 130A9. The central region 130A1 is a circular region having a center that coincides with the center C1 of the example image 122A1. The radial regions 130A2 to 130A9 are regions that radially exist from the central region 130A1 toward the outer edge of the example image 122A1, and are arranged along the circumferential direction CD1 of the example image 122A1 (in other words, around the center C1 of the example image 122A1).

[0090] FIG. 8 is a conceptual diagram showing an example of a method for generating teacher data 128A by the teacher data generation unit 106A associating correct answer data 126 with the example image 122A1. As shown in FIG. 8, with the example image 122A1 displayed on the screen 118A, the annotator 124 indicates to the teacher data generation unit 106A, via the reception device 116, the lumen correspondence position 139 which is the position within the example image 122A1 of the lumen 138 shown in the example image 122A1. The teacher data generation unit 106A superimposes and displays a circular frame 140 on the example image 122A1 and arranges the frame 140 at a position surrounding the lumen 138 shown in the example image 122A1 according to the instruction received by the reception device 116. The frame 140 is a mark that defines the lumen correspondence position 139 within the example image 122A1. That is, the position of the area surrounded by the frame 140 within the example image 122A1 is the lumen correspondence position 139. The size and position of the frame 140 are freely changed within the screen 118A according to the instruction received by the reception device 116. Here, the shape of the frame 140 is circular, but it may be a shape other than circular. Also, the size of the frame 140 can be changed according to the instruction received by the reception device 116.

[0091] With the frame 140 arranged at a position surrounding the lumen 138, the annotator 124 gives, via the reception device 116, to the teacher data generation unit 106A a confirmation instruction which is an instruction to confirm the lumen correspondence position 139. Thereby, the teacher data generation unit 106A confirms the lumen correspondence position 139.

[0092] The teacher data generation unit 106A specifies the division area 130A having the largest overlapping area with the frame 140 that defines the lumen correspondence position 139 from among the plurality of division areas 130A. Then, the teacher data generation unit 106A generates the teacher data 128A by associating the correct answer data 126 as an annotation capable of specifying the division area 130A in which the lumen 138 is shown with the division area 130A (the radial area 130A3 in the example shown in FIG. 8) specified from among the plurality of division areas 130A.

[0093] In the example shown in FIG. 8, a mode in which the correct data 126 is associated with the radial region 130A3 is shown, but this is merely an example. For example, as shown in FIG. 9, among the plurality of divided regions 130A, if the divided region 130A having the largest area overlapping with the frame 140 is the central region 130A1, the teacher data generation unit 106A generates the teacher data 128A by associating the correct data 126 with the central region 130A1.

[0094] FIG. 10 shows a conceptual diagram illustrating an example of a mode in which the lumen recognition model 92 is generated by performing machine learning using the teacher data 128A by the learning execution unit 106B. As shown in FIG. 10, in the information processing apparatus 100, the learning execution unit 106B acquires the teacher data 128A generated by the teacher data generation unit 106A. Then, the learning execution unit 106B performs machine learning using the teacher data 128A.

[0095] In the example shown in FIG. 10, the learning execution unit 106B has a model 142. An example of the model 142 includes a neural network. An example of the neural network includes a convolutional neural network. The learning execution unit 106B inputs the example image 122A1 included in the teacher data 128A into the model 142. When the example image 122A1 is input, the model 142 makes an inference and outputs an inference result 144. The learning execution unit 106B calculates an error 146 between the inference result 144 and the correct data 126 included in the teacher data 128A.

[0096] The learning execution unit 106B calculates a plurality of adjustment values 148 that minimize the error 146. Then, the learning execution unit 106B optimizes the model 142 by adjusting a plurality of optimization variables in the model 142 using the plurality of adjustment values 148. Here, the plurality of optimization variables refer to, for example, a plurality of connection weights and a plurality of offset values included in the model 142.

[0097] The learning execution unit 106B repeatedly performs learning processes including inputting the example image 122A1 to the model 142, calculating the error 146, calculating a plurality of adjustment values 148, and adjusting a plurality of optimization variables in the model 142, using a plurality of teacher data 128A. That is, for each of the plurality of example images 122A1 included in the plurality of teacher data 128A, the learning execution unit 106B adjusts a plurality of optimization variables in the model 142 using the plurality of adjustment values 148 calculated so that the error 146 is minimized, thereby optimizing the model 142. By optimizing the model 142 in this way, the lumen recognition model 92 is generated. The lumen recognition model 92 is transmitted from the information processing device 100 to the medical support device 24 via the external I / Fs 80 and 104 (see FIG. 5) and received by the medical support device 24. Then, in the medical support device 24, the lumen recognition model 92 is stored in the storage 86 by the processor 82 (see FIG. 4). The lumen recognition model 92 stored in the storage 86 is used by the recognition unit 82A (see FIG. 4).

[0098] Incidentally, when the lumen recognition model 92 is actually used, a frame 40 is input to the lumen recognition model 92. Then, the lumen recognition model 92 recognizes the lumen 42 (see FIG. 1) shown in the input frame 40. However, the central region 130A1 (see FIGS. 7 to 9) of the example image 122A1 used in the above-described teacher data 128A also includes features other than the central region 130A1, that is, features of the radial regions 130A2 to 130A9 (see FIGS. 7 to 9). As an example of the features of the radial regions 130A2 to 130A9, the morphological pattern of a plurality of folds 134 (see FIGS. 7 to 9) in the large intestine 132 can be mentioned.

[0099] As shown in FIGS. 7 to 9, when the example image 122A1 is classified into the central region 130A1 and the radial regions 130A2 to 130A9, useful information for learning the presence of the lumen 138 (for example, the morphological pattern of a plurality of folds 134 in the large intestine 132) in the radial regions 130A2 to 130A9 is cut off by the central region 130A1. Therefore, when machine learning is performed to cause the model 142 (FIG. 10) to recognize the lumen 138 (see FIGS. 7 to 9) in a state where the central region 130A1 and the radial regions 130A2 to 130A9 are classified, the presence of the central region 130A1 classified for machine learning inhibits the machine learning for the regions other than the central region 130A1, that is, the radial regions 130A2 to 130A9.

[0100] Thus, the lumen recognition model 92 generated by performing machine learning on the radial regions 130A2 to 130A9 from which information (for example, the morphological pattern of a plurality of folds 134 in the large intestine 132) has been cut off due to the presence of the central region 130A1 may recognize that the lumen 42 appears in the regions outside the central region of the frame 40 even though the lumen 42 does not appear in the regions outside the central region of the frame 40, or may recognize that the lumen 42 does not appear in the regions outside the central region of the frame 40 even though the lumen 42 appears in the regions outside the central region of the frame 40.

[0101] Therefore, in view of such circumstances, in the present embodiment, as shown in FIGS. 11 to 13, machine learning using the example image 122A2 instead of the example image 122A1 is performed on the model 142 so that the lumen recognition model 92 is generated. This will be described in detail below.

[0102] FIG. 11 is a conceptual diagram showing an example of the configuration of the example image 122A2. As shown in FIG. 11, the example image 122A2 is different in that it has a plurality of divided regions 150A instead of the plurality of divided regions 130A compared to the example image 122A1 (see FIGS. 7 to 9). The plurality of divided regions 150A are regions obtained by dividing the example image 122A2 along the circumferential direction CD2 (in other words, around the center C2 of the example image 122A2). In the example shown in FIG. 11, as an example of the plurality of divided regions 150A, the first to eighth divided regions 150A1 to 150A8 are shown. The first to eighth divided regions 150A1 to 150A8 are eight regions obtained by equally dividing the example image 122A2 into eight along the circumferential direction CD2. In other words, it can be said that the first to eighth divided regions 150A1 to 150A8 are eight regions radially existing from the center C2 of the example image 122A2 toward the outer edge of the example image 122A2. In the present embodiment, the first to eighth divided regions 150A1 to 150A8 are an example of the "plurality of regions" according to the present disclosure.

[0103] FIGS. 12 and 13 are conceptual diagrams showing an example of a method for generating the teacher data 128B by the teacher data generation unit 106A associating the correct answer data 126 with the example image 122A2. FIG. 12 shows an example of a mode in which the correct answer data 126 is associated with the divided region 150A when the lumen 138 appears at the same position as the lumen 138 shown in FIG. 8 in the example image 122A2. FIG. 13 shows an example of a mode in which the correct answer data 126 is associated with the divided region 150A when the lumen 138 appears at the same position as the lumen 138 shown in FIG. 9 in the example image 122A2.

[0104] As shown in FIG. 12, with the example image 122A2 displayed on the screen 118A, the annotator 124 instructs the teacher data generation unit 106A via the reception device 116, in the same manner as the examples shown in FIGS. 8 and 9, of the lumen correspondence position 149 which is the position within the example image 122A2 of the lumen 138 shown in the example image 122A2. The teacher data generation unit 106A, in the same manner as the examples shown in FIGS. 8 and 9, superimposes and displays a circular frame 140 on the example image 122A2 and arranges the frame 140 at a position surrounding the lumen 138 shown in the example image 122A2 according to the instruction received by the reception device 116. The frame 140 is a mark that defines the lumen correspondence position 149 within the example image 122A2. That is, the position of the area surrounded by the frame 140 within the example image 122A2 is the lumen correspondence position 149.

[0105] With the frame 140 arranged at a position surrounding the lumen 138, the annotator 124 gives an instruction to confirm, which is an instruction to confirm the lumen correspondence position 149, to the teacher data generation unit 106A via the reception device 116. Thereby, the teacher data generation unit 106A confirms the lumen correspondence position 149.

[0106] The teacher data generation unit 106A, in the same manner as the examples shown in FIGS. 8 and 9, identifies the division area 150A having the largest overlapping area with the frame 140 that defines the lumen correspondence position 149 from among the plurality of division areas 150A. Then, the teacher data generation unit 106A generates teacher data 128B by associating the correct data 126 as an annotation capable of identifying the division area 150A in which the lumen 138 is shown with the division area 150A (the second division area 150A2 in the example shown in FIG. 12) identified from among the plurality of division areas 150A.

[0107] On the one hand, as shown in FIG. 13, when the lumen 138 appears in the central region within the example image 122A (i.e., the region corresponding to the central region 130A1 shown in FIG. 9), the teacher data generation unit 106A generates teacher data 128B by associating correct answer data 126 with each of all the divided regions 150A (i.e., the first to eighth divided regions 150A1 to 150A8).

[0108] In the present embodiment, a plurality of teacher data 128B thus generated by the teacher data generation unit 106A (i.e., a plurality of teacher data 128B generated using all the example images 122A2 included in the example image set 122) are used for machine learning by the learning execution unit 106B in the same manner as the example shown in FIG. 10. The lumen recognition model 92 generated by performing machine learning on the model 142 using a plurality of teacher data 128B is used by the recognition unit 82A (see FIG. 14).

[0109] FIG. 14 is a conceptual diagram showing an example of the lumen recognition process 152 executed by the recognition unit 82A based on the lumen recognition model 92 generated by performing machine learning on the model 142 using a plurality of teacher data 128B generated in the manner shown in FIGS. 12 and 13. As shown in FIG. 14, the recognition unit 82A executes the lumen recognition process 152 on the frame 40 generated by imaging the intestinal wall 32 in the large intestine 28 including the lumen 42 by the camera 52. The lumen recognition process 152 is a process of recognizing the lumen 42 shown in the frame 40 by using the lumen recognition model 92 stored in the storage 86 (in other words, a process of specifying the position of the lumen 42 shown in the frame 40 within the frame 40 by using the lumen recognition model 92). The recognition unit 82A acquires the frame 40 from the camera 52 and inputs the acquired frame 40 into the lumen recognition model 92, thereby causing the lumen recognition model 92 to generate confidence information 154. The confidence information 154 is an example of the "confidence information" according to the present disclosure. Hereinafter, the details of the confidence information 154 will be described.

[0110] FIG. 15 is a conceptual diagram showing an example of the configuration of confidence information 154 generated by the lumen recognition model 92 when the lumen 42 appears in an area other than the central area of the frame 40. As shown in FIG. 15, the confidence information 154 is information in which a confidence level 158 (for example, the probability that the lumen 138 exists) is given to a map 156 corresponding to the frame 40. The map 156 is an example of the “image corresponding to the medical image” according to the present disclosure. Here, although the map 156 is illustrated, the frame 40 may be used instead of the map 156.

[0111] The map 156 has the same geometric features as the outer shape of the frame 40. That is, the map 156 can be said to be a circular image. It has a plurality of divided areas 160A corresponding to the plurality of divided areas 150A (see FIGS. 11 to 13). Each of the plurality of divided areas 160A is an area obtained by dividing along the circumferential direction CD3 of the map 156 (in other words, around the center C3 of the map 156). In the example shown in FIG. 15, as an example of the plurality of divided areas 160A, the first to eighth divided areas 160A1 to 160A8 are shown. The first to eighth divided areas 160A1 to 160A8 are eight areas obtained by equally dividing the map 156 along the circumferential direction CD3. In other words, each of the first to eighth divided areas 160A1 to 160A8 can also be said to be an area obtained by radially dividing the map 156. Further in other words, the first to eighth divided areas 160A1 to 160A8 can also be said to be eight areas radially existing from the center C3 of the map 156 toward the outer edge of the map 156. In the present embodiment, the first to eighth divided areas 160A1 to 160A8 are an example of the “plurality of divided areas” according to the present disclosure.

[0112] Map 156 is provided with a plurality of center lines CL. The plurality of center lines CL correspond to a plurality of divided regions 160A. Each of the plurality of center lines CL is a virtual line that extends from the center C3 and passes through the center of the arc of the corresponding divided region 160A. In the example shown in FIG. 15, for the first to eighth divided regions 160A1 to 160A8, first to eighth center lines CL1 to CL8 are provided as an example of the plurality of center lines CL. The first to eighth center lines CL1 to CL8 are arranged at intervals of 45 degrees around the center C3.

[0113] Each confidence level 158 assigned to the plurality of divided regions 160A is compared with a first threshold value TH1 (for example, 0.4). Then, the divided region 160A to which a value exceeding the first threshold value TH1 is assigned as the confidence level 158 is specified. The comparison between each confidence level 158 and the first threshold value TH1, and the specification of the divided region 160A to which a value exceeding the first threshold value TH1 is assigned as the confidence level 158 are performed by the control unit 82B. In the example shown in FIG. 15, the divided region 160A to which a value exceeding the first threshold value TH1 (0.7 in the example shown in FIG. 15) is assigned as the confidence level 158 is the second divided region 160A2. The first threshold value TH1 may be a fixed value that cannot be changed, or may be a variable value that can be changed according to an instruction or the like received by the reception device 64. In the present embodiment, the first threshold value TH1 is an example of the "threshold value" according to the present disclosure, and a value exceeding the first threshold value TH1 is an example of the "value exceeding the threshold value" according to the present disclosure.

[0114] FIG. 16 is a conceptual diagram showing an example of the processing content of the control unit 82B and an example of the display content of the screen 35 when the lumen 42 appears in a region other than the central region of the frame 40. As shown in FIG. 16, the control unit 82B acquires the frame 40 from the camera 52. The content shown in the frame 40 acquired by the control unit 82B is the same as the content shown in the frame 40 input to the lumen recognition model 92 by the recognition unit 82A. The control unit 82B displays the frame 40 acquired from the camera 52 in the first display area 35A.

[0115] Further, the control unit 82B displays the first visible information 162 in the first display area 35A as information indicating that the lumen 42 exists in any of the plurality of divided areas 160A based on the confidence information 154. For example, when the confidence information 154 is information in which a value exceeding the first threshold TH1 is given as the confidence 158 to a single divided area 160A, the control unit 82B determines that the lumen 42 appears in an area other than the central area of the frame 40, and displays the first visible information 162 in the first display area 35A.

[0116] In the example shown in FIG. 16, the first visible information 162 displayed in the first display area 35A is information that can visually identify that the divided area 160A to which a value exceeding the first threshold TH1 is given as the confidence 158 is the second divided area 160A2. In the example shown in FIG. 16, as the first visible information 162, a mark that outlines the outer periphery of the image area corresponding to the second divided area 160A2 (a fan-shaped image area in the example shown in FIG. 16) among all the image areas of the frame 40 is used. Since the lumen 42 appears in the image area corresponding to the second divided area 160A2 within the frame 40, it can be said that the first visible information 162 displayed in the first display area 35A is also information that can visually identify the position of the lumen 42 appearing in the frame 40. Further, the first visible information 162 is displayed within the frame 40 in the state of being displayed in the first display area 35A. In the example shown in FIG. 16, a mode in which the first visible information 162 is superimposed and displayed on the frame 40 is shown.

[0117] In the example shown in FIG. 16, further, the control unit 82B displays the text 44A in the second display area 35B as one of the auxiliary information 44. The text 44A is text indicating in which position within the frame 40 displayed in the first display area 35A the lumen 42 appears, and is displayed in the second display area 35B when the lumen 42 appears in an area other than the central area of the frame 40. In the example shown in FIG. 16, as the text 44A, text indicating that the lumen 42 appears in the upper right of the frame 40 is shown.

[0118] In the present embodiment, the first visible information 162 and the text 44A are examples of the "first information" and "visible information" according to the present disclosure.

[0119] FIG. 17 is a conceptual diagram showing a first example of the configuration of the confidence information 154 generated by the lumen recognition model 92 when the lumen 42 appears in the central region of the frame 40. As shown in FIG. 17, each confidence level 158 assigned to the plurality of divided regions 160A is compared with a second threshold TH2 (for example, 0.3). Then, the divided region 160A to which a value exceeding the second threshold TH2 is assigned as the confidence level 158 is specified. The comparison of each confidence level 158 with the second threshold TH2 and the specification of the divided region 160A to which a value exceeding the second threshold TH2 is assigned as the confidence level 158 are performed by the control unit 82B.

[0120] In the example shown in FIG. 17, the divided regions 160A to which values exceeding the second threshold TH2 (in the example shown in FIG. 17, 0.4) are assigned as the confidence level 158 are the first divided region 160A1 and the fourth divided region 160A4. The positional relationship between the first divided region 160A1 and the fourth divided region 160A4, to which values exceeding the second threshold TH2 are respectively assigned as the confidence level 158, is a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156 among the plurality of divided regions 160A. The positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156 among the plurality of divided regions 160A is synonymous with a positional relationship in which the angle formed by two center lines CL exceeds 90 degrees. In the example shown in FIG. 17, the angle formed by the center line CL1 of the first divided region 160A1 and the center line CL4 of the fourth divided region 160A4 is 135 degrees, which exceeds 90 degrees. Therefore, it can be said that the positional relationship between the first divided region 160A1 and the fourth divided region 160A4 is a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156 among the plurality of divided regions 160A.

[0121] Note that the second threshold value TH2 used for comparison with each confidence level 158 may be a fixed value that cannot be changed, or may be a variable value that can be changed according to an instruction or the like received by the receiving device 64. In the present embodiment, the second threshold value TH2 is an example of the "threshold value" according to the present disclosure, and a value exceeding the second threshold value TH2 is an example of the "value exceeding the threshold value" according to the present disclosure.

[0122] FIG. 18 is a conceptual diagram showing a second example of the configuration of the confidence information 154 generated by the lumen recognition model 92 when the lumen 42 appears in the central region of the frame 40. As shown in FIG. 18, each confidence level 158 assigned to a plurality of divided regions 160A is compared with a third threshold value TH3 (for example, 0.2). Then, the divided region 160A to which a value exceeding the third threshold value TH3 is assigned as the confidence level 158 is specified. The comparison between each confidence level 158 and the third threshold value TH3, and the specification of the divided region 160A to which a value exceeding the third threshold value TH3 is assigned as the confidence level 158 are performed by the control unit 82B.

[0123] In the example shown in FIG. 18, the divided regions 160A to which a value exceeding the third threshold value TH3 (0.3 in the example shown in FIG. 18) is assigned as the confidence level 158 are the first divided region 160A1, the third divided region 160A3, the fifth divided region 160A5, and the seventh divided region 160A7. The first divided region 160A1, the third divided region 160A3, the fifth divided region 160A5, and the seventh divided region 160A7 are four divided regions 160A arranged at regular intervals over the entire circumference of the map 156. In the example shown in FIG. 18, the regular interval over the entire circumference of the map 156 refers to an interval at which the angle formed by the center lines CL between the divided regions 160A is 90 degrees.

[0124] Note that the third threshold value TH3 used for comparison with each confidence level 158 may be a fixed value that cannot be changed, or may be a variable value that can be changed according to an instruction or the like received by the receiving device 64. In the present embodiment, the third threshold value TH3 is an example of the "threshold value" according to the present disclosure, and a value exceeding the third threshold value TH3 is an example of the "value exceeding the threshold value" according to the present disclosure.

[0125] FIG. 19 is a conceptual diagram showing a third example of the configuration of the confidence information 154 generated by the lumen recognition model 92 when the lumen 42 appears in the central region of the frame 40. As shown in FIG. 19, each confidence level 158 assigned to the plurality of divided regions 160A is compared with a fourth threshold value TH4 (for example, 0.1). Then, the divided region 160A to which a value exceeding the fourth threshold value TH4 is assigned as the confidence level 158 is specified. The comparison of each confidence level 158 with the fourth threshold value TH4 and the specification of the divided region 160A to which a value exceeding the fourth threshold value TH4 is assigned as the confidence level 158 are performed by the control unit 82B.

[0126] In the example shown in FIG. 19, the divided regions 160A to which a value exceeding the fourth threshold value TH4 (in the example shown in FIG. 19, 0.125) is assigned as the confidence level 158 are the first to eighth divided regions 160A1 to 160A8 (that is, all the divided regions 160A).

[0127] Note that the fourth threshold value TH4 used for comparison with each confidence level 158 may be a fixed value that cannot be changed, or may be a variable value that can be changed according to an instruction or the like received by the reception device 64. In the present embodiment, the fourth threshold value TH4 is an example of the "threshold value" according to the present disclosure, and a value exceeding the fourth threshold value TH4 is an example of the "value exceeding the threshold value" according to the present disclosure.

[0128] FIG. 20 is a conceptual diagram showing an example of the processing content of the control unit 82B and an example of the display content of the screen 35 when the lumen 42 appears in the central region of the frame 40. As shown in FIG. 20, the control unit 82B displays the frame 40 in the first display region 35A in the same manner as in the example shown in FIG. 16.

[0129] Further, when the confidence information 154 is information in which values exceeding the second threshold TH2 are given as the confidence 158 to two or more divided regions 160A having a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156 among all the divided regions 160A, the control unit 82B determines that the lumen 42 is shown in the central region of the frame 40, and displays the second visible information 164 in the first display region 35A as information indicating that the lumen 42 exists in the central region of the frame 40. Here, as an example of information in which values exceeding the second threshold TH2 are given as the confidence 158 to two or more divided regions 160A having a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156 among all the divided regions 160A, the confidence information 154 shown in FIG. 17 can be cited.

[0130] Further, when the confidence information 154 is information in which values exceeding the third threshold TH3 are given as the confidence 158 to two or more divided regions 160A arranged at regular intervals over the entire circumference of the map 156 among all the divided regions 160A, the control unit 82B determines that the lumen 42 is shown in the central region of the frame 40, and displays the second visible information 164 in the first display region 35A. Here, as an example of information in which values exceeding the third threshold TH3 are given as the confidence 158 to two or more divided regions 160A arranged at regular intervals over the entire circumference of the map 156 among all the divided regions 160A, the confidence information 154 shown in FIG. 18 can be cited.

[0131] Further, when the confidence information 154 is information in which values exceeding the fourth threshold TH4 are given as the confidence 158 to all the divided regions 160A, the control unit 82B determines that the lumen 42 is shown in the central region of the frame 40, and displays the second visible information 164 in the first display region 35A. Here, as an example of information in which values exceeding the fourth threshold TH4 are given as the confidence 158 to all the divided regions 160A, the confidence information 154 shown in FIG. 19 can be cited.

[0132] The second visible information 164 displayed in the first display area 35A is information that can visually identify that the lumen 42 exists in the central area of the frame 40. In the example shown in FIG. 20, as the second visible information 164, a mark that outlines the outer periphery of the central area (for example, the image area corresponding to the central area 130A1 shown in FIG. 9) of the entire image area of the frame 40 is used. Since the lumen 42 is shown in the central area of the entire image area of the frame 40, the second visible information 164 displayed in the first display area 35A can also be said to be information that can visually identify the position of the lumen 42 shown in the frame 40. Further, the second visible information 164 is displayed within the frame 40 in the state of being displayed in the first display area 35A. In the example shown in FIG. 20, a mode in which the second visible information 164 is superimposed and displayed on the frame 40 is shown.

[0133] In the example shown in FIG. 20, further, the control unit 82B displays the text 44B in the second display area 35B as one of the auxiliary information 44. The text 44B is displayed in the second display area 35B when the lumen 42 is shown in the central area of the frame 40. The text 44B is text indicating that the lumen 42 is shown in the central area of the frame 40 displayed in the first display area 35A.

[0134] Note that in the present embodiment, the second visible information 164 and the text 44B are examples of the "second information" and "visible information" according to the present disclosure.

[0135] Next, the operation of the information processing apparatus 100 will be described with reference to FIG. 21.

[0136] In the machine learning process shown in FIG. 21, first, in step ST10, the teacher data generation unit 106A acquires the unprocessed example image 122A2 from the example image set 122 stored in the storage 110. Here, the unprocessed example image 122A2 refers to an example image 122A2 that has not yet been used in the machine learning process. The teacher data generation unit 106A displays the example image 122A2 acquired from the example image set 122 on the screen 118A. After the process of step ST10 is executed, the machine learning process proceeds to step ST12.

[0137] In step ST12, the teacher data generation unit 106A receives an instruction for the lumen corresponding position 149. After the process of step ST12 is executed, the machine learning process proceeds to step ST14.

[0138] In step ST14, the teacher data generation unit 106A specifies the positional relationship between the lumen corresponding position 149 received in step ST12 and the plurality of divided regions 150A. After the process of step ST14 is executed, the machine learning process proceeds to step ST16.

[0139] In step ST16, the teacher data generation unit 106A associates the correct answer data 126 with the example image 122A2 acquired in step ST10 according to the positional relationship specified in step ST14. For example, when the lumen corresponding position 149 exists outside the central region of the example image 122A2, the correct answer data 126 is associated with the divided region 150A having the largest overlapping area with the lumen corresponding position 149 according to the instruction given from the annotator 124. Also, for example, when the lumen corresponding position 149 exists in the central region of the example image 122A2, the correct answer data 126 is associated with each of all the divided regions 150A according to the instruction given from the annotator 124. In this way, the teacher data generation unit 106A generates the teacher data 128B by associating the correct answer data 126 with the example image 122A2. The teacher data 128B generated in this way is stored in a predetermined storage medium (for example, the storage 110). After the process of step ST16 is executed, the machine learning process proceeds to step ST18.

[0140] In step ST18, the teacher data generation unit 106A determines whether there is an unprocessed example image 122A2. In step ST18, if there is an unprocessed example image 122A2, the determination is negative and the machine learning process proceeds to step ST10. In step ST18, if there is no unprocessed example image 122A2, the determination is positive and the machine learning process proceeds to step ST20.

[0141] In step ST20, the learning execution unit 106B generates the lumen recognition model 92 by performing machine learning using a plurality of teacher data 128B obtained by repeatedly executing the processes of steps ST10 to ST18 (see FIG. 10). The lumen recognition model 92 is stored in the storage 86 of the medical support device 24 (see FIG. 4). After the process of step ST20 is executed, the machine learning process ends.

[0142] Next, the operation of the part of the endoscope device 10 according to the present disclosure will be described with reference to FIG. 22. The flow of the medical support process shown in FIG. 22 is an example of the "medical support method" according to the present disclosure.

[0143] In the medical support process shown in FIG. 22, in step ST50, the recognition unit 82A and the control unit 82B acquire the frame 40 from the camera 52. The control unit 82B displays the frame 40 acquired from the camera 52 in the first display area 35A. After the process of step ST50 is executed, the medical support process proceeds to step ST52.

[0144] In step ST52, the recognition unit 82A generates the confidence information 154 by performing the lumen recognition process 152 using the lumen recognition model 92 stored in the storage 86 on the frame 40 acquired in step ST50. After the process of step ST52 is executed, the medical support process proceeds to step ST54.

[0145] In step ST54, the control unit 82B generates the first visual information 162 or the second visual information 164 based on the confidence information 154 generated in step ST52. For example, when the confidence information 154 (that is, the information indicating the distribution of the confidence levels 158 assigned to the map 156) is information of the type shown in FIG. 15 (that is, when the confidence information 154 is information in which a value exceeding the first threshold TH1 is assigned as the confidence level 158 to a single divided region 160A), the first visual information 162 is generated. Also, for example, when the confidence information 154 is information of the type shown in FIGS. 17 to 18, the second visual information 164 is generated. After the process of step ST54 is executed, the medical support process proceeds to step ST56.

[0146] In step ST56, the control unit 82B displays the first visual information 162 or the second visual information 164 generated in step ST54 on the screen 35 (see FIGS. 16 and 20). After the process of step ST56 is executed, the medical support process proceeds to step ST58.

[0147] In step ST58, the control unit 82B determines whether or not the condition for ending the medical support process is satisfied. As an example of the condition for ending the medical support process, there is a condition that an instruction to end the medical support process is given to the endoscope device 10 (for example, a condition that an instruction to end the medical support process is received by the reception device 64).

[0148] In step ST58, if the condition for ending the medical support process is not satisfied, the determination is negative and the medical support process proceeds to step ST50. In step ST58, if the condition for ending the medical support process is satisfied, the determination is affirmative and the medical support process ends.

[0149] Thus, in the medical support process, when the confidence information 154 (i.e., the information indicating the distribution of the confidence levels 158 assigned to the map 156) is the type of information shown in FIG. 15 (i.e., when the confidence information 154 is the information in which a value exceeding the first threshold TH1 is assigned as the confidence level 158 to a single divided region 160A), the first visible information 162 is displayed on the screen 35, and when the confidence information 154 is the type of information shown in FIGS. 17 to 18, the second visible information 164 is displayed on the screen 35. That is, the medical support process is a process capable of distinguishing between the case where the confidence information 154 is the type of information shown in FIG. 15 and the case where the confidence information 154 is the type of information shown in FIGS. 17 to 18.

[0150] As described above, in the present embodiment, the confidence information 154 is generated by inputting the frame 40 to the lumen recognition model 92. The confidence information 154 is information in which the confidence levels 158 indicating the presence of the lumen 42 are assigned to the first to eighth divided regions 160A1 to 160A8 obtained by dividing the map 156 along the circumferential direction CD3. In the present embodiment, when the lumen 42 appears in a region other than the central region of the frame 40, the first visible information 162 is displayed in the first display region 35A as information indicating that the lumen 42 exists in any of the divided regions 160A based on the confidence information 154. Further, when the lumen 42 appears in the central region of the frame 40, the second visible information 164 is displayed in the first display region 35A as information indicating that the lumen 42 appears in the central region of the frame 40.

[0151] By the way, in the comparative examples shown in FIGS. 7 to 9, the example image 122A1 is included in the teacher data 128A used for machine learning to generate the lumen recognition model 92. The teacher data 128A shown in FIGS. 8 and 9 as a comparative example with respect to the teacher data 128B according to the present embodiment is generated by associating the correct answer data 126 with the example image 122A1 in FIG. 7 shown as a comparative example with respect to the example image 122A2.

[0152] However, the central region 130A1 of the example image 122A1 also includes the features of the radial regions 130A2 to 130A9 (for example, the morphological pattern of the fold 134 shown in each of the radial regions 130A2 to 130A9). That is, when the example image 122A1 is classified into the central region 130A1 and the radial regions 130A2 to 130A9, the information for learning the presence of the lumen 138 in the radial regions 130A2 to 130A9 is eliminated by the central region 130A1. This means that the presence of the central region 130A1 inhibits the machine learning for the radial regions 130A2 to 130A9. The lumen recognition model 92 generated by such machine learning may recognize that the lumen 42 is shown in the region outside the central region of the frame 40 even though the lumen 42 is not shown in the region outside the central region of the frame 40, or may recognize that the lumen 42 is not shown in the region outside the central region of the frame 40 even though the lumen 42 is shown in the region outside the central region of the frame 40.

[0153] Therefore, in the present embodiment, the example image 122A2 included in the teacher data 128B used to generate the lumen recognition model 92 is not provided with a region corresponding to the central region 130A1. The example image 122A2 is divided into the first to eighth divided regions 150A1 to 150A8. Since the first to eighth divided regions 150A1 to 150A8 also have the information (for example, the morphological pattern of the fold 134) included in the central region 130A1 of the example image 122A1 shown as a comparative example, it is possible to perform machine learning with higher accuracy for the first to eighth divided regions 150A1 to 150A8 than for the radial regions 130A2 to 130A9.

[0154] In this embodiment, when the lumen 138 appears in the central region of the example image 122A2, correct data 126 is associated with all of the first to eighth divided regions 150A1 to 150A8. When the lumen 138 appears in a region other than the central region of the example image 122A2, the teacher data 128B is generated by associating the correct data 126 with the divided region 150A having the largest area overlapping with the lumen 138. In this embodiment, the position of the lumen 42 shown in the frame 40 is recognized for the lumen recognition model 92 obtained by machine learning using the teacher data 128B generated in this way within the frame 40. Therefore, according to this embodiment, compared with the case of recognizing the position of the lumen 42 shown in the frame 40 for the lumen recognition model 92 obtained by machine learning using the teacher data 128A generated in the manner shown in FIGS. 7 to 9, the position of the lumen 42 shown in the frame 40 can be recognized with high accuracy for the lumen recognition model 92. As a result, the endoscope apparatus 10 according to this embodiment enables the doctor 12 or the like to accurately grasp the position of the lumen 42 shown in the frame 40 within the frame 40 compared with the case of recognizing the position of the lumen 42 shown in the frame 40 for the lumen recognition model 92 obtained by machine learning using the teacher data 128A generated in the manner shown in FIGS. 7 to 9.

[0155] Also, in this embodiment, when the confidence information 154 is information in which a value exceeding the third threshold TH3 is given as the confidence 158 for the first divided region 160A1, the third divided region 160A3, the fifth divided region 160A5, and the seventh divided region 160A7 arranged at regular intervals over the entire circumference of the map 156 among the first to eighth divided regions 160A1 to 160A8 (see FIG. 18), the second visible information 164 is displayed in the first display region 35A. Thereby, it is possible to make the user or the like grasp that the lumen 42 exists in the central region of the frame 40.

[0156] Further, in the present embodiment, when the confidence information 154 is information in which a value exceeding the fourth threshold TH4 is given as the confidence 158 in the first to eighth divided regions 160A1 to 160A8 (see FIG. 19), the second visible information 164 is displayed in the first display region 35A. Thereby, it is possible to make the user or the like grasp that the lumen 42 exists in the central region of the frame 40.

[0157] Further, in the present embodiment, the first to eighth divided regions 160A1 to 160A8 are regions obtained by radially dividing the frame 40, and the confidence information 154 is generated by giving the confidence 158 to each of the first to eighth divided regions 160A1 to 160A8. Then, the first visible information 162 or the second visible information 164 is generated based on the confidence information 154 and displayed on the screen 35. Therefore, it is possible to make the user or the like grasp the position of the lumen 42 shown in the frame 40 in units of the divided regions 160A.

[0158] Further, in the present embodiment, when the lumen 42 appears in a region other than the central region of the frame 40, the first visible information 162 and the text 44A are displayed on the screen 35. Thereby, it is possible to visually recognize by the user or the like that the lumen 42 exists in a region other than the central region of the frame 40.

[0159] Further, in the present embodiment, when the lumen 42 appears in the central region of the frame 40, the first visible information 164 and the text 44B are displayed on the screen 35. Thereby, it is possible to visually grasp by the user or the like that the lumen 42 exists in the central region of the frame 40.

[0160] Also, in the present embodiment, the teacher data 128A used for the machine learning performed on the model 142 includes the example image 122A2 and the correct answer data 126 associated with the example image 122A2. As the example image 122A2, an image divided into the first to eighth divided regions 150A1 to 150A8 corresponding to the first to eighth divided regions 160A1 to 160A8 is used. The correct answer data 126 when the lumen 138 appears outside the central region of the example image 122A2 is an annotation capable of specifying the position of the divided region 150A where the lumen 138 appears. That is, when the lumen 138 appears outside the central region of the example image 122A2, the correct answer data 126 is associated with the divided region 150A where the lumen 138 appears. On the other hand, the correct answer data 126 when the lumen 138 appears in the central region of the example image 122A2 is an annotation capable of specifying all positions of the first to eighth divided regions 150A1 to 150A8. That is, when the lumen 138 appears outside the central region of the example image 122A2, the correct answer data 126 is associated with each of the first to eighth divided regions 150A1 to 150A8. In the present embodiment, by causing the lumen recognition model 92 generated by performing machine learning using the teacher data 128A obtained in this way on the model 142 to recognize the position of the lumen 42 in the frame 40 within the frame 40, the user or the like can accurately grasp the position of the lumen 42 in the frame 40 within the frame 40.

[0161] In the above embodiment, when the confidence information 154 is information of the type shown in FIGS. 17 to 18, an example of a form in which the first visible information 162 is not displayed on the screen 35 and the second visible information 164 is displayed on the screen 35 is given. However, this is merely an example. For example, when the confidence information 154 is information of the type shown in FIGS. 17 to 18, both the first visible information 162 and the second visible information 164 may not be output (for example, both the first visible information 162 and the second visible information 164 may not be displayed on the screen 35). By doing so, the user or the like can recognize that the first visible information 162 and the second visible information 164 are not output. Thereby, the user or the like can grasp that the lumen 42 exists in the central region of the frame 40.

[0162] Also, in the above embodiment, as the first visible information 162, a mark that borders the outer periphery of the image region corresponding to the divided region 160A in the entire image region of the frame 40 (the fan-shaped image region in the example shown in FIG. 16) is exemplified. However, this is merely an example, and any information may be used as long as it can visually identify in which divided region 160A the lumen 42 appears.

[0163] Also, in the above embodiment, as the second visible information 164, a mark that borders the outer periphery of the central region (for example, the image region corresponding to the central region 130A1 shown in FIG. 9) in the entire image region of the frame 40 is exemplified. However, this is merely an example, and any information may be used as long as it can visually identify that the lumen 42 appears in the central region of the entire image region of the frame 40.

[0164] Also, in the above embodiment, an example of a form in which the sample image 122A2 is divided into eight divided regions 150A is given. However, this is merely an example, and the sample image 122A2 may be divided into nine or more or seven or less divided regions 150A. In this case, the frame 40 may also be divided into the same number of divided regions 160A as the divided regions 150A.

[0165] Also, in the above embodiment, among the first to eighth divided regions 160A1 to 160A8, information in which a value exceeding the second threshold TH2 is given as the confidence level 158 to the first divided region 160A1 and the fourth divided region 160A4 that have a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156 is exemplified as one of the confidence level information 154. However, the present disclosure is not limited to this. For example, information in which a value exceeding the second threshold TH2 is given as the confidence level 158 to two or more divided regions 160A that have a positional relationship of 120 degrees or more in the circumferential direction CD3 of the map 156 among the first to eighth divided regions 160A1 to 160A8 may be used as one of the confidence level information 154. Also in this case, similar to the example shown in FIG. 16, by causing the first visible information 162 and / or the text 44A to be displayed on the screen 35, the same effects as those of the above embodiment can be obtained.

[0166] Also, in the above embodiment, when the confidence level information 154 is information of the type shown in FIGS. 17 to 19, an example of the form in which the second visible information 164 and the text 44B are displayed on the screen 35 is given. However, this is merely an example. For example, when the confidence level information 154 is information in which a value exceeding the second threshold TH2 is given as the confidence level 158 to two or more divided regions 160A that are equally spaced along the circumference around the center of the map 156 among all the divided regions 160A and have a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156, the second visible information 164 and / or the text 44B may be displayed on the screen 35.

[0167] As an example of two or more divided regions 160A that are equally spaced along the circumference around the center of the map 156 among all the divided regions 160A and have a positional relationship exceeding 90 degrees in the circumferential direction CD3 of the map 156, as shown in FIG. 23, there are a first divided region 160A1 and a fifth divided region 150A5 to which a value exceeding the second threshold TH2 (for example, 0.4) is assigned as the confidence level 158. In the example shown in FIG. 23, since the angle formed by the center line CL1 and the center line CL5 is 180 degrees, it can be said that the positional relationship between the first divided region 160A1 and the fifth divided region 160A5 is a positional relationship that is equally spaced along the circumferential direction CD3 of the map 156. Thus, even if the confidence information 154 is information of the type shown in FIG. 23, the same effects as those of the above-described embodiment can be obtained.

[0168] Also, in the above-described embodiment, an example of a form in which the first visible information 162 and the text 44A are displayed on the screen 35 when a confidence level 158 exceeding the first threshold TH1 is assigned to a single divided region 160A is given, but this is merely an example. For example, when the confidence information 154 is information in which a value exceeding a fifth threshold TH5 (for example, 0.3) is assigned to two or more divided regions 160A having a positional relationship of 90 degrees or less in the circumferential direction CD3 of the map 156 among all the divided regions 160A, the first visible information 162 and / or the text 44A may be displayed on the screen 35. For example, as shown in FIG. 24, since the angle formed by the center line CL1 and the center line CL2 is 45 degrees, it can be said that the positional relationship between the first divided region 160A1 and the second divided region 160A2 is a positional relationship of 90 degrees or less in the circumferential direction CD3 of the map 156. Thus, when a value exceeding the fifth threshold TH5 is assigned to the first divided region 160A1 and the second divided region 160A2, for example, a mark that outlines the outer periphery of a fan-shaped image region (that is, a fan-shaped image region with a central angle of 90 degrees) combining the first divided region 160A1 and the second divided region 160A2 among all the image regions of the frame 40 may be superimposed and displayed on the frame 40 as the second visible information 164. Also in this case, the same effects as those of the above-described embodiment can be obtained. Note that the fifth threshold TH5 is an example of the "threshold" according to the present disclosure, and a value exceeding the fifth threshold TH5 is an example of the "value exceeding the threshold" according to the present disclosure.

[0169] In the above-described embodiment, an example form in which the control unit 82B outputs visually identifiable information such as the text 44A, the text 44B, the first visible information 162, and the second visible information 164 has been given. However, the present disclosure is not limited to this, and audible information (for example, voice) capable of identifying the position of the lumen 42 within the frame 40 may be output to a speaker (not shown), or information capable of identifying the position of the lumen 42 within the frame 40 may be stored in a storage medium (for example, the storage 76, the storage 86, or a storage provided in an external device such as a server).

[0170] In the above-described embodiment, an example form in which the medical support process is performed by the computer 78 has been described. However, the present disclosure is not limited to this, and at least a part of the processes included in the medical support process may be performed by a device provided outside the computer 78. Hereinafter, an example in this case will be described with reference to FIG. 25.

[0171] FIG. 25 is a conceptual diagram showing an example of the configuration of the endoscope device 166. The endoscope device 166 is an example of the "endoscope device" according to the present disclosure. The endoscope device 166 is different from the endoscope device 10 described in the above-described embodiment in that it has an external device 168.

[0172] For example, the external device 168 is a server and is communicably connected to the computer 78 via a network 170 (for example, a WAN and / or a LAN, etc.). Here, a server is exemplified, but at least one personal computer or the like may be used as the external device 168 instead of the server.

[0173] As an example of the external device 168, there is at least one server that directly or indirectly transmits and receives data to and from the computer 78 via the network 170. The external device 168 receives a processing execution instruction given from the processor 82 of the computer 78 via the network 170. Then, the external device 168 executes a process according to the received processing execution instruction, and transmits the processing result to the computer 78 via the network 170. In the computer 78, the processor 82 receives the processing result transmitted from the external device 168 via the network 170, and executes a process using the received processing result.

[0174] As an example of the processing execution instruction, there is an instruction to cause the external device 168 to execute at least a part of the medical support process. As a first example of at least a part of the medical support process (that is, the process to be executed by the external device 168), there is the lumen recognition process 152. In this case, the external device 168 executes the lumen recognition process 152 according to the processing execution instruction given from the processor 82 via the network 170, and transmits information including the confidence information 154 as the first processing result to the computer 78 via the network 170. In the computer 78, the processor 82 receives the first processing result, and executes the same process as in the above embodiment using the received first processing result.

[0175] As a second example of at least a part of the medical support process (that is, the process to be executed by the external device 168), there is the process by the control unit 82B. In this case, the external device 168 executes the process by the control unit 82B according to the processing execution instruction given from the processor 82 via the network 170, and transmits the second processing result (for example, the text 44A, the text 44B, the first visible information 162, and / or the second visible information 164, etc.) to the computer 78 via the network 170. In the computer 78, the processor 82 receives the second processing result, and executes the same process as in the above embodiment (for example, display using the display device 18, etc.) using the received second processing result.

[0176] Note that the external device 168 may be implemented by cloud computing. Cloud computing is merely an example, and the external device 168 may be implemented by network computing such as fog computing, edge computing, or grid computing.

[0177] In the above embodiment, an example form in which the medical support program 90 is stored in the storage 86 has been described, but the present disclosure is not limited to this. For example, the medical support program 90 may be stored in a portable computer-readable non-transitory storage medium such as an SSD or a USB memory. The medical support program 90 stored in the non-transitory storage medium is installed in the computer 78 of the endoscope device 10. The processor 82 executes medical support processing according to the medical support program 90.

[0178] Alternatively, the medical support program 90 may be stored in a storage device such as another computer or server connected to the endoscope device 10 via a network, and the medical support program 90 may be downloaded and installed in the computer 78 in response to a request from the endoscope device 10.

[0179] Note that it is not necessary to store all of the medical support program 90 in a storage device such as another computer or server connected to the endoscope device 10, or to store all of the medical support program 90 in the storage 86. A part of the medical support program 90 may be stored.

[0180] As hardware resources for executing medical support processing, various types of processors shown below can be used. As the processor, for example, there is a CPU which is a general-purpose processor that functions as a hardware resource for executing medical support processing by executing software, that is, a program. Further, as the processor, for example, there is a dedicated electric circuit which is a processor having a circuit configuration dedicatedly designed for executing specific processing such as an FPGA, a PLD, or an ASIC. A memory is built in or connected to any of these processors, and any of these processors executes medical support processing by using the memory.

[0181] The hardware resources for executing medical support processing may be constituted by one of these various types of processors, or may be constituted by a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resources for executing medical support processing may be one processor.

[0182] As an example of being constituted by one processor, first, there is a form in which one processor is constituted by a combination of one or more CPUs and software, and this processor functions as a hardware resource for executing medical support processing. Second, there is a form in which a processor that realizes the functions of the entire system including a plurality of hardware resources for executing medical support processing with one IC chip, as represented by an SoC or the like, is used. Thus, medical support processing is realized as a hardware resource by using one or more of the above various types of processors.

[0183] Furthermore, as a hardware structure of these various types of processors, more specifically, an electric circuit combining circuit elements such as semiconductor elements can be used. Also, the above medical support processing is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist.

[0184] The above-described description and illustrated content are detailed descriptions of the part related to the present disclosure and are merely examples of the present disclosure. For example, the descriptions regarding the above-described configuration, function, operation, and effect are descriptions of an example of the configuration, function, operation, and effect of the part related to the present disclosure. Therefore, it goes without saying that within the scope not departing from the gist of the present disclosure, the above-described description and illustrated content may be deleted of unnecessary parts, new elements may be added, or replacements may be made. Also, in order to avoid complication and facilitate the understanding of the part related to the present disclosure, in the above-described description and illustrated content, descriptions regarding common technical knowledge and the like that do not particularly require explanation for implementing the present disclosure are omitted.

[0185] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually stated to be incorporated by reference.

[0186] Regarding the above embodiments, the following additional remarks are disclosed.

[0187] (Supplementary Note 1) A teacher data generation method for generating teacher data by associating correct answer data with an example image, wherein the above example image is an image showing a sample of a medical image (for example, Frame 40) and is an image divided into a plurality of divided regions (for example, the 1st to 8th divided regions 150A1 to 150A8) along the circumferential direction, when a sample of a lumen (for example, Lumen 138) appears outside the central region of the above example image (for example, Example Image 122A2), associating correct answer data (for example, Correct Answer Data 126) with the divided region in which the sample of the lumen appears among the above plurality of divided regions, and when the sample of the lumen appears in the central region of the above example image, including associating the correct answer data with each of the above plurality of divided regions Teacher data generation method.

[0188] (Supplementary Note 2) Each of the plurality of divided regions is a region obtained by radially dividing the example image. The teacher data generation method according to Supplementary Note 1.

[0189] (Supplementary Note 3) There are eight of the plurality of divided regions radially. The teacher data generation method according to Supplementary Note 2.

[0190] (Supplementary Note 4) An example image showing a sample of a medical image (for example, Frame 40), the example image (for example, Example Image 122A2) divided into a plurality of divided regions (for example, First to Eighth Divided Regions 150A1 to 150A8) along the circumferential direction, when a sample of a lumen (for example, Lumen 138) appears outside the central region of the example image, associating correct answer data (for example, Correct Answer Data 126) with the divided region in which the sample of the lumen appears among the plurality of divided regions, When the sample of the lumen appears in the central region of the example image, associating the correct answer data with each of the plurality of divided regions, and, Optimizing the model (for example, Model 142) by performing machine learning using teacher data (for example, Teacher Data 128B) obtained by associating the correct answer data with the example image to generate a learned model (for example, Lumen Recognition Model 92). A method for generating a learned model.

[0191] (Supplementary Note 5) Each of the plurality of divided regions is a region obtained by radially dividing the example image. The learned model generation method according to Supplementary Note 4.

[0192] (Supplementary Note 6) There are eight of the plurality of divided regions radially. The learned model generation method according to Supplementary Note 5.

[0193] (Supplementary Note 7) Comprising a processor, The above-mentioned processor By inputting a medical image generated by imaging the inside of a luminal organ including the lumen into the learned model, confidence information in which the confidence that the lumen exists in each of a plurality of divided regions obtained by dividing the medical image or an image corresponding to the medical image along the circumferential direction is assigned to the plurality of divided regions is generated, Outputs first information indicating that the lumen exists in any of the plurality of divided regions based on the confidence information, When the confidence information is information in which values exceeding a threshold are assigned as the confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, second information indicating that the lumen exists in the central region of the medical image is output Medical support device.

[0194] (Appendix 8) The above-mentioned processor outputs the second information when the confidence information is information in which the values are assigned to two or more divided regions that are equally spaced along the circumference around the center of the medical image or the image among the plurality of divided regions and have a positional relationship exceeding 90 degrees in the circumferential direction The medical support device according to Appendix 7.

[0195] (Appendix 9) The above-mentioned processor outputs the second information when the confidence information is information in which the values are assigned to two or more divided regions having a positional relationship of 120 degrees or more in the circumferential direction among the plurality of divided regions The medical support device according to Appendix 7.

[0196] (Appendix 10) The above-mentioned processor outputs the second information when the confidence information is information in which the values are assigned to two or more divided regions arranged at regular intervals over the entire circumference of the medical image or the image among the plurality of divided regions The medical support device according to Appendix 7.

[0197] (Appendix 11) The processor outputs the second information when the confidence information is information in which the value is assigned to all of the plurality of divided regions. The medical support device according to Supplementary Note 7.

[0198] (Supplementary Note 12) The processor outputs the first information when the confidence information is information in which the value is assigned to a single divided region among the plurality of divided regions, and when the confidence information is information in which the value is assigned to two or more divided regions having a positional relationship of 90 degrees or less in the circumferential direction among the plurality of divided regions. The medical support device according to any one of Supplementary Notes 7 to 11.

[0199] (Supplementary Note 13) Each of the plurality of divided regions is a region obtained by radially dividing the medical image. The medical support device according to any one of Supplementary Notes 7 to 12.

[0200] (Supplementary Note 14) There are eight of the plurality of divided regions radially. The medical support device according to Supplementary Note 13.

[0201] (Supplementary Note 15) The learned model is obtained by machine learning using teacher data including an example image showing a sample of the medical image, the example image being divided into a plurality of regions corresponding to the plurality of divided regions, and correct answer data associated with the example image. When the sample of the lumen appears outside the central region of the example image, the correct answer data is an annotation capable of specifying the position of the region where the sample of the lumen appears among the plurality of regions. When the sample of the lumen appears in the central region of the example image, the correct answer data is an annotation capable of specifying all positions of the plurality of regions. The medical support device according to any one of Supplementary Notes 7 to 14.

[0202] (Supplementary Note 16) The above processor outputs the first information by displaying the first information on the screen, and outputs the second information by displaying the second information on the screen The medical support device according to any one of Appendices 7 to 15.

[0203] (Appendix 17) The above processor displays the medical image on the screen, displays the first information in the medical image in the state displayed on the screen, and displays the second information in the medical image in the state displayed on the screen The medical support device according to Appendix 16.

[0204] (Appendix 18) The first information and the second information are visible information capable of visually identifying the position of the lumen shown in the medical image. The medical support device according to any one of Appendices 7 to 17.

[0205] (Appendix 19) The medical image is an endoscopic image generated by imaging the inside of the luminal organ including the lumen with an endoscope. The medical support device according to any one of Appendices 7 to 18.

[0206] (Appendix 20) The medical support device according to any one of Appendices 7 to 19, and an endoscope, and the medical image is generated by imaging the inside of the luminal organ including the lumen with the endoscope. Endoscope device.

[0207] (Appendix 21) By inputting a medical image generated by imaging the inside of a luminal organ including the lumen into a learned model, confidence information in which the confidence that the lumen exists in each of a plurality of divided regions obtained by dividing the medical image or an image corresponding to the medical image along the circumferential direction is given to the plurality of divided regions is generated. Outputting first information indicating that the lumen exists in any one of the plurality of divided regions based on the confidence information, and When the confidence information is information in which values exceeding a threshold are given as the confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions, outputting second information indicating that the lumen exists in the central region of the medical image. Medical support method.

[0208] (Appendix 22) By inputting a medical image generated by imaging the inside of a luminal organ including the lumen into a learned model, confidence information in which the confidence that the lumen exists in each of a plurality of divided regions obtained by dividing the medical image or an image corresponding to the medical image along the circumferential direction is given to the plurality of divided regions is generated. Outputting first information indicating that the lumen exists in any one of the plurality of divided regions based on the confidence information, and A program for causing a computer to execute a process including outputting second information indicating that the lumen exists in the central region of the medical image when the confidence information is information in which values exceeding a threshold are given as the confidence to two or more divided regions having a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of divided regions.

Explanation of Signs

[0209] 10,166 Endoscope device 12 Physician 14 Nurse 16 Endoscope scope 18,118 Display device 20 Light source device 22 Control device 24 Medical support device 26 Subject 28, 132 Large intestine 30 Light 32, 136 Intestinal wall 34 Cart 35, 118A Screen 35A First display area 35B Second display area 39 Endoscopic moving image 40 Frame 42, 138 Lumen 43, 134 Fold 44 Auxiliary information 44A, 44B Text 46 Operation unit 48 Insertion part 50 Tip part 50A Tip surface 52 Camera 54 Lighting device 54A, 54B Lighting window 56 Treatment opening 58 Treatment tool 60 Treatment tool insertion port 62 Universal code 64, 116 Reception device 66, 78, 102 Computer 68, 88, 112 Bus 70, 80, 104 External I / F 72, 82, 106 Processor 74, 84, 108 Memory 76, 86, 110 Storage 82A Recognition part 82B Control part 90 Medical support program 92 Lumen recognition model 100 Information processing device 106A Teacher data generation part 106B Learning execution part 116A Keyboard 116B Mouse 120 Machine learning processing program 122 Example Image Set 122A, 122A1, 122A2 Example Images 124 Annotator 126 Correct Answer Data 128, 128A, 128B Teacher Data 130A, 150A, 160A Division Regions 130A1 Central Region 130A2 - 130A9 Radial Regions 139, 149 Lumen Corresponding Positions 140 Frame 142 Model 144 Inference Result 146 Error 148 Adjustment Value 150A1 - 150A8, 160A1 - 160A8 1st - 8th Division Regions 152 Lumen Recognition Process 154 Confidence Information 156 Map 158 Confidence 162 First Visual Information 164 Second Visual Information 168 External Device 170 Network C1, C2, C3 Centers CD1, CD2, CD3 Circumferential Directions CL Center Line CL1 - CL8 1st - 8th Center Lines TH1 First Threshold TH2 Second Threshold TH3 Third Threshold TH4 Fourth Threshold TH5 Fifth Threshold

Claims

1. A processor is provided. The processor, A medical image generated by imaging the inside of a tubular organ including the lumen is input to the trained model, and the medical image or an image corresponding to the medical image is divided in a circumferential direction to obtain a plurality of divided regions, and certainty information is generated in which a certainty that the lumen exists in each of the plurality of divided regions is assigned to the plurality of divided regions; An output process is performed to distinguish between a case where the certainty information is not information in which a value exceeding a threshold is assigned as the certainty to two or more divided regions among the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, and a case where the certainty information is information in which a value exceeding a threshold is assigned as the certainty to two or more divided regions among the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction. Medical support equipment.

2. The output process includes: outputting first information indicating that the lumen is present in any of the plurality of segmented regions based on the certainty information when the certainty information is not information in which a value exceeding a threshold has been assigned as the certainty to two or more segmented regions among the plurality of segmented regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, and not outputting the first information when the certainty information is information in which a value exceeding a threshold has been assigned as the certainty to two or more segmented regions among the plurality of segmented regions that are in a positional relationship exceeding 90 degrees in the circumferential direction. The medical support device according to claim 1 .

3. The output process includes: When the certainty information is information in which a value exceeding a threshold is assigned as the certainty to two or more divided regions among the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, second information indicating that the lumen is present in a central region of the medical image is output, or a process of not outputting the second information is performed. The medical support device according to claim 1 .

4. The output process includes: When the certainty information is not information in which a value exceeding a threshold is assigned as the certainty to two or more partitioned regions that are in a positional relationship exceeding 90 degrees in the circumferential direction among the plurality of partitioned regions, a process of not outputting the second information is performed. The medical support device according to claim 3.

5. The output process includes a process of outputting the second information when the certainty information is information in which the values ​​are assigned to two or more partitioned regions of the plurality of partitioned regions that are equally spaced around a center of the medical image or the image and that are in a positional relationship that exceeds 90 degrees in the circumferential direction. The medical support device according to claim 3.

6. The output process includes a process of outputting the second information when the certainty information is information in which the values ​​are assigned to two or more partitioned regions that are in a positional relationship of 120 degrees or more in the circumferential direction among the plurality of partitioned regions. The medical support device according to claim 3.

7. The output process includes a process of outputting the second information when the certainty level information is information in which the values ​​are assigned to two or more partitioned regions that are arranged at regular intervals around the medical image or the entire circumference of the medical image among the plurality of partitioned regions. The medical support device according to claim 3.

8. The output process includes a process of outputting the second information when the certainty information is information in which the value is assigned to all of the plurality of segmented regions. The medical support device according to claim 3.

9. The output process includes a process of outputting the first information when the certainty information is information in which the value is assigned to a single segmented area among the plurality of segmented areas, and when the certainty information is information in which the value is assigned to two or more segmented areas among the plurality of segmented areas that are in a positional relationship of 90 degrees or less in the circumferential direction. The medical support device according to claim 2.

10. Each of the plurality of divided regions is an area obtained by radially dividing the medical image. The medical support device according to claim 1 .

11. The plurality of divided regions are arranged radially in eight. The medical support device according to claim 10.

12. the trained model is obtained by machine learning using training data including example images showing samples of the medical images, the example images being divided into a plurality of regions corresponding to the plurality of divided regions, and correct answer data corresponding to the example images; When the sample of the lumen is shown in a region other than the central region of the example image, the correct answer data is an annotation capable of identifying the position of the region in which the sample of the lumen is shown among the plurality of regions, When the sample of the lumen is shown in the central region of the sample image, the correct answer data is an annotation capable of identifying all positions of the multiple regions. The medical support device according to claim 1 .

13. The output process includes: when the certainty information is not information in which a value exceeding a threshold is assigned as the certainty to two or more of the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, among the plurality of divided regions, outputting the first information by displaying on a screen first information indicating that the lumen is present in any of the plurality of divided regions based on the certainty information; When the certainty information is information in which a value exceeding a threshold is assigned as the certainty to two or more divided regions among the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, the second information indicating that the lumen is present in a central region of the medical image is displayed on the screen, thereby outputting the second information. The medical support device according to claim 1 .

14. The processor, Displaying the medical image on the screen; Displaying the first information within the medical image displayed on the screen; Displaying the second information within the medical image displayed on the screen The medical support device according to claim 13.

15. The output process includes: outputting first information indicating that the lumen is present in any of the plurality of segmented regions based on the certainty information when the certainty information is not information in which a value exceeding a threshold has been assigned as the certainty to two or more segmented regions among the plurality of segmented regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, and outputting second information indicating that the lumen is present in a central region of the medical image when the certainty information is information in which a value exceeding a threshold has been assigned as the certainty to two or more segmented regions among the plurality of segmented regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, The first information and the second information are visible information that can visually identify the position of the lumen shown in the medical image. The medical support device according to claim 1 .

16. The medical image is an endoscopic image generated by imaging the inside of the hollow organ including the lumen with an endoscopic scope. The medical support device according to claim 1 .

17. A medical support device according to any one of claims 1 to 16; An endoscope, The medical image is generated by imaging the inside of the hollow organ including the lumen with the endoscope. Endoscopic device.

18. A medical image generated by imaging the inside of a tubular organ including the lumen is input to the trained model, and the medical image or an image corresponding to the medical image is divided in a circumferential direction to generate certainty information in which a certainty that the lumen exists in each of the plurality of divided regions is assigned to the plurality of divided regions; and performing an output process capable of distinguishing between a case in which the certainty information is not information in which a value exceeding a threshold has been assigned as the certainty to two or more divided regions among the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction, and a case in which the certainty information is information in which a value exceeding a threshold has been assigned as the certainty to two or more divided regions among the plurality of divided regions that are in a positional relationship exceeding 90 degrees in the circumferential direction. Medical assistance methods.

19. A medical image generated by imaging the inside of a tubular organ including the lumen is input to the trained model, and the medical image or an image corresponding to the medical image is divided in a circumferential direction to generate certainty information in which a certainty that the lumen exists in each of the plurality of divided regions is assigned to the plurality of divided regions; and A program for causing a computer to execute a process including performing an output process capable of distinguishing between a case in which the certainty information is not information in which a value exceeding a threshold has been assigned as the certainty information to two or more divided areas among the plurality of divided areas that are positioned more than 90 degrees in the circumferential direction, and a case in which the certainty information is information in which a value exceeding a threshold has been assigned as the certainty information to two or more divided areas among the plurality of divided areas that are positioned more than 90 degrees in the circumferential direction.

Images