Medical support device, endoscope system, medical support method, and program

Abstract

To provide a medical support device, an endoscope system, a medical support method, and a program capable of supporting intubation to the duodenal papilla when difficulty of the intubation to the duodenal papilla is a reference difficulty or more.SOLUTION: A medical support device includes a processor. The processor obtains difficulty of the intubation to the duodenal papilla, which is the difficulty determined on the basis of the duodenal papilla and / or a peripheral area of the duodenal papilla shown in an intestinal wall image obtained when the intestinal wall of the duodenal papilla is imaged by an endoscope scope. When the difficulty is the reference difficulty or more, the processor outputs support information for supporting the intubation.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] The present disclosure relates to a medical support device, an endoscope system, a medical support method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology for displaying an image from a camera on a flexible elongated member (see Figures 6A and 6B, 6D and 6E) and detecting and identifying the nipple for medical personnel. The controller described in Patent Document 1 also determines a target trajectory based on the nipple image and overlays it on the image to provide a visual intubation path for medical personnel. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-105685 Summary of the Invention

[0004] One embodiment of the present disclosure provides a medical support device, an endoscopic system, a medical support method, and a program that can support intubation of the duodenal papilla when the difficulty of intubation of the duodenal papilla is equal to or greater than a standard difficulty level. [Means for solving the problem]

[0005] A first aspect of the present disclosure is a medical support device that includes a processor, which acquires a level of difficulty of intubation of the duodenal papilla, the level of difficulty being determined based on the duodenal papilla and / or the area surrounding the duodenal papilla that is visible in an image of the intestinal wall obtained by imaging the duodenal wall with an endoscope, and outputs support information to assist in intubation if the level of difficulty is equal to or greater than a standard level of difficulty.

[0006] A second aspect of the present disclosure is the medical support device according to the first aspect, in which, when the difficulty level is determined based on the duodenal papilla, the difficulty level is determined based on the aspect of the duodenal papilla.

[0007] A third aspect of the present disclosure is the medical support device according to the second aspect, in which the aspect of the duodenal papilla includes a shape of the duodenal papilla.

[0008] A fourth aspect of the present disclosure is the medical support device according to the second or third aspect, in which the aspect of the duodenal papilla includes a size of the duodenal papilla.

[0009] A fifth aspect of the present disclosure is the medical support device according to any one of the second to fourth aspects, in which the aspect of the duodenal papilla includes a type of opening of the duodenal papilla.

[0010] A sixth aspect of the present disclosure is a medical support device according to any one of the second to fifth aspects, in which the aspect of the duodenal papilla includes a confluence of the bile duct and the pancreatic duct within the duodenal papilla.

[0011] A seventh aspect of the present disclosure is the medical support device according to any one of the second to sixth aspects, in which the aspect of the duodenal papilla includes the length of the oral protuberance.

[0012] An eighth aspect of the present disclosure is a medical support device according to any one of the first to seventh aspects, in which, when the difficulty level is determined based on the surrounding area, the difficulty level is determined based on the aspect of the surrounding area.

[0013] A ninth aspect of the present disclosure is the medical support device according to the eighth aspect, in which the aspect of the surrounding area includes the presence or absence of a diverticulum.

[0014] A tenth aspect of the present disclosure is the medical support device according to the eighth or ninth aspect, in which the aspect of the surrounding area includes deformation of the duodenum.

[0015] An eleventh aspect of the present disclosure is a medical support device according to any one of the first to tenth aspects, wherein the support information includes information indicating that the difficulty level is equal to or higher than a standard difficulty level.

[0016] A twelfth aspect of the present disclosure is a medical support device according to any one of the first to eleventh aspects, in which the support information includes incision area identification information capable of identifying an incision area for incising the duodenal papilla.

[0017] A thirteenth aspect of the present disclosure is the medical support device according to the twelfth aspect, in which the incision area is included in an area that avoids blood vessels that are included in the duodenal papilla and require a certain level of caution or more.

[0018] A fourteenth aspect of the present disclosure is a medical support device according to the twelfth or thirteenth aspect, in which an incision area is predicted by performing image processing on an intestinal wall image, and the incision area identification information is the predicted result of the incision area by the image processing.

[0019] A fifteenth aspect of the present disclosure is a medical support device according to the fourteenth aspect, in which the image processing is a process of inputting an intestinal wall image into the first trained model to generate a prediction result for the first trained model.

[0020] A 16th aspect of the present disclosure is a medical support device according to the 14th or 15th aspect, in which the intestinal wall image includes a first intestinal wall image in which blood vessels in the duodenal papilla that are at a certain level of attention or higher are identifiable, and the incision area is predicted within an area that avoids the blood vessels by performing image processing on the first intestinal wall image.

[0021] A seventeenth aspect of the present disclosure is a medical support device according to any one of the first to sixteenth aspects, in which the support information includes operation support information that supports the operation of an endoscope.

[0022] An 18th aspect of the present disclosure is a medical support device according to any one of the 1st to 17th aspects, in which the difficulty level is determined based on the duodenal papilla and / or surrounding area recognized by performing a recognition process on an intestinal wall image.

[0023] A 19th aspect of the present disclosure is a medical support device according to the 18th aspect, in which the recognition process is a process in which the intestinal wall image is input into the second trained model, thereby causing the second trained model to recognize the duodenal papilla and / or surrounding area that is shown in the intestinal wall image.

[0024] A twentieth aspect of the present disclosure is an endoscope system including a medical support device according to any one of the first to nineteenth aspects and an endoscope.

[0025] A 21st aspect of the present disclosure is a medical support method that includes obtaining a difficulty level for intubation of the duodenal papilla, the difficulty level being determined based on the duodenal papilla and / or the area surrounding the duodenal papilla that is visible in an image of the intestinal wall obtained by imaging the duodenal intestinal wall with an endoscopic scope, and outputting support information that supports intubation if the difficulty level is equal to or greater than a standard difficulty level.

[0026] A 22nd aspect of the present disclosure is a program for causing a computer to execute medical support processing, including obtaining a difficulty level for intubation of the duodenal papilla, the difficulty level being determined based on the duodenal papilla and / or the area surrounding the duodenal papilla that is visible in an image of the intestinal wall obtained by imaging the duodenal intestinal wall with an endoscopic scope, and outputting support information to assist in intubation if the difficulty level is equal to or greater than a standard difficulty level. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a conceptual diagram showing an example of an embodiment in which the duodenoscope system is used. [Figure 2] 1 is a conceptual diagram showing an example of the overall configuration of a duodenoscope system. [Figure 3] FIG. 2 is a block diagram showing an example of a hardware configuration of an electrical system of the duodenoscope system. [Figure 4] FIG. 1 is a conceptual diagram showing an example of the duodenum, bile duct, and pancreatic duct. [Figure 5] 2 is a block diagram showing an example of the main functions of a processor included in the medical support device and an example of information stored in a storage. FIG. [Figure 6] 10 is a conceptual diagram showing an example of a recognition process using a papilla recognition model by a recognition unit and a display process of an intestinal wall image by a control unit. FIG. [Figure 7] FIG. 10 is a conceptual diagram illustrating an example of a recognition process using a difficulty recognition model by a recognition unit. [Figure 8] FIG. 10 is a conceptual diagram illustrating an example of a prediction process using an insertion direction prediction model by a recognition unit. [Figure 9] 10 is a conceptual diagram illustrating an example of a process in which the control unit acquires support information from the support information table, and a process in which the control unit generates high-difficulty notification information. FIG. [Figure 10] 10 is a conceptual diagram showing an example of a display process using insertion direction information, support information, and high-difficulty notification information by a control unit. FIG. [Figure 11A] 10 is a flowchart showing an example of the flow of medical support processing. [Figure 11B] This is a continuation of the flowchart shown in FIG. 11A. [Figure 12] FIG. 10 is a conceptual diagram showing a modified example of the recognition process using the difficulty recognition model by the recognition unit. [Figure 13] 13 is a conceptual diagram showing an example of a mode in which high-difficulty notification information is generated and displayed on a screen based on a plurality of difficulty levels obtained by a recognition process using the difficulty level recognition model shown in FIG. 12. FIG. [Figure 14] FIG. 10 is a conceptual diagram showing an example of the processing contents of the control unit when the difficulty level of intubation into the duodenal papilla is determined based on the confluence type and deformation of the duodenum. [Figure 15] FIG. 10 is a conceptual diagram showing an example of image processing using an incision region prediction model by a recognition unit. [Figure 16] This is a conceptual diagram showing an example of a manner in which visible information (i.e., a mark of the planned incision area) based on incision area identification information obtained by image processing using an incision area prediction model by the recognition unit is superimposed on an intestinal wall image displayed on the screen. [Figure 17] A conceptual diagram showing an example of a series of processes in which a processor included in a computer issues a processing execution request to an external device via a network, the external device executes processing in response to the processing execution request, and the processor included in the computer receives the processing result from the external device. DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, examples of embodiments of a medical support device, an endoscope system, a medical support method, and a program according to the present disclosure will be described with reference to the accompanying drawings.

[0029] First, the terms used in the following description will be explained.

[0030] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". GPGPU is an abbreviation for "General-Purpose computing on Graphics Processing Units". APU is an abbreviation for "Accelerated Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". RAM is an abbreviation for "Random Access Memory". NVM is an abbreviation for "Non-volatile memory". EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". AI is an abbreviation for "Artificial Intelligence". BLI is an abbreviation for "Blue Light Imaging". LCI is an abbreviation for "Linked Color Imaging". I / F is an abbreviation for "Interface". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network".5G is an abbreviation for "5th Generation Mobile Communication System." IC is an abbreviation for "Integrated Circuit."

[0031] In the following description, a coded processor (hereinafter simply referred to as a "processor") may be a single physical or virtual computing device, or a combination of multiple physical or virtual computing devices. Furthermore, a processor may be a single type of computing device, or a combination of multiple types of computing devices. Examples of computing devices include a CPU, a GPU, a GPGPU, an APU, or a TPU.

[0032] In the following description, a signed memory is a memory such as a RAM in which information is temporarily stored, and is used as a work memory by a processor.

[0033] In the following description, the term "storage" refers to one or more nonvolatile storage devices that store various programs, various parameters, etc. Examples of nonvolatile storage devices include flash memory, magnetic disks, and magnetic tapes. Another example of storage is cloud storage.

[0034] In the following embodiments, the external I / F with a symbol controls the exchange of various information between multiple devices connected to each other. An example of the external I / F is a USB interface. A communication I / F including a communication processor, an antenna, etc. may be applied to the external I / F. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0035] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0036] As an example, as shown in Figure 1, a duodenoscope system 10 includes a duodenoscope 12 and a display device 13. The duodenoscope 12 is used by a doctor 14 in an endoscopic examination. In this embodiment, the duodenoscope system 10 is an example of an "endoscopic system" according to the present disclosure, and the duodenoscope 12 is an example of an "endoscopic scope" according to the present disclosure.

[0037] The duodenoscope system 10 is communicably connected to a communication device (not shown), and information obtained by the duodenoscope system 10 is transmitted to the communication device. The communication device receives the information transmitted from the duodenoscope system 10 and executes processing using the received information (for example, processing for recording in an electronic medical record, etc.).

[0038] The duodenoscope 12 is inserted into the upper gastrointestinal tract of a subject 20 (e.g., a patient). The duodenoscope 12 is an endoscope with an optical imaging function that irradiates light into the upper gastrointestinal tract of the subject 20 and captures an image of the light reflected by an intestinal wall 30, which is part of the upper gastrointestinal tract of the subject 20. The duodenoscope 12 captures an image of the intestinal wall 30, thereby obtaining an image showing the appearance of the intestinal wall 30 and outputting the image to the display device 13. The image of the intestinal wall 30 captured by the duodenoscope 12 is observed by a doctor 14 via the display device 13.

[0039] The duodenoscope 12 is equipped with a light source device 22, a control device 24, and a medical support device 25. The light source device 22, the control device 24, and the medical support device 25 are installed on a wagon 34. The wagon 34 has a plurality of stands arranged vertically, and the medical support device 25, the light source device 22, and the control device 24 are installed from the lower stand to the upper stand. In addition, a display device 13 is installed on the top stand of the wagon 34.

[0040] The control device 24 controls the entire duodenoscope 12. The medical support device 25, under the control of the control device 24, performs various image processing on the images obtained by capturing images of the intestinal wall 30 by the duodenoscope 12.

[0041] The display device 13 displays various information including images. Examples of the display device 13 include a liquid crystal display and an EL display. Alternatively, instead of the display device 13, or together with the display device 13, a tablet terminal with a display may be used.

[0042] A plurality of screens are displayed side by side on the display device 13. In the example shown in Fig. 1, screens 36A and 36B are shown as examples of the plurality of screens. Screen 36A displays an intestinal wall image 40 obtained by capturing an image of the intestinal wall 30 using the duodenoscope 12. The intestinal wall image 40 shows the intestinal wall 30. In the example shown in Fig. 1, the intestinal wall 30 includes a duodenal papilla 30A and a peripheral area 30B of the duodenal papilla 30A.

[0043] In this embodiment, the intestinal wall image 40 is an example of an "intestinal wall image" according to the present disclosure. Also, in this embodiment, the duodenal papilla 30A is an example of a "duodenal papilla" according to the present disclosure. Also, in this embodiment, the surrounding region 30B is an example of a "region surrounding the duodenal papilla" according to the present disclosure.

[0044] The intestinal wall image 40 is a moving image, and is configured to include a plurality of time-series frames 41. The plurality of time-series frames 41 are displayed on the screen 36A at a predetermined frame rate (for example, several tens of frames per second).

[0045] Screen 36A is the main screen, while screen 36B is a sub-screen. Screen 36B displays various information that assists the physician 14 in performing a procedure using the duodenoscope 12. The sizes of screens 36A and 36B may be fixed, or may be changeable in accordance with instructions given to the duodenoscope system 10 by the physician 14 or various conditions.

[0046] 2, the duodenoscope 12 includes an operating section 42 and an insertion section 44. The insertion section 44 is partially curved by operating the operating section 42. As the physician 14 operates the operating section 42, the insertion section 44 is inserted while curving in accordance with the shape of the upper gastrointestinal tract (for example, the shape of the duodenum).

[0047] A camera 48, an illumination device 50, a treatment opening 51, and an erection mechanism 52 are provided at the tip 46 of the insertion section 44. The camera 48 and the illumination device 50 are provided on the side of the tip 46. In other words, the duodenoscope 12 is configured as a side-viewing endoscope, which makes it easy to observe the inner wall of the duodenum, i.e., the intestinal wall 30.

[0048] The camera 48 is a device that captures images of the inside of the subject 20, i.e., the inside of the upper gastrointestinal tract, to generate an intestinal wall image 40 as a medical image. An example of the camera 48 is a CMOS camera. However, this is merely an example, and other types of cameras such as a CCD camera may also be used.

[0049] The illumination device 50 has an illumination window 50A. The illumination device 50 emits light through the illumination window 50A. Examples of the light emitted from the illumination device 50 include visible light (e.g., white light) and invisible light (e.g., near-infrared light). The illumination device 50 also emits special light through the illumination window 50A. Examples of the special light include light for BLI and / or light for LCI. The camera 48 captures images of the inside of the subject 20 by an optical method while light is being emitted from the illumination device 50 inside the subject 20.

[0050] The treatment opening 51 is used as a treatment tool ejection port for ejecting a treatment tool 54 from the distal end portion 46, a suction port for sucking blood and internal waste, and a delivery port for delivering a fluid.

[0051] A treatment tool 54 protrudes from the treatment opening 51 in accordance with the operation of the doctor 14. The treatment tool 54 is inserted into the insertion section 44 from a treatment tool insertion port 58. The treatment tool 54 passes through the insertion section 44 via the treatment tool insertion port 58 and protrudes from the treatment opening 51 into the body of the subject 20. In the example shown in FIG. 2 , a cannula 54A protrudes from the treatment opening 51 as the treatment tool 54. The cannula 54A is merely one example of the treatment tool 54, and other examples of the treatment tool 54 include a catheter, a guide wire, a papillotomy knife, and a snare.

[0052] The raising mechanism 52 changes the protruding direction of the treatment tool 54 protruding from the treatment opening 51. The raising mechanism 52 is provided with a guide 52A, and the guide 52A rises relative to the protruding direction of the treatment tool 54, thereby changing the protruding direction of the treatment tool 54 along the guide 52A. This makes it easier to protrude the treatment tool 54 toward the intestinal wall 30. In the example shown in FIG. 2, the raising mechanism 52 changes the protruding direction of the treatment tool 54 to a direction perpendicular to the traveling direction of the tip portion 46. The raising mechanism 52 is operated by the doctor 14 via the operating unit 42. This adjusts the degree of change in the protruding direction of the treatment tool 54.

[0053] The duodenoscope 12 is connected to the light source device 22 and the control device 24 via a universal cord 60. A reception device 62 is connected to the control device 24. A medical support device 25 is also connected to the control device 24. A display device 13 is also connected to the medical support device 25. That is, the control device 24 is connected to the display device 13 via the medical support device 25.

[0054] Here, the medical support device 25 is exemplified as an external device for expanding the functions performed by the control device 24, and therefore an example is given in which the control device 24 and the display device 13 are indirectly connected via the medical support device 25, but this is merely one example. For example, the display device 13 may be directly connected to the control device 24. In this case, for example, the functions of the medical support device 25 may be installed in the control device 24, or the control device 24 may be equipped with a function to cause a server (not shown) to execute the same processing as that executed by the medical support device 25 (for example, the medical support processing described below), and receive and use the processing results from the server.

[0055] The reception device 62 receives instructions from a user (for example, the doctor 14) and outputs the received instructions as an electrical signal to the control device 24. Examples of the reception device 62 include a keyboard, a mouse, a touch panel, a foot switch, and a microphone.

[0056] The control device 24 controls the light source device 22, exchanges various signals with the camera 48, and exchanges various signals with the medical support device 25.

[0057] The light source device 22 emits light under the control of the control device 24 and supplies the light to the illumination device 50. A light guide is built into the illumination device 50, and the light supplied from the light source device 22 passes through the light guide and is irradiated from an illumination window 50A. The control device 24 causes the camera 48 to capture an image, acquires an intestinal wall image 40 (see FIG. 1) from the camera 48, and outputs it to a predetermined output destination (for example, the medical support device 25).

[0058] The medical support device 25 performs various image processing on the intestinal wall image 40 input from the control device 24. The medical support device 25 outputs the intestinal wall image 40 that has been subjected to various image processing to a predetermined output destination (for example, the display device 13).

[0059] Although the embodiment in which the intestinal wall image 40 output from the control device 24 is output to the display device 13 via the medical support device 25 has been described above, this is merely one example. The control device 24 and the display device 13 may be connected, and the intestinal wall image 40 that has been subjected to image processing by the medical support device 25 may be displayed on the display device 13 via the control device 24.

[0060] 3, the control device 24 includes a computer 64, a bus 66, and an external I / F 68. The computer 64 includes a processor 70, a memory 72, and a storage 74. The processor 70, the memory 72, the storage 74, and the external I / F 68 are connected to the bus 66.

[0061] The external I / F 68 controls the exchange of various information between the processor 70 and one or more devices (hereinafter also referred to as "first external devices") that exist outside the control device 24.

[0062] A camera 48 is connected to the external I / F 68 as one of the first external devices, and the external I / F 68 controls the exchange of various information between the camera 48 and the processor 70. The processor 70 controls the camera 48 via the external I / F 68. The processor 70 also acquires, via the external I / F 68, an intestinal wall image 40 (see FIG. 1 ) obtained by the camera 48 capturing an image of the inside of the body of the subject 20.

[0063] The light source device 22 is connected to the external I / F 68 as one of the first external devices, and the external I / F 68 controls the exchange of various information between the light source device 22 and the processor 70. The light source device 22 supplies light to the illumination device 50 under the control of the processor 70. The illumination device 50 irradiates the light supplied from the light source device 22.

[0064] A reception device 62 is connected to the external I / F 68 as one of the first external devices, and the processor 70 acquires instructions accepted by the reception device 62 via the external I / F 68 and executes processing according to the acquired instructions.

[0065] The medical support device 25 includes a computer 76 and an external I / F 78. The computer 76 includes a processor 80, a memory 82, and a storage 84. The processor 80, the memory 82, the storage 84, and the external I / F 78 are connected to a bus 86. In this embodiment, the medical support device 25 is an example of a "medical support device" according to the present disclosure, the computer 76 is an example of a "computer" according to the present disclosure, and the processor 80 is an example of a "processor" according to the present disclosure.

[0066] The external I / F 78 controls the exchange of various information between the processor 80 and one or more devices (hereinafter also referred to as "second external devices") that exist outside the medical support device 25.

[0067] The control device 24 is connected to the external I / F 78 as one of the second external devices. In the example shown in Fig. 3, the external I / F 68 of the control device 24 is connected to the external I / F 78. The external I / F 78 controls the exchange of various information between the processor 80 of the medical support device 25 and the processor 70 of the control device 24. For example, the processor 80 acquires an intestinal wall image 40 (see Fig. 1) from the processor 70 of the control device 24 via the external I / Fs 68 and 78, and performs various image processing on the acquired intestinal wall image 40.

[0068] The display device 13, which serves as one of the second external devices, is connected to the external I / F 78. The processor 80 controls the display device 13 via the external I / F 78, thereby causing the display device 13 to display various information (for example, an intestinal wall image 40 that has been subjected to various image processing).

[0069] Incidentally, a procedure called ERCP (endoscopic retrograde cholangiopancreatography) examination is known as one medical procedure for the duodenum using a duodenoscope 12. As shown in FIG. 4 as an example, in an ERCP examination, for example, the duodenoscope 12 is first inserted into the duodenum 88 via the esophagus and stomach. In this case, the insertion state of the duodenoscope 12 may be confirmed using an X-ray image obtained by X-ray imaging. Then, the tip 46 of the duodenoscope 12 reaches the vicinity of the duodenal papilla 30A present in the intestinal wall 30.

[0070] In an ERCP examination, for example, a cannula 54A is inserted into the duodenal papilla 30A from the luminal side of the duodenum 88. Here, the duodenal papilla 30A is a portion that protrudes from the intestinal wall 30. A papillary prominence 90A, which is the tip of the duodenal papilla 30A, is provided with the ends of one or more ducts 92 that lead to internal organs (e.g., the gallbladder and pancreas), i.e., openings 90A1 leading to the ducts 92. In other words, the ducts 92 lead to the openings 90A1 present in the papillary prominence 90A.

[0071] Examples of the one or more ducts 92 include a bile duct 92A and a pancreatic duct 92B. The opening 90A1 may be provided separately for each of the bile duct 92A and the pancreatic duct 92B, or may be provided in common for both the bile duct 92A and the pancreatic duct 92B.

[0072] In an ERCP examination, X-ray imaging is performed with a contrast agent injected into the tube 92 through the opening 90A1. When inserting the cannula 54A into the tube 92, the physician 14 must accurately grasp the direction 94 of the tube 92. In particular, since the direction 94 near the opening 90A1 is approximately the same as the insertion direction of the cannula 54A relative to the opening 90A1, it is extremely important for the physician 14 to visually grasp the direction 94 near the opening 90A1.

[0073] Examples of the running direction 94 of the duct 92 include the running direction 94A of the bile duct 92A and the running direction 94B of the pancreatic duct 92B. When inserting the cannula 54A into the bile duct 92A, it is effective for the physician 14 to visually understand the running direction 94A, and when inserting the cannula 54A into the pancreatic duct 92B, it is effective for the physician 14 to visually understand the running direction 94B.

[0074] The difficulty of inserting the cannula 54A into the duct 92, i.e., the difficulty of intubating the duodenal papilla 30A, varies greatly depending on the conditions. For example, the difficulty of intubating the duodenal papilla 30A increases depending on the combination of the type of junction of the bile duct 92A and the pancreatic duct 92B within the duodenal papilla 30A (hereinafter simply referred to as the "junction type") and the type of opening 90A1. The difficulty of intubating the duodenal papilla 30A also increases depending on the shape of the duodenal papilla 30A, the size of the duodenal papilla 30A, the length of the oral protuberance of the duodenal papilla 30A, the state of the surrounding region 30B (e.g., whether or not a diverticulum is present), and / or deformation of the duodenum.

[0075] Therefore, when the difficulty of intubating the duodenal papilla 30A is above a certain level (for example, a level of difficulty at which a doctor with average ability to intubate the duodenal papilla would like information useful for assisting intubation of the duodenal papilla), it is preferable to display support information, which is information useful for assisting intubation of the duodenal papilla 30A, on screens 36A and / or 36B.

[0076] However, the doctor 14 performing the procedure does not have the time to judge for himself whether the difficulty of intubating the duodenal papilla 30A is above a certain level and to perform operations to display the necessary support information on the screens 36A and / or 36B.

[0077] In view of these circumstances, in this embodiment, as an example shown in FIG. 5, medical support processing is performed by the processor 80 of the medical support device 25. A medical support program 96 is stored in the storage 84. In this embodiment, the medical support program 96 is an example of a "program" according to the present disclosure. The processor 80 reads the medical support program 96 from the storage 84 and executes the read medical support program 96 on the memory 82, thereby performing the medical support processing. The medical support processing is realized by the processor 80 operating as a recognition unit 80A and a control unit 80B in accordance with the medical support program 96 executed on the memory 82.

[0078] The storage 84 stores a nipple recognition model 98, a difficulty recognition model 100, an insertion direction prediction model 102, and a support information table 104. As will be described in detail later, the difficulty recognition model 100 and the insertion direction prediction model 102 are used by the recognition unit 80A, and the support information table 104 is used by the control unit 80B.

[0079] 6, the recognition unit 80A and the control unit 80B acquire an intestinal wall image 40. For example, the recognition unit 80A and the control unit 80B acquire the intestinal wall image 40, which is generated by the camera 48 capturing images at an imaging frame rate (e.g., several tens of frames per second), from the camera 48, on a frame-by-frame basis. The control unit 80B displays the intestinal wall image 40 acquired from the camera 48 on the screen 36A.

[0080] On the other hand, the recognition unit 80A recognizes the duodenal papilla 30A and the surrounding area 30B shown in each of the multiple frames 41 included in the intestinal wall image 40 acquired from the camera 48. To achieve this, in this embodiment, the recognition unit 80A recognizes the duodenal papilla 30A and the surrounding area 30B using an AI-based method. Here, the recognition process is performed using a papilla recognition model 98. In this embodiment, the papilla recognition model 98 is an example of a "second trained model" according to the present disclosure.

[0081] The nipple recognition model 98 is a trained model for object recognition using an AI segmentation method, and is obtained by performing machine learning for frame 41. The nipple recognition model 98 is optimized by performing machine learning on a neural network using first training data, which is a data set including a plurality of data (i.e., data for a plurality of frames) in which first example data and first supervised data are associated with each other. In other words, the nipple recognition model 98 is a trained model optimized so that first supervised data is generated when the first example data is input.

[0082] The first example data is an image corresponding to frame 41 (in other words, a sample image that simulates frame 41). A first example of an image corresponding to frame 41 is an optical image actually obtained by a duodenoscope having the same configuration as duodenoscope 12. A second example of an image corresponding to frame 41 is a virtually created image (for example, an image generated by a generation AI).

[0083] The first correct answer data is correct answer data (i.e., annotations) for the first example data. That is, the first correct answer data is information that can distinguish and identify the duodenal papilla and the surrounding area of ​​the duodenal papilla that appear in the image used as the first example data. Here, as an example of the first correct answer data, annotations that identify the geometric characteristics (e.g., position, size, and shape) of the duodenal papilla that appear in the image used as the first example data, the anatomical characteristics of the duodenal papilla (e.g., the type of opening and the length of the oral protuberance), the geometric characteristics (e.g., position, size, and shape) of the surrounding area of ​​the duodenal papilla, and the anatomical characteristics of the surrounding area of ​​the duodenal papilla (e.g., the presence or absence of a diverticulum) are used.

[0084] The recognition unit 80A inputs frames 41 to the papilla recognition model 98 at the frame rate at which the frames 41 are displayed on the screen 36A. Each time a frame 41 is input, the papilla recognition model 98 recognizes the duodenal papilla 30A and the surrounding area 30B shown in the input frame 41, generates a papilla recognition result 106 as the recognition result, and outputs the generated result to the control unit 80B. The papilla recognition result 106 includes information indicating the geometric characteristics of the duodenal papilla 30A, the anatomical characteristics of the duodenal papilla 30A (e.g., the type of opening 90A1 and the length of the oral ridge), the geometric characteristics of the surrounding area 30B, and the anatomical characteristics of the surrounding area 30B (e.g., the presence or absence of a diverticulum).

[0085] 7, the recognition unit 80A determines whether the duodenal papilla 30A and the surrounding area 30B are included in the frame 41 input to the papilla recognition model 98 to obtain the papilla recognition result 106, based on the papilla recognition result 106. Here, if the duodenal papilla 30A and the surrounding area 30B are included in the frame 41 input to the papilla recognition model 98 to obtain the papilla recognition result 106, the recognition unit 80A inputs the frame 41 input to the papilla recognition model 98 to obtain the papilla recognition result 106 to the difficulty recognition model 100.

[0086] The difficulty recognition model 100 is a trained model for image processing by AI, and is obtained by performing machine learning for frame 41. The difficulty recognition model 100 is optimized by performing machine learning on a neural network using second training data, which is a data set including a plurality of data (i.e., data for a plurality of frames) in which second example data and second correct answer data are associated with each other. In other words, the nipple recognition model 98 is a trained model optimized so that first correct answer data is generated when first example data is input.

[0087] 7, the second training data is training data 108. Training data 108 includes a sample image 110, which is an example of the second example data, and difficulty level data 112, which is an example of the second correct answer data, and sample image 110 and difficulty level data 112 are associated with each other.

[0088] The sample image 110 is the same image as the image used in the above-mentioned first example data. The sample image 110 shows a duodenal wall 111, a duodenal papilla 111A, and a surrounding area 111B of the duodenal papilla 111A.

[0089] The difficulty data 112 is data indicating the difficulty of intubating the duodenal papilla 111A. The difficulty of intubating the duodenal papilla 111A is determined based on the duodenal papilla 111A and the surrounding area 111B. In the example shown in Fig. 7, difficulty X is indicated as the difficulty of intubating the duodenal papilla 111A. The difficulty data 112 is data indicating difficulty X.

[0090] 7, the difficulty level X is determined based on a papilla aspect 114 and a surrounding area aspect 116. The papilla aspect 114 is the aspect of the duodenal papilla 111A. The papilla aspect 114 includes a shape 114A of the duodenal papilla 111A, a size 114B of the duodenal papilla 111A, an opening type 114C (i.e., the type of opening of the duodenal papilla 111A), and an oral ridge length 114D (i.e., the length of the oral ridge of the duodenal papilla 111A).

[0091] In contrast, the peripheral region aspect 116 is the aspect of the peripheral region 111 B. The peripheral region aspect 116 includes the presence or absence of a diverticulum 116A (that is, the presence or absence of a diverticulum).

[0092] Shape 114A is assigned a difficulty level of x1. Size 114B is assigned a difficulty level of x2. Orifice type 114C is assigned a difficulty level of x3. Oral ridge length 114D is assigned a difficulty level of x4. Diverticulum presence / absence 116A is assigned a difficulty level of x5.

[0093] Difficulty x1 refers to the difficulty of intubating the duodenal papilla 111A in the case of shape 114A. Difficulty x2 refers to the difficulty of intubating the duodenal papilla 111A in the case of size 114B. Difficulty x3 refers to the difficulty of intubating the duodenal papilla 111A in the case of opening type 114C. Difficulty x4 refers to the difficulty of intubating the duodenal papilla 111A in the case of oral ridge length 114D. Difficulty x5 refers to the difficulty of intubating the duodenal papilla 111A in the case of presence or absence of diverticulum 116A.

[0094] For example, difficulty level X is a difficulty level determined by an annotator (i.e., the creator of the difficulty level data 112) comprehensively from the difficulties x1 to x5. An example of a difficulty level determined comprehensively from the difficulties x1 to x5 is an average of the difficulties x1 to x5.

[0095] The difficulty recognition model 100 is a trained model in which a model 107 (for example, a neural network) is optimized by performing machine learning using training data 108 on the model 107.

[0096] The recognition unit 80A inputs frames 41 to the difficulty recognition model 100 at the frame rate at which the frames 41 are displayed on the screen 36A. Each time a frame 41 is input, the difficulty recognition model 100 recognizes the difficulty of intubating the duodenal papilla 30A shown in the input frame 41, and generates a difficulty recognition result 118 as the recognition result. The difficulty recognition result 118 is output to the control unit 80B.

[0097] The difficulty recognition result 118 includes a difficulty Y. The difficulty Y indicates the difficulty of intubating the duodenal papilla 30A shown in the frame 41 input to the difficulty recognition model 100.

[0098] The difficulty recognition model 100 is a trained model obtained by machine learning using training data 108 including difficulty data 112 determined based on the duodenal papilla 111A and the surrounding area 111B. Therefore, the difficulty Y included in the difficulty recognition result 118 generated by the difficulty recognition model 100 can be said to be a difficulty determined based on the duodenal papilla 30A and the surrounding area 30B.

[0099] Moreover, the difficulty recognition model 100 is a trained model obtained by machine learning using training data 108 including difficulty data 112 determined based on a papilla aspect 114 and a surrounding area aspect 116. Therefore, the difficulty Y included in the difficulty recognition result 118 generated by the difficulty recognition model 100 can be said to be a difficulty determined based on the aspect of the duodenal papilla 30A and the aspect of the surrounding area 30B.

[0100] Moreover, the difficulty recognition model 100 is a trained model obtained by machine learning using training data 108 including difficulty data 112 determined based on a shape 114A, a size 114B, an opening type 114C, an oral bulge length 114D, and the presence or absence of a diverticulum 116A. Therefore, the difficulty Y included in the difficulty recognition result 118 generated by the difficulty recognition model 100 can be said to be a difficulty determined based on the shape of the duodenal papilla 30A, the size of the duodenal papilla 30A, the type of the opening 90A1 of the duodenal papilla 30A, the length of the oral bulge of the duodenal papilla 30A, and the presence or absence of a diverticulum in the surrounding area 30B.

[0101] 8, the recognition unit 80A determines whether a specific opening is included in a frame 41 input to the papilla recognition model 98 to obtain the papilla recognition result 106, based on the papilla recognition result 106. Here, the specific opening refers to an opening 90A1 that is generally recognized as being difficult to intubate the duodenal papilla 30A (for example, a nodular or villous opening 90A1). The recognition unit 80A also determines whether the difficulty level Y included in the difficulty level recognition result 118 is equal to or greater than a reference difficulty level.

[0102] Here, a first example of the standard difficulty level is the level of difficulty at which a doctor with below-average ability to intubate the duodenal papilla is expected to take a certain amount of time (e.g., 10 minutes) or more from start to finish. A second example of the standard difficulty level is the level of difficulty at which a doctor with average ability to intubate the duodenal papilla is expected to take a certain amount of time (e.g., 5 minutes) or more from start to finish. A third example of the standard difficulty level is the level of difficulty at which a doctor with above-average ability to intubate the duodenal papilla is expected to take a certain amount of time (e.g., 2 minutes) or more from start to finish. The standard difficulty level is statistically derived in advance based on multiple cases in which multiple doctors have actually performed intubation at the duodenal papilla.

[0103] If a specific opening is included in the frame 41 input to the papilla recognition model 98 to obtain the papilla recognition result 106, and the difficulty Y included in the difficulty recognition result 118 is equal to or greater than the reference difficulty level, the recognition unit 80A predicts the direction in which the cannula 54A is inserted into the duodenal papilla 30A (hereinafter simply referred to as the "insertion direction") based on the insertion direction prediction model 102. The insertion direction includes the direction in which the cannula 54A is inserted into the bile duct 92A and the direction in which the cannula 54A is inserted into the pancreatic duct 92B.

[0104] The insertion direction prediction model 102 is a trained model for prediction by AI, and is obtained by performing machine learning for frame 41. The insertion direction prediction model 102 is optimized by performing machine learning on a neural network using third training data, which is a data set including a plurality of data (i.e., data for a plurality of frames) in which third example data and third supervised data are associated with each other. In other words, the insertion direction prediction model 102 is a trained model optimized so that third supervised data is generated when third example data is input.

[0105] The third example data is a merged format with the image used in the first example data described above. The third correct answer data is correct answer data (i.e., annotation) for the third example data. That is, the third correct answer data is information that can identify the insertion direction that is suitable for the duodenal papilla shown in the image used as the third example data (for example, the insertion direction applied to the bile duct 92A and the insertion direction applied to the pancreatic duct 92B).

[0106] The recognition unit 80A inputs the same frames 41 as those input to the nipple recognition model 98 and junction format information 120 to the insertion direction prediction model 102 at the frame rate at which the frames 41 are displayed on the screen 36A in order to obtain the nipple recognition result 106. Here, the junction format information 120 is received by the reception device 62. The junction format information 120 is information indicating a junction format that has been identified in advance from examination results obtained by performing a CT examination, an MRI examination, or the like on the subject 20, for example.

[0107] When the frame 41 and the merging format information 120 are input to the insertion direction prediction model 102, the insertion direction prediction model 102 generates and outputs insertion direction information 122, which is information that can identify the insertion direction. In this embodiment, the insertion direction information 122 is an example of "support information" according to the present disclosure.

[0108] As an example, as shown in FIG. 9 , the support information table 104 includes support information 124. The support information 124 includes a plurality of opening specification information 124A, a plurality of junction type specification information 124B, and a plurality of operation support information 124C. The opening specification information 124A is information that can specify the opening 90A1. The junction type specification information 124B is information that can specify the junction type. The operation support information 124C is information that assists in operating the duodenoscope 12. In this embodiment, the operation support information 124C is an example of "operation support information" according to the present disclosure.

[0109] Opening identification information 124A is provided for each type of opening 90A1 in the support information table 104. The opening identification information 124A is information in which an opening schema 124A1, which is a schema capable of identifying the type of opening 90A1, is associated with opening text 124A2, which is text capable of identifying the type of opening 90A1.

[0110] The merging format specifying information 124B is provided for each merging format. Furthermore, the merging format specifying information 124B is associated with the opening specifying information 124A for each type of opening 90A1. The merging format specifying information 124B is information in which a merging format schema 124B1, which is a schema that can identify the merging format, is associated with a merging format text 124B2, which is text that can identify the merging format.

[0111] The operation support information 124C is associated with opening specification information 124A for each type of opening 90A1. Furthermore, the operation support information 124C is associated with junction type specification information 124B for each junction type. The operation support information 124C is information in which scope position information 124C1 and approach method information 124C2 are associated with each other. The scope position information 124C1 is information (here, as an example, text) that can identify a recommended position for the duodenoscope 12 (for example, a position where the objective lens of the camera 48 faces the opening 90A1 in a front view). The approach method information 124C2 is information (here, as an example, text) that can identify a method of approaching the opening 90A1 (for example, the close-up method, the upward method, etc.).

[0112] If the difficulty level Y included in the difficulty level recognition result 118 acquired from the recognition unit 80A is equal to or greater than the standard difficulty level, the control unit 80B generates high difficulty notification information 126. The high difficulty notification information 126 is information that notifies the user that the difficulty level Y is equal to or greater than the standard difficulty level. An example of the high difficulty notification information 126 is text indicating that the difficulty level Y is equal to or greater than the standard difficulty level. In this embodiment, the high difficulty notification information 126 is an example of "information indicating that the difficulty level is equal to or greater than the standard difficulty level" according to the present disclosure. Furthermore, in this embodiment, the high difficulty notification information 126 is an example of "support information" according to the present disclosure.

[0113] When the difficulty level Y included in the difficulty level recognition result 118 acquired from the recognition unit 80A is equal to or greater than the reference difficulty level, a specific opening is captured in the frame 41 input to the nipple recognition model 98 to obtain the nipple recognition result 106, and the merging type identified by the merging type information 120 accepted by the accepting device 62 is a specific merging type (e.g., partition type or common pipe type), the control unit 80B acquires support information 124 that meets the conditions from the support information table 104. Here, the support information 124 that meets the conditions refers to, for example, opening identification information 124A corresponding to the specific opening, merging type identification information 124B corresponding to the specific merging type, and operation support information 124C corresponding to these.

[0114] As an example, as shown in FIG. 10, the control unit 80B outputs to the display device 13 the insertion direction information 122 obtained from the insertion direction prediction model 102, the support information 124 obtained from the support information table 104, and the high difficulty notification information 126.

[0115] That is, when the difficulty Y included in the difficulty recognition result 118 is equal to or greater than the standard difficulty and a specific opening appears in the frame 41 input to the nipple recognition model 98, insertion direction information 122 is output to the display device 13. Also, when the difficulty Y included in the difficulty recognition result 118 is equal to or greater than the standard difficulty, a specific opening appears in the frame 41 input to the nipple recognition model 98, and the merging format identified from the merging format information 120 accepted by the accepting device 62 is a specific merging format, support information 124 is output to the display device 13. Furthermore, when the difficulty Y included in the difficulty recognition result 118 is equal to or greater than the standard difficulty, high difficulty notification information 126 is output to the display device 13.

[0116] In this manner, the insertion direction information 122, the support information 124, and the highly difficult notification information 126 are output to the display device 13, whereby visible information is displayed on the screens 36A and 36B of the display device 13. For example, the control unit 80B visualizes the insertion direction information 122 and displays it on the screen 36A. In the example shown in FIG. 10 , arrow marks 122A and 122B, which are visualizations of the insertion direction information 122, are superimposed on the frame 41 displayed on the screen 36A. The direction indicated by the arrow mark 122A is the direction in which the cannula 54A is inserted into the bile duct 92A, and the direction indicated by the arrow mark 122B is the direction in which the cannula 54A is inserted into the pancreatic duct 92B. The control unit 80B also displays the support information 124 on the screen 36B. Furthermore, the control unit 80B displays the highly difficult notification information 126 as text TX1 on the screen 36B.

[0117] Next, the operation of the portion of the duodenoscope system 10 according to the present disclosure will be described with reference to FIGS. 11A and 11B.

[0118] 11A and 11B show an example of the flow of medical support processing performed by the processor 80. The flow of medical support processing shown in Fig. 11A and 11B is an example of the "medical support method" according to the present disclosure.

[0119] In the medical support processing shown in Fig. 11A, first, in step ST10, the control unit 80B determines whether or not one frame of image data has been captured by the camera 48 of the intestinal wall 30. If one frame of image data has not been captured by the camera 48 in step ST10, the determination is negative, and the medical support processing proceeds to step ST42 shown in Fig. 11B. If one frame of image data has been captured by the camera 48 in step ST10, the determination is positive, and the medical support processing proceeds to step ST12.

[0120] In step ST12, the recognition unit 80A and the control unit 80B acquire the intestinal wall image 40 from the camera 48. The control unit 80B displays the intestinal wall image 40 on the screen 36A. After the processing of step ST12 is executed, the medical support processing proceeds to step ST14.

[0121] In step ST14, the recognition unit 80A inputs the frame 41 included in the intestinal wall image 40 acquired in step ST12 into the papilla recognition model 98. As a result, the papilla recognition model 98 generates a papilla recognition result 106. After the processing of step ST14 is executed, the medical support processing proceeds to step ST16.

[0122] In step ST16, the recognition unit 80A acquires the nipple recognition result 106 generated by the nipple recognition model 98. After the processing of step ST16 is executed, the medical support processing proceeds to step ST18.

[0123] In step ST18, the recognition unit 80A determines whether the duodenal papilla 30A is captured in the frame 41 input to the papilla recognition model 98 in step ST14, based on the papilla recognition result 106 acquired in step ST16. If the duodenal papilla 30A is not captured in the frame 41 input to the papilla recognition model 98 in step ST18, the determination is negative, and the medical support processing proceeds to step ST42 shown in Figure 11B. If the duodenal papilla 30A is captured in the frame 41 input to the papilla recognition model 98 in step ST18, the determination is positive, and the medical support processing proceeds to step ST20.

[0124] In step ST20, the recognition unit 80A inputs the frame 41 that was input to the nipple recognition model 98 in step ST14 to obtain the nipple recognition result 106 in step ST16 into the difficulty recognition model 100. As a result, the difficulty recognition model 100 generates the difficulty recognition result 118. After the processing of step ST20 is executed, the medical support processing proceeds to step ST22.

[0125] In step ST22, the recognition unit 80A acquires the difficulty level recognition result 118 generated by the difficulty level recognition model 100. After the processing of step ST22 is executed, the medical support processing proceeds to step ST24.

[0126] In step ST24, the control unit 80B determines whether the difficulty level Y included in the difficulty level recognition result 118 acquired in step ST22 is equal to or greater than the standard difficulty level. If the difficulty level Y included in the difficulty level recognition result 118 acquired in step ST22 is less than the standard difficulty level in step ST24, the determination is negative, and the medical support processing proceeds to step ST42 shown in Fig. 11B. If the difficulty level Y included in the difficulty level recognition result 118 acquired in step ST22 is equal to or greater than the standard difficulty in step ST24, the determination is positive, and the medical support processing proceeds to step ST26.

[0127] In step ST26, the control unit 80B generates high-difficulty notification information 126. Then, the control unit 80B displays the high-difficulty notification information 126 as text TX1 on the screen 36B. After the processing of step ST26 is executed, the medical support processing proceeds to step ST28.

[0128] In step ST28, the recognition unit 80A determines whether a specific opening is captured in the frame 41 input to the nipple recognition model 98 in step ST14 to obtain the nipple recognition result 106 in step ST16, based on the nipple recognition result 106 acquired in step ST16. If a specific opening is not captured in the frame 41 input to the nipple recognition model 98 in step ST14 to obtain the nipple recognition result 106 in step ST16, the determination in step ST28 is negative, and the medical support processing proceeds to step ST42 shown in Figure 11B. If a specific opening is captured in the frame 41 input to the nipple recognition model 98 in step ST14 to obtain the nipple recognition result 106 in step ST16, the determination in step ST28 is positive, and the medical support processing proceeds to step ST30.

[0129] In step ST30, the recognition unit 80A acquires the junction format information 120 accepted by the acceptance device 62. After the processing of step ST30 is executed, the medical support processing proceeds to step ST32.

[0130] In step ST32, the recognition unit 80A inputs the frame 41 input to the nipple recognition model 98 in step ST14 to obtain the nipple recognition result 106 in step ST16, and the merging format information 120 acquired in step ST30, to the insertion direction prediction model 102. As a result, the insertion direction prediction model 102 generates insertion direction information 122. After the processing of step ST32 is executed, the medical support processing proceeds to step ST34.

[0131] In step ST34, the control unit 80B acquires insertion direction information 122 generated by the insertion direction prediction model 102 to which the frame 41 and the merging format information 120 have been input in step ST32. Then, the control unit 80B displays the insertion direction information 122 as arrow marks 122A and 122B superimposed on the frame 41 displayed on the screen 36A. After the processing of step ST34 is executed, the medical support processing proceeds to step ST36 shown in FIG. 11B.

[0132] In step ST36, the control unit 80B determines whether all of the first, second, and third conditions are satisfied. The first condition refers to the condition that the difficulty level Y included in the difficulty level recognition result 118 acquired in step ST22 is equal to or greater than the reference difficulty level. The second condition refers to the condition that a specific opening is included in the frame 41 input to the nipple recognition model 98 in step ST14 to acquire the nipple recognition result 106 in step ST16. The third condition refers to the condition that the merging type identified from the merging type information 120 acquired in step ST30 is a specific merging type. If all of the first, second, and third conditions are not satisfied in step ST36, the determination is negative, and the medical support process proceeds to step ST42. If all of the first, second, and third conditions are satisfied in step ST36, the determination is positive, and the medical support process proceeds to step ST38.

[0133] In step ST38, the control section 80B acquires the support information 124 that meets the conditions from the support information table 104. After the processing of step ST38 is executed, the medical support processing proceeds to step ST40.

[0134] In step ST40, the control unit 80B displays the support information 124 acquired in step ST38 on the screen 36B. After the processing of step ST40 is executed, the medical support processing proceeds to step ST42.

[0135] In step ST42, the control unit 80B determines whether a condition for terminating the medical support process is satisfied. An example of the condition for terminating the medical support process is that an instruction to terminate the medical support process has been given to the duodenoscope system 10 (for example, that an instruction to terminate the medical support process has been accepted by the acceptance device 62).

[0136] In step ST42, if the condition for terminating the medical support process is not satisfied, the determination is negative, and the medical support process proceeds to step ST10 shown in Fig. 11A. In step ST42, if the condition for terminating the medical support process is satisfied, the determination is positive, and the medical support process ends.

[0137] As described above, in this embodiment, when the difficulty level Y determined based on the duodenal papilla 30A and the surrounding area 30B shown in the intestinal wall image 40 obtained by imaging the intestinal wall 30 with the duodenoscope 12 is equal to or greater than the standard difficulty level, the support information 124 is displayed on the screen 36B. Therefore, when the difficulty level Y of intubation of the duodenal papilla 30A is equal to or greater than the standard difficulty level, support for intubation of the duodenal papilla 30A can be provided. That is, the physician 14 performing the procedure does not need to determine for himself or herself whether the difficulty level Y of intubation of the duodenal papilla 30A is equal to or greater than the standard difficulty level or perform an operation to display the support information 124 on the screens 36A and / or 36B. However, since the support information 124 is displayed on the screen 36B when the difficulty level Y is equal to or greater than the standard difficulty level, the physician 14 can accurately intubate the duodenal papilla 30A in a short time.

[0138] Furthermore, in this embodiment, the support information 124 displayed on the screen 36B includes operation support information 124C. Therefore, compared to when the operation support information 124C is not displayed, it is possible to make it easier for the doctor 14 performing the procedure to perform the operation required for intubation of the duodenal papilla 30A as the operation of the duodenoscope 12.

[0139] Furthermore, in this embodiment, the difficulty level Y is determined based on the duodenal papilla 30A and the surrounding area 30B recognized by performing AI recognition processing (i.e., recognition processing using the papilla recognition model 98) on the intestinal wall image 40. Therefore, the difficulty level Y of intubation for the duodenal papilla 30A can be determined with higher accuracy than when the doctor 14 during the procedure identifies the duodenal papilla 30A and the surrounding area 30B only by visual inspection and determines the difficulty level of intubation for the duodenal papilla 30A.

[0140] Furthermore, in this embodiment, when the difficulty level Y determined based on the duodenal papilla 30A and the surrounding area 30B shown in the intestinal wall image 40 obtained by imaging the intestinal wall 30 with the duodenoscope 12 is equal to or greater than the reference difficulty level, the insertion direction information 122 is displayed as arrow marks 122A and 122B superimposed on the intestinal wall image 40. This allows the doctor 14 to insert the cannula 54A into the bile duct 92A and the pancreatic duct 92B accurately in a short time.

[0141] Furthermore, in this embodiment, when the difficulty level Y determined based on the duodenal papilla 30A and the surrounding area 30B shown in the intestinal wall image 40 obtained by imaging the intestinal wall 30 with the duodenoscope 12 is equal to or higher than the standard difficulty level, high difficulty notification information 126 is displayed on the screen 36B. This allows the doctor 14 performing the procedure to visually recognize that the difficulty level Y of intubation of the duodenal papilla 30A is equal to or higher than the standard difficulty level.

[0142] In this embodiment, the difficulty level Y is determined based on the state of the duodenal papilla 30A. Therefore, the difficulty level of intubation into the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation into the duodenal papilla 30A is determined without taking into account the state of the duodenal papilla 30A.

[0143] In this embodiment, the difficulty level Y is determined based on the shape of the duodenal papilla 30A. Therefore, the difficulty level of intubation into the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation into the duodenal papilla 30A is determined without taking into account the shape of the duodenal papilla 30A.

[0144] In this embodiment, the difficulty level Y is determined based on the size of the duodenal papilla 30A. Therefore, the difficulty level of intubation of the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation of the duodenal papilla 30A is determined without taking the size of the duodenal papilla 30A into consideration.

[0145] In this embodiment, the difficulty level Y is determined based on the type of the opening 90A1 of the duodenal papilla 30A. Therefore, the difficulty level of intubation of the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation of the duodenal papilla 30A is determined without taking into account the type of the opening 90A1 of the duodenal papilla 30A.

[0146] In this embodiment, the difficulty level Y is determined based on the length of the oral bulge of the duodenal papilla 30A. Therefore, the difficulty level of intubation of the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation of the duodenal papilla 30A is determined without taking into account the length of the oral bulge of the duodenal papilla 30A.

[0147] In this embodiment, the difficulty level Y is determined based on the state of the surrounding area 30B. Therefore, the difficulty level of intubation into the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation into the duodenal papilla 30A is determined without taking into account the state of the surrounding area 30B.

[0148] In this embodiment, the difficulty level Y is determined based on the presence or absence of a diverticulum in the surrounding region 30B. Therefore, the difficulty level of intubation into the duodenal papilla 30A can be determined with higher accuracy than when the difficulty level of intubation into the duodenal papilla 30A is determined without considering the presence or absence of a diverticulum in the surrounding region 30B.

[0149] In the above embodiment, data indicating difficulty X has been given as an example of the luminance data 112 included in the training data 108 used in machine learning to create the difficulty recognition model 100, but this is merely an example. For example, as shown in FIG. 12 , the difficulty data 112 may include data indicating difficulty x1, data indicating difficulty x2, data indicating difficulty x3, data indicating difficulty x4, and data indicating difficulty x5, in addition to data indicating difficulty X. When the difficulty recognition model 100 is generated by performing machine learning on the model 107 using training data 110 including the difficulty data 112 configured in this manner, the difficulty recognition model 100 generates and outputs a difficulty recognition result 128.

[0150] 13, the difficulty recognition result 128 includes difficulty levels y1, y2, y3, y4, and y5 in addition to difficulty level Y. Difficulty level Y included in the difficulty recognition result 128 is a difficulty level obtained by comprehensively evaluating difficulty levels y1, y2, y3, y4, and y5 as the difficulty of intubation of the duodenal papilla 30A. An example of the difficulty level comprehensively evaluated by evaluating difficulty levels y1, y2, y3, y4, and y5 is an average of difficulty levels y1, y2, y3, y4, and y5.

[0151] Difficulty y1 corresponds to the shape 128A of the duodenal papilla 30A. That is, difficulty y1 refers to the difficulty of intubating the duodenal papilla 30A for the shape 128A. Difficulty y2 corresponds to the size 128B of the duodenal papilla 30A. That is, difficulty y2 refers to the difficulty of intubating the duodenal papilla 30A for the size 128B. Difficulty y3 corresponds to the opening type 128C (i.e., the type of opening 90A1 of the duodenal papilla 30A). That is, difficulty y3 refers to the difficulty of intubating the duodenal papilla 30A for the opening type 128C. Difficulty y4 corresponds to the oral ridge length 128D (i.e., the length of the oral ridge of the duodenal papilla 30A). That is, difficulty level y4 refers to the difficulty level of intubation into the duodenal papilla 30A when the oral bulge length is 128D. Difficulty level y5 corresponds to the presence or absence of a diverticulum 128E (i.e., the presence or absence of a diverticulum). That is, difficulty level y5 refers to the difficulty level of intubation into the duodenal papilla 30A when the diverticulum is present or absent 128E.

[0152] The control unit 80B generates high difficulty notification information 129 based on the difficulty recognition result 128. The high difficulty notification information 129 includes the high difficulty notification information 126 described in the above embodiment and also includes element identification information. The element identification information refers to information that can identify which of multiple elements (here, as an example, shape 128A, size 128B, opening type 128C, oral bulge length 128D, and presence or absence of diverticulum 128E) increases the difficulty of intubation of the duodenal papilla 30A. Of the multiple elements, elements that increase the difficulty of intubation of the duodenal papilla 30A are elements with a difficulty level equal to or higher than the standard difficulty level. The control unit 80B identifies elements with a difficulty level equal to or higher than the standard difficulty level based on the difficulty recognition result 128, generates element identification information based on the identification result, and generates high difficulty notification information 129 including the element identification information. Then, the control unit 80B displays the high difficulty notification information 129 in text format on the screen 36B. As a result, the screen 36B displays the text TX1 as in the above embodiment, and also displays the text TX2. The text TX2 is text that can identify which of a plurality of elements (here, as examples, the shape 128A, the size 128B, the opening type 128C, the oral bulge length 128D, and the presence or absence of a diverticulum 128E) increases the difficulty of intubation of the duodenal papilla 30A. Here, the text TX1 and TX2 are illustrated, but these are merely examples, and an image (for example, a mark) or the like may be displayed visually instead of text.

[0153] 13, the difficulty level Y is determined based on the shape 128A, size 128B, opening type 128C, oral bulge length 128D, and presence or absence of diverticulum 128E, but this is merely an example. For example, the difficulty level Y may be determined based on the confluence type, which is one aspect of the duodenal papilla 30A, and the deformation of the duodenum, which is one aspect of the surrounding region 30B. The deformation of the duodenum occurs due to a medical procedure on the upper gastrointestinal tract (e.g., resection of at least a portion of the stomach, etc.).

[0154] As an example of determining the difficulty level Y based on the merging pattern and the duodenal deformation, there is a case in which the control unit 80B adjusts the difficulty level recognition result 128 based on the merging pattern information 120 and the duodenal deformation information 130, as shown in Fig. 14. In the example shown in Fig. 14, the merging pattern information 120 and the duodenal deformation information 130 are received by the reception device 62. The duodenal deformation information 130 includes information related to the deformation of the duodenum, such as information indicating that the duodenum is deformed, information indicating the degree of the duodenal deformation, and information that can identify a portion of the duodenum where deformation of a certain level or more has occurred.

[0155] The control unit 80B adjusts the difficulty levels y1, y2, y3, y4, and y5 in accordance with the merging format information 120 and the duodenum deformation information 130. For example, the control unit 80B adjusts the difficulty levels y1, y2, y3, y4, and y5 by multiplying the difficulty levels y1, y2, y3, y4, and y5 by coefficients determined in accordance with the merging format information 120 and the duodenum deformation information 130. For example, the coefficients by which the difficulty levels y1, y2, y3, y4, and y5 are multiplied may be calculated from an arithmetic expression in which the merging format information 120 and the duodenum deformation information 130 are independent variables and the coefficients are dependent variables. Here, an example is given in which both the junction format information 120 and the duodenal deformation information 130 are used, but this is merely an example, and it is also possible to use either the junction format information 120 or the duodenal deformation information 130.

[0156] 14, the difficulty recognition result 128 is adjusted based on the confluence type, which is one aspect of the duodenal papilla 30A, so the difficulty Y of intubation for the duodenal papilla 30A can be determined with higher accuracy than when the difficulty of intubation for the duodenal papilla 30A is determined without considering the confluence type. Also, in the example shown in Fig. 14, the difficulty recognition result 128 is adjusted based on the deformation of the duodenum, which is one aspect of the surrounding area 30B, so the difficulty Y of intubation for the duodenal papilla 30A can be determined with higher accuracy than when the difficulty of intubation for the duodenal papilla 30A is determined without considering the deformation of the duodenum.

[0157] 14 shows an example in which the duodenum deformation information 130 received by the reception device 62 is acquired by the control unit 80B, but this is merely one example. For example, in the process from when the duodenoscope 12 is inserted into the upper gastrointestinal tract to when it reaches the duodenum, an AI-based recognition process for the region of the upper gastrointestinal tract may be performed, and duodenum deformation information 130 may be generated based on the recognition result of the region of the upper gastrointestinal tract (for example, non-recognition of the region, a confidence level below a threshold, recognition of a deformed stomach, and / or recognition of a deformed duodenum).

[0158] Incidentally, when intubation through the duodenal papilla 30A is difficult, the doctor 14 may make an incision in the duodenal papilla 30A to expose the tube 92 from the duodenal papilla 30A. In this case, to achieve high-precision incision of the duodenal papilla 30A, it is advisable to superimpose information (e.g., visible information such as a line) that can identify the incision region for incising the duodenal papilla 30A (i.e., the region where the incision is planned) on the intestinal wall image 40 displayed on the screen 36A. To achieve such a superimposed display, it is important to first predict the incision region for incising the duodenal papilla 30A.

[0159] Therefore, in order to realize the prediction of the incision area, as shown in FIG. 15 as an example, when a duodenal papilla 30A and a surrounding area 30B are included in a frame 41 input to the papilla recognition model 98 to obtain a papilla recognition result 106, the recognition unit 80A performs prediction processing using an AI method. The prediction processing using the AI ​​method is realized by image processing using an incision area prediction model 132. That is, the recognition unit 80A inputs the frame 41 input to the papilla recognition model 98 to obtain the papilla recognition result 106 into the incision area prediction model 132, thereby causing the incision area prediction model 132 to perform image processing. In this embodiment, the incision area prediction model 132 is an example of a "first trained model" according to the present disclosure. Furthermore, in this embodiment, the image processing using the incision area prediction model 132 is an example of "image processing" according to the present disclosure.

[0160] The incision area prediction model 132 is a trained model for prediction by AI, and is obtained by performing machine learning for frame 41. The incision area prediction model 132 is optimized by performing machine learning on a neural network using fourth training data, which is a data set including a plurality of data (i.e., data for a plurality of frames) in which fourth example data and fourth correct answer data are associated with each other. In other words, the insertion direction prediction model 132 is a trained model optimized so that fourth correct answer data is generated when the fourth example data is input.

[0161] 15, the fourth training data is training data 133. Training data 133 includes sample image 135, which is an example of the fourth example data, and incision area data 137, which is an example of the fourth correct answer data, and sample image 135 and incision area data 137 are associated with each other.

[0162] The sample image 135 is the same image as the image used in the above-mentioned first example data. The sample image 135 shows a duodenal wall 134, a duodenal papilla 134A, and a surrounding area 134B of the duodenal papilla 134A.

[0163] The incision region data 137 is data that can identify the incision region 136 for incising the duodenal papilla 134A. The data that can identify the incision region 136 is data (e.g., coordinates) that can identify the geometric characteristics (e.g., position, size, and shape) of the incision region 136 within the sample image 135.

[0164] The incision region 136 is included in an area that avoids a plurality of blood vessels 134C in the duodenal papilla 134A. The plurality of blood vessels 134C are blood vessels included in the duodenal papilla 134A that require a certain level of attention. The certain level of attention refers to the level of attention required for the thinnest blood vessels that are expected to cause bleeding associated with blood vessel damage to make the duct (e.g., bile duct and / or pancreatic duct) invisible. If the sample image 135 is an image obtained by irradiating the intestinal wall 134 with special light and capturing reflected light, the locations of blood vessels included in the duodenal papilla 134A that require a certain level of attention can be visually identified by an annotator.

[0165] The incision region prediction model 132 is optimized by performing machine learning using training data 133 on a model 139 (for example, a neural network).

[0166] The recognition unit 80A inputs frames 41 to the incision region prediction model 132 at the frame rate at which the frames 41 are displayed on the screen 36A. An example of the frames 41 input to the incision region prediction model 132 is an image obtained by capturing reflected light obtained by irradiating the intestinal wall 30 with special light. The image obtained by capturing reflected light obtained by irradiating the intestinal wall 30 with special light is an image in which blood vessels of a certain level or higher contained in the duodenal papilla 30A are captured in a state in which they can be identified. In the example shown in FIG. 15, the frames 41 input to the incision region prediction model 132 are an example of a "first intestinal wall image" according to the present disclosure.

[0167] Each time a frame 41 is input, the incision area prediction model 132 predicts the incision area for incising the duodenal papilla 30A shown in the input frame 41 (i.e., the area planned for incision). The incision area for incising the duodenal papilla 30A is predicted within an area that avoids blood vessels included in the duodenal papilla 30A that require a certain level of attention by performing image processing on the frame 41 using the incision area prediction model 132. The incision area prediction model 132 generates incision area identification information 138 as a prediction result. The incision area identification information 138 refers to information that can identify the incision area for incising the duodenal papilla 30A. In this embodiment, the incision area identification information 138 is an example of the "incision area identification information" and "support information" according to the present disclosure.

[0168] As an example, as shown in FIG. 16 , the control unit 80B acquires incision region identification information 138 from the recognition unit 80A. The control unit 80B superimposes the incision region identification information 138 on the frame 41 displayed on the screen 36A as a planned incision region mark 142, which is a mark that can identify the incision region, in a display mode that allows it to be distinguished from other image regions. In the example shown in FIG. 16 , the planned incision region mark 142 is a line-shaped mark. The incision region identified by the planned incision region mark 142 is included in an area that avoids multiple blood vessels 140 included in the duodenal papilla 30A. The multiple blood vessels 140 are blood vessels included in the duodenal papilla 30A that require a certain level of attention or higher. In this embodiment, the multiple blood vessels 140 are an example of "blood vessels included in the duodenal papilla that require a certain level of attention or higher" according to the present disclosure.

[0169] In this way, in the examples shown in Figures 15 and 16, the incision area identification information 138 is displayed superimposed on the frame 41 displayed on the screen 36A as the planned incision area mark 142 in a display manner that makes it distinguishable from other image areas, so that the doctor 14 can perform the incision of the duodenal papilla 30A with greater precision than when the incision area identification information 138 is not displayed.

[0170] 15, the incision area prediction model 132 predicts an incision area in an area that avoids multiple blood vessels 140 that are included in the duodenal papilla 30A and that require a certain level of attention. Therefore, the doctor 14 can incise the duodenal papilla 30A without damaging the blood vessels 140.

[0171] 15, an incision area is predicted by executing image processing on frame 21 using incision area prediction model 132, and incision area identification information 138 is obtained as a prediction result of the image processing. Therefore, compared to when doctor 14 identifies the incision area only by visual inspection, doctor 14 can identify the incision area with higher accuracy.

[0172] 15, an image in which blood vessels in the duodenal papilla 30A that require a certain level of attention can be identified is used as the frame 21 input to the incision area prediction model 132. Therefore, compared to when the doctor 14 identifies the incision area only visually, the doctor 14 can accurately identify the incision area that avoids blood vessels in the duodenal papilla 30A that require a certain level of attention.

[0173] In the above embodiment, an example was given in which the difficulty data 112 is determined based on the nipple area configuration 114 and the surrounding area configuration 116, but this is merely one example, and the difficulty data 112 may also be determined based on either the nipple area configuration 114 or the surrounding area configuration 116.

[0174] In the above embodiment, an example was given in which nipple configuration 114 includes shape 114A, size 114B, opening type 114C, and mouth-side protuberance length 114D, but this is merely one example, and nipple configuration 114 may include at least one of shape 114A, size 114B, opening type 114C, and mouth-side protuberance length 114D.

[0175] In the above embodiment, the explanation is given on the assumption that the doctor 14 completes the intubation of the duodenal papilla 30A, but when it is recognized by the AI ​​recognition processing that the intubation of the duodenal papilla 30A has not been completed even after a certain time (for example, 10 minutes) has elapsed, the processor 80 may display information prompting a change of the operator and / or information prompting a change of the surgical procedure on the screens 36A and / or 36B. Furthermore, the information prompting a change of the operator and / or information prompting a change of the surgical procedure may be output by voice.

[0176] In the above embodiment, an example was given in which the insertion direction information 122, the support information 124, and the high-difficulty notification information 126 are displayed as visible information, but this is merely one example, and the insertion direction information 122, at least a portion of the support information 124, and / or the high-difficulty notification information 126 may be output as audio.

[0177] In the above embodiment, a trained model for object recognition using an AI segmentation method was exemplified as the nipple recognition model 98, but the present disclosure also applies when a trained model for object recognition using an AI bounding box method is used.

[0178] In the above embodiment, an example has been given in which the insertion direction information 122 is superimposed on the frame 21 as visible information, but the present disclosure is not limited to this. For example, the position of the opening 90A1 may be identified by image processing using the nipple recognition model 98, and an image (for example, a mark) that can identify the position of the opening 90A1 may be superimposed on the frame 21.

[0179] In the above embodiment, information indicating a junction type previously identified from examination results is given as an example of the junction type information 120, but the present disclosure is not limited to this. For example, the junction type may be identified visually by the doctor 14 from images obtained by X-ray imaging during the ERCP examination, or the junction type may be identified by performing image analysis processing on images obtained by X-ray imaging during the ERCP examination by the computer 76 or the like, and the junction type information 120 may be generated based on the identification results.

[0180] In the above embodiment, an example in which the medical support processing is performed by the computer 76 has been described, but the present disclosure is not limited to this, and at least a part of the processing included in the medical support processing may be performed by a device provided outside the computer 76. An example of this case will be described below with reference to FIG.

[0181] 17 is a conceptual diagram showing an example of the configuration of a duodenoscope system 144. The duodenoscope system 144 is an example of an "endoscopic system" according to the present disclosure. The duodenoscope system 144 differs from the duodenoscope system 10 described in the above embodiment in that it includes an external device 146.

[0182] The external device 146 is communicatively coupled to the computer 76 via a network 148 (eg, a WAN and / or a LAN, etc.).

[0183] An example of the external device 146 is at least one server that directly or indirectly transmits and receives data to and from the computer 76 via the network 148. The external device 146 receives a processing execution instruction provided from the processor 80 of the computer 76 via the network 148. The external device 146 then executes processing in accordance with the received processing execution instruction and transmits the processing result to the computer 76 via the network 148. In the computer 76, the processor 80 receives the processing result transmitted from the external device 146 via the network 148 and executes processing using the received processing result.

[0184] An example of the processing execution instruction is an instruction to cause the external device 146 to execute at least a part of the medical support processing. A first example of at least a part of the medical support processing (i.e., the processing to be executed by the external device 146) is AI processing using the nipple recognition model 98, the difficulty recognition model 100, the insertion direction prediction model 102, and / or the incision area prediction model 132 described above. In this case, the external device 146 executes the AI ​​processing in accordance with the processing execution instruction provided from the processor 80 via the network 148, and transmits the AI ​​processing result to the computer 76 via the network 148. In the computer 76, the processor 80 receives the AI ​​processing result and executes processing similar to that in the above embodiment using the received AI processing result.

[0185] A second example of at least a portion of the medical support processing (i.e., processing to be executed by the external device 146) is a portion of processing by the control unit 80B. For example, processing to acquire support information 124 from the support information table 104, processing to make various determinations, and / or processing to generate information can be cited. In this case, the external device 146 executes processing to acquire support information 124 from the support information table 104, processing to make various determinations, and / or processing to generate information in accordance with a processing execution instruction provided from the processor 80 via the network 148, and transmits the processing results to the computer 76 via the network 148. In the computer 76, the processor 80 receives the processing results and executes processing similar to that of the above embodiment using the received processing results.

[0186] For example, the external device 146 is realized by cloud computing. Note that cloud computing is merely one example, and the external device 146 may be realized by network computing such as fog computing, edge computing, or grid computing. Instead of a server, at least one personal computer or the like may be used as the external device 146. Alternatively, the external device 146 may be a computing device with a communication function and equipped with multiple types of AI functions.

[0187] In the above embodiment, an example has been described in which the medical support program 96 is stored in the storage 84, but the present disclosure is not limited to this. For example, the medical support program 96 may be stored in a portable, computer-readable, non-transitory storage medium such as an SSD or a USB memory. The medical support program 96 stored in the non-transitory storage medium is installed in the computer 76 of the duodenoscope system 10. The processor 80 executes medical support processing in accordance with the medical support program 96.

[0188] Alternatively, the medical support program 96 may be stored in a storage device such as another computer or server connected to the duodenoscope system 10 via a network, and the medical support program 96 may be downloaded and installed on the computer 76 in response to a request from the duodenoscope system 10.

[0189] It is not necessary to store the entire medical support program 96 in a storage device such as another computer or server device connected to the duodenoscope system 10, or to store the entire medical support program 96 in the storage 84; only a portion of the medical support program 96 may be stored.

[0190] The hardware resources that execute the medical support processing can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource that executes medical support processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with circuit configurations specifically designed to execute specific processes. Each processor has built-in or connected memory, and executes medical support processing by using the memory.

[0191] The hardware resource that executes the medical support processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the medical support processing may be a single processor.

[0192] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes medical support processing. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute medical support processing, on a single IC chip, as typified by SoCs. In this way, medical support processing is realized using one or more of the above-mentioned various processors as hardware resources.

[0193] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The above medical support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0194] The above-described description and illustrations are a detailed explanation of the parts related to the present disclosure and are merely an example of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or elements may be replaced with other parts from the above-described description and illustrations, as long as they do not deviate from the gist of the present disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the parts related to the present disclosure, the above-described description and illustrations omit explanations of common general technical knowledge that do not require particular explanation to enable the implementation of the present disclosure.

[0195] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0196] 10,144 Duodenoscope System 12 Duodenoscope 13 Display device 14. Doctor 20 Subject 22 Light source device 24 Control device 25 Medical support equipment 30,111,134 intestinal wall 30A,111A,134A Duodenal papilla 30B, 111B, 134B surrounding areas 34 Wagon 36A,36B screen 40 Intestinal wall images 41 frames 42 Operation section 44 Insertion section 46 Tip 48 Camera 50 Lighting equipment 50A Lighting window 51 Treatment opening 52 Orthostatic mechanism 52A Guide 54 Treatment tools 54A Cannula 58 Treatment tool insertion port 60 Universal Code 62 Reception device 64,76 Computer 66,86 bus 68,78 External I / F 70,80 processor 72,82 memory 74,84 storage 80A recognition part 80B Control section 88 Duodenum 90A Nipple protuberance 90A1 opening 92 tube 92A Bile duct 92B Pancreatic duct 94 Direction of travel 94A Bile duct direction 94B Pancreatic duct direction 96 Medical Assistance Program 98 Nipple Recognition Model 100 Difficulty Recognition Model 102 Insertion direction prediction model 104 Support Information Table 106 Nipple recognition results 107,139 models 108 Teacher Data 110,135 Sample images 112 Difficulty Data 114 Nipple aspect 114A,128A shape 114B, 128B size 114C,128C Opening type 114D,128D Oral ridge length 116 Surrounding area aspect 116A,128E Presence or absence of diverticulum 118 Difficulty recognition results 120 Confluence format information 122 Insertion direction information 122A, 122B arrow mark 124 Support Information 124A Aperture specific information 124A1 Opening Schema 124A2 Opening Text 124B Merging format specific information 124B1 Confluence form schema 124B2 Confluence format text 124C Operational assistance information 124C1 Scope Position Information 124C2 Approach Information 126,129 High difficulty notification information 128 Difficulty recognition result 130 Information on duodenal deformation 132 Incision Area Prediction Model 134C,140 Blood vessels 136 Incision area 137 incision area data 138 Incision area identification information 146 External device 148 Network TX1,TX2 text x1,x2,x3,x4,x5,X,y1,y2,y3,y4,y5,Y Difficulty

Claims

1. a processor; The processor: a difficulty level of intubation of the duodenal papilla, the difficulty level being determined based on the duodenal papilla and / or the area surrounding the duodenal papilla that is captured in an intestinal wall image obtained by capturing an image of the duodenal wall using an endoscope; If the difficulty is equal to or greater than a reference difficulty, support information for supporting the intubation is output. Medical support equipment.

2. When the difficulty level is determined based on the duodenal papilla, The difficulty level is determined based on the state of the duodenal papilla. The medical support device according to claim 1 .

3. The aspect of the duodenal papilla includes the shape of the duodenal papilla. The medical support device according to claim 2 .

4. The aspect of the duodenal papilla includes the size of the duodenal papilla. The medical support device according to claim 2 .

5. The aspect of the duodenal papilla includes the type of opening of the duodenal papilla. The medical support device according to claim 2 .

6. The configuration of the duodenal papilla includes a confluence of the bile duct and the pancreatic duct in the duodenal papilla. The medical support device according to claim 2 .

7. The aspect of the duodenal papilla includes the length of the oral ridge. The medical support device according to claim 2 .

8. When the difficulty level is determined based on the surrounding area, The difficulty level is determined based on the state of the surrounding area. The medical support device according to claim 1 .

9. The state of the surrounding area includes the presence or absence of diverticula. The medical support device according to claim 8.

10. The aspect of the surrounding area includes a modification of the duodenum. The medical support device according to claim 8.

11. The support information includes information indicating that the difficulty level is equal to or higher than the reference difficulty level. The medical support device according to claim 1 .

12. The support information includes incision area specifying information that can specify an incision area for incising the duodenal papilla. The medical support device according to claim 1 .

13. The incision area is included in the area avoiding blood vessels of a certain level or more that are included in the duodenal papilla. The medical support device according to claim 12.

14. image processing is performed on the intestinal wall image to predict the incision area; The incision area specifying information is a prediction result of the incision area by the image processing. The medical support device according to claim 12.

15. The image processing is a process of inputting the intestinal wall image into a first trained model to cause the first trained model to generate the prediction result. The medical support device according to claim 14.

16. The intestinal wall image includes a first intestinal wall image in which a blood vessel having a certain level of attention or more included in the duodenal papilla is captured in an identifiable state, The incision region is predicted within a region avoiding the blood vessels by performing the image processing on the first intestinal wall image. The medical support device according to claim 14.

17. The support information includes operation support information that supports the operation of the endoscope. The medical support device according to claim 1 .

18. The difficulty level is determined based on the duodenal papilla and / or the surrounding area recognized by executing a recognition process on the intestinal wall image. The medical support device according to claim 1 .

19. The recognition process is a process of inputting the intestinal wall image into a second trained model, thereby causing the second trained model to recognize the duodenal papilla and / or the surrounding area that appear in the intestinal wall image. The medical support device according to claim 18.

20. A medical support device according to any one of claims 1 to 19; The endoscope Endoscopy system.

21. Acquiring a degree of difficulty of intubation of the duodenal papilla, the degree of difficulty being determined based on the duodenal papilla and / or the area surrounding the duodenal papilla that is captured in an intestinal wall image obtained by capturing an image of the duodenal wall using an endoscope; and and outputting support information for supporting the intubation when the difficulty level is equal to or greater than a reference difficulty level. Medical support methods.

22. Acquiring a degree of difficulty of intubation of the duodenal papilla, the degree of difficulty being determined based on the duodenal papilla and / or the area surrounding the duodenal papilla that is captured in an intestinal wall image obtained by capturing an image of the duodenal wall using an endoscope; and A program for causing a computer to execute a medical support process that includes outputting support information for supporting the intubation when the difficulty level is equal to or higher than a reference difficulty level.

Images