Medical assistance device, endoscope device, medical assistance method, and program

Abstract

This medical assistance device comprises a processor. The processor acquires a plurality of frames obtained on a time-series basis by imaging the inside of a luminal organ with a camera while hemostatic forceps are inserted into the luminal organ, the plurality of frames including the steps of hemostatic treatment using the hemostatic forceps. In addition, the processor outputs hemostasis period specification information that can specify that a hemostasis period in which hemostatic treatment is performed is a period until a fixed time elapses from when the steps shown in the acquired plurality of frames change from the step for preparing hemostasis with the hemostatic forceps to the step for starting hemostasis with the hemostatic forceps.

Description

Medical support device, endoscope device, medical support method, and program

[0001] The present disclosure relates to a medical support device, an endoscope device, a medical support method, and a program.

[0002] Japanese Patent No. 6875038 discloses a method for analyzing a plurality of frame images constituting an image obtained from an endoscopic camera to determine whether or not a hemostatic device is included in the surgical field image. The technology described in Japanese Patent No. 6875038 controls the playback of a partial video if the surgical field image does not include a hemostatic device, and prevents the playback of the partial video if a hemostatic device appears in the surgical field image.

[0003] Japanese Patent Application Publication No. 2016-062488 describes a method for detecting a treatment tool used inside a patient's body during an endoscopic examination by image processing from frame images included in a video captured inside the patient's body using an endoscope. Japanese Patent Application Publication No. 2016-062488 also describes that treatment tools to be detected include hemostatic forceps, and that treatment tools may be detected using color information about the sheath. Japanese Patent Application Publication No. 2016-062488 further describes that images from the period from when the treatment tool enters the frame until when the treatment tool leaves the frame are recorded as video data.

[0004] International Publication No. 2022 / 181714A1 describes a method for using a trained model to recognize multiple steps that make up a surgery and identify instruments such as forceps from a series of predetermined frames in surgical images (video or still images captured continuously in chronological order) obtained during endoscopic surgery or the like.

[0005] Republished Publication No. 2019-181432 describes a surgery assistance system that uses a trained classifier to predict the occurrence of a dangerous situation. Republished Publication No. 2019-181432 also describes an aspect in which, in generating training data used to train the classifier, it is determined that hemostasis is being performed by detecting characteristics of a surgical instrument used for hemostasis from surgical images obtained from an endoscopic camera or the like.

[0006] Japanese Patent Application Publication No. 2020-081332 discloses an endoscopy-related business system that creates reports on endoscopic examinations. Japanese Patent Application Publication No. 2020-081332 describes an embodiment in which a procedure recognition unit included in the endoscopy-related business system recognizes the presence or absence of a biopsy by recognizing the forceps used for the biopsy (biopsy forceps) using one or more endoscopic images. Japanese Patent Application Publication No. 2020-081332 also describes that the recognition of the type of procedure includes hemostasis in addition to biopsy.

[0007] One embodiment of the present disclosure provides a medical support device, an endoscopic device, a medical support method, and a program that allow a user or the like to know the period during which hemostasis is being performed using hemostatic forceps.

[0008] A first aspect of the present disclosure is a medical support device that includes a processor, which acquires a plurality of frames obtained in chronological order by a camera capturing images of the inside of a tubular organ into which hemostatic forceps are inserted, the plurality of frames capturing stages of hemostatic treatment using hemostatic forceps, and outputs hemostatic period identification information that can identify that the period from when the stage captured in the acquired plurality of frames changes from the stage of preparing to stop hemostatic treatment using hemostatic forceps to the stage of starting hemostatic treatment using hemostatic forceps until a certain amount of time has passed is the hemostatic period during which hemostatic treatment is being performed.

[0009] A second aspect of the present disclosure is a medical support device according to the first aspect, in which a processor detects a change from the hemostasis preparation stage to the hemostasis initiation stage by performing object recognition processing based on multiple acquired frames, and the hemostasis period is the period from when the change is detected by the processor until a certain amount of time has elapsed.

[0010] A third aspect of the present disclosure is the medical support device according to the second aspect, in which the object recognition processing is realized by using a model obtained by performing machine learning on a neural network.

[0011] A fourth aspect of the present disclosure is a medical support device according to any one of the first to third aspects, in which the hemostatic forceps have a sheath, the plurality of frames include a first frame and a second frame, the first frame is a frame in which the sheath is visible, and the second frame is a frame in which the sheath is not visible, the hemostasis preparation stage is a stage in which the first frame is obtained, and the hemostasis initiation stage is a stage in which the second frame is obtained following the first frame.

[0012] A fifth aspect of the present disclosure is a medical support device according to any one of the first to third aspects, in which the hemostatic forceps have a sheath and a clip pin, the plurality of frames include a first frame and a second frame, the first frame is a frame in which the sheath is visible, and the second frame is a frame in which the clip pin is visible, the hemostasis preparation stage is a stage in which the first frame is obtained, and the hemostasis initiation stage is a stage in which the second frame is obtained following the first frame.

[0013] A sixth aspect of the present disclosure is a medical support device according to any one of the first to third aspects, in which the hemostatic forceps have a sheath and a clip pin, the plurality of frames include a first frame and a second frame, the first frame is a frame in which the sheath is visible, the second frame is a frame in which the clip pin is visible but the sheath is not visible, the hemostasis preparation stage is a stage in which the first frame is obtained, and the hemostasis initiation stage is a stage in which the second frame is obtained following the first frame.

[0014] A seventh aspect of the present disclosure is a medical support device according to any one of the fourth to sixth aspects, wherein the plurality of frames further includes a third frame, the third frame being a frame showing a sheath, and the device is reset if the third frame is obtained before a certain time has elapsed since the transition from the hemostasis preparation stage to the hemostasis initiation stage.

[0015] An eighth aspect of the present disclosure is a medical support device according to any one of the first to seventh aspects, in which hemostatic treatment-related information regarding hemostatic treatment within a hemostatic period is stored in a storage area.

[0016] A ninth aspect of the present disclosure is the medical support device according to the eighth aspect, in which the hemostasis treatment-related information includes hemostasis period specifying information or a part of the hemostasis period specifying information.

[0017] A tenth aspect of the present disclosure is a medical support device according to the ninth aspect, the device including, in part, start timing information that can identify the start timing of the hemostasis period and / or end timing information that can identify the end timing of the hemostasis period.

[0018] An eleventh aspect of the present disclosure is a medical support device according to any one of the eighth to tenth aspects, in which the hemostatic treatment-related information includes hemostatic location-related information regarding the location where the hemostatic treatment was performed within the tubular organ.

[0019] A twelfth aspect of the present disclosure is the medical support device according to the eleventh aspect, in which the hemostasis location-related information includes lesion-related information regarding a lesion that was present at the location where the hemostasis procedure was performed.

[0020] A thirteenth aspect of the present disclosure is the medical support device according to the twelfth aspect, in which the lesion-related information includes a lesion image obtained by capturing an image of the lesion with a camera.

[0021] A fourteenth aspect of the present disclosure is a medical support device according to any one of the eleventh to thirteenth aspects, in which the information related to the hemostasis location includes a post-hemostasis image obtained by capturing an image of the location where the hemostasis procedure was performed with a camera.

[0022] A fifteenth aspect of the present disclosure is a medical support device according to any one of the eleventh to fourteenth aspects, in which the information related to the hemostasis location includes a pre-hemostasis image obtained by capturing an image of the location where the hemostasis procedure was performed using a camera before the hemostasis procedure.

[0023] A sixteenth aspect of the present disclosure is a medical support device according to any one of the eighth to fifteenth aspects, in which the hemostatic treatment-related information includes count information that can identify the number of times hemostatic treatment has been performed within a tubular organ.

[0024] A seventeenth aspect of the present disclosure is a medical support device according to any one of the first to sixteenth aspects, in which the certain period of time is a statistically determined time required to perform one hemostatic procedure using hemostatic forceps.

[0025] An eighteenth aspect of the present disclosure is an endoscope device including the medical support device according to any one of the first to seventeenth aspects and a camera.

[0026] A nineteenth aspect of the present disclosure is a medical support method including acquiring a plurality of frames obtained in chronological order by a camera capturing images of the inside of a tubular organ into which hemostatic forceps are inserted, the plurality of frames capturing stages of hemostatic treatment using hemostatic forceps, and outputting hemostatic period identification information that can identify that the period from when the stage captured in the acquired plurality of frames changes from the stage of preparing to stop hemostatic treatment using hemostatic forceps to the stage of starting hemostatic treatment using hemostatic forceps until a certain amount of time has passed is the hemostatic period during which hemostatic treatment is being performed.

[0027] A twentieth aspect of the present disclosure is a program for causing a computer to execute processing including acquiring a plurality of frames obtained in chronological order by a camera capturing images of the inside of a tubular organ into which hemostatic forceps are inserted, the plurality of frames depicting stages of hemostatic treatment using hemostatic forceps, and outputting hemostatic period identification information that can identify that the period from when the stage depicted in the acquired plurality of frames changes from the stage of preparing to stop hemostatic treatment using hemostatic forceps to the stage of starting hemostatic treatment using hemostatic forceps until a certain amount of time has passed is the hemostatic period during which hemostatic treatment is being performed.

[0028] 1 is a conceptual diagram showing an example of a manner in which an endoscopic device is used. FIG. 1 is a conceptual diagram showing an example of the overall configuration of an endoscopic device. FIG. 2 is a block diagram showing an example of the hardware configuration of the electrical system of an endoscopic device. FIG. 3 is a block diagram showing an example of main functions of an embodiment of a processor included in a medical support device, and an example of information stored in a storage. FIG. 4 is a conceptual diagram showing an example of processing content of a recognition unit and a control unit. FIG. 5 is a graph (i.e., a histogram) showing an example of the relationship between the number of frames required for one hemostasis treatment using a clip pin in an endoscopic examination (i.e., the number of frames captured by the camera from the start to the end of one hemostasis treatment using a clip pin) and the number of hemostasis treatments using hemostatic forceps. FIG. 6 is a conceptual diagram showing an example of processing content in which a hemostasis treatment period is specified by the control unit. FIG. 7 is a conceptual diagram showing an example of processing content by the control unit when a sheath appears in a frame after a change from a hemostasis preparation stage to a hemostasis start stage and before the hemostasis treatment time has elapsed. FIG. 8 is a flowchart showing an example of the flow of medical support processing. FIG. 9 is a conceptual diagram showing an example of a manner in which hemostasis treatment-related information is stored in a storage area. FIG. 10 is a conceptual diagram showing an example of the content of hemostasis location-related information. FIG. 1 is a conceptual diagram showing an example of a series of processes in which a computer issues a processing execution request to an external device via a network, the external device executes processing in response to the processing execution request, and the computer receives the processing result from the external device.

[0029] Hereinafter, examples of embodiments of a medical support device, an endoscope device, a medical support method, and a program according to the present disclosure will be described with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". GPGPU is an abbreviation for "General-Purpose computing on Graphics Processing Units". APU is an abbreviation for "Accelerated Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". RAM is an abbreviation for "Random Access Memory". NVM is an abbreviation for "Non-volatile memory". EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". AI is an abbreviation for "Artificial Intelligence". BLI is an abbreviation for "Blue Light Imaging". LCI is an abbreviation for "Linked Color Imaging". I / F is an abbreviation for "Interface". SSL is an abbreviation for "Sessile Serrated Lesion". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network".5G is an abbreviation for "5th Generation Mobile Communication System."

[0032] In the following description, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU, a GPU, a GPGPU, an APU, and a TPU.

[0033] In the following description, a coded memory is at least one memory (e.g., volatile memory) such as a RAM in which information is temporarily stored and which is used by a processor as a working memory.

[0034] In the following description, the term "storage" refers to one or more nonvolatile storage devices that store various programs, various parameters, etc. Examples of nonvolatile storage devices include flash memory, magnetic disks, and magnetic tapes. Another example of storage is cloud storage.

[0035] In the following embodiments, the external I / F with a symbol controls the exchange of various information between multiple devices connected to each other. An example of the external I / F is a USB interface. A communication I / F including a communication processor, an antenna, etc. may be applied to the external I / F. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0036] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0037] Fig. 1 is a conceptual diagram showing an example of how an endoscopic device 10 is used. As shown in Fig. 1, the endoscopic device 10 is used by a doctor 12 in an endoscopic examination. The endoscopic examination is assisted by staff such as a nurse 14. In this embodiment, the endoscopic device 10 is an example of an "endoscopic device" according to the present disclosure.

[0038] The endoscope device 10 is communicatively connected to a communication device (not shown), and information obtained by the endoscope device 10 is transmitted to the communication device. An example of the communication device is a server and / or a client terminal (e.g., a personal computer and / or a tablet terminal) that manages various information such as electronic medical records. The communication device receives the information transmitted from the endoscope device 10 and performs processing using the received information (e.g., processing to store the information in an electronic medical record, etc.).

[0039] The endoscope device 10 includes an endoscope 16 , a display device 18 , a light source device 20 , a control device 22 , and a medical support device 24 .

[0040] The endoscope device 10 is a modality for performing medical examination on a large intestine 28 contained in the body of a subject 26 (e.g., a patient) using an endoscope 16. In this embodiment, the large intestine 28 is an object observed by a physician 12 during an endoscopic examination.

[0041] The endoscope 16 is used by the doctor 12 and inserted into a hollow organ of the subject 26. In this embodiment, the endoscope 16 is inserted into the large intestine 28 of the subject 26. The endoscope device 10 causes the endoscope 16 inserted into the large intestine 28 of the subject 26 to capture images of the inside of the large intestine 28 of the subject 26, and performs various medical procedures on the large intestine 28 as necessary.

[0042] The endoscopic device 10 captures images of the inside of the large intestine 28 of the subject 26, thereby acquiring and outputting images showing the state of the inside of the large intestine 28. In this embodiment, the endoscopic device 10 is an endoscope having an optical imaging function that captures images of reflected light obtained by irradiating the inside of the large intestine 28 with light 30 and reflecting it off a wall 32 of the large intestine 28.

[0043] Although an endoscopic examination of the large intestine 28 is illustrated here, this is merely one example, and the present disclosure also applies to an endoscopic examination of a hollow organ such as the esophagus, stomach, duodenum, or trachea.

[0044] The light source device 20, the control device 22, and the medical support device 24 are installed on a wagon 34. The wagon 34 has a plurality of stands arranged vertically, and the medical support device 24, the control device 22, and the light source device 20 are installed from the lower stand to the upper stand. In addition, the display device 18 is installed on the top stand of the wagon 34.

[0045] The control device 22 controls the entire endoscope device 10. The medical support device 24, under the control of the control device 22, performs various image processing on the images obtained by capturing images of the intestinal wall 32 by the endoscope 16.

[0046] The display device 18 displays various information including images. Examples of the display device 18 include a liquid crystal display and an EL display. Alternatively, a tablet terminal with a display may be used instead of the display device 18 or together with the display device 18.

[0047] A screen 35 is displayed on the display device 18. The screen 35 includes a plurality of display areas. The plurality of display areas are arranged side by side within the screen 35. In the example shown in FIG. 1 , a first display area 36 and a second display area 38 are shown as examples of the plurality of display areas. The size of the first display area 36 is larger than the size of the second display area 38. The first display area 36 is used as a main display area, and the second display area 38 is used as a sub-display area. Note that the size relationship between the first display area 36 and the second display area 38 is not limited to this, and may be any size relationship that fits within the screen 35.

[0048] An endoscopic moving image 39 is displayed in the first display area 36. The endoscopic moving image 39 is a moving image acquired by imaging the intestinal wall 32 inside the large intestine 28 of the subject 26 with the endoscope 16. In the example shown in Fig. 1, a moving image showing the intestinal wall 32 is shown as an example of the endoscopic moving image 39.

[0049] The intestinal wall 32 shown in the endoscopic video 39 includes at least one lesion 41 (for example, one lesion 41 in the example shown in FIG. 1 ) as a region of interest (i.e., a region to be observed) that is gazed upon by the physician 12, and the physician 12 can visually recognize the appearance of the intestinal wall 32, including the lesion 41, through the endoscopic video 39. In this embodiment, the lesion 41 is an example of a "lesion" according to the present disclosure.

[0050] There are various types of lesions 41, and examples of the types of lesions 41 include neoplastic polyps and non-neoplastic polyps. Examples of the types of neoplastic polyps include adenomatous polyps (e.g., SSL). Examples of the types of non-neoplastic polyps include hamartomatous polyps, hyperplastic polyps, and inflammatory polyps. Note that the types exemplified here are types that are anticipated in advance as types of lesions 41 when an endoscopic examination is performed on the large intestine 28, and the types of lesions will differ depending on the organ that is subjected to the endoscopic examination.

[0051] The image displayed in the first display area 36 is one frame 40 included in a moving image that includes a plurality of frames 40 arranged in chronological order. In other words, the plurality of frames 40 arranged in chronological order are displayed in the first display area 36 at a predetermined frame rate (e.g., several tens of frames per second). In this embodiment, the frame 40 is an example of a "frame" according to the present disclosure.

[0052] An example of a moving image displayed in the first display area 36 is a moving image in a live view format. The live view format is merely one example, and a moving image that is temporarily stored in a memory or the like and then displayed, such as a moving image in a post-view format, may also be used. Furthermore, each frame included in a recording moving image stored in a memory or the like may be reproduced and displayed on the screen 35 (e.g., the first display area 36) as an endoscopic moving image 39.

[0053] Within the screen 35, the second display area 38 exists outside the first display area 36. In the example shown in Fig. 1, the second display area 38 is adjacent to the first display area 36 and is displayed on the right side of the screen 35 when viewed from the front. The display position of the second display area 38 may be anywhere as long as it is different from the first display area 36, ​​but it is preferable that the second display area 38 be displayed in a position that allows comparison with the endoscopic moving image 39 displayed within the first display area 36.

[0054] The second display area 38 displays medical information 44, which is information related to medical care. Examples of the medical information 44 include information that assists the doctor 12 in making medical decisions. A first example of the information that assists the doctor 12 in making medical decisions includes various types of visible information about the subject 26 into which the endoscope 16 is inserted (e.g., name, gender, medications taken, medical history, blood pressure, and / or heart rate). A second example of the information that assists the doctor 12 in making medical decisions includes visible information such as text and / or images obtained by processing the endoscopic video 39 using AI (e.g., feature maps and / or information obtained by processing the feature maps).

[0055] Fig. 2 is a conceptual diagram showing an example of the overall configuration of the endoscope device 10. As shown in Fig. 2, the endoscope 16 includes an operation unit 46 and an insertion unit 48. The insertion unit 48 is partially curved by operating the operation unit 46. The insertion unit 48 is inserted into the large intestine 28 (see Fig. 1) while curving in accordance with the shape of the large intestine 28 in accordance with operation of the operation unit 46 by the physician 12 (see Fig. 1).

[0056] The distal end portion 50 of the insertion section 48 is provided with a camera 52, an illumination device 54, and a treatment tool opening 56. The camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end portion 50. Note that, although an example in which the camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end portion 50 is given here, this is merely one example, and the camera 52 and the illumination device 54 may be provided on the side surface of the distal end portion 50, so that the endoscope 16 is configured as a side-viewing scope.

[0057] The camera 52 is inserted into a body cavity of the subject 26 to capture an image of the observation target region. In this embodiment, the camera 52 captures an image of the inside of the subject 26 (for example, the inside of the large intestine 28) to obtain an endoscopic moving image 39. An example of the camera 52 is a CMOS camera. However, this is merely an example, and other types of cameras such as a CCD camera may also be used. In this embodiment, the camera 52 is an example of a "camera" according to the present disclosure.

[0058] The illumination device 54 has illumination windows 54A and 54B. The illumination device 54 emits light 30 (see FIG. 1 ) through the illumination windows 54A and 54B. Examples of the light 30 emitted from the illumination device 54 include visible light (e.g., white light) and invisible light (e.g., near-infrared light). The illumination device 54 also emits special light through the illumination windows 54A and 54B. Examples of the special light include light for BLI and / or light for LCI. The camera 52 captures images of the inside of the large intestine 28 using an optical method while the light 30 is being emitted by the illumination device 54 inside the large intestine 28.

[0059] The treatment tool opening 56 is an opening for allowing a treatment tool 58 to protrude from the distal end portion 50. The treatment tool opening 56 is also used as a suction port for sucking blood, internal waste, etc., and as a delivery port for delivering fluid.

[0060] A treatment tool insertion port 60 is formed in the operation unit 46, and the treatment tool 58 is inserted into the insertion section 48 from the treatment tool insertion port 60. The treatment tool 58 passes through the insertion section 48 and protrudes to the outside from the treatment tool opening 56. Examples of the treatment tool 58 include a hemostatic forceps 58A, a puncture needle, a high-frequency knife, a snare, a catheter, a guidewire, and a cannula. In the example shown in FIG. 2 , the treatment tool 58 is a hemostatic forceps 58A protruding from the treatment tool opening 56. The hemostatic forceps 58A is an example of a "hemostatic forceps" according to the present disclosure.

[0061] The hemostatic forceps 58A are used for hemostasis. Hemostasis is a procedure to stop bleeding from a bleeding site (e.g., a bleeding excision site by excising the lesion 41 with a snare or high-frequency knife, etc.). For example, clip hemostasis is used for hemostasis. Clip hemostasis is a method of stopping bleeding by pinching and compressing the bleeding site (e.g., a blood vessel or mucous membrane, etc.) without using high-frequency current, etc. Depending on the size and / or amount of bleeding of the bleeding site, 10 or more hemostasis procedures may be performed on a single lesion 41.

[0062] The hemostatic forceps 58A has a sheath 58A1 and a clip pin 58A2. The clip pin 58A2 protrudes from the distal end of the sheath 58A1 in response to operation of a lever (not shown) or the like by the physician 12, and clips the bleeding site, thereby stopping bleeding from the bleeding site. Once the clip pin 58A2 has clipped the bleeding site, it detaches from the sheath 58A1 and is left in the large intestine 28. Note that, in general, the clip pin 58A2 clipped to the bleeding site is expelled from the large intestine 28 together with stool after the endoscopic examination.

[0063] The endoscope 16 is connected to a light source device 20 and a control device 22 via a universal cord 62. A medical support device 24 and a reception device 64 are connected to the control device 22. A display device 18 is also connected to the medical support device 24. That is, the control device 22 is connected to the display device 18 via the medical support device 24.

[0064] Note that, because the medical support device 24 is exemplified here as an external device for expanding the functions performed by the control device 22, an example in which the control device 22 and the display device 18 are indirectly connected via the medical support device 24 is given, but this is merely one example. For example, the display device 18 may be directly connected to the control device 22. In this case, for example, the functions of the medical support device 24 may be installed in the control device 22, or the control device 22 may be equipped with a function for causing a server (not shown) to execute the same processing as that executed by the medical support device 24 (e.g., the medical support processing described below) and receiving and using the processing results from the server.

[0065] The reception device 64 receives instructions from the doctor 12 and outputs the received instructions as an electrical signal to the control device 22. Examples of the reception device 64 include a keyboard, a mouse, a touch panel, a foot switch, a microphone, and / or a remote control device.

[0066] The control device 22 controls the light source device 20 , exchanges various signals with the camera 52 , and exchanges various signals with the medical support device 24 .

[0067] The light source device 20 emits light under the control of the control device 22 and supplies the light to the illumination device 54. The illumination device 54 has a built-in light guide, and the light supplied from the light source device 20 passes through the light guide and is irradiated from illumination windows 54A and 54B. The control device 22 causes the camera 52 to capture an image, acquires an endoscopic moving image 39 (see FIG. 1 ) from the camera 52, and outputs it to a predetermined output destination (for example, the medical support device 24).

[0068] The medical support device 24 supports medical care (here, as an example, endoscopic examination) by performing various image processing on the endoscopic moving image 39 input from the control device 22. The medical support device 24 outputs the endoscopic moving image 39 that has been subjected to various image processing to a predetermined output destination (for example, the display device 18).

[0069] Although the embodiment in which the endoscopic moving image 39 output from the control device 22 is output to the display device 18 via the medical support device 24 has been described above, this is merely one example. For example, the control device 22 and the display device 18 may be connected, and the endoscopic moving image 39 that has been subjected to image processing by the medical support device 24 may be displayed on the display device 18 via the control device 22.

[0070] Fig. 3 is a block diagram showing an example of the hardware configuration of the electrical system of the endoscope device 10. As shown in Fig. 3, the control device 22 includes a computer 66, a bus 68, and an external I / F 70. The computer 66 includes a processor 72, a memory 74, and a storage 76. The processor 72, the memory 74, the storage 76, and the external I / F 70 are connected to the bus 68. The processor 72 controls the entire control device 22. The memory 74 and the storage 76 are used by the processor 72.

[0071] The external I / F 70 controls the exchange of various information between the processor 72 and one or more devices (hereinafter also referred to as “first external devices”) that exist outside the control device 22 .

[0072] The external I / F 70 is connected to the camera 52 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the camera 52 and the processor 72. The processor 72 controls the camera 52 via the external I / F 70. The processor 72 also acquires, via the external I / F 70, endoscopic moving images 39 (see FIG. 1) obtained by the camera 52 capturing images of the inside of the large intestine 28 (see FIG. 1).

[0073] The light source device 20 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the light source device 20 and the processor 72. The light source device 20 supplies light to the illumination device 54 under the control of the processor 72. The illumination device 54 irradiates the light supplied from the light source device 20.

[0074] A reception device 64 is connected to the external I / F 70 as one of the first external devices, and the processor 72 acquires instructions accepted by the reception device 64 via the external I / F 70 and executes processing according to the acquired instructions.

[0075] The medical support device 24 includes a computer 78 and an external I / F 80. The computer 78 includes a processor 82, a memory 84, and a storage 86. The processor 82, the memory 84, the storage 86, and the external I / F 80 are connected to a bus 88. In the present embodiment, the medical support device 24 is an example of a "medical support device" according to the present disclosure, the computer 78 is an example of a "computer" according to the present disclosure, and the processor 82 is an example of a "processor" according to the present disclosure.

[0076] The hardware configuration of the computer 78 (i.e., the processor 82, the memory 84, and the storage 86) is basically the same as the hardware configuration of the computer 66, so a description of the hardware configuration of the computer 78 will be omitted here.

[0077] The external I / F 80 controls the exchange of various information between the processor 82 and one or more devices (hereinafter also referred to as "second external devices") that exist outside the medical support device 24.

[0078] The control device 22 is connected to the external I / F 80 as one of the second external devices. In the example shown in Fig. 3, the external I / F 70 of the control device 22 is connected to the external I / F 80. The external I / F 80 controls the exchange of various information between the processor 82 of the medical support device 24 and the processor 72 of the control device 22. For example, the processor 82 acquires endoscopic moving images 39 (see Fig. 1 ) from the processor 72 of the control device 22 via the external I / Fs 70 and 80, and performs various image processing on the acquired endoscopic moving images 39.

[0079] The display device 18, which serves as one of the second external devices, is connected to the external I / F 80. The processor 82 controls the display device 18 via the external I / F 80, thereby causing the display device 18 to display various information (e.g., an endoscopic moving image 39 that has been subjected to various image processing).

[0080] During endoscopic examination, after the lesion 41 is excised, hemostatic treatment is performed to stop bleeding from the bleeding site (i.e., the site where the lesion 41 was excised) using hemostatic forceps 58A. After the hemostatic treatment is completed, the doctor 12 determines whether or not a next medical treatment (e.g., further hemostatic treatment) is necessary and determines the timing of performing the next medical treatment. Therefore, during endoscopic examination, it is important for the doctor 12 to accurately determine whether or not hemostatic treatment is currently being performed (especially whether or not the hemostatic treatment has been completed) in order to perform appropriate medical treatment.

[0081] In view of the above circumstances, in this embodiment, medical support processing is performed by a processor 82 of the medical support device 24, as shown in Fig. 4. Fig. 4 is a block diagram showing an example of the main functions of the processor 82 included in the medical support device 24 and an example of information stored in a storage 86.

[0082] A medical support program 90 is stored in the storage 86. The medical support program 90 is an example of a "program" according to the present disclosure. The processor 82 reads the medical support program 90 from the storage 86 and executes the read medical support program 90 on the memory 84 to perform medical support processing. The medical support processing is realized by the processor 82 operating as a recognition unit 82A and a control unit 82B in accordance with the medical support program 90 executed on the memory 84.

[0083] The storage 86 stores a recognition model 92. As will be described in detail later, the recognition model 92 is used by the recognition unit 82A.

[0084] FIG. 5 is a conceptual diagram illustrating an example of the processing performed by the recognition unit 82A and the control unit 82B. As shown in FIG. 5 , the recognition unit 82A and the control unit 82B acquire, frame by frame, a plurality of frames 40 in time series from the camera 52 included in an endoscopic moving image 39 generated by capturing images of the inside of the large intestine 28 at an imaging frame rate (e.g., several tens of frames per second) using the camera 52. In the example shown in FIG. 5 , the recognition unit 82A acquires a plurality of frames 40 each showing a bleeding site 42 (e.g., a site of bleeding caused by the resection of a lesion 41) and a stage of hemostasis treatment using hemostatic forceps 58A. The frames 40 acquired from the camera 52 by the recognition unit 82A are acquired by the control unit 82B in synchronization with the recognition unit 82A. The stage of hemostasis treatment using the hemostatic forceps 58A refers to, for example, a stage of preparing for hemostasis using the hemostatic forceps 58A and a stage of starting hemostasis using the hemostatic forceps 58A.

[0085] The hemostasis preparation stage is a stage in which the bleeding site 42 is being targeted (i.e., a stage in which hemostasis treatment is being prepared). During the stage in which the bleeding site 42 is being targeted, the sheath 58A1 (e.g., the sheath 58A1 in a state in which the clip pin 58A2 does not protrude from the tip) is shown in the frame 40. Therefore, the hemostasis preparation stage can also be said to be a stage in which the sheath 58A1 (e.g., the sheath 58A1 in a state in which the clip pin 58A2 does not protrude from the tip) is shown in the frame 40.

[0086] The hemostasis initiation stage is a stage at which hemostasis treatment for the bleeding site 42 is initiated. At the stage at which hemostasis treatment for the bleeding site 42 is initiated, the frame 40 shows the clip pin 58A2 to which the bleeding site 42 is clipped, but the sheath 58A1 is not. The hemostasis initiation stage can also be considered a stage at which the frame 40 shows the clip pin 58A2 to which the bleeding site 42 is clipped, but the sheath 58A1 is not. In this embodiment, a stage at which the state changes from the first state to the second state is used as an example of the hemostasis initiation stage. Here, the first state refers to a state in which the sheath 58A1 is captured in the frame 40 obtained immediately before the latest frame 40. The second state refers to a state in which the latest frame 40 shows the clip pin 58A2 to which the bleeding site 42 is clipped, but the sheath 58A1 is not captured.

[0087] In this embodiment, the frame 40 showing the sheath 58A1 is an example of a "first frame" according to the present disclosure, and the frame 40 showing the clip pin 58A2 detached from the sheath 58A1 is an example of a "second frame" according to the present disclosure.

[0088] The control unit 82B outputs the endoscopic moving image 39 to the display device 18. For example, the control unit 82B displays the endoscopic moving image 39 as a live view image in the first display area 36. That is, each time the control unit 82B acquires a frame 40 from the camera 52, the control unit 82B displays the acquired frame 40 in order in the first display area 36 at a display frame rate (e.g., several tens of frames per second). The control unit 82B also displays medical information 44 in the second display area 38. For example, the control unit 82B also updates the display content of the second display area 38 (e.g., the medical information 44) in accordance with the display content of the first display area 36.

[0089] The recognition unit 82A uses the endoscopic moving image 39 acquired from the camera 52 to recognize the hemostatic forceps 58A appearing in the endoscopic moving image 39. That is, the recognition unit 82A recognizes the hemostatic forceps 58A appearing in the frame 40 by sequentially performing a recognition process 96 on each of a plurality of frames 40 in chronological order included in the endoscopic moving image 39 acquired from the camera 52. For example, the recognition unit 82A individually recognizes the sheath 58A1 and the clip pin 58A2. For example, recognition of the clip pin 58A2 is recognition of the clip pin 58A2 in a state where it has been detached from the sheath 58A1.

[0090] The recognition process 96 is performed by the recognition unit 82A on the acquired frame 40 each time the frame 40 is acquired. The recognition process 96 is a process for recognizing each of the sheath 58A1 and the clip pin 58A2 using an AI-based method (i.e., object recognition process using machine learning). In the present embodiment, for example, the recognition process 96 is an object recognition process using AI using a segmentation method (e.g., semantic segmentation, instance segmentation, and / or panoptic segmentation).

[0091] Here, processing using a recognition model 92 is performed as the recognition processing 96. The recognition model 92 is a trained model for object recognition using an AI segmentation method. An example of a trained model for object recognition using an AI segmentation method is a model for semantic segmentation. An example of a model for semantic segmentation is a model with an encoder-decoder structure. An example of a model with an encoder-decoder structure is U-Net or HRNet. In this embodiment, the recognition processing 96 is an example of the "object recognition processing" according to the present disclosure.

[0092] The recognition model 92 is optimized by performing machine learning on the neural network using training data. The training data is a data set including a plurality of data (i.e., data for a plurality of frames) in which example data and correct answer data are associated with each other.

[0093] The example data are various images that are assumed to be frame 40. The correct answer data are correct answer data (i.e., annotations) for the example data. Here, annotations that identify the sheaths of various hemostats that appear in the images used as the example data and annotations that identify the clip pins of various hemostats that appear in the images used as the example data are used as examples of correct answer data.

[0094] The recognition unit 82A acquires a frame 40 from the camera 52 and inputs the acquired frame 40 to the recognition model 92. As a result, each time a frame 40 is input, the recognition model 92 recognizes whether the sheath 58A1 is captured in the input frame 40 and whether the clip pin 58A2 is captured in the input frame 40. The recognition model 92 then outputs a recognition result 98.

[0095] The recognition result 98 includes sheath presence / absence information 98A and clip pin presence / absence information 98B. The sheath presence / absence information 98A is information indicating whether a sheath 58A1 is present in the corresponding frame 40 (i.e., the frame 40 input to the recognition model 92). The clip pin presence / absence information 98B is information indicating whether a clip pin 58A2 is present in the corresponding frame 40 (i.e., the frame 40 input to the recognition model 92).

[0096] The recognition unit 82A acquires the recognition result 98 output from the recognition model 92 and outputs the acquired recognition result 98 to the control unit 82B. The control unit 82B executes processing (see, for example, FIGS. 7 and 8 ) using the recognition result 98 input from the recognition unit 82A.

[0097] Fig. 6 is a graph (i.e., a histogram) showing an example of the relationship between the number of frames required for one hemostatic procedure using the clip pin 58A2 during endoscopic examination (i.e., the number of frames captured by the camera 52 from the start to the end of one hemostatic procedure using the clip pin 58A2) and the number of hemostatic procedures using the hemostatic forceps 58A. The vertical axis in Fig. 6 represents the number of hemostatic procedures using the hemostatic forceps 58A (i.e., the number of procedures performed using the clip hemostatic method using the clip pin 58A2), and the horizontal axis in Fig. 6 represents the number of frames. Note that the graph shown in Fig. 6 was created from data obtained by performing multiple endoscopic examinations on the large intestines 28 of multiple subjects 26.

[0098] 6 , the medical support process according to this embodiment uses a hemostatic treatment time 100. The hemostatic treatment time 100 is an example of the “fixed time” according to the present disclosure. The hemostatic treatment time 100 is a predetermined fixed time required for one hemostatic treatment, and is statistically determined as the time required for one hemostatic treatment.

[0099] In the example shown in FIG. 6, in most endoscopic examinations, one hemostasis treatment is completed between the start of one hemostasis treatment using the clip pin 58A2 and the time when the camera 52 has captured approximately 1800 frames of images.

[0100] 6, the timing at which one hemostatic treatment using the clip pin 58A2 starts is the timing at which the bleeding site 42 is clipped by the clip pin 58A2 and the sheath 58A1 moves out of the frame 40. Also, in the example shown in FIG. 6, the timing at which one hemostatic treatment using the clip pin 58A2 ends is the timing at which approximately 1800 frames of image capturing have been performed by the camera 52 since the start of one hemostatic treatment using the clip pin 58A2.

[0101] In most endoscopic examinations, the timing at which approximately 1,800 frames are captured corresponds to the timing at which the sheath 58A1 re-enters the frame (the timing at which the sheath 58A1 is again captured in the frame 40). The sheath 58A1 re-enters the frame means that the Nth hemostasis procedure using the clip pin 58A2 has been completed and preparation for the N+1th hemostasis procedure has begun (for example, a stage in which a target for clipping with the clip pin 58A2 in the N+1th endoscopic examination (i.e., the bleeding site 42 where the previous hemostasis procedure was performed, or a bleeding site 42 different from the bleeding site 42 where the previous hemostasis procedure was performed) has begun). The timing at which approximately 1,800 frames are captured by the camera 52 after the start of the Nth hemostasis procedure using the clip pin 58A2 is sufficient as the time from the start to the end of the Nth hemostasis procedure. In the example shown in Figure 6, the sporadic hemostatic procedures performed after approximately 1,800 frames of imaging are highly likely to be irregular hemostatic procedures that are unlikely to occur in normal endoscopic examinations, and can therefore be considered statistically as noise.

[0102] In the medical support processing according to this embodiment, the hemostasis treatment time 100 is the time required for the camera 52 to capture 1,800 frames. However, this is merely an example. For example, the hemostasis treatment time 100 may be the time required for the camera 52 to capture the number of frames corresponding to the maximum number of hemostasis treatments in the graph shown in FIG. 6 , the time required for the camera 52 to capture the number of frames corresponding to the average number of hemostasis treatments in the graph shown in FIG. 6 , or the time required for the camera 52 to capture the number of frames corresponding to the median number of hemostasis treatments in the graph shown in FIG. 6 . While the time required to capture the number of frames corresponding to the statistical value is illustrated here, this is merely an example, and the hemostasis treatment time 100 may be a time predetermined for the statistical value.

[0103] Alternatively, the time required for the camera 52 to capture fewer than 1800 frames or the time required for the camera 52 to capture more than 1800 frames may be used as the hemostasis treatment time 100. Alternatively, the hemostasis treatment time 100 set as a default (e.g., the hemostasis treatment time 100 set as described above) may be adjusted according to instructions from the physician 12 or the like (e.g., instructions received by the reception device 64 or the like). In any case, the hemostasis treatment time 100 may be set as the start timing of the hemostasis treatment when the bleeding site 42 is clipped by the clip pin 58A2 and the sheath 58A1 moves out of the frame 40, and the end timing of the hemostasis treatment when the hemostasis treatment is completed. The hemostasis treatment time 100 is preferably a time statistically derived in advance from a history of multiple hemostasis treatments actually performed (e.g., collected data on the time required for each of multiple hemostasis treatments), or an adjusted time derived in this manner.

[0104] The timing to start the hemostatic treatment does not have to be the timing when the bleeding site 42 is clipped by the clip pin 58A2 and the sheath 58A1 is moved out of the frame 40, but may be the timing when the bleeding site 42 changes from a state where it is not clipped by the clip pin 58A2 to a state where it is clipped by the clip pin 58A2. In this case, for example, the state where the bleeding site 42 is not clipped by the clip pin 58A2 and the state where the bleeding site 42 is clipped by the clip pin 58A2 may be recognized by an AI-based object recognition process (e.g., an object recognition process using the recognition model 92) and / or a non-AI-based object recognition process (e.g., an object recognition process using template matching).

[0105] Furthermore, the timing to start the hemostatic treatment may be the timing when the state changes from one in which the sheath 58A1 is visible in the frame 40 to one in which the sheath 58A1 is not visible in the frame 40. In this case, for example, the state in which the sheath 58A1 is visible in the frame 40 and the state in which the sheath 58A1 is not visible in the frame 40 may be recognized by AI object recognition processing and / or non-AI object recognition processing.

[0106] Hemostatic treatment is a medical treatment performed during the period from when the hemostatic preparation stage changes to when the hemostatic start stage is reached until the hemostatic treatment time 100 has elapsed. In this embodiment, the start timing of the hemostatic treatment is the timing when the hemostatic preparation stage changes to when the hemostatic start stage is reached, and the end timing of the hemostatic treatment is the timing when the hemostatic treatment time 100 has elapsed since the hemostatic preparation stage changed to when the hemostatic start stage is reached. In other words, in the medical support processing according to this embodiment, the period from when the hemostatic treatment starts to when the hemostatic treatment ends is used as the hemostatic period during which the hemostatic treatment is being performed.

[0107] 7 is a conceptual diagram showing an example of processing details by which the control unit 82B determines the hemostasis treatment time 100. As shown in FIG. 7, the control unit 82B determines the hemostasis preparation stage and the hemostasis initiation stage based on the recognition result 98 input from the recognition unit 82A. That is, the control unit 82B determines whether the sheath 58A1 is shown in the frame 40 by referencing sheath presence / absence information 98A included in the recognition result 98. If the sheath 58A1 is shown in the frame 40, the control unit 82B determines that the stage in which the sheath 58A1 is shown in the frame 40 is the hemostasis preparation stage.

[0108] Next, the control unit 82B determines whether the sheath 58A1 is not captured in the frame 40 but the clip pin 58A2 is captured in the frame 40 by referring to the sheath presence / absence information 98A and the clip pin presence / absence information 98B included in the recognition result 98. Then, when a frame 40 in which the clip pin 58A2 is captured but the sheath 58A1 is not captured is obtained following a frame 40 in which the sheath 58A1 is captured, the control unit 82B determines that the stage in which the frame 40 in which the sheath 58A1 is captured but the clip pin 58A2 is captured is the hemostasis start stage.

[0109] Here, the control unit 82B determines that the period from the change from the hemostasis preparation stage to the hemostasis initiation stage until the elapse of the hemostasis treatment time 100 is the hemostasis period 101. That is, the control unit 82B detects the change from the hemostasis preparation stage to the hemostasis initiation stage based on the recognition result 98 and starts timing the hemostasis treatment time 100. The control unit 82B then identifies the period from the detection of the change from the hemostasis preparation stage to the hemostasis initiation stage until the elapse of the hemostasis treatment time 100 as the hemostasis period 101. When the control unit 82B identifies the hemostasis period 101, it outputs hemostasis period identification information 102, which can identify the hemostasis period 101, to the display device 18.

[0110] An example of the hemostasis period identification information 102 is text 44A that can identify that the current time is the hemostasis period 101. An example of the text 44A is the text "Hemostasis is in progress." The content of the text 44A may be any content that can identify that the current time is the hemostasis period 101. Although the text 44A is shown here as an example, it may be an image that can identify that the current time is the hemostasis period 101.

[0111] When the hemostasis period specifying information 102 is output to the display device 18, text 44A is displayed in the second display area 38 as part of the medical information 44 during the hemostasis period 101 (i.e., until the hemostasis treatment time 100 has elapsed). While an example in which the text 44A is visually displayed is given here, this is merely an example. Alternatively, a sound identifying the hemostasis period 101 may be output from a speaker (not shown), or information identifying the hemostasis period 101 may be printed on a medium (e.g., paper) by a printer. While an example in which the hemostasis period specifying information 102 is output to the display device 18 is given here, the hemostasis period specifying information 102 may be output to a server, a personal computer, a tablet terminal, or the like. The hemostasis period specifying information 102 may also be recorded on an electronic medium such as an electronic medical record.

[0112] 8 is a conceptual diagram illustrating an example of processing performed by the control unit 82B when the sheath 58A1 appears in the frame 40 before the hemostasis treatment time 100 has elapsed since the transition from the hemostasis preparation stage to the hemostasis initiation stage. As shown in FIG. 8 , the sheath 58A1 may appear in the frame 40 before the hemostasis treatment time 100 has elapsed since the transition from the hemostasis preparation stage to the hemostasis initiation stage. For example, due to body movement and / or hand shaking by the physician 12, the sheath 58A1 may appear in the frame 40 at a timing not intended by the physician 12 (i.e., a timing before the hemostasis treatment is complete). Furthermore, for example, the sheath 58A1 may appear in the frame 40 when the physician 12 attempts to perform the next hemostasis treatment even though the hemostasis treatment is not complete.

[0113] In this way, if the sheath 58A1 appears in the frame 40 before the hemostasis treatment time 100 has elapsed since the transition from the hemostasis preparation stage to the hemostasis start stage, it is preferable to make the doctor 12 aware that the current time is the hemostasis period 101.

[0114] Therefore, by referring to the recognition result 98, the control unit 82B determines whether the frame 40 in which the sheath 58A1 appears was obtained by the recognition unit 82A before the hemostasis treatment time 100 elapsed after the transition from the hemostasis preparation stage to the hemostasis initiation stage. If the frame 40 in which the sheath 58A1 appears is obtained by the recognition unit 82A (i.e., the recognition unit 82A recognizes that the sheath 58A1 has entered the frame), the control unit 82B resets the hemostasis treatment time 100. Next, if the recognition unit 82A obtains a frame 40 in which the sheath 58A1 is not visible (i.e., the recognition unit 82A recognizes that the sheath 58A1 has exited the frame), the control unit 82B resumes timing the hemostasis treatment time 100, thereby restarting the hemostasis period 101.

[0115] In this embodiment, the frame 40 in which the sheath 58A1 is captured before the hemostasis treatment time 100 has elapsed since the transition from the hemostasis preparation stage to the hemostasis initiation stage is an example of the "third frame" according to the present disclosure.

[0116] Next, the operation of the portion of the endoscopic device 10 according to the present disclosure will be described with reference to Fig. 9. The flow of medical support processing shown in Fig. 9 is an example of a "medical support method" according to the present disclosure. For ease of explanation, a case will be described in which the medical support processing is performed by the processor 82 when a bleeding site 42 occurs due to the resection of a lesion 41 from the intestinal wall 32 during endoscopic examination.

[0117] 9 , first, in step ST10, the recognition unit 82A determines whether or not one frame of image data has been captured by the camera 52 inside the large intestine 28. If one frame of image data has not been captured by the camera 52 inside the large intestine 28 in step ST10, the determination is negative, and the medical support process proceeds to step ST32. If one frame of image data has been captured by the camera 52 inside the large intestine 28 in step ST10, the determination is positive, and the medical support process proceeds to step ST12.

[0118] In step ST12, the recognition unit 82A and the control unit 82B acquire a frame 40 obtained by imaging the large intestine 28 with the camera 52. The control unit 82B then displays the frame 40 in the first display area 36 (see FIGS. 5, 7, and 8). After the processing of step ST12 is executed, the medical support processing proceeds to step ST14.

[0119] In step ST14, the recognition unit 82A performs recognition processing 96 on the frame 40 acquired in step ST12. The control unit 82B acquires a recognition result 98 obtained by the recognition unit 82A performing the recognition processing 96. After the processing of step ST14 is executed, the medical support processing proceeds to step ST16.

[0120] In step ST16, the control unit 82B refers to the recognition result 98 acquired in step ST14 and determines whether the sheath 58A1 is captured in the frame 40 acquired in step ST12. If the sheath 58A1 is not captured in the frame 40 acquired in step ST12 in step ST16, the determination is negative, and the medical support processing proceeds to step ST22. If the sheath 58A1 is captured in the frame 40 acquired in step ST12 in step ST16, the determination is positive, and the medical support processing proceeds to step ST18.

[0121] In step ST18, the control unit 82B determines whether the endoscopic examination is in the hemostasis period 101. If the endoscopic examination is not in the hemostasis period 101 in step ST18 (i.e., if the current time is within the hemostasis period 101), the determination is negative, and the medical support processing proceeds to step ST32. If the endoscopic examination is in the hemostasis period 101 in step ST18, the determination is positive, and the medical support processing proceeds to step ST20.

[0122] In step ST20, the control unit 82B resets the hemostasis treatment time 100. After the processing of step ST20 is executed, the medical support processing proceeds to step ST32.

[0123] In step ST22, the control unit 82B determines whether or not the clip pin 58A2 is captured in the frame 40 acquired in step ST12 by referring to the recognition result 98 acquired in step ST14. If the clip pin 58A2 is not captured in the frame 40 acquired in step ST12 in step ST22, the determination is negative, and the medical support processing proceeds to step ST32. If the clip pin 58A2 is captured in the frame 40 acquired in step ST12 in step ST22, the determination is positive, and the medical support processing proceeds to step ST24.

[0124] In step ST24, the control unit 82B determines whether the sheath 58A1 was captured in the frame 40 acquired in step ST12 one frame before the frame 40 used in the determination in step ST22. If the sheath 58A1 was not captured in the frame 40 acquired in step ST12 one frame before the frame 40 used in the determination in step ST22 in step ST24, the determination is negative, and the medical support processing proceeds to step ST32. If the sheath 58A1 was captured in the frame 40 acquired in step ST12 one frame before the frame 40 used in the determination in step ST22 in step ST24, the determination is positive, and the medical support processing proceeds to step ST26.

[0125] In step ST26, the control unit 82B starts a hemostasis period 101. The start of the hemostasis period 101 is realized by the control unit 82B starting to measure a hemostasis treatment time 100. The control unit 82B then outputs hemostasis period specification information 102 to the display device 18. As a result, text 44A is displayed in the second display area 38 (see FIG. 7). After the processing of step ST26 is executed, the medical support processing proceeds to step ST28.

[0126] In step ST28, the control unit 82B determines whether or not the hemostasis treatment time 100 has elapsed since the start of timing the hemostasis treatment time 100. If the hemostasis treatment time 100 has not elapsed since the start of timing the hemostasis treatment time 100 in step ST28, the determination is negative, and the medical support processing proceeds to step ST32. If the hemostasis treatment time 100 has elapsed since the start of timing the hemostasis treatment time 100 in step ST28, the determination is positive, and the medical support processing proceeds to step ST30.

[0127] In step ST30, the control unit 82B ends the hemostasis period 101. The end of the hemostasis period 101 is realized by ending the timing of the hemostasis treatment time 100 by the control unit 82B. Then, the control unit 82B hides the text 44A in the second display area 38. In other words, the text 44A is erased from the second display area 38. After the processing of step ST30 is executed, the medical support processing proceeds to step ST32.

[0128] In step ST32, the control unit 82B determines whether a condition for terminating the medical support process is satisfied. One example of the condition for terminating the medical support process is that an instruction to terminate the medical support process has been given to the endoscope device 10 (for example, that an instruction to terminate the medical support process has been accepted by the acceptance device 64).

[0129] In step ST32, if the condition for terminating the medical support process is not satisfied, the determination is negative and the medical support process proceeds to step ST10. In step ST32, if the condition for terminating the medical support process is satisfied, the determination is positive and the medical support process ends.

[0130] As described above, in the endoscopic device 10, the recognition unit 82A acquires a plurality of frames 40 that are obtained in chronological order by capturing images of the inside of the large intestine 28 using the camera 52, and the plurality of frames 40 that show stages of the hemostatic treatment are acquired. Then, the control unit 82B outputs to the display device 18 hemostatic period identification information 102 that can identify the hemostatic period 101 from when the stage of the hemostatic treatment captured in the plurality of frames 40 acquired by the recognition unit 82A changes from the hemostatic preparation stage to the hemostatic start stage until the hemostatic treatment time 100 has elapsed. Text 44A that can identify the hemostatic period 101 is displayed on the screen 35 (here, as an example, the second display area 38) of the display device 18. This allows the physician 12 to understand that the current period is the hemostatic period 101 (i.e., the period during which the hemostatic treatment is being performed using the hemostatic forceps 58).

[0131] Furthermore, the hemostatic procedure time 100 is a statistically determined time required to perform one hemostatic procedure using the hemostatic forceps 58 (here, as an example, a time determined based on the graph shown in FIG. 6 ). This makes it difficult for the (N+1)th hemostatic procedure to be performed before the Nth hemostatic procedure has been completed, for example. Furthermore, because the physician 12 can accurately determine when one hemostatic procedure has been completed, the number of hemostatic procedures performed can also be counted accurately.

[0132] Furthermore, in the endoscope device 10, the control unit 82B detects a change from the hemostasis preparation stage to the hemostasis start stage based on a recognition result 98 obtained by the recognition unit 82A performing recognition processing 96 using a recognition model 92 on multiple frames 40. This makes it possible to detect a change from the hemostasis preparation stage to the hemostasis start stage without placing a burden on the doctor 12, compared to when the doctor 12 determines the change from the hemostasis preparation stage to the hemostasis start stage by observing multiple frames 40.

[0133] The period from when the change from the hemostasis preparation stage to the hemostasis initiation stage is detected until the hemostasis treatment time 100 has elapsed is set to hemostasis period 101. The physician 12 is notified that hemostasis period 101 has elapsed by displaying text 44A in the second display area 38. In this way, the period from when the change from the hemostasis preparation stage to the hemostasis initiation stage is detected until the hemostasis treatment time 100 has elapsed is set to hemostasis period 101 and notified to the physician 12, so that the physician 12 can be made aware of hemostasis period 101 without imposing a burden on the physician 12.

[0134] Furthermore, in the endoscope device 10, the stage at which a frame 40 showing the sheath 58A1 is obtained is regarded as the hemostasis preparation stage, and the stage at which a frame 40 showing the sheath 58A1 is obtained followed by a frame 40 not showing the sheath 58A1 is regarded as the hemostasis initiation stage. Therefore, the timing at which the hemostasis procedure is started, i.e., the start timing of the hemostasis period 101, can be determined simply. As a result, the hemostasis period 101 can also be determined simply.

[0135] Furthermore, in the endoscopic device 10, the stage at which a frame 40 showing the sheath 58A1 is obtained is considered to be the hemostasis preparation stage, and the stage at which a frame 40 showing the clip pin 58A2 detached from the sheath 58A1 is obtained following the frame 40 showing the sheath 58A1 is considered to be the hemostasis initiation stage. Therefore, the timing at which the hemostasis procedure is initiated, i.e., the start timing of the hemostasis period 101, can be determined simply. As a result, the hemostasis period 101 can also be determined simply. Note that, while the frame 40 showing the clip pin 58A2 detached from the sheath 58A1 is illustrated here, this is merely an example, and the present disclosure also applies to a frame 40 showing the clip pin 58A2 not detached from the sheath 58A1.

[0136] Furthermore, in the endoscope device 10, the stage at which a frame 40 showing the sheath 58A1 is obtained is regarded as the hemostasis preparation stage, and the stage at which a frame 40 showing the sheath 58A1 is followed by a frame 40 showing the clip pin 58A2 detached from the sheath 58A1 but not the sheath 58A1 is regarded as the hemostasis start stage. Therefore, the timing at which the hemostasis procedure is started, i.e., the start timing of the hemostasis period 101, can be determined simply. As a result, the hemostasis period 101 can also be determined simply.

[0137] Furthermore, in the endoscopic device 10, if a frame 40 showing the sheath 58A1 is obtained while the hemostasis treatment time 100 is being measured (i.e., during the hemostasis period 101), the hemostasis treatment time 100 is reset. That is, under the condition that the hemostasis treatment time 100 is predetermined as the time required for one hemostasis treatment, the hemostasis treatment time 100 is reset if a frame 40 showing the sheath 58A1 is obtained after the transition from the hemostasis preparation stage to the hemostasis start stage and before the hemostasis treatment time 100 has elapsed. This prevents the doctor 12 from starting the next hemostasis treatment when the hemostasis treatment is not yet complete, or prevents the doctor 12 from mistakenly determining that one hemostasis treatment has been performed when the hemostasis treatment is not yet complete.

[0138] In the above embodiment, the control unit 82B outputs the hemostasis period specifying information 102 to the display device 18 during the hemostasis period 101. However, the present disclosure is not limited to this. For example, as shown in Fig. 10, the control unit 82B may store hemostasis treatment-related information 104 related to the hemostasis treatment during the hemostasis period 101 in the storage area 84A of the memory 84 upon the start of the hemostasis period 101. This facilitates processing using the hemostasis treatment-related information 104.

[0139] The hemostasis treatment-related information 104 may include the hemostasis period specifying information 102 or a part of the hemostasis period specifying information 102. This makes it easier to perform processing using the hemostasis period specifying information 102 or a part of the hemostasis period specifying information 102.

[0140] Furthermore, examples of information included as part of the hemostasis period identification information 102 contained in the hemostasis treatment-related information 104 include hemostasis start timing information 104A (e.g., information indicating the start time of the hemostasis period 101) that can identify the start timing of the hemostasis period 101, and / or hemostasis end timing information 104B (e.g., information indicating the end time of the hemostasis period 101) that can identify the end timing of the hemostasis period 101. Hemostasis start timing information 104A is an example of "start timing information" according to the present disclosure, and hemostasis end timing information 104B is an example of "end timing information" according to the present disclosure.

[0141] If the hemostasis start timing information 104A is stored in the storage area 84A, the doctor 12 or the like can ascertain the start timing of the hemostasis period 101 from the hemostasis start timing information 104A stored in the storage area 84A. Also, if the hemostasis end timing information 104B is stored in the storage area 84A, the doctor 12 or the like can ascertain the end timing of the hemostasis period 101 from the hemostasis end timing information 104B stored in the storage area 84A.

[0142] Furthermore, hemostasis procedure-related information 104 may include hemostasis location-related information 104C. Hemostasis location-related information 104C is an example of "hemostasis location-related information" according to the present disclosure. Hemostasis location-related information 104C refers to information relating to the location in large intestine 28 where hemostasis procedure was performed. If hemostasis location-related information 104C is stored in storage area 84A, doctor 12 or the like can refer to hemostasis location-related information 104C stored in storage area 84A to determine what medical procedure is required for the location in large intestine 28 where hemostasis procedure was performed.

[0143] An example of information included in the hemostasis location-related information 104C is lesion-related information 104C1. The lesion-related information 104C1 is an example of "lesion-related information" according to the present disclosure. The lesion-related information 104C1 refers to information related to the lesion 41 present at the site where hemostasis was performed. In this way, if the hemostasis location-related information 104C includes the lesion-related information 104C1, the doctor 12 or the like can refer to the lesion-related information 104C1 stored in the storage area 84A to determine what medical treatment is required for the lesion 41 present at the site where hemostasis was performed, what medical treatment is required for the site where the lesion 41 was present (e.g., the bleeding site 42), or what medical treatment is required for the site where hemostasis was completed (e.g., the site clipped with the clip pin 58A2).

[0144] The hemostatic procedure-related information 104 may also include count information 104D. The count information 104D is an example of "count information" according to the present disclosure. The count information 104D is information that can identify the number of times hemostatic procedures have been performed in the large intestine 28 (e.g., hemostatic procedures performed on the bleeding site 42 of one lesion 41, or the number of clip pins 58A2 placed in the large intestine 28). In this way, if the hemostatic procedure-related information 104 includes the count information 104D, the doctor 12 or the like can easily determine the number of times hemostatic procedures have been performed in the large intestine 28 by referring to the count information 104D stored in the storage area 84A. As a result, the necessary medical procedures can be determined based on the number of times hemostatic procedures have been performed in the large intestine 28.

[0145] 11 , the hemostasis location-related information 104C may include a pre-hemostasis image 40A. The pre-hemostasis image 40A is an example of a “pre-hemostasis image” according to the present disclosure. The pre-hemostasis image 40A is a frame 40 obtained by capturing, with the camera 52, the state of the bleeding location 42 after hemostasis treatment before hemostasis. If the pre-hemostasis image 40A is included in the hemostasis location-related information 104C, the doctor 12 or the like can understand the state of the bleeding location 42 before hemostasis by referring to the pre-hemostasis image 40A stored in the storage area 84A.

[0146] Furthermore, the hemostasis location-related information 104C may include a post-hemostasis image 40B. The post-hemostasis image 40B is an example of a "post-hemostasis image" according to the present disclosure. The post-hemostasis image 40B is a frame 40 obtained by capturing an image of the location where hemostasis treatment has been performed by the camera 52. If the post-hemostasis image 40B is included in the hemostasis location-related information 104C, the doctor 12 or the like can understand the state of the location where hemostasis treatment has been performed by referring to the post-hemostasis image 40B stored in the storage area 84A.

[0147] The lesion-related information 104C1 may also include a lesion image 40C. The lesion image 40C is an example of a “lesion image” according to the present disclosure. The lesion image 40C is a frame 40 obtained by capturing an image of a lesion 41 using the camera 52. If the lesion image 40C is included in the lesion-related information 104C1, the physician 12 or the like can refer to the frame 40 included in the lesion image 40C stored in the storage area 84A to determine what medical treatment is required for the lesion 41 that existed at the location where hemostasis was performed, what medical treatment is required for the location where the lesion 41 existed (e.g., bleeding location 42), or what medical treatment is required for the location where hemostasis was completed (e.g., the location clipped with the clip pin 58A2).

[0148] The lesion-related information 104C1 may include lesion type information 104C1a and / or lesion type information 104C1b. The lesion type information 104C1a is information indicating the type of lesion 41. Examples of types of lesion 41 include neoplastic polyps and non-neoplastic polyps. Examples of types of neoplastic polyps include adenomatous polyps (e.g., SSL). Examples of types of non-neoplastic polyps include hamartomatous polyps, hyperplastic polyps, and inflammatory polyps. Note that the types exemplified here are types that are anticipated as types of lesion 41 when an endoscopic examination is performed on the large intestine 28, and the types of lesion 41 will differ depending on the organ that is subjected to the endoscopic examination.

[0149] The lesion type information 104C1b is information indicating the type of the lesion 41. Examples of the types of the lesion 41 include pedunculated, subpedunculated, sessile, surface elevated, surface flat, and surface depressed.

[0150] Lesion-related information 104C1 may include geometric property information 104C1c. Geometric property information 104C1c is information indicating the geometric property of lesion 41. An example of the geometric property of lesion 41 is information (e.g., coordinates) that can identify the position and / or shape of an image area indicating lesion 41 within lesion image 40C.

[0151] For example, the lesion type information 104C1a, the lesion type information 104C1b, and / or the geometric property information 104C1c are information obtained by performing object recognition processing (e.g., object recognition processing using AI and / or object recognition processing using template matching) on ​​the frame 40. The lesion type information 104C1a, the lesion type information 104C1b, and / or the geometric property information 104C1c may be accepted by the accepting device 64. In this case, the control unit 82B causes the lesion type information 104C1a, the lesion type information 104C1b, and / or the geometric property information 104C1c accepted by the accepting device 64 to be included in the lesion-related information 104C1.

[0152] The lesion type information 104C1a, the lesion type information 104C1b, and / or the geometric property information 104C1c may be associated with the lesion image 40C. In this manner, the control unit 82B displays the lesion image 40C on the screen 35, and also displays the lesion type information 104C1a, the lesion type information 104C1b, and / or the geometric property information 104C1c associated with the lesion image 40C. Thus, the physician 12 can observe the lesion image 40C through the screen 35 and can grasp the type, the type, and / or the geometric property of the lesion 41 depicted in the lesion image 40C.

[0153] The hemostasis start timing information 104A and / or hemostasis end timing information 104B may be associated with hemostasis location-related information 104C related to the hemostasis start timing information 104A and / or hemostasis end timing information 104B (e.g., hemostasis location-related information 104C obtained during the hemostasis period 101 specified by the hemostasis start timing information 104A and / or hemostasis end timing information 104B). In this manner, the doctor 12 can grasp the hemostasis location-related information 104C through the screen 35 or the like, and can also grasp the hemostasis start timing information 104A and / or hemostasis end timing information 104B associated with the hemostasis location-related information 104C through the screen 35 or the like.

[0154] In the example shown in FIG. 10 , the hemostatic treatment-related information 104 is stored in the storage area 84A, but this is merely one example. At least a portion of the information included in the hemostatic treatment-related information 104 may be stored in the storage 76 and / or 86, etc. At least a portion of the information included in the hemostatic treatment-related information 104 may be stored in an electronic medical record. At least a portion of the information included in the hemostatic treatment-related information 104 may be stored in a server, a personal computer, and / or a tablet terminal. At least a portion of the information included in the hemostatic treatment-related information 104 may be printed on a medium by a printer. At least a portion of the information included in the hemostatic treatment-related information 104 may be output as audio from a speaker.

[0155] Furthermore, at least a portion of the information included in the hemostatic treatment-related information 104 may be used as input information for the so-called generation AI. Information output from the generation AI may be stored in the various storage areas described above (e.g., storage 76 and / or 86), displayed as medical information 44 on the display device 18, printed on a medium by a printer, or output as audio from a speaker. An example of the generation AI is ChatGPT, which uses GPT-4 (Internet search <https: / / openai.com / gpt-4>) or GPT-4-32k. When at least a portion of the information included in the hemostatic treatment-related information 104 is used for ChatGPT, for example, at least a portion of the information included in the hemostatic treatment-related information 104 is used as instruction data (e.g., prompts) to be input into ChatGPT. An example of information output from ChatGPT is information regarding medical treatment derived from hemostatic treatment-related information 104 input into ChatGPT (e.g., information indicating the contents of medical treatment recommended to be performed after hemostatic treatment).

[0156] In the above embodiment, the recognition process 96 using AI in a segmentation method is exemplified, but the present disclosure is not limited to this. For example, the present disclosure can also be applied to recognition process using AI in a bounding box method.

[0157] In the above embodiment, an example in which the medical support processing is performed by the computer 78 has been described, but the present disclosure is not limited to this, and at least a part of the processing included in the medical support processing may be performed by a device provided outside the computer 78. An example of this case will be described below with reference to FIG.

[0158] 12 is a conceptual diagram showing an example of the configuration of an endoscope device 106. The endoscope device 106 is an example of an "endoscope device" according to the present disclosure. The endoscope device 106 differs from the endoscope device 10 described in the above embodiment in that it includes an external device 108.

[0159] The external device 108 is communicatively coupled to the computer 78 via a network 110 (eg, a WAN and / or a LAN, etc.).

[0160] An example of the external device 108 is at least one server that directly or indirectly transmits and receives data to and from the computer 78 via the network 110. The external device 108 receives a processing execution instruction provided from the processor 82 of the computer 78 via the network 110. The external device 108 then executes processing in accordance with the received processing execution instruction and transmits the processing results to the computer 78 via the network 110. In the computer 78, the processor 82 receives the processing results transmitted from the external device 108 via the network 110 and executes processing using the received processing results.

[0161] An example of the processing execution instruction is an instruction to cause the external device 108 to execute at least a part of the medical support processing. A first example of at least a part of the medical support processing (i.e., a processing to be executed by the external device 108) is a recognition processing 96. In this case, the external device 108 executes the recognition processing 96 in accordance with the processing execution instruction provided from the processor 82 via the network 110, and transmits a recognition result 98 to the computer 78 via the network 110. In the computer 78, the processor 82 receives the recognition result 98 and executes the same processing as in the above embodiment using the received recognition result 98.

[0162] A second example of at least a portion of the medical support processing (i.e., processing to be executed by the external device 108) is processing by the control unit 82B. In this case, the external device 108 executes processing by the control unit 82B in accordance with processing execution instructions provided by the processor 82 via the network 110, and transmits the processing results (e.g., hemostasis period specification information 102 and / or hemostasis treatment-related information 104) to the computer 78 via the network 110. In the computer 78, the processor 82 receives the processing results and uses the received processing results to execute processing similar to that of the above embodiment (e.g., displaying using the display device 18 and / or storing in the storage area 84A, etc.).

[0163] For example, the external device 108 is realized by cloud computing. Note that cloud computing is merely one example, and the external device 108 may be realized by network computing such as fog computing, edge computing, or grid computing. Instead of a server, at least one personal computer or the like may be used as the external device 108. Alternatively, the external device 108 may be a computing device with a communication function and equipped with multiple types of AI functions.

[0164] In the above embodiment, an example in which the medical support program 90 is stored in the storage 86 has been described, but the present disclosure is not limited to this. For example, the medical support program 90 may be stored in a portable, computer-readable, non-transitory storage medium such as an SSD or USB memory. The medical support program 90 stored in the non-transitory storage medium is installed in the computer 78 of the endoscope device 10. The processor 82 executes medical support processing in accordance with the medical support program 90.

[0165] In addition, the medical support program 90 may be stored in a storage device such as another computer or server connected to the endoscopic device 10 via a network, and the medical support program 90 may be downloaded and installed on the computer 78 in response to a request from the endoscopic device 10.

[0166] It is not necessary to store the entire medical support program 90 in a storage device such as another computer or server device connected to the endoscopic device 10, or to store the entire medical support program 90 in the storage 86; only a portion of the medical support program 90 may be stored.

[0167] The hardware resources that execute the medical support processing can be various processors, as listed below. Examples of processors include a CPU, which is a general-purpose processor that functions as a hardware resource that executes medical support processing by executing software, i.e., a program. Examples of processors include dedicated electrical circuits, such as FPGAs, PLDs, or ASICs, which are processors with a circuit configuration specifically designed to execute specific processing. Each processor has built-in or connected memory, and executes medical support processing by using the memory.

[0168] The hardware resource for executing the medical support processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource for executing the medical support processing may be a single processor.

[0169] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes medical support processing. Second, there is a system that uses a processor that realizes the functions of the entire system, including multiple hardware resources that execute medical support processing, on a single IC chip, as typified by SoC. In this way, medical support processing is realized using one or more of the various processors described above as hardware resources.

[0170] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The above medical support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the process.

[0171] The above-described description and illustrations are a detailed explanation of the parts related to the present disclosure and are merely an example of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or elements may be replaced with other parts from the above-described description and illustrations, as long as they do not deviate from the gist of the present disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the parts related to the present disclosure, the above-described description and illustrations omit explanations of common general technical knowledge that do not require particular explanation to enable the implementation of the present disclosure.

[0172] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A medical support device comprising a processor, which acquires a plurality of frames obtained in a time series by a camera capturing images of the inside of a tubular organ into which a hemostatic forceps is inserted, the frames depicting stages of a hemostatic procedure using the hemostatic forceps, and outputs hemostatic period identification information capable of identifying that the period from when the stage depicted in the acquired plurality of frames changes from a stage of preparing to hemostatically using the hemostatic forceps to a stage of initiating hemostatic action using the hemostatic forceps until a certain amount of time has elapsed is the hemostatic period during which the hemostatic procedure is being performed.

2. The medical support device of claim 1, wherein the processor detects a change from the hemostasis preparation stage to the hemostasis initiation stage by performing object recognition processing based on the acquired multiple frames, and the hemostasis period is the period from when the change is detected by the processor to when the certain time has elapsed.

3. The medical support device according to claim 2, wherein the object recognition processing is realized by using a model obtained by performing machine learning on a neural network.

4. The medical support device of claim 1, wherein the hemostatic forceps has a sheath, the plurality of frames include a first frame and a second frame, the first frame is a frame in which the sheath is depicted, the second frame is a frame in which the sheath is not depicted, the hemostasis preparation stage is a stage in which the first frame is obtained, and the hemostasis initiation stage is a stage in which the second frame is obtained subsequent to the first frame.

5. The medical support device of claim 1, wherein the hemostatic forceps has a sheath and a clip pin, the plurality of frames include a first frame and a second frame, the first frame is a frame in which the sheath is depicted, the second frame is a frame in which the clip pin is depicted, the hemostasis preparation stage is a stage in which the first frame is obtained, and the hemostasis initiation stage is a stage in which the second frame is obtained subsequent to the first frame.

6. The medical support device of claim 1, wherein the hemostatic forceps has a sheath and a clip pin, the plurality of frames include a first frame and a second frame, the first frame is a frame in which the sheath is depicted, the second frame is a frame in which the clip pin is depicted but the sheath is not depicted, the hemostasis preparation stage is a stage in which the first frame is obtained, and the hemostasis initiation stage is a stage in which the second frame is obtained subsequent to the first frame.

7. The medical support device of claim 4, wherein the plurality of frames further includes a third frame, the third frame being a frame in which the sheath is shown, and the certain period of time is reset when the third frame is obtained before the certain period of time has elapsed since the change from the hemostasis preparation stage to the hemostasis initiation stage.

8. The medical support device according to claim 1, wherein hemostasis treatment-related information regarding the hemostasis treatment during the hemostasis period is stored in a storage area.

9. The medical support device according to claim 8, wherein the hemostasis treatment related information includes the hemostasis period specifying information or a part of the hemostasis period specifying information.

10. A medical support device as described in claim 9, wherein the portion includes start timing information capable of identifying a start timing of the hemostasis period and / or end timing information capable of identifying an end timing of the hemostasis period.

11. The medical support device according to claim 8, wherein the hemostasis treatment-related information includes hemostasis location-related information regarding the location in the tubular organ where the hemostasis treatment was performed.

12. The medical support device according to claim 11, wherein the hemostasis location related information includes lesion related information regarding a lesion that was present at the location where the hemostasis procedure was performed.

13. The medical support device according to claim 12, wherein the lesion-related information includes a lesion image obtained by capturing an image of the lesion by the camera.

14. The medical support device according to claim 11, wherein the information relating to the hemostasis location includes a post-hemostasis image obtained by imaging the location where the hemostasis procedure was performed with the camera.

15. The medical support device according to claim 11, wherein the information relating to the hemostasis location includes a pre-hemostasis image obtained by capturing an image of the location where the hemostasis procedure was performed before hemostasis using the camera.

16. The medical support device according to claim 8, wherein the hemostasis treatment-related information includes number information that can identify the number of times the hemostasis treatment has been performed within the tubular organ.

17. The medical support device according to claim 1, wherein the certain period of time is a statistically determined period of time required for one hemostasis procedure to be performed using the hemostatic forceps.

18. An endoscope apparatus comprising: a medical support device according to any one of claims 1 to 17; and the camera.

19. A medical support method comprising: acquiring a plurality of frames obtained in a chronological order by imaging the inside of a tubular organ into which a hemostatic forceps is inserted using a camera, the plurality of frames showing stages of a hemostatic procedure using the hemostatic forceps; and outputting hemostatic period identification information capable of identifying that the period from when the stage shown in the plurality of acquired frames changes from a stage of preparing to hemostatically using the hemostatic forceps to a stage of starting to hemostatically using the hemostatic forceps until a certain amount of time has elapsed is the hemostatic period during which the hemostatic procedure is being performed.

20. A program for causing a computer to execute a process including: acquiring a plurality of frames obtained in a chronological order by imaging the inside of a tubular organ into which a hemostatic forceps is inserted using a camera, the plurality of frames showing stages of a hemostatic procedure using the hemostatic forceps; and outputting hemostatic period identification information capable of identifying that the period from when the stage shown in the plurality of acquired frames changes from the stage of preparing for hemostatic use using the hemostatic forceps to the stage of starting hemostatic use using the hemostatic forceps until a certain amount of time has elapsed is the hemostatic period during which the hemostatic procedure is being performed.

Images