Medical support device, endoscope apparatus, medical support system, medical support method and program

Abstract

To provide a medical support device, an endoscope apparatus, a medical support system, a medical support method, and a program which allow a user to recognize a time lag degree caused by transmission / reception between the medical support device and a processing device.SOLUTION: A medical support device comprises: a communication device; and a processor. The communication device transmits a first medical image to a processing device, and receives a processing result which is obtained by executing processing on the first medical image by the processing device. The processor performs a first control of outputting time information related to a time taken from transmission of the first medical image to the processing device by the communication device to reception of the processing result, and / or a second control of making an output aspect of the processing result different depending on the time.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to a medical support device, an endoscope device, a medical support system, a medical support method, and a program. [Background technology]

[0002] Patent Document 1 discloses an electronic endoscope system including an electronic endoscope that captures an image of an object to be observed and generates an image signal, a processor to which the electronic endoscope is detachably connected, and an image processing device that is detachably connected to the processor. In the electronic endoscope system described in Patent Document 1, the image processing device includes a first image processing means that performs a first image processing on the image signal to generate a first video signal, and a processing time output means that outputs information about the processing time in the first image processing to the processor. In the electronic endoscope system described in Patent Document 1, the processor includes an image signal output means that outputs the image signal to the image processing device, a second image processing means that performs a second image processing on the image signal to generate a second video signal, and a first timing control means that controls the timing of output of the second video signal based on the information about the processing time.

[0003] Patent Document 2 discloses an examination support device that is used by connecting to an endoscope system that includes a camera unit that is inserted into the body of a subject. The examination support device described in Patent Document 2 includes an acquisition unit, a detection unit, a diagnosis unit, and a display control unit. In the examination support device described in Patent Document 2, the acquisition unit sequentially acquires image signals that the endoscope system provides to an external device. The detection unit detects whether frame images sequentially generated from the image signals are still images or moving images. When the detection unit detects that the frame image is a still image, the diagnosis unit performs a diagnosis of the inside of the body by inputting a diagnostic image obtained by cutting out at least a part of the image area of ​​the frame image into a trained model. The display control unit displays the diagnosis result output by the diagnosis unit on the display unit.

[0004] In the examination support device described in Patent Document 2, the display control unit does not display the diagnosis result until a preset hold time has elapsed since the detection unit detected that the frame image is a still image, and even if the hold time has elapsed, the display control unit does not display the diagnosis result if the detection unit detects that the frame image generated from the image signal acquired by the acquisition unit at the time the hold time has elapsed is a moving image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2011-036488 A [Patent Document 2] Patent Publication No. 2022-142990 Summary of the Invention

[0006] One embodiment of the present disclosure provides a medical support device, an endoscopic device, a medical support system, a medical support method, and a program that enable a user to recognize the degree of time lag caused by transmission and reception between a medical support device and a processing device. [Means for solving the problem]

[0007] A first aspect of the present disclosure is a medical support device comprising a communication device and a processor, wherein the communication device transmits a first medical image to a processing device and receives a processing result obtained by the processing device performing processing on the first medical image, and the processor performs a first control to output time information regarding the time required from when the communication device transmits the first medical image to the processing device to when the processing result is received, and / or a second control to vary the output manner of the processing result depending on the time.

[0008] A second aspect of the present disclosure is the medical support device according to the first aspect, in which the processor outputting the time information includes the processor displaying the time information on the first screen.

[0009] A third aspect of the present disclosure is a medical support device according to the second aspect, wherein the first control includes control to display on the second screen a first medical image and / or a second medical image obtained later in time than the first medical image in a state that can be compared with time information.

[0010] A fourth aspect of the present disclosure is a medical support device according to any one of the first to third aspects, in which the time information includes first identification information capable of identifying the time itself.

[0011] A fifth aspect of the present disclosure is a medical support device according to any one of the first to fourth aspects, in which the time information includes second identification information capable of identifying a difference between the time and a first reference time.

[0012] A sixth aspect of the present disclosure is the medical support device according to the fifth aspect, in which the second identification information includes length information capable of identifying whether a time is long or short with respect to the first reference time.

[0013] A seventh aspect of the present disclosure is the medical support device according to any one of the first to sixth aspects, in which the time information is visualized information.

[0014] An eighth aspect of the present disclosure is the medical support device according to the seventh aspect, in which the time information is information visualized by text and / or marks.

[0015] A ninth aspect of the present disclosure is the medical support device according to the seventh or eighth aspect, in which the processor changes a visualization format in which the time information is visualized, depending on time.

[0016] A tenth aspect of the present disclosure is a medical support device according to the ninth aspect, in which the processor makes the visibility of the time information when the time is equal to or greater than a second reference time higher than the visibility of the time information when the time is less than the second reference time.

[0017] An eleventh aspect of the present disclosure is a medical support device according to any one of the first to tenth aspects, in which the output aspect is a display aspect in which the processor displays the processing result on a third screen.

[0018] A twelfth aspect of the present disclosure is a medical support device according to the eleventh aspect, in which the display mode includes a first display mode and a second display mode, the first display mode is used when the time is equal to or greater than a third reference time and is capable of identifying that the time is equal to or greater than the third reference time, and the second display mode is used when the time is less than the third reference time and is capable of identifying that the time is less than the third reference time.

[0019] A thirteenth aspect of the present disclosure is the medical support device according to the twelfth aspect, in which the second display aspect has higher visibility than the first display aspect.

[0020] A fourteenth aspect of the present disclosure is the medical support device according to the thirteenth aspect, in which the processing result is hidden in the first display aspect and is displayed in the second display aspect.

[0021] A fifteenth aspect of the present disclosure is a medical support device relating to any one of the eleventh to fourteenth aspects, wherein the second control includes control to display on the fourth screen the first medical image and / or a third medical image obtained later in time than the first medical image in a state that allows comparison with the processing result.

[0022] A sixteenth aspect of the present disclosure is a medical support device according to any one of the eleventh to fourteenth aspects, wherein the second control includes control to display the first medical image on the fourth screen in a state in which it can be compared with the processing result when the time is less than a fourth reference time.

[0023] A seventeenth aspect of the present disclosure is the medical support device according to the sixteenth aspect, wherein the fourth reference time is the longest time for which the degree of affinity between the first medical image and the processing result falls within an acceptable range.

[0024] An eighteenth aspect of the present disclosure is a medical support device according to any one of the first to seventeenth aspects, wherein the processing is AI processing on the first medical image.

[0025] A 19th aspect of the present disclosure is a medical support device according to any one of the first to eighteenth aspects, wherein the processing includes a recognition process for recognizing a characteristic area appearing in the first medical image.

[0026] A twentieth aspect of the present disclosure is a medical support device according to the nineteenth aspect, in which the characteristic region is a lesion.

[0027] A 21st aspect of the present disclosure is a medical support device relating to any one of the first to 20th aspects, wherein the first medical image is an endoscopic image generated by capturing an image of the inside of the body with a camera mounted on an endoscopic scope.

[0028] A 22nd aspect of the present disclosure is an endoscopic device comprising a medical support device relating to any one of the 1st to 21st aspects and a camera inserted into the body to generate a first medical image by capturing images of the inside of the body.

[0029] A 23rd aspect of the present disclosure is a medical support system comprising a medical support device and a processing device, wherein the medical support device transmits a first medical image to the processing device, the processing device performs processing on the first medical image and transmits a processing result obtained by performing the processing to the medical support device, and the medical support device receives the processing result transmitted from the processing device and performs a first control to output time information regarding the time required from when the first medical image is transmitted to the server to when the processing result is received by the medical support device, and / or a second control to vary the output manner of the processing result depending on time.

[0030] A 24th aspect of the present disclosure is a medical support method including transmitting a first medical image to a processing device, receiving a processing result obtained by the processing device performing processing on the first medical image, and performing a first control to output time information regarding the time required from when the first medical image is transmitted to the processing device to when the processing result is received, and / or a second control to vary the output manner of the processing result depending on the time.

[0031] A 25th aspect of the present disclosure is a program for causing a computer to execute medical support processing including transmitting a first medical image to a processing device, receiving a processing result obtained by the processing device performing processing on the first medical image, and performing a first control to output time information regarding the time required from when the first medical image is transmitted to the processing device to when the processing result is received, and / or a second control to vary the output manner of the processing result depending on time. [Brief description of the drawings]

[0032] [Figure 1] FIG. 1 is a conceptual diagram showing an example of how the medical support system is used by a doctor. [Diagram 2] 1 is a conceptual diagram showing an example of an overall configuration of a medical support system. [Diagram 3] 2 is a block diagram showing an example of a hardware configuration of an electrical system of the medical support system. FIG. [Figure 4] 2 is a block diagram showing an example of main functions of a processor included in the medical support device, and an example of information stored in a storage. [Diagram 5] FIG. 2 is a conceptual diagram showing an example of processing contents of a medical support device and a processing device. [Figure 6] 4 is a conceptual diagram showing an example of processing contents of a control unit and an example of contents displayed on a screen of a display device. FIG. [Figure 7] 13 is a flowchart showing an example of the flow of a medical support process. [Figure 8] 13 is a conceptual diagram showing a modified example of the content displayed in the second display area when the transmission / reception time is equal to or longer than a reference time. FIG. [Figure 9] 13A and 13B are conceptual diagrams showing modified examples of the contents displayed on the screen of the display device. [Figure 10] This is a conceptual diagram showing an example of a display in which a frame obtained later in time than a frame displayed in the first display area when the transmission and reception time is less than the reference time is used as the frame displayed in the first display area when the transmission and reception time is equal to or longer than the reference time. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] Hereinafter, examples of embodiments of a medical support device, an endoscopic device, a medical support system, a medical support method, and a program according to the present disclosure will be described with reference to the accompanying drawings.

[0034] First, the terms used in the following description will be explained.

[0035] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". GPGPU is an abbreviation for "General-Purpose computing on Graphics Processing Units". APU is an abbreviation for "Accelerated Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". RAM is an abbreviation for "Random Access Memory". NVM is an abbreviation for "Non-volatile memory". EEPROM is an abbreviation for "Electrically Erasable Programmable Read-Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". AI is an abbreviation for "Artificial Intelligence". BLI is an abbreviation for "Blue Light Imaging". LCI is an abbreviation for "Linked Color Imaging". I / F is an abbreviation for "Interface". SSL is an abbreviation for "Sessile Serrated Lesion". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network".5G is an abbreviation for "5th Generation Mobile Communication System." RTC is an abbreviation for "Real Time Clock." IC is an abbreviation for "Integrated Circuit." MRI is an abbreviation for "Magnetic Resonance Imaging." CT is an abbreviation for "Computed Tomography."

[0036] In the following description, a coded processor (hereinafter simply referred to as a "processor") may be one physical or virtual arithmetic device, or a combination of multiple physical or virtual arithmetic devices. Also, a processor may be one type of arithmetic device, or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU, a GPU, a GPGPU, an APU, and a TPU.

[0037] In the following description, a signed memory is a memory, such as a RAM, in which information is temporarily stored and which is used as a working memory by a processor.

[0038] In the following description, a storage with a symbol is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory, magnetic disks, and magnetic tapes. Another example of storage is cloud storage.

[0039] In the following embodiments, the external I / F with a symbol controls the transmission and reception of various information between multiple devices connected to each other. An example of the external I / F is a USB interface. A communication I / F including a communication processor and an antenna may be applied to the external I / F. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0040] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0041] Fig. 1 is a conceptual diagram showing an example of an aspect in which a medical support system 1 is used. As shown in Fig. 1, the medical support system 1 includes a processing device 2 and an endoscope device 10, and the processing device 2 and the endoscope device 10 are communicatively connected via a network 3. An example of the network 3 is the Internet. However, the Internet is merely an example, and other examples of the network 3 include a WAN and / or a LAN. In this embodiment, the medical support system 1 is an example of a "medical support system" according to the present disclosure, and the processing device 2 is an example of a "processing device" according to the present disclosure.

[0042] The endoscope device 10 is used by a doctor 12 in an endoscopic examination or the like. The endoscopic examination is assisted by staff such as a nurse 14.

[0043] Information obtained by the endoscope device 10 is transmitted to the processing device 2 via the network 3. An example of the processing device 2 is a cloud server. However, the cloud server is merely one example, and the processing device 2 may be an on-premise server or a personal computer. The processing device 2 receives the information transmitted from the endoscope device 10, executes processing using the received information, and transmits the processing results obtained by executing the processing to the endoscope device 10.

[0044] The endoscopic device 10 includes an endoscope 16, a display device 18, a light source device 20, a control device 22, and a medical support device 24. In this embodiment, the endoscope scope 16 is an example of an "endoscope scope" according to the present disclosure, the endoscopic device 10 is an example of an "endoscope device" according to the present disclosure, and the medical support device 24 is an example of a "medical support device" according to the present disclosure.

[0045] The endoscope device 10 is a modality for performing medical treatment on a large intestine 28 contained in the body of a subject 26 (e.g., a patient) using an endoscope scope 16. In this embodiment, the large intestine 28 is an object to be observed by a doctor 12.

[0046] The endoscope 16 is used by the doctor 12 and inserted into the body of a subject 26. In this embodiment, the endoscope 16 is inserted into a large intestine 28, which is a hollow organ of the subject 26.

[0047] The endoscope device 10 causes the endoscope scope 16 inserted into the large intestine 28 of the subject 26 to capture images of the inside of the large intestine 28 of the subject 26, and also performs various medical procedures on the large intestine 28 as necessary.

[0048] The endoscope device 10 captures an image of the inside of the large intestine 28 of the subject 26, thereby acquiring and outputting an image showing the state inside the large intestine 28. In this embodiment, the endoscope device 10 is an endoscope having an optical imaging function for capturing an image of reflected light obtained by irradiating the inside of the large intestine 28 with light 30 and reflecting it off a wall 32 of the large intestine 28.

[0049] Although an endoscopic examination of the large intestine 28 is illustrated here, this is merely one example, and the present disclosure also applies to an endoscopic examination of a hollow organ such as the esophagus, stomach, duodenum, or trachea.

[0050] The light source device 20, the control device 22, and the medical support device 24 are installed on a wagon 34. The wagon 34 has a plurality of tables arranged vertically, and the medical support device 24, the control device 22, and the light source device 20 are installed from the lower table to the upper table. In addition, the display device 18 is installed on the top table of the wagon 34.

[0051] The control device 22 controls the entire endoscope device 10. The medical support device 24, under the control of the control device 22, performs various image processing on images obtained by imaging the intestinal wall 32 with the endoscope scope 16. The medical support device 24 is also communicatively connected to the processing device 2 via the network 3, and by requesting the processing device 2 to provide a service, receives the requested service from the processing device 2.

[0052] The display device 18 displays various information including images. Examples of the display device 18 include a liquid crystal display and an EL display. Also, instead of the display device 18 or together with the display device 18, a tablet terminal with a display may be used.

[0053] The display device 18 displays a screen 35. The screen 35 includes a plurality of display areas. The plurality of display areas are arranged side by side within the screen 35. In the example shown in FIG. 1, a first display area 35A, a second display area 35B, and a third display area 35C are shown as examples of the plurality of display areas. The size of the first display area 35A is larger than the size of the second display area 35B. The first display area 35A is used as a main display area, and the second display area 35B is used as a sub display area. The size relationship between the first display area 35A and the second display area 35B is not limited thereto, and may be any size relationship that fits within the screen 35. The third display area 35C is a display area other than the first display area 35A and the second display area 35B within the screen 35, and is used as a sub display area. In this embodiment, the screen 35 is an example of the "first screen", "second screen", "third screen", and "fourth screen" according to the present disclosure.

[0054] An endoscopic moving image 39 is displayed in the first display region 35A. The endoscopic moving image 39 is a moving image acquired by imaging an intestinal wall 32 in the large intestine 28 of the subject 26 with the endoscope scope 16. In the example shown in Fig. 1, a moving image showing the intestinal wall 32 is shown as an example of the endoscopic moving image 39.

[0055] The intestinal wall 32 shown in the endoscopic video 39 includes a lesion 42 (e.g., one lesion 42 in the example shown in FIG. 1) as an area of ​​interest (i.e., an area to be observed) gazed upon by the doctor 12, and the doctor 12 can visually recognize the appearance of the intestinal wall 32 including the lesion 42 through the endoscopic video 39. In this embodiment, the lesion 42 is an example of a "characteristic area" and a "lesion" according to the present disclosure.

[0056] There are various types of lesions 42, and examples of the types of lesions 42 include neoplastic polyps and non-neoplastic polyps. Examples of the types of neoplastic polyps include adenomatous polyps (e.g., SSL). Examples of the types of non-neoplastic polyps include hamartomatous polyps, hyperplastic polyps, and inflammatory polyps. Note that the types exemplified here are types that are assumed in advance as types of lesions 42 when an endoscopic examination is performed on the large intestine 28, and the types of lesions 42 will differ if the organ in which the endoscopic examination is performed is different.

[0057] In this embodiment, for ease of explanation, an example is given in which one lesion 42 is captured in the endoscopic video 39, but the present disclosure is not limited to this, and the present disclosure also applies when multiple lesions 42 are captured in the endoscopic video 39.

[0058] In this embodiment, a lesion 42 is illustrated, however, this is merely one example, and the area of ​​interest (i.e., the area to be observed) gazed upon by the doctor 12 may be a characteristic area having some unique feature, such as an organ (e.g., the duodenal papilla), a mark, an artificial treatment device (e.g., an artificial clip), or a treated area (e.g., an area where traces remain after the removal of a polyp, etc.), etc.

[0059] The image displayed in first display area 35A is one frame 40 included in a moving image that includes a plurality of frames 40 arranged in chronological order. In other words, in first display area 35A, a plurality of frames 40 arranged in chronological order are displayed at a default frame rate (e.g., several tens of frames per second).

[0060] An example of a moving image displayed in the first display area 35A is a moving image in a live view format. The live view format is merely one example, and a moving image that is temporarily stored in a memory or the like and then displayed, such as a moving image in a post-view format, may also be used. Furthermore, each frame included in a moving image for recording stored in a memory or the like may be reproduced and displayed on the screen 35 (for example, the first display area 35A) as an endoscopic moving image 39.

[0061] Within the screen 35, the second display area 35B is adjacent to the first display area 35A, and is displayed in the lower right corner as viewed from the front within the screen 35. The display position of the second display area 35B may be anywhere within the screen 35 of the display device 18, but it is preferable that it is displayed in a position where it can be contrasted with the endoscopic moving image 39.

[0062] In the second display area 35B, auxiliary information 44 that assists the doctor 12 in making medical decisions during an endoscopic examination is displayed. The auxiliary information 44 is information that is referred to by the doctor 12. Examples of the auxiliary information 44 include various types of information related to the subject 26 into whose body the endoscope 16 is inserted, and / or various types of information obtained by performing medical support processing, which will be described later.

[0063] The third display area 35C displays visual information obtained by performing AI-based processing on the frame 40. This will be described in detail later.

[0064] Fig. 2 is a conceptual diagram showing an example of the overall configuration of the endoscope device 10. As shown in Fig. 2, the endoscope scope 16 includes an operation section 46 and an insertion section 48. The insertion section 48 is partially curved by operating the operation section 46. The insertion section 48 is inserted into the large intestine 28 (see Fig. 1) while curving in accordance with the shape of the large intestine 28 (see Fig. 1) in accordance with the operation of the operation section 46 by the doctor 12 (see Fig. 1).

[0065] The distal end 50 of the insertion section 48 is provided with a camera 52, an illumination device 54, and a treatment tool opening 56. The camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end 50. Note that, although an example in which the camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end 50 is given here, this is merely one example, and the camera 52 and the illumination device 54 may be provided on the side surface of the distal end 50, so that the endoscope 16 is configured as a side-viewing scope.

[0066] The camera 52 is mounted on the endoscope 16, and is inserted into the body cavity of the subject 26 to capture an image of an observation target region, thereby generating a frame 40 as an endoscopic image. In this embodiment, the camera 52 captures an image of the inside of the subject 26 (e.g., inside the large intestine 28) to generate an endoscopic video image 39 including a plurality of frames 40 in a time series. An example of the camera 52 is a CMOS camera. However, this is merely an example, and other types of cameras such as a CCD camera may also be used. In this embodiment, the camera 52 is an example of a "camera" according to the present disclosure, and the frame 40 is an example of a "first medical image", a "second medical image", a "third medical image", and an "endoscopic image" according to the present disclosure.

[0067] The illumination device 54 has illumination windows 54A and 54B. The illumination device 54 irradiates light 30 (see FIG. 1) through the illumination windows 54A and 54B. Examples of the light 30 irradiated from the illumination device 54 include visible light (e.g., white light) and non-visible light (e.g., near-infrared light). The illumination device 54 also irradiates special light through the illumination windows 54A and 54B. Examples of the special light include light for BLI and / or light for LCI. The camera 52 captures an image of the inside of the large intestine 28 by an optical method while the light 30 is irradiated inside the large intestine 28 by the illumination device 54.

[0068] The treatment tool opening 56 is an opening for allowing a treatment tool 58 to protrude from the distal end portion 50. The treatment tool opening 56 is also used as a suction port for sucking blood, internal waste, and the like, and as a delivery port for delivering fluid.

[0069] A treatment tool insertion port 60 is formed in the operation section 46, and the treatment tool 58 is inserted into the insertion section 48 from the treatment tool insertion port 60. The treatment tool 58 passes through the insertion section 48 and protrudes to the outside from the treatment tool opening 56. In the example shown in FIG. 2, a puncture needle is shown as the treatment tool 58 protruding from the treatment tool opening 56. Here, a puncture needle is shown as the treatment tool 58, but this is merely one example, and the treatment tool 58 may be a grasping forceps, a papillotomy knife, a snare, a catheter, a guidewire, a cannula, and / or a puncture needle with a guide sheath, etc.

[0070] The endoscope 16 is connected to a light source device 20 and a control device 22 via a universal cord 62. A medical support device 24 and a reception device 64 are connected to the control device 22. In addition to the processing device 2, the display device 18 is also connected to the medical support device 24. That is, the control device 22 is connected to the processing device 2 and the display device 18 via the medical support device 24.

[0071] Here, the medical support device 24 is exemplified as an external device for expanding the functions performed by the control device 22, and therefore an example is given in which the control device 22 and the display device 18 are indirectly connected via the medical support device 24, but this is merely one example. For example, the display device 18 may be directly connected to the control device 22. In this case, for example, it is sufficient that the functions of the medical support device 24 are installed in the control device 22.

[0072] The reception device 64 receives instructions from the doctor 12 and outputs the received instructions as an electrical signal to the control device 22. Examples of the reception device 64 include a keyboard, a mouse, a touch panel, a foot switch, a microphone, and / or a remote control device.

[0073] The control device 22 controls the light source device 20, exchanges various signals with the camera 52, and exchanges various signals with the medical support device 24.

[0074] The light source device 20 emits light under the control of the control device 22, and supplies the light to the illumination device 54. A light guide is built into the illumination device 54, and the light supplied from the light source device 20 passes through the light guide and is irradiated from illumination windows 54A and 54B. The light irradiated from the illumination windows 54A and 54B is light 30 (see FIG. 1). The control device 22 causes the camera 52 to capture an image, acquires an endoscopic video image 39 (see FIG. 1) from the camera 52, and outputs it to a predetermined output destination (for example, the medical support device 24).

[0075] The medical support device 24 performs various types of image processing on the endoscopic moving image 39 input from the control device 22 to provide support for medical care (here, as an example, an endoscopic examination). The medical support device 24 outputs the endoscopic moving image 39 that has been subjected to various types of image processing to a predetermined output destination (for example, the display device 18).

[0076] Although an example in which the endoscopic moving image 39 output from the control device 22 is output to the display device 18 via the medical support device 24 has been described here, this is merely one example. For example, the control device 22 and the display device 18 may be connected, and the endoscopic moving image 39 that has been subjected to image processing by the medical support device 24 may be displayed on the display device 18 via the control device 22.

[0077] Fig. 3 is a block diagram showing an example of a hardware configuration of an electrical system of the endoscope device 10. As shown in Fig. 3, the control device 22 includes a computer 66, a bus 68, and an external I / F 70. The computer 66 includes a processor 72, a memory 74, and a storage 76. The processor 72, the memory 74, the storage 76, and the external I / F 70 are connected to the bus 68. The processor 72 controls the entire control device 22. The memory 74 and the storage 76 are used by the processor 72.

[0078] The external I / F 70 is responsible for transmitting and receiving various types of information between the processor 72 and one or more devices (hereinafter, also referred to as “first external devices”) that exist outside the control device 22.

[0079] The camera 52 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 is responsible for transmitting and receiving various types of information between the camera 52 and the processor 72. The processor 72 controls the camera 52 via the external I / F 70. The processor 72 also acquires, via the external I / F 70, an endoscopic moving image 39 (see FIG. 1) obtained by the camera 52 capturing an image of the inside of the large intestine 28 (see FIG. 1).

[0080] The light source device 20 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the light source device 20 and the processor 72. The light source device 20 supplies light to the illumination device 54 under the control of the processor 72. The illumination device 54 irradiates the light supplied from the light source device 20.

[0081] A reception device 64 is connected to the external I / F 70 as one of the first external devices, and the processor 72 acquires instructions accepted by the reception device 64 via the external I / F 70 and executes processing according to the acquired instructions.

[0082] The medical support device 24 includes a computer 78, an RTC 79, and an external I / F 80. The computer 78 includes a processor 82, a memory 84, and a storage 86. The processor 82, the memory 84, the storage 86, the RTC 79, and the external I / F 80 are connected to a bus 88. In this embodiment, the computer 78 is an example of a "computer" according to the present disclosure, and the processor 82 is an example of a "processor" according to the present disclosure.

[0083] The hardware configuration of the computer 78 (i.e., the processor 82, the memory 84, and the storage 86) is basically the same as the hardware configuration of the computer 66, so a description of the hardware configuration of the computer 78 will be omitted here.

[0084] The RTC 79 is an IC that generates and outputs data such as the current time. The processor 82 obtains data such as the current time from the RTC 79. Note that, although an example in which the processor 82 obtains data of the current time from the RTC 79 is given here, this is merely one example, and the processor 82 may obtain data of the current time from a server on the network 3 or the like.

[0085] The external I / F 80 controls the exchange of various information between the processor 82 and one or more devices (hereinafter, also referred to as “second external devices”) that exist outside the medical support device 24.

[0086] The control device 22 is connected to the external I / F 80 as one of the second external devices. In the example shown in Fig. 3, the external I / F 70 of the control device 22 is connected to the external I / F 80. The external I / F 80 controls the exchange of various information between the processor 82 of the medical support device 24 and the processor 72 of the control device 22. For example, the processor 82 acquires endoscopic moving images 39 (see Fig. 1) from the processor 72 of the control device 22 via the external I / Fs 70 and 80, and performs various image processing on the acquired endoscopic moving images 39.

[0087] The display device 18, which serves as one of the second external devices, is connected to the external I / F 80. The processor 82 controls the display device 18 via the external I / F 80 to cause the display device 18 to display various information (e.g., an endoscopic moving image 39 that has been subjected to various image processing, etc.).

[0088] The processing device 2 is connected to the external I / F 80 via the network 3 as one of the second external devices. The processor 82 exchanges various information with the processing device 2 via the external I / F 80. For example, the external I / F 80 transmits the endoscopic moving image 39 acquired from the camera 52 by the processor 82 to the processing device 2. The processing device 2 receives the endoscopic moving image 39 transmitted from the external I / F 80, executes processing using at least one frame 40 included in the received endoscopic moving image 39, and transmits the processing result to the medical support device 24. The external I / F 80 receives the processing result transmitted from the processing device 2. The processor 82 acquires the processing result received by the external I / F 80. In this embodiment, the external I / F 80 is an example of a "communication device" according to the present disclosure.

[0089] In recent years, there has been progress in the development of a technology in which a trained model optimized by performing machine learning on a model (e.g., a neural network) recognizes lesions 42 by performing object recognition processing on each frame 40 included in an endoscopic video image 39, and the recognition result of the object recognition processing or information based on the recognition result of the object recognition processing is displayed on a screen 35.

[0090] However, the object recognition process performed on each frame 40 included in the endoscopic video 39 imposes a very large processing load. Therefore, currently, a technology is being considered in which the endoscopic device 10 causes a processing device 2 communicably connected to the endoscopic device 10 via a network 3 to execute object recognition processing, and transmits the processing results of the object recognition processing to the endoscopic device 10, thereby reducing the processing load on the endoscopic device 10.

[0091] When this technology is used, a time lag (hereinafter also simply referred to as "time lag") occurs due to transmission and reception between the endoscope device 10 and the processing device 2. If a time lag occurs, there is a risk that the content of the frame 40 displayed in the first display area 35A of the endoscope device 10 and the content of the processing result by the processing device 2 (for example, the recognition result by the object recognition processing, or information based on the recognition result by the object recognition processing) will not match. Furthermore, if the time lag is very short, the content of the frame 40 displayed in the first display area 35A of the endoscope device 10 and the content of the processing result by the processing device 2 will almost match, so there will be no significant impact on the diagnosis by the doctor 12.

[0092] If object recognition processing is performed by the processing device 2 communicably connected to the endoscope device 10 via the network 3, it is very important for the doctor 12 to recognize the extent of the time lag in order for the doctor 12 to make a correct diagnosis in an endoscopic examination. This is not limited to endoscopic examinations, but also applies to other types of medical examinations (for example, radiographic examinations, MRI examinations, CT examinations, etc.).

[0093] In view of such circumstances, in this embodiment, as an example shown in Fig. 4, medical support processing is performed by a processor 82 of the medical support device 24. Note that Fig. 4 is a block diagram showing an example of the main functions of the processor 82 included in the medical support device 24 and an example of information stored in a storage 86.

[0094] A medical support program 90 is stored in the storage 86. The medical support program 90 is an example of a "program" according to the present disclosure. The processor 82 reads the medical support program 90 from the storage 86 and executes the read medical support program 90 on the memory 84 to perform medical support processing. The medical support processing is realized by the processor 82 operating as a communication unit 82A and a control unit 82B in accordance with the medical support program 90 executed on the memory 84.

[0095] Fig. 5 is a conceptual diagram showing an example of the processing contents performed by the medical support device 24 and the processing device 2. As shown in Fig. 5, the communication unit 82A and the control unit 82B acquire each of a plurality of frames 40 in time series included in the endoscopic moving image 39 generated by the camera 52 capturing images at an imaging frame rate (e.g., several tens of frames / second) from the camera 52, one frame at a time. The communication unit 82A and the control unit 82B acquire the frames 40 at the same timing.

[0096] The control unit 82B outputs the endoscopic moving image 39 to the display device 18. For example, the control unit 82B displays the endoscopic moving image 39 as a live view image in the first display area 35A. That is, every time the control unit 82B acquires a frame 40 from the camera 52, the control unit 82B displays the acquired frame 40 in sequence in the first display area 35A according to a display frame rate (e.g., several tens of frames / second). The control unit 82B also displays auxiliary information 44 in the second display area 35B. The control unit 82B also updates the display content of the second display area 35B (e.g., auxiliary information 44) in accordance with the display content of the first display area 35A.

[0097] The communication unit 82A communicates (i.e., transmits and receives various information) with the processing device 2 via the external I / F 80 (see FIG. 3). The communication unit 82A transmits the endoscopic video 39 acquired from the camera 52 to the processing device 2 via the external I / F 80 (see FIG. 3). Here, the communication unit 82A acquires a transmission time 92 which is the time when the endoscopic video 39 is transmitted to the processing device 2. The transmission time 92 is, for example, the time acquired by the communication unit 82A from the RTC 79 at the timing when the endoscopic video 39 is transmitted from the external I / F 80 to the processing device 2.

[0098] The processing device 2 receives the endoscopic video 39 transmitted from the communication unit 82A. The processing device 2 has a recognition model 94, and executes a recognition process 95 using the recognition model 94 on the endoscopic video 39. The processing device 2 may be a device that executes only the recognition process 95, or may be a device that executes multiple processes including the recognition process 95. For example, the multiple processes may include a process of processing at least one frame 40 included in the endoscopic video 39. In this embodiment, the recognition process 95 is an example of the object recognition process described above, and is also an example of the "AI processing" and "recognition processing" according to the present disclosure.

[0099] The processing device 2 recognizes a lesion 42 in the endoscopic moving image 39 based on the endoscopic moving image 39. That is, the processing device 2 sequentially performs a recognition process 95 on each of a plurality of frames 40 in chronological order included in the endoscopic moving image 39 acquired from the camera 52, thereby recognizing the lesion 42 appearing in the frames 40.

[0100] For example, the processing device 2 recognizes the geometric characteristics of the lesion 42 (e.g., position, size, shape, etc.), the type of the lesion 42, and the type of the lesion 42 (e.g., pedunculated, subpedunculated, sessile, surface elevated, surface flat, surface depressed, etc.). Furthermore, the processing device 2 recognizes a region (hereinafter simply referred to as a "region") in the endoscopic video image 39 based on the endoscopic video image 39 acquired from the camera 52. Here, a region refers to a region of the large intestine 28. Examples of the region of the large intestine 28 include the anal canal, lower rectum, upper rectum, sigmoid colon, descending colon, transverse colon, ascending colon, and ileocecal area.

[0101] The recognition process 95 is performed by the processing device 2 on the acquired frame 40 every time the frame 40 is acquired. The recognition process 95 is a process for recognizing the lesion 42 and the body part by a method using AI. Here, the recognition process 95 is a process using a recognition model 94.

[0102] The recognition model 94 is a trained model for object recognition by an AI bounding box method. The recognition model 94 is optimized by performing machine learning using training data on a neural network. The training data is a data set including a plurality of data (i.e., data for a plurality of frames) in which example data and correct answer data are associated with each other.

[0103] The example data is an image that assumes frame 40. A first example of an image that assumes frame 40 is an image obtained by actually capturing an image of the inside of the large intestine with a camera. A second example of an image that assumes frame 40 is a virtually created image (e.g., an image generated by a generation AI). The correct answer data is correct answer data (i.e., annotations) for the example data. Here, annotations that identify the geometric characteristics, type, model, and location of the lesion depicted in the image used as the example data are used as an example of correct answer data.

[0104] The processing device 2 acquires a frame 40 from the camera 52, and inputs the acquired frame 40 to the recognition model 94. As a result, each time a frame 40 is input, the recognition model 94 recognizes the lesion 42 and the area shown in the input frame 40, and generates and outputs a recognition result 96. In this embodiment, the recognition result 96 is an example of a "processing result" according to the present disclosure.

[0105] The processing device 2 transmits the recognition result 96 itself to the medical support device 24, or transmits information generated based on the recognition result 96 to the medical support device 24. In the medical support device 24, the external I / F 80 receives the recognition result 96 transmitted from the processing device 2. The communication unit 82A acquires the recognition result 96 received by the external I / F 80.

[0106] Here, the communication unit 82A acquires a reception time 98 which is the time when the endoscopic video 39 is received by the external I / F 80. The reception time 98 is, for example, the time acquired by the communication unit 82A from the RTC 79 at the timing when the recognition result 96 transmitted from the processing device 2 is received by the external I / F 80.

[0107] The control unit 82B acquires the recognition result 96, the transmission time 92, and the reception time 98 from the communication unit 82A. Based on the recognition result 96, the transmission time 92, and the reception time 98, the control unit 82B displays the recognition result 96 itself on the screen 35 (for example, the first display area 35A, the second display area 35B, and / or the third display area 35C) of the display device 18, or displays information generated based on the recognition result 96 on the screen 35 (for example, the first display area 35A, the second display area 35B, and / or the third display area 35C).

[0108] The recognition result 96 includes lesion presence / absence information 100. The lesion presence / absence information 100 is information indicating whether or not a lesion 42 is captured in the frame 40 input to the recognition model 94. The recognition result 96 also includes geometric property information 102, a lesion location map 106, lesion feature information 108, and the like.

[0109] The geometric characteristic information 102 is information (e.g., coordinates) capable of identifying the shape, size, and position of the lesion 42 within the frame 40. The geometric characteristic information 102 or information based on the geometric characteristic information 102 may be displayed in the first display area 35A, may be displayed as a piece of auxiliary information 44 in the second display area 35B, or may be displayed in the third display area 35C.

[0110] The lesion location map 106 is a map capable of identifying the location of the lesion 42 within the frame 40. The lesion characteristic information 108 is information capable of identifying the type and model of the lesion 42 appearing in the frame 40 that has been input to the recognition model 94.

[0111] The geometric characteristics of the lesion location map 106 (e.g., the shape and size of the outer contour) correspond to the geometric characteristics of the frame 40 (e.g., the shape and size of the outer contour). The lesion location map 106 is displayed on the screen 35 by the control unit 82B. In this embodiment, the lesion location map 106 is displayed in the second display area 35B by the control unit 82B as one of the auxiliary information 44. For example, the display and non-display of the lesion location map 106 is switched according to an instruction and / or various conditions received by the reception device 64. The lesion location map 106 displayed on the screen 35 is updated according to the display frame rate applied to the first display area 35A. With this configuration, the doctor 12 can grasp the general position of the lesion 42 in the endoscopic moving image 39 displayed in the first display area 35A by referring to the lesion location map 106 displayed in the second display area 35B while observing the endoscopic moving image 39 displayed in the first display area 35A.

[0112] The control unit 82B displays the lesion characteristic information 108 on the screen 35. For example, the lesion characteristic information 108 may be displayed as one piece of auxiliary information 44 in the second display area 35B, or in the first display area 35A and / or the third display area 35C.

[0113] The recognition result 96 also includes body part information 110. The body part information 110 is information that can identify a body part captured in the frame 40. The body part information 110 may be displayed in the first display area 35A, may be displayed as a piece of auxiliary information 44 in the second display area 35B, or may be displayed in the third display area 35C.

[0114] 6 is a conceptual diagram showing an example of the processing contents of the communication unit 82A and the control unit 82B, and an example of the display contents displayed on the screen 35. As shown in FIG. 6, the control unit 82B calculates a transmission / reception time 112 based on a transmission time 92 and a reception time 98 acquired from the communication unit 82A. Here, the transmission / reception time 112 refers to the time required from when a frame 40 included in an endoscopic moving image 39 is transmitted to the processing device 2 by the external I / F 80 until a recognition result 96 obtained by executing a recognition process 95 on the frame 40 is received by the external I / F 80. The transmission / reception time 112 can be said to be an index representing the degree of a time lag caused by transmission / reception between the medical support device 24 and the processing device 2. In this embodiment, the transmission / reception time 112 is an example of the "time" according to the present disclosure.

[0115] The control unit 82B identifies the difference between the transmission and reception time 112 and the reference time 114 by comparing the transmission and reception time 112 with the reference time 114. In other words, the difference between the transmission and reception time 112 and the reference time 114 indicates whether the transmission and reception time 112 is longer or shorter than the reference time 114. In this embodiment, the reference time 114 is an example of the "first reference time," "second reference time," "third reference time," and "fourth reference time" according to the present disclosure.

[0116] An example of the reference time 114 is the longest time during which the degree of affinity between the frame 40 displayed in the first display area 35A and the recognition result 96 falls within an acceptable range. For example, the longest time during which the degree of affinity between the frame 40 displayed in the first display area 35A and the recognition result 96 falls within an acceptable range is the time obtained in advance by testing with an actual device and / or computer simulation as the longest time during which the doctor 12 or the like recognizes that the content of the frame 40 displayed in the first display area 35A and the content of the recognition result 96 match (i.e., are consistent).

[0117] Here, the longest time at which the degree of affinity between the frame 40 displayed in the first display area 35A and the recognition result 96 falls within an allowable range is illustrated as an example of the reference time 114, but this is merely an example, and the reference time 114 may be a statistical time (e.g., a mode, an average, or a median) obtained in advance by a test using an actual device and / or a computer simulation as the time at which the doctor 12 or the like recognizes that the contents of the frame 40 displayed in the first display area 35A and the contents of the recognition result 96 match (i.e., are consistent). Also, the reference time 114 may be a fixed value, or may be a variable value that is changed according to an instruction accepted by the acceptance device 64 or the like and / or various conditions.

[0118] The control unit 82B determines whether or not to perform display based on the recognition result 96 based on the transmission / reception time 112, and controls the display device 18 to perform or not perform display based on the recognition result 96 based on the determination result.

[0119] The control unit 82B acquires the recognition result 96 from the processing device 2, and generates and outputs processing result information 116, which is information relating to the acquired recognition result 96. In the example shown in Fig. 6, when the control unit 82B outputs the processing result information 116, it means that the control unit 82B displays the processing result information 116 on the screen 35. The processing result information 116 includes, for example, information capable of identifying the lesion 42, information indicating the characteristics of the lesion 42, and / or information capable of identifying the area where the lesion 42 is present.

[0120] The longer the transmission / reception time 112, the greater the degree of the time lag caused by transmission / reception between the medical support device 24 and the processing device 2. This means that the greater the degree of the time lag, the greater the possibility that the contents of the frame 40 displayed on the screen 35 and the contents of the processing result information 116 are not consistent, and conversely, the smaller the degree of the time lag, the greater the possibility that the contents of the frame 40 displayed on the screen 35 and the contents of the processing result information 116 are consistent.

[0121] In this embodiment, in order for the control unit 82B to determine whether the transmission / reception time 112, which can be said to be an index showing the degree of time lag, is a time when the contents of the frame 40 displayed on the screen 35 and the contents of the processing result information 116 match, the control unit 82B compares the transmission / reception time 112 with a reference time 114 to determine whether the transmission / reception time 112 is equal to or greater than the reference time 114. That is, if the transmission / reception time 112 is equal to or greater than the reference time 114, the control unit 82B determines that the contents of the frame 40 displayed on the screen 35 and the contents of the processing result information 116 do not match, and if the transmission / reception time 112 is less than the reference time 114, the control unit 82B determines that the contents of the frame 40 displayed on the screen 35 and the contents of the processing result information 116 match. The determination result by the control unit 82B is expressed in the output form of the processing result information 116 or by outputting information related to the transmission / reception time 112.

[0122] For example, the control unit 82B performs a first control for outputting information related to the transmission / reception time 112 and a second control for changing the output mode of the processing result information 116 depending on the transmission / reception time 112.

[0123] Here, an example of the second control will be described first. An example of the output mode of the processing result information 116 is a display mode in which the control unit 82B displays the processing result information 116 on the screen 35. The display mode includes a first display mode and a second display mode. The first display mode is used when the transmission / reception time 112 is equal to or greater than the reference time 114, and is capable of identifying that the transmission / reception time 112 is equal to or greater than the reference time 114, and the second display mode is used when the transmission / reception time 112 is less than the reference time 114, and is capable of identifying that the transmission / reception time 112 is less than the reference time 114. In the example shown in FIG. 6, when the transmission / reception time 112 is equal to or greater than the reference time 114, the control unit 82B displays the processing result information 116 on the screen 35 in the first display mode, and when the transmission / reception time 112 is less than the reference time 114, the control unit 82B displays the processing result information 116 on the screen 35 in the second display mode. The second display mode has higher visibility than the first display mode. For example, in the first display mode, the processing result information 116 is not displayed, and in the second display mode, the processing result information 116 is displayed.

[0124] Further, the second control performed by the control unit 82B includes control to display the frame 40 on the screen 35 (first display area 35A in the example shown in Fig. 6) in a state in which it can be compared with the processing result information 116. In the example shown in Fig. 6, when the transmission / reception time 112 is less than the reference time 114, the processing result information 116 is displayed in the second display area 35B and the third display area 35C, and the same frame 40 as the frame 40 inputted into the recognition model 94 to obtain the recognition result 96 used to generate the processing result information 116 is displayed in the first display area 35A.

[0125] Here, an example has been given in which the same frame 40 as the frame 40 input into the recognition model 94 to obtain the recognition result 96 used to generate the processing result information 116 is displayed in the first display area 35A, but this is merely one example, and a frame 40 obtained later in time than the frame 40 input into the recognition model 94 to obtain the recognition result 96 used to generate the processing result information 116 (for example, a frame 40 obtained several frames to several tens of frames later) may be displayed in the first display area 35A.

[0126] In the example shown in FIG. 6, when the transmission / reception time 112 is less than the reference time 114, the control unit 82B generates part name information 118, lesion identification information 120, differentiation information 122, and size information 124 as processing result information 116, and displays them in the third display area 35C (in the example shown in FIG. 6, the area below the first display area 35A).

[0127] The region name information 118 is information indicating the name of the region recognized by the recognition process 95, and is generated based on the region information 110 (see FIG. 5). Here, the region information 110 is illustrated as an example, but any information other than the region information 110 may be used as long as it is information capable of identifying the region where the lesion 42 is present.

[0128] Lesion identification information 120 is information capable of identifying lesion 42 recognized by performing recognition process 95. Differentiation information 122 is information indicating the result of differentiating lesion 42 (e.g., the degree of malignancy), and is generated based on lesion characteristic information 108 (see FIG. 5). Size information 124 is information indicating the size of lesion 42 recognized by performing recognition process 95, and is generated based on geometric property information 102 (see FIG. 5). Although differentiation information 122 and size information 124 are exemplified here, information indicating the characteristics of lesion 42 other than differentiation information 122 and size information 124 may be used.

[0129] The lesion position map 106 (see FIG. 6) includes a bounding box BB, and when the transmission / reception time 112 is less than the reference time 114, the control unit 82B displays the bounding box BB in the first display area 35A as the processing result information 116. The bounding box BB is a rectangular frame (e.g., a rectangular frame circumscribing an image area showing the lesion 42) capable of identifying the position recognized by the recognition model 94 as the position in the frame 40 of the lesion 42 captured in the frame 40. In the example shown in FIG. 6, the bounding box BB is superimposed and displayed at the position where the lesion 42 is captured in the frame 40 displayed in the first display area 35A. The display of the bounding box BB is updated in synchronization with the display timing of the endoscopic moving image 39 displayed in the first display area 35A.

[0130] Note that although a bounding box BB is given here, this is merely one example, and the control unit 82B may display an identifier in place of the bounding box BB (for example, a mark defined by the four corners of a rectangular frame) on the screen 35 (in the example shown in FIG. 6, the first display area 35A within the screen 35).

[0131] 6, the control unit 82B generates past information 126 as one piece of auxiliary information 44 and displays it in the second display area 35B. The past information 126 is information generated based on a recognition result 96 obtained in the past. For example, the past information 126 includes various pieces of information previously displayed in the third display area 35C, and images or text that can identify areas that have not been observed or treated in the current endoscopic examination.

[0132] Furthermore, when the transmission / reception time 112 is less than the reference time 114, the control unit 82B displays the lesion location map 106 as one piece of the processing result information 116 and as one piece of the auxiliary information 44 in the second display region 35B.

[0133] Here, an example of the first control will be described. The control unit 82B performs control to output time information 130 related to the transmission / reception time 112 as the first control, regardless of whether the transmission / reception time 112 is equal to or longer than the reference time 114. The time information 130 is an example of the "time information" according to the present disclosure.

[0134] In the example shown in FIG. 6, as one example of a mode in which the control unit 82B outputs the time information 130, a mode in which the control unit 82B displays the time information 130 in the second display area 35B is shown.

[0135] The first control performed by the control unit 82B includes control to display the frame 40 on the screen 35 (in the example shown in Fig. 6, the first display area 35A) in a state in which it can be compared with the time information 130. In the example shown in Fig. 6, the time information 130 is displayed in the second display area 35B, and the same frame 40 as the frame 40 inputted to the recognition model 94 to obtain the recognition result 96 received by the external I / F 80 at the reception time 98 used to calculate the transmission / reception time 112 corresponding to the time information 130 is displayed in the first display area 35A.

[0136] Here, an example is given in which the same frame 40 as the frame 40 input into the recognition model 94 to obtain the recognition result 96 received by the external I / F 80 at the reception time 98 used to calculate the transmission / reception time 112 corresponding to the time information 130 is displayed in the first display area 35A, but this is merely one example, and a frame 40 obtained later in time (for example, a frame 40 obtained several to several tens of frames later) than the frame 40 input into the recognition model 94 to obtain the recognition result 96 received by the external I / F 80 at the reception time 98 used to calculate the transmission / reception time 112 corresponding to the time information 130 may be displayed in the first display area 35A.

[0137] The time information 130 includes first identification information 130A capable of identifying the transmission / reception time 112 itself, and second identification information 130B (in the example shown in FIG. 6 ) capable of identifying the difference between the transmission / reception time 112 and the reference time 114. The first identification information 130A and the second identification information 130B are information visualized by text. The first identification information 130A is an example of the "first identification information" according to the present disclosure, and the second identification information 130B is an example of the "second identification information" according to the present disclosure.

[0138] 6, the transmission / reception time 112 itself is displayed in text format in the second display area 35B as the first identification information 130A. Also, the first identification information 130A is displayed in the second display area 35B with content that can be identified as information indicating a time lag (i.e., a communication delay time).

[0139] In the example shown in FIG. 6, as the second identification information 130B, length information capable of identifying whether the transmission / reception time 112 is longer or shorter than the reference time 114 is displayed in the second display area 35B in text format. An example of the length information is information capable of identifying whether the transmission / reception time 112 is longer or shorter than the reference time 114. In the example shown in FIG. 6, if the transmission / reception time 112 is longer than the reference time 114, the text "high" is displayed as the length information, and if the transmission / reception time 112 is shorter than the reference time 114, the text "low" is displayed as the length information. In addition, the second identification information 130B is displayed in the second display area 35B with content capable of identifying that it is information indicating the degree of time lag. Note that here, the text "high" and the text "low" are examples of "length information" according to the present disclosure.

[0140] Also, here, the text "high" and the text "low" are exemplified as the length information, but this is merely an example. When the transmission / reception time 112 is equal to or greater than the reference time 114, some information capable of identifying that the transmission / reception time 112 is equal to or greater than the reference time 114 (for example, an indicator capable of identifying that the transmission / reception time 112 is equal to or greater than the reference time 114) may be output, and when the transmission / reception time 112 is less than the reference time 114, some information capable of identifying that the transmission / reception time 112 is less than the reference time 114 (for example, an indicator capable of identifying that the transmission / reception time 112 is less than the reference time 114) may be output.

[0141] When the transmission / reception time 112 is less than the reference time 114, it is not necessary to output any information that can identify that the transmission / reception time 112 is less than the reference time 114. This is because, when the transmission / reception time 112 is less than the reference time 114, the content of the frame 40 displayed in the first display area 35A and the processing result information 116 are consistent, and even if the time information 130 is not output, there is no particular hindrance to the doctor 12 in performing the endoscopic examination.

[0142] The control unit 82B changes the visualization format in which the time information 130 is visualized, depending on the transmission and reception time 112. For example, the control unit 82B increases the visibility of the time information 130 when the transmission and reception time 112 is equal to or greater than the reference time 114, compared to the visibility of the time information 130 when the transmission and reception time 112 is less than the reference time 114. In the example shown in FIG. 6, the control unit 82B displays the time information 130 at different visibility levels when the transmission and reception time 112 is less than the reference time 114 and when the transmission and reception time 112 is equal to or greater than the reference time 114. For example, the control unit 82B displays the time information 130 when the transmission and reception time 112 is less than the reference time 114 at a first visibility level, and displays the time information 130 when the transmission and reception time 112 is equal to or greater than the reference time 114 at a second visibility level.

[0143] The second visibility level is a visibility level higher than the first visibility level. For example, the height of the visibility level is determined by the size of the text of the time information 130. In the example shown in FIG. 6, the size of the text of the time information 130 at the second visibility level is larger than the size of the text of the time information 130 at the first visibility level. That is, the control unit 82B increases the size of the text of the time information 130 when the transmission / reception time 112 is equal to or greater than the reference time 114 compared to the size of the text of the time information 130 when the transmission / reception time 112 is less than the reference time 114, thereby increasing the visibility of the time information 130 when the transmission / reception time 112 is equal to or greater than the visibility of the time information 130 when the transmission / reception time 112 is less than the reference time 114.

[0144] Note that, although an example of changing the visibility level by changing the size of the text of the time information 130 is given here, this is merely one example. For example, the visibility level may be changed by changing the transparency of the text of the time information 130, the color density of the text of the time information 130, and / or the brightness of the text of the time information 130. Also, the visibility of the time information 130 may be changed by displaying the time information 130 when the transmission / reception time 112 is equal to or greater than the reference time 114, and by hiding the time information 130 when the transmission / reception time 112 is less than the reference time 114.

[0145] Next, the operation of the portion of the medical support system 1 according to the present disclosure will be described with reference to Fig. 7. The flow of the medical support process shown in Fig. 7 is an example of the "medical support method" according to the present disclosure.

[0146] 7, first, in step ST10, the control unit 82B determines whether or not one frame of images has been captured by the camera 52 inside the large intestine 28. If one frame of images has not been captured by the camera 52 inside the large intestine 28 in step ST10, the determination is negative, and the medical support process proceeds to step ST28. If one frame of images has been captured by the camera 52 inside the large intestine 28 in step ST10, the determination is positive, and the medical support process proceeds to step ST12.

[0147] In step ST12, the communication unit 82A and the control unit 82B acquire a frame 40 obtained by imaging the inside of the large intestine 28 (e.g., the intestinal wall 32) by the camera 52. Then, the control unit 82B displays the frame 40 in the first display area 35A. Note that, if the frame 40 is already displayed in the first display area 35A, the control unit 82B updates the frame 40 displayed in the first display area 35A. That is, the processing of step ST12 is repeatedly executed, and the frame 40 is displayed in the first display area 35A in a live view manner. After the processing of step ST12 is executed, the medical support processing proceeds to step ST14.

[0148] In step ST14, the communication unit 82A causes the external I / F 80 to transmit the frame 40 acquired in step ST12 to the processing device 2. Then, the communication unit 82A acquires the transmission time 92. After the processing of step ST14 is executed, the medical support processing proceeds to step ST16.

[0149] When the frame 40 is transmitted to the processing device 2 by executing the process of step ST14, the processing device 2 executes a recognition process 95 on the frame 40 and transmits a recognition result 96 by the recognition process 95 to the medical support device 24.

[0150] In step ST16, the communication unit 82A determines whether or not the recognition result 96 has been received by the external I / F 80. If the recognition result 96 has not been received by the external I / F 80 in step ST16, the determination is negative, and the determination of step ST16 is performed again. If the recognition result 96 has been received by the external I / F 80 in step ST16, the determination is positive, and the medical support process proceeds to step ST18. Furthermore, if the recognition result 96 has been received by the external I / F 80 in step ST16, the communication unit 82A acquires the reception time 98.

[0151] In step ST18, the control unit 82B acquires the transmission time 92 and the reception time 98 from the communication unit 82A, and calculates the transmission / reception time 112 based on the transmission time 92 and the reception time 98. After the processing of step ST18 is executed, the medical support processing proceeds to step ST20.

[0152] In step ST20, the control unit 82B determines whether or not the transmission / reception time 112 calculated in step ST18 is shorter than the reference time 114. In step ST20, if the transmission / reception time 112 calculated in step ST18 is not shorter than the reference time 114, the determination is negative, and the medical support processing proceeds to step ST26. In step ST20, if the transmission / reception time 112 calculated in step ST18 is shorter than the reference time 114, the determination is positive, and the medical support processing proceeds to step ST22.

[0153] In step ST22, the control section 82B displays the processing result information 116 on the screen 35. After the processing in step ST22 is executed, the medical support processing proceeds to step ST24.

[0154] In step ST24, the control unit 82B displays the time information 130 in the second display area 35B at the first visibility level. After the process of step ST24 is executed, the medical support process proceeds to step ST28.

[0155] In step ST26, the control unit 82B displays the time information 130 at the second visibility level in the second display area 35B. After the process of step ST26 is executed, the medical support process proceeds to step ST28.

[0156] In step ST28, the control unit 82B judges whether or not a condition for terminating the medical support process is satisfied. One example of the condition for terminating the medical support process is that an instruction to terminate the medical support process is given to the medical support system 1 (for example, that an instruction to terminate the medical support process is accepted by the acceptance device 64).

[0157] In step ST28, if the condition for terminating the medical support process is not satisfied, the determination is negative, and the medical support process proceeds to step ST10. In step ST28, if the condition for terminating the medical support process is satisfied, the determination is positive, and the medical support process ends.

[0158] As described above, in the medical support system 1, the time information 130 is displayed on the screen 35. The time information 130 is information on the transmission / reception time 112 required from when the frame 40 is transmitted from the external I / F 80 of the medical support device 24 to the processing device 2 until the recognition result 96 obtained by the processing device 2 executing the recognition process 95 on the frame 40 is received by the external I / F 80 of the medical support device 24. The transmission / reception time 112 can be said to be an index representing the degree of the time lag caused by the transmission / reception between the medical support device 24 and the processing device 2. Therefore, in the medical support system 1, the display mode of the processing result information 116 is made different depending on the transmission / reception time 112. In this way, the doctor 12 can visually recognize the degree of the time lag caused by the transmission / reception between the medical support device 24 and the processing device 2 from the time information 130 displayed on the screen 35 and / or the display mode of the processing result information 116.

[0159] Furthermore, in the medical support system 1, the time information 130 and the frame 40 are displayed on the screen 35 in a comparative state. Therefore, the doctor 12 can determine whether or not the contents of the frame 40 and the processing result information 116 are consistent with each other while referring to the time information 130.

[0160] In addition, in the medical support system 1, the first identification information 130A is displayed in the second display area 35B. The first identification information 130A is information that can identify the transmission / reception time 112 itself. Therefore, the doctor 12 can visually recognize the transmission / reception time 112 itself, which indicates the degree of time lag caused by transmission / reception between the medical support device 24 and the processing device 2.

[0161] Furthermore, in the medical support system 1, the second identification information 130B is displayed in the second display area 35B. The second identification information 130B is information that can identify the difference between the transmission / reception time 112 and the reference time 114 (i.e., whether the transmission / reception time 112 is longer or shorter than the reference time 114). Therefore, the doctor 12 can visually recognize the difference between the transmission / reception time 112 and the reference time 114 (i.e., whether the transmission / reception time 112 is longer or shorter than the reference time 114), which indicates the degree of time lag caused by transmission / reception between the medical support device 24 and the processing device 2.

[0162] Furthermore, in the medical support system 1, the time information 130 is displayed as visualized information in the second display area 35B. For example, the time information 130 is visualized by text. Therefore, the degree of the time lag caused by transmission and reception between the medical support device 24 and the processing device 2 can be visualized and made perceptible to the doctor 12.

[0163] Furthermore, in the medical support system 1, the visualization format is changed according to the transmission / reception time 112. For example, the size of the text of the time information 130 is changed according to the transmission / reception time 112. In this case, for example, the control unit 82B makes the size of the text of the time information 130 displayed in the second display area 35B larger when the transmission / reception time 112 is equal to or greater than the reference time 114 than the size of the text of the time information 130 when the transmission / reception time 112 is less than the reference time 114. This allows the doctor 12 to visually recognize whether the transmission / reception time 112 is equal to or greater than the reference time 114.

[0164] Furthermore, in the medical support system 1, the display mode of the processing result information 116 is changed depending on whether the transmission / reception time 112 is equal to or longer than the reference time 114. Therefore, the doctor 12 can determine whether the transmission / reception time 112 is equal to or longer than the reference time 114 from the display mode of the processing result information 116.

[0165] Furthermore, in the medical support system 1, when the transmission / reception time 112 is less than the reference time 114, the processing result information 116 is displayed on the screen 35, and when the transmission / reception time 112 is equal to or greater than the reference time 114, the processing result information 116 is not displayed on the screen 35. This allows the doctor 12 to visually distinguish between a case where the transmission / reception time 112 is less than the reference time 114 and a case where the transmission / reception time 112 is equal to or greater than the reference time 114. Furthermore, since the processing result information 116 is not displayed on the screen 35, the doctor 12 can recognize that the content of the frame 40 and the processing result information 116 are not consistent.

[0166] Furthermore, in the medical support system 1, the frame 40 and the processing result information 116 are displayed on the screen 35 in a comparative manner. Therefore, the doctor 12 can determine whether or not the frame 40 and the processing result information 116 are consistent with each other.

[0167] Furthermore, in the medical support system 1, when the transmission / reception time 112 is less than the reference time 114, the frame 40 and the processing result information 116 are displayed on the screen 35 in a comparable state. Therefore, the doctor 12 can visually grasp that the transmission / reception time 112 is less than the reference time 114, and can determine whether the frame 40 and the processing result information 116 are consistent with each other.

[0168] In addition, in the medical support system 1, the longest time at which the degree of affinity between the frame 40 displayed in the first display area 35A and the recognition result 96 falls within an allowable range is used as the reference time 114 to be compared with the transmission / reception time 112 by the control unit 82B. Therefore, compared to a case in which a time at which the degree of affinity between the frame 40 displayed in the first display area 35A and the recognition result 96 does not fall within an allowable range is used as the reference time 114, it is possible to suppress the doctor 12 from erroneously recognizing that the frame 40 currently displayed in the first display area 35A and the processing result information 116 displayed on the screen 35 are consistent, even if a recognition result 96 that is not consistent with the frame 40 currently displayed in the first display area 35A is obtained due to an excessively long time lag.

[0169] In the above embodiment, an example has been given in which, when the transmission / reception time 112 is less than the reference time 114, the lesion location map 106 is displayed in the second display area 35B as one piece of processing result information 116 and as one piece of auxiliary information 44, but the present disclosure is not limited to this. For example, as shown in Fig. 8, the lesion location map 106 may be displayed in the second display area 35B regardless of whether the transmission / reception time 112 is less than the reference time 114.

[0170] The consistency between the recognition result 96 and the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is equal to or longer than the reference time 114 is lower than the consistency between the recognition result 96 and the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is shorter than the reference time 114. In order to allow the doctor 12 to visually recognize this fact, the control unit 82B makes the visibility of the lesion location map 106 displayed in the second display area 35B lower when the transmission / reception time 112 is equal to or longer than the reference time 114 than the visibility of the lesion location map 106 displayed in the second display area 35B when the transmission / reception time 112 is shorter than the reference time 114. This allows the doctor 12 to visually recognize whether the degree of the time lag is equal to or longer than the reference time 114 from the visibility of the lesion location map 106. Also, the doctor 12 can visually grasp the degree of consistency between the recognition result 96 and the frame 40 displayed in the first display area 35A from the visibility of the lesion location map 106.

[0171] In the above embodiment, an example in which the processing result information 116 is not displayed on the screen 35 when the transmission / reception time 112 is equal to or longer than the reference time 114 has been given, but this is merely an example. For example, as shown in FIG. 9, even when the transmission / reception time 112 is equal to or longer than the reference time 114, the processing result information 116 may be displayed on the screen 35. In this case, the visibility of the processing result information 116 is lowered. That is, when the transmission / reception time 112 is equal to or longer than the reference time 114, the visibility of the processing result information 116 displayed on the screen 35 is lowered than when the transmission / reception time 112 is shorter than the reference time 114. In the example shown in FIG. 9, when the transmission / reception time 112 is equal to or longer than the reference time 114, the site name information 118, the lesion identification information 120, the differentiation information 122, and the size information 124 are displayed in a grayed-out state in the third display area 35C, and the line of the bounding box BB is also displayed thinner than the line of the bounding box BB displayed when the transmission / reception time 112 is shorter than the reference time 114. There are various methods for reducing visibility, such as a method for reducing visibility by increasing the transparency of the display, a method for reducing visibility by covering the display area with some kind of mask, and a method for reducing visibility by blinking the display.

[0172] 9, when the transmission / reception time 112 is less than the reference time 114, the time information 130 may not be displayed on the screen 35. By not displaying the time information 130 on the screen 35, the doctor 12 can proceed with medical treatment without paying attention to the time information 130.

[0173] Furthermore, as shown in FIG. 9 as an example, when the transmission / reception time 112 is equal to or longer than the reference time 114, a notification mark 132 may be displayed on the screen 35 (in the example shown in FIG. 9 ) as one piece of visualized information of the time information 130. The notification mark 132 is a mark that notifies that the transmission / reception time 112 is equal to or longer than the reference time 114 (in other words, that the processing result information 116 displayed on the screen 35 and the content of the frame 40 displayed in the first display area 35A are not consistent). In this way, by displaying the notification mark 132 on the screen 35, the doctor 12 can visually grasp from the notification mark 132 that the transmission / reception time 112 is equal to or longer than the reference time 114 and that the processing result information 116 displayed on the screen 35 and the content of the frame 40 displayed in the first display area 35A are not consistent. The notification mark 132 is an example of the “time information” and the “mark” according to the present disclosure.

[0174] 9, when the transmission / reception time 112 is equal to or greater than the reference time 114, the visibility of the bounding box BB displayed on the screen 35 is lower than when the transmission / reception time 112 is less than the reference time 114, but this is merely one example, and the display mode (e.g., color, line type, or shape, etc.) of the bounding box BB may be made different when the transmission / reception time 112 is equal to or greater than the reference time 114 and when the transmission / reception time 112 is less than the reference time 114. In addition, the display mode (e.g., color, line type, or shape, etc.) of the bounding box BB may be made different depending on the type and / or model of the lesion 42.

[0175] In the above embodiment, an example in which the time information 130 and the processing result information 116 are displayed on the screen 35 has been given, but this is merely one example, and the time information 130 and / or the processing result information 116 may be output as sound from a speaker (not shown). In this case, for example, the control unit 82B may adjust the sound quality and / or volume of the sound output from the speaker so that the sound when the transmission / reception time 112 is equal to or longer than the reference time 114 is easier to hear than the sound when the transmission / reception time 112 is shorter than the reference time 114.

[0176] In the above embodiment, an example is given on the premise that the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is less than the reference time 114 is the same as the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is equal to or greater than the reference time 114, but the present disclosure is not limited thereto. For example, as shown in FIG. 10, the control unit 82B may use a frame 40 obtained later in time than the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is less than the reference time 114 as the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is equal to or greater than the reference time 114. Here, the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is equal to or greater than the reference time 114 is an example of the "second medical image" and the "third medical image" according to the present disclosure.

[0177] In this way, by using a frame 40 obtained temporally later than the frame 40 displayed in the first display area 35A when the transmission / reception time 112 is less than the reference time 114 as the frame 40 to be displayed in the first display area 35A when the transmission / reception time 112 is equal to or greater than the reference time 114, the contents of the frame 40 displayed in the first display area 35A can be consistent with the processing result information 116 displayed on the screen 35, even when the transmission / reception time 112 is equal to or greater than the reference time 114.

[0178] Furthermore, when the transmission / reception time 112 is equal to or greater than the reference time 114, guide information may be displayed on the screen 35, recommending stopping the camera 52 inserted into the large intestine 28 or slowing down the speed of inserting or removing the camera 52. In this way, the content of the frame 40 displayed in the first display area 35A and the processing result information 116 displayed on the screen 35 can be made consistent.

[0179] In the above embodiment, the time information 130 is displayed in the second display area 35B, but this is merely an example, and the time information 130 may be displayed in a location other than the second display area 35B. The same can be said for the processing result information 116.

[0180] Moreover, the processing result information 116 and the time information 130 do not have to be displayed only on the screen 35 of the display device 18. For example, the processing result information 116 may be distributed and displayed on a plurality of display devices, and the time information 130 may also be distributed and displayed on a plurality of display devices.

[0181] In the above embodiment, lesion 42 is exemplified as an example of a "characteristic region" according to the present disclosure, but the present disclosure is not limited thereto, and the technology of the present disclosure can be applied even if a resection region, a bleeding region, a marking region, an organ, or a treatment tool (e.g., a hemostatic clip placed inside the body) is applied instead of lesion 42.

[0182] In the above embodiment, the camera 52 mounted on the endoscope 16 is exemplified, but the present disclosure is not limited to this. For example, the technology of the present disclosure is also applicable to a fundus camera, a camera mounted on a medical microscope, or the like.

[0183] In the above embodiment, a frame 40 is illustrated as being obtained by imaging the inside of the large intestine 28 using a camera 52 mounted on the endoscope scope 16, but this is merely one example, and the frame 40 may be a medical image obtained using various modalities such as a radiation imaging device, an MRI, a CT, or an ultrasound probe.

[0184] In the above embodiment, the recognition model 94 is exemplified, but a so-called generation AI may be used as the recognition model 94. For example, instruction data (so-called prompt) including the frame 40 may be input to the generation AI, and the generation AI may output the recognition result 96 or the processing result information 116. Furthermore, the generation AI may output auxiliary information 44 in addition to the recognition result 96 or the processing result information 116 in response to the input prompt. One example of the generation AI is GPT-4 (Internet search<https: / / openai.com / gpt-4> ) is used in ChatGPT.

[0185] Instruction data including at least a part of the processing result information 116 may be used as input information for the generation AI. Examples of information output from the generation AI include information regarding the operation of the camera 52, information indicating the contents of medical procedures recommended to be performed during the endoscopic examination, and / or information indicating the contents of medical procedures recommended to be performed after the endoscopic examination. The information output from the generation AI may be stored in various storage areas (e.g., storage 76 and / or 86), displayed as auxiliary information 44 on the display device 18, printed on a medium by a printer, or output as audio from a speaker.

[0186] In the above embodiment, recognition processing 95 using AI in a bounding box method is exemplified, but this is merely one example, and for example, recognition processing using AI in a segmentation method may be performed instead of recognition processing using AI in a bounding box method 95. Also, instead of recognition processing using an AI method, recognition processing using a non-AI method (for example, a template matching method) may be performed, or recognition processing using a combination of a non-AI method and an AI method may be performed.

[0187] In the above embodiment, an example was given of a form in which medical support processing is performed by computer 78, but the present disclosure is not limited to this, and at least a portion of the processing included in the medical support processing may be performed by a device provided outside computer 78.

[0188] In the above embodiment, the processing device 2 is realized by cloud computing, but this is merely an example. The processing device 2 may be realized by network computing such as fog computing, edge computing, or grid computing.

[0189] In the above embodiment, an example in which the medical support program 90 is stored in the storage 86 has been described, but the present disclosure is not limited to this. For example, the medical support program 90 may be stored in a portable computer-readable non-transitory storage medium such as an SSD or a USB memory. The medical support program 90 stored in the non-transitory storage medium is installed in the computer 78 of the endoscope device 10. The processor 82 executes medical support processing in accordance with the medical support program 90.

[0190] In addition, the medical support program 90 may be stored in a storage device such as another computer or server connected to the endoscopic device 10 via a network, and the medical support program 90 may be downloaded and installed on the computer 78 upon request from the endoscopic device 10.

[0191] It is not necessary to store all of the medical support program 90 in a storage device such as another computer or server device connected to the endoscopic device 10, or to store all of the medical support program 90 in the storage 86; only a portion of the medical support program 90 may be stored.

[0192] The hardware resources for executing the medical support processing can be various processors as shown below. An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the medical support processing by executing software, i.e., a program. Another example of a processor is a dedicated electric circuit, which is a processor having a circuit configuration designed specifically for executing a specific process, such as an FPGA, a PLD, or an ASIC. Each processor has a built-in or connected memory, and each processor executes the medical support processing by using the memory.

[0193] The hardware resource for executing the medical support process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource for executing the medical support process may be a single processor.

[0194] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes medical support processing. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute medical support processing on a single IC chip, as typified by SoC. In this way, the medical support processing is realized using one or more of the above various processors as hardware resources.

[0195] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. Also, the above medical support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processing may be changed, without departing from the spirit of the invention.

[0196] The above description and illustrations are detailed descriptions of the parts related to the present disclosure and are merely an example of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations within the scope of the gist of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the present disclosure.

[0197] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0198] The following supplementary notes are further disclosed regarding the above embodiment.

[0199] (Appendix 1) A communication device; A processor, The communication device includes: Transmitting the medical image to a processing device; receiving a processing result obtained by the processing device executing processing on the medical image; The processor is A first control is performed in which the communication device outputs time information regarding the time required from when the communication device transmits the medical image to the processing device until when the communication device receives the processing result, and / or a second control is performed in which the communication device changes the output mode of the processing result depending on the time. Medical support equipment.

[0200] (Appendix 2) The processor outputting the time information includes the processor displaying the time information on a screen. 2. A medical support device as described in appendix 1.

[0201] (Appendix 3) The first control includes a control for displaying the medical image on a screen in a state in which the time can be compared with the time information when the time is less than a reference time. 3. A medical support device as described in appendix 2.

[0202] This allows a user (for example, a doctor) to easily understand the relationship between the medical image processed by the processing device and the time lag that occurs during transmission and reception.

[0203] (Appendix 4) The first control includes a control for displaying the medical image on a second screen in a state in which the medical image can be compared with the time information when the time is equal to or greater than a reference time. 3. A medical support device as described in appendix 2.

[0204] This allows a user (for example, a doctor) to easily understand the relationship between the medical image processed by the processing device and the time lag that occurs during transmission and reception. [Explanation of symbols]

[0205] 1 Medical Support System 2 Processing equipment 3. Network 10 Endoscopic device 12. Doctor 14. Nurse 16 Endoscope 18 Display device 20 Light source device 22 Control device 24 Medical Support Equipment 26 Subject 28 Large intestine 30 light 32 Intestinal wall 34 Wagon 35 screens 35A 1st display area 35B 2nd display area 35C 3rd display area 39 Endoscopic video images 40 frames 42 Lesions 44 Supporting Information 46 Control section 48 Insertion section 50 Tip 50A Tip surface 52 Camera 54 Lighting Equipment 54A, 54B Lighting window 56 Treatment opening 58 Treatment tools 60 Treatment tool insertion port 62 Universal Code 64 Reception device 66,78 Computer 68,88 Bus 70,80 External I / F 72,82 processor 74,84 Memory 76,86 Storage 79 RTC 82A Communications Department 82B Control section 90 Medical Assistance Program 92 Transmission time 94 Recognition Model 95 Recognition Processing 96 Recognition results 98 Received Time 100 Lesion presence / absence information 102 Geometric property information 106 Lesion Location Map 108 Lesion characteristic information 110 Part information 112 Transmission Time 114 Reference Time 116 Processing result information 118 Part name information 120 Lesion Identification Information 122 Identification information 124 Size Information 126 Past information 130 Time Information 130A First Specific Information 130B Second specific information 132 Notification mark BB Bounding Box

Claims

1. A communication device; a processor; The communication device includes: Transmitting the first medical image to a processing device; receiving a processing result obtained by the processing device executing processing on the first medical image; The processor, A first control is performed in which the communication device outputs time information regarding a time required from when the communication device transmits the first medical image to the processing device until when the communication device receives the processing result, and / or a second control is performed in which an output mode of the processing result is changed depending on the time. Medical support equipment.

2. The processor outputting the time information includes the processor displaying the time information on a first screen. The medical support device according to claim 1 .

3. The first control includes control of displaying, on a second screen, the first medical image and / or a second medical image obtained later in time than the first medical image in a state that can be compared with the time information. The medical support device according to claim 2.

4. The time information includes first identification information capable of identifying the time itself. The medical support device according to claim 1 .

5. The time information includes second identification information capable of identifying a difference between the time and a first reference time. The medical support device according to claim 1 .

6. The second specification information includes length / shortage information capable of specifying whether the time is longer or shorter than the first reference time. The medical support device according to claim 5.

7. The time information is visualized information. The medical support device according to claim 1 .

8. The time information is visualized by text and / or marks. The medical support device according to claim 7.

9. The processor changes a visualization format, which is a format for visualizing the time information, according to the time. The medical support device according to claim 7.

10. The processor increases visibility of the time information when the time is equal to or greater than a second reference time, compared to visibility of the time information when the time is less than the second reference time. The medical support device according to claim 9.

11. The output mode is a display mode in which the processor displays the processing result on a third screen. The medical support device according to claim 1 .

12. The display mode includes a first display mode and a second display mode, the first display manner is used when the time is equal to or longer than a third reference time, and is capable of identifying that the time is equal to or longer than the third reference time; The second display manner is used when the time is less than the third reference time, and enables identification that the time is less than the third reference time. The medical support device according to claim 11.

13. The second display mode has higher visibility than the first display mode. The medical support device according to claim 12.

14. In the first display mode, the processing result is not displayed, In the second display mode, the processing result is displayed. The medical support device according to claim 13.

15. The second control includes a control of displaying, on a fourth screen, the first medical image and / or a third medical image obtained later in time than the first medical image in a state where the third medical image can be compared with the processing result. The medical support device according to claim 11.

16. The second control includes a control of displaying the first medical image on a fourth screen in a state in which the first medical image can be compared with the processing result when the time is less than a fourth reference time. The medical support device according to claim 11.

17. The fourth reference time is the longest time for which the degree of affinity between the first medical image and the processing result falls within an allowable range. The medical support device according to claim 16.

18. The processing is AI processing for the first medical image. The medical support device according to claim 1 .

19. The processing includes a recognition process for recognizing a characteristic area in the first medical image. The medical support device according to claim 1 .

20. The characteristic region is a lesion.

20. The medical support device according to claim 19.

21. The first medical image is an endoscopic image generated by capturing an image of the inside of a body using a camera mounted on an endoscope. The medical support device according to claim 1 .

22. A medical support device according to any one of claims 1 to 21; a camera that is inserted into a body and captures an image of the inside of the body to generate the first medical image. Endoscopic device.

23. A medical support device; A processing device, the medical support device transmits a first medical image to the processing device; The processing device includes: performing processing on the first medical image; A processing result obtained by executing the processing is transmitted to the medical support device; The medical support device includes: receiving the processing result transmitted from the processing device; A first control is performed to output time information regarding a time required from when the first medical image is transmitted to the processing device until the processing result is received by the medical support device, and / or a second control is performed to vary an output mode of the processing result depending on the time. Medical support system.

24. transmitting the first medical image to a processing device; receiving a processing result obtained by the processing device performing processing on the first medical image; and The method includes performing a first control for outputting time information regarding a time required from when the first medical image is transmitted to the processing device until the processing result is received, and / or performing a second control for changing an output mode of the processing result depending on the time. Medical assistance methods.

25. transmitting the first medical image to a processing device; receiving a processing result obtained by the processing device performing processing on the first medical image; and A program for causing a computer to execute medical support processing, including performing a first control to output time information regarding the time required from when the first medical image is transmitted to the processing device to when the processing result is received, and / or a second control to vary the output mode of the processing result depending on the time.

Images