Medical support device, medical support system, medical support method, and program

Abstract

To provide a medical support device, a medical support system, a medical support method, and a program which can suppress a failure caused when recognition processing is performed on each of a plurality of medical images corresponding to a plurality of medical examinations.SOLUTION: A medical support device includes a processor. The processor acquires a plurality of medical images corresponding to a plurality of medical examinations, and acquires a plurality of pieces of medical examination information, which correspond to the plurality of medical examinations and which are related to each of the corresponding medical examinations, so as to make a processing device perform recognition processing for each of the plurality of medical images in order determined based on the plurality of pieces of medical examination information.SELECTED DRAWING: Figure 11

Description

[Technical field]

[0001] The present disclosure relates to a medical support device, a medical support system, a medical support method, and a program. [Background technology]

[0002] Patent Document 1 discloses a control method for controlling processing of data acquired from a medical imaging modality by using a plurality of data processors connected to the plurality of medical imaging modalities via a computer network. The control method described in Patent Document 1 includes acquiring imaging information for imaging performed in one of the plurality of medical imaging modalities, acquiring load information of the plurality of data processors before the imaging is completed, allocating at least a part of the plurality of data processors to processing of data acquired in imaging based on the imaging information based on the acquired load information, and executing processing of the acquired data in the assigned data processor.

[0003] Patent Document 2 discloses an image database system including a storage unit for storing medical image data, a control unit for managing the storage unit, an image database server that stores attribute information including key information associated with the medical image data stored in the storage unit and relays the medical image data between externally connected devices, and a DICOM gateway that relays the medical image data between the image database server and multiple externally connected modality devices using the DICOM protocol, all of which are connected to one another via a switch. The image database described in Patent Document 2 includes at least a plurality of control units, image database servers, and DICOM gateways, forming a control unit group, an image database server group, and a DICOM gateway group, and is provided with a load balancer that controls the load distribution for each group based on the header information of a request. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-046376 A [Patent Document 2] International Publication No. 2005 / 003977 Summary of the Invention

[0005] One embodiment of the present disclosure provides a medical support device, a medical support system, a medical support method, and a program that can accurately suppress problems that occur when recognition processing is performed on each of multiple medical images corresponding to multiple medical tests. [Means for solving the problem]

[0006] A first aspect of the present disclosure is a medical support device including a processor that acquires a plurality of medical images corresponding to a plurality of medical tests, acquires a plurality of pieces of medical test information corresponding to the plurality of medical tests, each piece of medical test information being related to the corresponding medical tests, and causes a processing device to perform recognition processing on each of the plurality of medical images in an order determined based on the plurality of pieces of medical test information.

[0007] A second aspect of the present disclosure is the medical support device according to the first aspect, in which the medical test information includes test-in-progress information that is known within a period during which the corresponding medical test is performed.

[0008] A third aspect of the present disclosure is the medical support device according to the first or second aspect, in which the medical test information includes pre-test information that is known before the implementation period of the corresponding medical test.

[0009] A fourth aspect of the present disclosure is the medical support device according to any one of the first to third aspects, wherein the medical test information includes subject information related to a subject undergoing a corresponding medical test.

[0010] A fifth aspect of the present disclosure is a medical support device according to any one of the first to fourth aspects, in which the medical test information includes operator information related to an operator who performs the corresponding medical test.

[0011] A sixth aspect of the present disclosure is a medical support device according to any one of the first to fifth aspects, wherein the medical image is obtained in a corresponding medical examination, and the medical examination information includes modality information regarding the modality used for imaging to obtain the medical image in the corresponding medical examination.

[0012] A seventh aspect of the present disclosure is a medical support device according to any one of the first to sixth aspects, wherein the medical images are obtained in a corresponding medical test, the medical test information includes intermediate results of the corresponding medical test, and the intermediate results are information identified based on the medical images obtained in the corresponding medical test.

[0013] An eighth aspect of the present disclosure is the medical support device according to the seventh aspect, in which the intermediate result includes feature region information that is information on a first feature region appearing in the medical image.

[0014] A ninth aspect of the present disclosure is the medical support device according to the eighth aspect, in which the first feature region is a lesion, and the intermediate result is a screening result of the lesion or a differentiation result of the lesion.

[0015] A tenth aspect of the present disclosure is a medical support device according to any one of the first to ninth aspects, in which a processor transmits medical support information including an execution result obtained by executing a recognition process to an apparatus used in a medical examination corresponding to a medical image that was the subject of the recognition process executed to obtain the execution result included in the medical support information.

[0016] An eleventh aspect of the present disclosure is a medical support device according to the tenth aspect, in which the medical support information includes information associating a medical image that was the subject of recognition processing executed to obtain the execution result included in the medical support information with the execution result.

[0017] A twelfth aspect of the present disclosure is a medical support device according to the tenth or eleventh aspect, in which the processor transmits corresponding medical support information to the device in the order in which execution results are obtained by performing recognition processing on each of a plurality of medical images in sequence.

[0018] A thirteenth aspect of the present disclosure is a medical support device according to any one of the first to twelfth aspects, in which the recognition process is a process of generating information related to the second feature region by inputting a medical image into a trained model that generates information related to the first region by inputting an image containing a first region corresponding to the second feature region.

[0019] A fourteenth aspect of the present disclosure is a medical support device according to any one of the first to thirteenth aspects, wherein each of the plurality of medical tests is an endoscopic examination, and each of the plurality of medical images is an endoscopic image obtained in the endoscopic examination.

[0020] A fifteenth aspect of the present disclosure is a medical support system comprising a medical support device relating to any one of the first to fourteenth aspects, and a communication device that transmits medical images to the medical support device and receives execution results obtained by executing the recognition processing.

[0021] A sixteenth aspect of the present disclosure is a medical support method including acquiring a plurality of medical images corresponding to a plurality of medical tests, acquiring a plurality of pieces of medical test information corresponding to the plurality of medical tests, each piece of medical test information being related to the corresponding medical tests, and having a processing device perform recognition processing on each of the plurality of medical images in an order determined based on the plurality of pieces of medical test information.

[0022] A seventeenth aspect of the present disclosure is a program for causing a computer to execute a process including acquiring a plurality of medical images corresponding to a plurality of medical tests, acquiring a plurality of pieces of medical test information corresponding to the plurality of medical tests, each piece of medical test information being related to the corresponding medical tests, and causing a processing device to execute recognition processing for each of the plurality of medical images in an order determined based on the plurality of pieces of medical test information. [Brief description of the drawings]

[0023] [Figure 1] 1 is a schematic diagram showing an example of the overall configuration of a medical support system; [Diagram 2] 1 is a conceptual diagram showing an example of an overall configuration of a medical support system. [Diagram 3] 2 is a block diagram showing an example of a hardware configuration of an electrical system of the medical support system. FIG. [Figure 4] 2 is a block diagram showing an example of main functions of a processor included in the server, and an example of information stored in a storage included in the server. FIG. [Diagram 5] 4 is a conceptual diagram showing an example of processing contents of an endoscope device and a server. FIG. [Figure 6] 4A to 4C are conceptual diagrams showing an example of processing contents of a server and an example of contents displayed on a screen of a display device included in an endoscope device. [Figure 7] 10 is a conceptual diagram showing an example of processing content executed by a processor included in an endoscope device when the endoscope device acquires server processing information from a server. [Figure 8]A conceptual diagram showing an example of the processing content for determining the order in which a plurality of medical images (in the example shown in FIG. 8, the medical images in the processing target information obtained from the processor of the endoscope device used in the first endoscope examination and the medical images in the processing target information obtained from the processor of the endoscope device used in the second endoscope examination) are used for recognition processing based on the processing request information in the metadata included in each of the plurality of processing target information (in the example shown in FIG. 8, the processing target information obtained from the processor of the endoscope device used in the first endoscope examination and the processing target information obtained from the processor of the endoscope device used in the second endoscope examination) corresponding to the plurality of endoscope devices. [Figure 9] A conceptual diagram showing an example of the processing content of the control unit and the recognition unit of the server. [Figure 10] A conceptual diagram showing an example of the mode in which server processing information including information in which a medical image and processing identification information are associated is transmitted from the server to the endoscope device. [Figure 11] A flowchart showing an example of the flow of medical support processing executed by a processor included in the server. [Figure 12] A conceptual diagram showing an example of the processing content for determining the order in which a plurality of medical images (in the example shown in FIG. 12, the medical images in the processing target information obtained from the processor of the endoscope device used in the first endoscope examination and the medical images in the processing target information obtained from the processor of the endoscope device used in the second endoscope examination) are used for recognition processing based on the subject information in the metadata included in each of the plurality of processing target information (in the example shown in FIG. 12, the processing target information obtained from the processor of the endoscope device used in the first endoscope examination and the processing target information obtained from the processor of the endoscope device used in the second endoscope examination) corresponding to the plurality of endoscope devices. [Figure 13]This is a conceptual diagram showing an example of processing content that determines the order in which multiple medical images (in the example shown in Figure 13, the medical images in the processing target information obtained from the processor of the endoscopic device used in the first endoscopic examination and the medical images in the processing target information obtained from the processor of the endoscopic device used in the second endoscopic examination) are used for recognition processing based on operator information in metadata contained in each of multiple processing target information corresponding to multiple endoscopic devices (in the example shown in Figure 13, the processing target information obtained from the processor of the endoscopic device used in the first endoscopic examination and the processing target information obtained from the processor of the endoscopic device used in the second endoscopic examination). [Figure 14] This is a conceptual diagram showing an example of processing content that determines the order in which multiple medical images (in the example shown in Figure 14, the medical images in the processing target information obtained from the processor of the endoscopic device used in the first endoscopic examination, and the medical images in the processing target information obtained from the processor of the endoscopic device used in the second endoscopic examination) are used for recognition processing based on modality information in metadata contained in each of multiple processing target information corresponding to multiple endoscopic devices (in the example shown in Figure 14, the processing target information obtained from the processor of the endoscopic device used in the first endoscopic examination, and the processing target information obtained from the processor of the endoscopic device used in the second endoscopic examination). [Figure 15] This is a conceptual diagram showing an example of processing content that determines the order in which multiple medical images (in the example shown in Figure 15, the medical images in the processing target information obtained from the processor of the endoscopic device used in the first endoscopic examination, and the medical images in the processing target information obtained from the processor of the endoscopic device used in the second endoscopic examination) are used for recognition processing based on the recognition results in the metadata contained in each of multiple processing target information corresponding to multiple endoscopic devices (in the example shown in Figure 15, the processing target information obtained from the processor of the endoscopic device used in the first endoscopic examination, and the processing target information obtained from the processor of the endoscopic device used in the second endoscopic examination). [Figure 16]11 is a conceptual diagram showing an example of a mode in which server processing information including information in which a medical image is associated with a bounding box as one of the recognition results is transmitted from a server to an endoscope device. FIG. [Figure 17] FIG. 13 is a schematic diagram showing a modified example of the overall configuration of the medical support system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, an example of an embodiment of a medical support device, a medical support system, a medical support method, and a program according to the present disclosure will be described with reference to the attached drawings.

[0025] First, the terms used in the following description will be explained.

[0026] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". GPGPU is an abbreviation for "General-Purpose computing on Graphics Processing Units". APU is an abbreviation for "Accelerated Processing Unit". TPU is an abbreviation for "Tensor Processing Unit". RAM is an abbreviation for "Random Access Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EL is an abbreviation for "Electro-Luminescence". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". AI is an abbreviation for "Artificial Intelligence". BLI is an abbreviation for "Blue Light Imaging". LCI is an abbreviation for "Linked Color Imaging". WLI is an abbreviation for "White Light Image". I / F is an abbreviation for "Interface". SSL is an abbreviation for "Sessile Serrated Lesion". LAN is an abbreviation for "Local Area Network". WAN is an abbreviation for "Wide Area Network". 5G is an abbreviation for "5th Generation Mobile Communication System".IC is an abbreviation for "Integrated Circuit." MRI is an abbreviation for "Magnetic Resonance Imaging." CT is an abbreviation for "Computed Tomography."

[0027] In the following description, a coded processor (hereinafter simply referred to as a "processor") may be one physical or virtual arithmetic device, or a combination of multiple physical or virtual arithmetic devices. Also, a processor may be one type of arithmetic device, or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU, a GPU, a GPGPU, an APU, and a TPU.

[0028] In the following description, a signed memory is a memory, such as a RAM, in which information is temporarily stored and which is used as a working memory by a processor.

[0029] In the following description, a storage with a symbol is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory, magnetic disks, and magnetic tapes. Another example of storage is cloud storage.

[0030] In the following embodiments, the external I / F with a symbol controls the transmission and reception of various information between multiple devices connected to each other. An example of the external I / F is a USB interface. A communication I / F including a communication processor and an antenna may be applied to the external I / F. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G, Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0032] FIG. 1 is a schematic diagram showing an example of the overall configuration of a medical support system 1. As shown in FIG. 1, the medical support system 1 includes a server 2 and a plurality of endoscopic devices 10, which are communicatively connected via a network 3. An example of the network 3 is the Internet. However, the Internet is merely an example, and other examples of the network 3 include a WAN and / or a LAN. In this embodiment, the medical support system 1 is an example of a "medical support system" according to the present disclosure, the server 2 is an example of a "processing device" and a "medical support device" according to the present disclosure, and the endoscopic device 10 is an example of an "device," "modality," and "communication device" according to the present disclosure.

[0033] A plurality of endoscopic devices 10 are used by a plurality of doctors 12 in a plurality of endoscopic examinations 5 performed in a plurality of endoscopic examination rooms 4. The endoscopic examinations 5 are assisted by staff such as a nurse 14. In the example shown in FIG. 1, a first endoscopic examination room 4A and a second endoscopic examination room 4B are shown as an example of a plurality of endoscopic examinations 4. In addition, in the example shown in FIG. 1, a first endoscopic examination 5A and a second endoscopic examination 5B are shown as an example of a plurality of endoscopic examinations 5. In this embodiment, the endoscopic device 10 is used by a different doctor 12 in each of the first endoscopic examination 5A and the second endoscopic examination 5B. Here, the doctor 12 is an example of a "technician" according to the present disclosure. The first endoscopic examination 5A and the second endoscopic examination 5B are examples of "multiple medical examinations" according to the present disclosure. Also, each of the first endoscopic examination 5A and the second endoscopic examination 5B is an example of an "endoscopic examination" according to the present disclosure.

[0034] In this embodiment, for the sake of easy understanding of the present disclosure, the first endoscopy examination room 4A and the second endoscopy examination room 4B are cited as a plurality of endoscopy examination rooms 4, and the first endoscopy examination 5A and the second endoscopy examination 5B are cited as a plurality of endoscopy examinations 5. However, this is merely an example, and the present disclosure is still valid even if each of the endoscopy examination room 4 and the endoscopy examination 5 has three or more.

[0035] The information obtained by the endoscope device 10 is transmitted to the server 2 via the network 3. An example of the server 2 is a cloud server. However, the cloud server is merely an example, and the server 2 may be an on-premises server or a personal computer. The server 2 receives the information transmitted from the endoscope device 10, executes processing using the received information, and transmits the processing result obtained by executing the processing to the endoscope device 10.

[0036] The endoscope device 10 includes an endoscope scope 16, a display device 18, a light source device 20, a control device 22, and a function expansion device 24.

[0037] The endoscope device 10 is a modality applied to the subject 26 who undergoes the endoscopy examination 5. That is, the endoscope device 10 is a modality for performing medical treatment on the large intestine 28 of the subject 26 (for example, a patient) using the endoscope scope 16. In this embodiment, the large intestine 28 is the object to be observed by the doctor 12. The subject 26 is an example of the "subject" according to the present disclosure.

[0038] The endoscope 16 is used by the doctor 12 and inserted into the body of the subject 26. In this embodiment, the endoscope 16 is inserted into the large intestine 28, which is a tubular organ of the subject 26. The endoscope device 10 causes the endoscope 16 inserted into the large intestine 28 of the subject 26 to capture images of the inside of the large intestine 28 of the subject 26, and performs various medical procedures on the large intestine 28 as necessary. The endoscope device 10 captures images of the inside of the large intestine 28 of the subject 26, and outputs the images. In this embodiment, the endoscope device 10 is an endoscope having an optical imaging function that captures reflected light obtained by irradiating light 30 inside the large intestine 28 and reflecting it off a wall 32 of the large intestine 28.

[0039] Although an endoscopic examination of the large intestine 28 is illustrated here, this is merely one example, and the present disclosure also applies to an endoscopic examination of a hollow organ such as the upper digestive tract (e.g., the esophagus, stomach, and duodenum) or the trachea.

[0040] The light source device 20, the control device 22, and the function expansion device 24 are installed on a wagon 34. The wagon 34 has a plurality of platforms arranged vertically, and the function expansion device 24, the control device 22, and the light source device 20 are installed from the lower platform to the upper platform. In addition, the display device 18 is installed on the top platform of the wagon 34.

[0041] The control device 22 controls the entire endoscope device 10. The function expansion device 24 is a device that expands the functions of the endoscope device 10, and is attached to the control device 22 later. The function expansion device 24 operates according to instructions from the control device 22. The function expansion device 24 is communicatively connected to the server 2 via the network 3, and receives the requested service from the server 2 by requesting the server 2 to provide the service. The function expansion device 24 performs various processes according to instructions from the control device 22 on information obtained from the control device 22 and information obtained from the server 2 (e.g., an image obtained by imaging the intestinal wall 32 with the endoscope scope 16, and the processing results by the server 2, etc.).

[0042] The display device 18 displays various information including images. Examples of the display device 18 include a liquid crystal display and an EL display. Also, instead of the display device 18 or together with the display device 18, a tablet terminal with a display may be used.

[0043] The display device 18 displays a screen 35. The screen 35 includes a plurality of display areas. The plurality of display areas are arranged side by side within the screen 35. In the example shown in FIG. 1, a first display area 35A and a second display area 35B are shown as an example of the plurality of display areas. The display device 18 includes a plurality of display areas. In the example shown in FIG. 1, a first display area 35A and a second display area 35B are shown as an example of the plurality of display areas. The size of the first display area 35A is larger than the size of the second display area 35B. The first display area 35A is used as a main display area, and the second display area 35B is used as a sub display area. The size relationship between the first display area 35A and the second display area 35B is not limited to this, and may be any size relationship that fits within the screen 35.

[0044] A medical image 40 is displayed in the first display area 35A. The medical image 40 is an endoscopic image obtained in the endoscopic examination 5, i.e., an endoscopic image acquired by imaging an intestinal wall 32 in the large intestine 28 of the subject 26 by the endoscope scope 16 during the endoscopic examination 5. In the example shown in FIG. 1, an image showing the intestinal wall 32 is shown as an example of the medical image 40. The medical image 40 is displayed as a moving image or a still image in the first display area 35A. Here, the medical image 40 is an example of a "medical image" and an "endoscopic image" according to the present disclosure.

[0045] The intestinal wall 32 shown in the medical image 40 includes a lesion 42 (e.g., one lesion 42 in the example shown in FIG. 1) as a region of interest (i.e., a region to be observed) gazed upon by the physician 12, and the physician 12 can visually recognize the appearance of the intestinal wall 32 including the lesion 42 through the medical image 40. In this embodiment, the lesion 42 is an example of a "first characteristic region," a "second characteristic region," and a "lesion" according to the present disclosure.

[0046] There are various types of lesions 42, and examples of the types of lesions 42 include neoplastic polyps and non-neoplastic polyps. Examples of the types of neoplastic polyps include adenomatous polyps (e.g., SSL). Examples of the types of non-neoplastic polyps include hamartomatous polyps, hyperplastic polyps, and inflammatory polyps. Note that the types exemplified here are types that are assumed in advance as types of lesions 42 when an endoscopic examination is performed on the large intestine 28, and the types of lesions 42 will differ if the organ in which the endoscopic examination is performed is different.

[0047] In this embodiment, for ease of explanation, an example is given in which a single lesion 42 is shown in the medical image 40, but the present disclosure is not limited to this, and the present disclosure also applies when multiple lesions 42 are shown in the medical image 40.

[0048] In this embodiment, a lesion 42 is illustrated, however, this is merely one example, and the area of ​​interest (i.e., the area to be observed) gazed upon by the doctor 12 may be a characteristic area having some unique feature, such as an organ (e.g., the duodenal papilla), a mark, an artificial treatment device (e.g., an artificial clip), or a treated area (e.g., an area where traces remain after the removal of a polyp, etc.), etc.

[0049] The image displayed in the first display area 35A is one medical image 40 included in a moving image that includes a plurality of chronological medical images 40. In other words, the first display area 35A displays a plurality of chronological medical images 40 at a default frame rate (e.g., several tens of frames per second).

[0050] An example of a moving image displayed in the first display area 35A is a moving image in a live view format. The live view format is merely one example, and a moving image that is temporarily stored in a memory or the like and then displayed, such as a moving image in a post-view format, may also be used. Furthermore, each frame included in a moving image for recording stored in a memory or the like may be reproduced and displayed on the screen 35 (for example, the first display area 35A) as a medical image 40.

[0051] Within the screen 35, the second display area 35B is adjacent to the first display area 35A, and is displayed in the lower right corner as viewed from the front within the screen 35. The display position of the second display area 35B may be anywhere within the screen 35 of the display device 18, but it is preferable that it be displayed in a position where it can be contrasted with the medical image 40.

[0052] In the second display area 35B, auxiliary information 44 that assists the doctor 12 in making medical decisions during an endoscopic examination is displayed. The auxiliary information 44 is information that is referred to by the doctor 12. Examples of the auxiliary information 44 include various types of information related to the subject 26 into whose body the endoscope 16 is inserted, and / or various types of information obtained by performing medical support processing, which will be described later.

[0053] Fig. 2 is a conceptual diagram showing an example of the overall configuration of the endoscope device 10. As shown in Fig. 2, the endoscope scope 16 includes an operation section 46 and an insertion section 48. The insertion section 48 is partially curved by operating the operation section 46. The insertion section 48 is inserted into the large intestine 28 (see Fig. 1) while curving in accordance with the shape of the large intestine 28 (see Fig. 1) in accordance with the operation of the operation section 46 by the doctor 12 (see Fig. 1).

[0054] The distal end 50 of the insertion section 48 is provided with a camera 52, an illumination device 54, and a treatment tool opening 56. The camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end 50. Note that, although an example in which the camera 52 and the illumination device 54 are provided on the distal end surface 50A of the distal end 50 is given here, this is merely one example, and the camera 52 and the illumination device 54 may be provided on the side surface of the distal end 50, so that the endoscope 16 is configured as a side-viewing scope.

[0055] The camera 52 is mounted on the endoscope 16, and is inserted into the body cavity of the subject 26 to capture an image of an observation target region, thereby generating a medical image 40 as an endoscopic image. In this embodiment, the camera 52 captures images of the inside of the subject 26 (e.g., inside the large intestine 28) at a preset frame rate, thereby generating a moving image including a plurality of medical images 40 in a time series. An example of the camera 52 is a CMOS camera. However, this is merely an example, and other types of cameras such as a CCD camera may also be used.

[0056] The illumination device 54 has illumination windows 54A and 54B. The illumination device 54 irradiates light 30 (see FIG. 1) through the illumination windows 54A and 54B. Examples of the light 30 irradiated from the illumination device 54 include visible light (e.g., white light) and non-visible light (e.g., near-infrared light). The illumination device 54 also irradiates special light through the illumination windows 54A and 54B. Examples of the special light include light for BLI and / or light for LCI. The camera 52 captures an image of the inside of the large intestine 28 by an optical method while the light 30 is irradiated inside the large intestine 28 by the illumination device 54.

[0057] The treatment tool opening 56 is an opening for allowing a treatment tool 58 to protrude from the distal end portion 50. The treatment tool opening 56 is also used as a suction port for sucking blood, internal waste, and the like, and as a delivery port for delivering fluid.

[0058] A treatment tool insertion port 60 is formed in the operation section 46, and the treatment tool 58 is inserted into the insertion section 48 from the treatment tool insertion port 60. The treatment tool 58 passes through the insertion section 48 and protrudes to the outside from the treatment tool opening 56. In the example shown in FIG. 2, a puncture needle is shown as the treatment tool 58 protruding from the treatment tool opening 56. Here, a puncture needle is shown as the treatment tool 58, but this is merely one example, and the treatment tool 58 may be a grasping forceps, a papillotomy knife, a snare, a catheter, a guidewire, a cannula, and / or a puncture needle with a guide sheath, etc.

[0059] The endoscope 16 is connected to a light source device 20 and a control device 22 via a universal cord 62. A function expansion device 24 and a reception device 64 are connected to the control device 22. In addition to the server 2, the display device 18 is also connected to the function expansion device 24. That is, the control device 22 is connected to the server 2 and the display device 18 via the function expansion device 24.

[0060] Here, the function expansion device 24 is exemplified as an external device for expanding the functions performed by the control device 22, and therefore an example is given in which the control device 22 and the display device 18 are indirectly connected via the function expansion device 24, but this is merely one example. For example, the display device 18 may be directly connected to the control device 22. In this case, it is sufficient that the functions of the function expansion device 24 are installed in the control device 22, for example.

[0061] The reception device 64 receives instructions from the doctor 12 and outputs the received instructions as an electrical signal to the control device 22. Examples of the reception device 64 include a keyboard, a mouse, a touch panel, a foot switch, a microphone, and / or a remote control device.

[0062] The control device 22 controls the light source device 20, exchanges various signals with the camera 52, and exchanges various signals with the function expansion device 24.

[0063] The light source device 20 emits light under the control of the control device 22, and supplies the light to the illumination device 54. A light guide is built into the illumination device 54, and the light supplied from the light source device 20 passes through the light guide and is irradiated from illumination windows 54A and 54B. The light irradiated from the illumination windows 54A and 54B is light 30 (see FIG. 1). The control device 22 causes the camera 52 to capture an image, acquires a medical image 40 (see FIG. 1) from the camera 52, and outputs it to a predetermined output destination (for example, the function expansion device 24).

[0064] The function expansion device 24 supports medical care (here, as an example, endoscopic examination) by performing various processes on the medical image 40 input from the control device 22. The function expansion device 24 outputs the medical image 40 that has been subjected to various processes to a predetermined output destination (for example, the display device 18).

[0065] Although the embodiment in which the medical image 40 output from the control device 22 is output to the display device 18 via the function expansion device 24 has been described above, this is merely one example. For example, the control device 22 and the display device 18 may be connected, and the medical image 40 that has been subjected to image processing by the function expansion device 24 may be displayed on the display device 18 via the control device 22.

[0066] Fig. 3 is a block diagram showing an example of a hardware configuration of an electrical system of the endoscope device 10. As shown in Fig. 3, the control device 22 includes a computer 66 and an external I / F 70. The computer 66 includes a processor 72, a memory 74, and a storage 76. The processor 72, the memory 74, the storage 76, and the external I / F 70 are connected to a bus 68. The processor 72 controls the entire control device 22. The memory 74 and the storage 76 are used by the processor 72.

[0067] The external I / F 70 is responsible for transmitting and receiving various types of information between the processor 72 and one or more devices (hereinafter, also referred to as “first external devices”) that exist outside the control device 22.

[0068] The camera 52 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 is responsible for transmitting various types of information between the camera 52 and the processor 72. The processor 72 controls the camera 52 via the external I / F 70. The processor 72 also acquires, via the external I / F 70, medical images 40 (see FIG. 1) obtained by the camera 52 capturing an image of the inside of the large intestine 28 (see FIG. 1).

[0069] The light source device 20 is connected to the external I / F 70 as one of the first external devices, and the external I / F 70 controls the exchange of various information between the light source device 20 and the processor 72. The light source device 20 supplies light to the illumination device 54 under the control of the processor 72. The illumination device 54 irradiates the light supplied from the light source device 20.

[0070] A reception device 64 is connected to the external I / F 70 as one of the first external devices, and the processor 72 acquires instructions accepted by the reception device 64 via the external I / F 70 and executes processing according to the acquired instructions.

[0071] The function expansion device 24 includes a computer 78 and an external I / F 80. The computer 78 includes a processor 82, a memory 84, and a storage 86. The processor 82, the memory 84, the storage 86, and the external I / F 80 are connected to a bus 88. Note that the hardware configuration of the computer 78 (i.e., the processor 82, the memory 84, and the storage 86) is basically the same as the hardware configuration of the computer 66, so a description of the hardware configuration of the computer 78 will be omitted here.

[0072] The external I / F 80 controls the exchange of various information between the processor 82 and one or more devices (hereinafter, also referred to as “second external devices”) existing outside the function extension device 24.

[0073] The control device 22 is connected to the external I / F 80 as one of the second external devices. In the example shown in Fig. 3, the external I / F 70 of the control device 22 is connected to the external I / F 80. The external I / F 80 controls the exchange of various information between the processor 82 of the function expansion device 24 and the processor 72 of the control device 22. For example, the processor 82 acquires a medical image 40 (see Fig. 1) from the processor 72 of the control device 22 via the external I / Fs 70 and 80, and performs various processes on the acquired medical image 40.

[0074] The display device 18, which serves as one of the second external devices, is connected to the external I / F 80. The processor 82 controls the display device 18 via the external I / F 80 to cause the display device 18 to display various information (e.g., medical images 40, etc.).

[0075] The external I / F 80 is connected to the server 2 via the network 3 as one of the second external devices. The processor 82 exchanges various information with the server 2 via the external I / F 80. For example, the external I / F 80 transmits medical images 40 acquired by the processor 82 from the camera 52 to the server 2. The server 2 receives the medical images 40 transmitted from the external I / F 80, executes processing using at least one medical image 40 included in the received medical images 40, and transmits the processing result to the function extension device 24. The external I / F 80 receives the processing result transmitted from the server 2. The processor 82 acquires the processing result received by the external I / F 80.

[0076] FIG. 4 is a block diagram showing an example of the hardware configuration of the electrical system of the server 2. As shown in FIG. 4, the server 2 includes a computer 96 and an external I / F 98. The computer 96 includes a processor 100, a memory 102, and a storage 104. The processor 100, the memory 102, the storage 104, and the external I / F 98 are connected to a bus 106. Here, the computer 96 is an example of a "computer" according to the present disclosure, and the processor 100 is an example of a "processor" according to the present disclosure. Note that the hardware configuration of the computer 96 (i.e., the processor 100, the memory 102, and the storage 104) is basically the same as the hardware configuration of the computer 66, so a description of the hardware configuration of the computer 96 will be omitted here.

[0077] The external I / F 98 is responsible for transmitting various types of information between the processor 100 and one or more devices (hereinafter also referred to as "third external devices") existing outside the server 2. A function expansion device 24 is connected to the external I / F 98 via the network 3 as one of the third external devices. The processor 100 transmits and receives various types of information to and from the function expansion device 24 via the external I / F 98. For example, the external I / F 98 receives various types of information such as medical images 40 transmitted from the function expansion device 24. The processor 100 executes processing using the information received by the external I / F 98, and transmits the processing results to the function expansion device 24.

[0078] In recent years, there has been progress in the development of a technology in which a trained model optimized by performing machine learning on a model (e.g., a neural network) recognizes a lesion 42 by performing object recognition processing on a medical image 40, and the recognition result of the object recognition processing or information based on the recognition result of the object recognition processing is displayed on a screen 35.

[0079] The object recognition process performed on the medical image 40 imposes a very large processing load. Therefore, currently, a technology is being considered for reducing the processing load and traffic that occurs when the endoscope device 10 causes the server 2, which is communicably connected to the endoscope device 10 via the network 3, to execute the object recognition process.

[0080] However, when multiple endoscope devices 10 share the server 2, it is assumed that the server 2 performs object recognition processing simultaneously on multiple medical images 40 transmitted from the multiple endoscope devices 10. When multiple endoscope devices 10 access the server 2 in a concentrated manner, the processing load and traffic on the server 2 become very large, and as a result, the time lag between when each of the multiple endoscope devices 10 requests the server 2 to perform object recognition processing and when the server 2 receives the processing result (i.e., the processing result of the object recognition processing) becomes long. This causes, for example, a discrepancy between the content of the medical image 40 displayed on the screen 35 of each endoscope device 10 and the content of the processing result (for example, the content of the processing result unrelated to the content of the medical image 40 is displayed on the screen 35). In addition, there is a risk that the server 2 will go down due to an overload on the server 2 caused by the simultaneous object recognition processing on multiple medical images 40 transmitted from the multiple endoscope devices 10. On the other hand, it is necessary for the doctor 12 performing the endoscopic examination 5 with high urgency to understand the processing result by the server 2 before the doctor 12 performing the endoscopic examination 5 with low urgency.

[0081] In order to solve such problems, in this embodiment, as an example shown in FIG. 4, a medical support process is performed by the processor 100 of the server 2. A medical support program 108, a first recognition model 110A, and a second recognition model 110B are stored in the storage 104. The medical support program 108 is an example of a "program" according to the present disclosure. The first recognition model 110A and the second recognition model 110B are an example of a "trained model" according to the present disclosure.

[0082] The processor 100 performs medical support processing by reading a medical support program 108 from the storage 104 and executing the read medical support program 108 on the memory 102. The medical support processing is realized by the processor 100 operating as a control unit 100A and a recognition unit 100B in accordance with the medical support program 108 executed on the memory 102. As will be described in detail later, a first recognition model 110A and a second recognition model 110B are used by the recognition unit 100B.

[0083] Fig. 5 is a conceptual diagram showing an example of the processing contents performed by the server 2 and the endoscope device 10. As shown in Fig. 5, the server 2 communicates with the endoscope device 10. The endoscope device 10 transmits processing target information 111 corresponding to an endoscopic examination 5 to the server 2. The processing target information 111 is an example of "medical examination information" according to the present disclosure.

[0084] The processing target information 111 is information to be processed by the server 2, and is information related to the corresponding endoscopic examination 5. In the server 2, the control unit 100A acquires the processing target information 111 transmitted from the endoscope device 10 via the external I / F 98 (see FIG. 4). The processing target information 111 includes a medical image 40 and metadata 112. The metadata 112 is data related to the medical image 40, and is associated with the medical image 40.

[0085] In the server 2, the control unit 100A outputs the medical image 40 to the recognition unit 100B. The recognition unit 100B recognizes a characteristic area (e.g., a lesion 42 appearing in the medical image 40) in the medical image 40 based on the medical image 40 input from the control unit 100A. To achieve this, the recognition unit 100B executes a recognition process 116 on the medical image 40 input from the control unit 100A. The recognition process 116 is an example of the "recognition process" according to the present disclosure.

[0086] The recognition process 116 is a process using AI (hereinafter, also referred to as "AI process"). The recognition process 116 is performed by the server 2 on the acquired medical image 40 every time the medical image 40 is acquired. In the example shown in FIG. 5, a first recognition process 116A and a second recognition process 116B are shown as examples of the recognition process 116. The control unit 100A causes the recognition unit 100B to selectively execute the first recognition process 116A and the second recognition process 116B. The first recognition process 116A is a process for recognizing a lesion 42 appearing in the medical image 40 (so-called screening), and the second recognition process 116B is a process for distinguishing the lesion 42 appearing in the medical image 40 (i.e., a process for identifying the specific type and / or model of the lesion 42).

[0087] The metadata 112 associated with the medical image 40 includes multiple types of information such as device identification information 114 and processing request information 115. The device identification information 114 is information capable of identifying the endoscope device 10. Examples of the device identification information 114 include an address capable of identifying the location of the endoscope device 10 on the network 3, and the model number of the endoscope device 10. For example, the device identification information 114 is referenced by the server 2 when the server 2 performs unique processing for each of the multiple endoscope devices 10 (for example, when the server 2 performs a recognition processing 116 in response to a request from the endoscope device 10 and transmits the processing result of the recognition processing 116 to the endoscope device 10 that made the request).

[0088] The processing request information 115 is information that requests the recognition unit 100B to execute the recognition process 116. The information that requests the recognition unit 100B to execute the recognition process 116 refers to, for example, information that requests the recognition unit 100B to execute either the first recognition process 116A or the second recognition process 116B. The control unit 100A causes the recognition unit 100B to selectively execute the first recognition process 116A or the second recognition process 116B according to the processing request information 115 in the metadata 112 associated with the medical image 40 that is the processing target.

[0089] In the first recognition process 116A, the first recognition model 110A is used. The first recognition model 110A is a trained model for object recognition by an AI bounding box method. The first recognition model 110A is optimized by performing machine learning using the first teacher data on the neural network. In other words, the first recognition model 110A is a trained model optimized so that the first correct answer data is generated by inputting the first example data.

[0090] The first teacher data is a data set including a plurality of data (i.e., a plurality of frames of data) in which the first example data and the first correct answer data are associated with each other. The first example data is an image corresponding to the medical image 40 (in other words, an image assuming the medical image 40). A first example of an image corresponding to the medical image 40 is an image obtained by actually capturing an image of the inside of the large intestine with a camera. A second example of an image assuming the medical image 40 is an image created virtually (e.g., an image generated by a generation AI). The first correct answer data is correct answer data (i.e., annotation) for the first example data. Here, an annotation that identifies the position of a lesion in the image used as the first example data is used as an example of the first correct answer data. Here, the lesion in the image used as the first example data is an example of the "first region" according to the present disclosure, the first example data is an example of the "image in which the first region is captured" according to the present disclosure, and the first correct answer data is an example of the "information related to the first region" according to the present disclosure.

[0091] When the recognition unit 100B executes the first recognition process 116A on the medical image 40 in accordance with an instruction from the control unit 100A, the recognition unit 100B inputs the medical image 40 input from the control unit 100A to the first recognition model 110A. As a result, the first recognition model 110A recognizes the position of the lesion 42 shown in the input medical image 40 within the medical image 40, and generates and outputs the first recognition result 122A, which is the result of the recognition.

[0092] In the second recognition process 116B, the second recognition model 110B is used. The second recognition model 110B is a trained model for object recognition by an AI bounding box method. The second recognition model 110B is optimized by performing machine learning using the second teacher data on the neural network. In other words, the second recognition model 110B is a trained model optimized so that the second correct answer data is generated by inputting the second example data.

[0093] The second teacher data is a data set including a plurality of data (i.e., a plurality of frames of data) in which the second example data and the second correct answer data are associated with each other. A first example of the second example data is the same data as the first example data (i.e., an image corresponding to the medical image 40). A second example of the second example data is an image in which an area showing a lesion is cut out from an image corresponding to the medical image 40 by a predetermined frame (e.g., a circumscribing rectangular frame, etc.). A third example of the second example data is an image showing the lesion itself corresponding to the lesion 42. The second correct answer data is correct answer data (i.e., annotation) for the second example data. Here, as an example of the second correct answer data, annotations are used that specify the geometric characteristics (e.g., position, size, shape, etc.) of the lesion shown in the image used as the second example data, the type of the lesion 42, and the type of the lesion 42 (e.g., pedunculated, subpedunculated, sessile, surface elevated, surface flat, surface depressed, etc.).

[0094] When the recognition unit 100B executes the second recognition process 116B on the medical image 40 in accordance with an instruction from the control unit 100A, the recognition unit 100B inputs the medical image 40 input from the control unit 100A to the second recognition model 110B. As a result, the second recognition model 110B recognizes the geometric characteristics of the lesion 42, the type of the lesion 42, the form of the lesion 42, and the like, which are shown in the input medical image 40, and generates and outputs the second recognition result 122B, which is the recognition result. In the following, when there is no need to distinguish between the first recognition result 122A and the second recognition result 122B, they will be referred to as the "recognition result 122". In this embodiment, the recognition result 122 is an example of the "execution result" and "information related to the second feature region" according to the present disclosure.

[0095] Fig. 6 is a conceptual diagram showing an example of the processing contents performed by the server 2 and the multiple endoscope devices 10. As shown in Fig. 6, the control unit 100A acquires the recognition result 122 from the recognition unit 100B. Then, the control unit 100A generates server processing information 124. The server processing information 124 is an example of "medical support information" according to the present disclosure.

[0096] The server processing information 124 includes the recognition result 122 and the processing specification information 126 acquired from the recognition unit 100B, and the recognition result 122 and the processing specification information 126 are associated with each other.

[0097] The process specification information 126 is information capable of specifying the type of the recognition process 116 corresponding to the recognition result 122, that is, information capable of specifying the type of the recognition process 116 executed by the recognition unit 100B to obtain the recognition result 122 (that is, the first recognition process 116A or the second recognition process 116B). When the recognition result 122 acquired by the control unit 100A from the recognition unit 100B is the first recognition result 122A, information capable of specifying that the recognition process 116 performed by the recognition unit 100B is the first recognition process 116A is generated by the control unit 100A as the process specification information 126. When the recognition result 122 acquired by the control unit 100A from the recognition unit 100B is the second recognition result 122B, information capable of specifying that the recognition process 116 performed by the recognition unit 100B is the second recognition process 116B is generated by the control unit 100A as the process specification information 126.

[0098] The control unit 100A refers to the processing request information 115 (see FIG. 5) and transmits server processing information 124 via the external I / F 98 (see FIG. 4) to the endoscope device 10, which is the request source that requested the server 2 to execute the recognition processing 116. In the endoscope device 10, the processor 82 acquires the server processing information 124 transmitted from the server 2 via the external I / F 80 (see FIG. 3).

[0099] The processor 82 specifies whether the recognition process 116 performed by the recognition unit 100B is the first recognition process 116A or the second recognition process 116B from the process specification information 126 included in the server process information 124 acquired via the external I / F 80 (see FIG. 3). Then, when the recognition process 116 performed by the recognition unit 100B is the first recognition process 116A, the processor 82 displays information based on the first recognition result 122A included in the server process information 124 on the screen 35. For example, the processor 82 superimposes and displays a bounding box BB based on the first recognition result 122A on the medical image 40 displayed in the first display area 35A. The bounding box BB is a rectangular frame-shaped mark that can specify the position of the lesion 42 in the medical image 40. The processor 82 also displays text 44A in the second display area 35B. An example of the text 44A is text that can identify that a lesion 42 is present in the medical image 40 displayed in the first display region 35A.

[0100] Furthermore, when the recognition process 116 performed by the recognition unit 100B is the second recognition process 116B, the processor 82 displays information based on the second recognition result 122B included in the server processing information 124 on the screen 35. For example, the processor 82 superimposes and displays a bounding box BB based on the second recognition result 122B (in the example shown in FIG. 6, a bounding box BB capable of identifying the position in the medical image 40 of the lesion 42 that is the processing target of the second recognition process 116B) on the medical image 40 displayed in the first display area 35A. Furthermore, the processor 82 displays texts 44B, 44C, and 44D in the second display area 35B.

[0101] An example of the text 44B is text capable of identifying the type of lesion 42 appearing in the medical image 40 displayed in the first display area 35A. An example of the text 44C is text capable of identifying the type of lesion 42 appearing in the medical image 40 displayed in the first display area 35A. An example of the text 44D is text capable of identifying the size of the lesion 42 appearing in the medical image 40 displayed in the first display area 35A. Here, the texts 44A to 44D are illustrated as one piece of auxiliary information 44 displayed in the second display area 35B, but visible information other than text (for example, an image) may be displayed in the second display area 35B, or the information expressed by the texts 44A to 44D may be output as audio.

[0102] Fig. 7 is a conceptual diagram showing an example of the processing contents performed in the endoscope device 10. As shown in Fig. 7, when the processor 82 newly generates processing target information 111, the processor 82 generates processing request information 115 included in the metadata 112 of the processing target information 111 by referring to processing specification information 126 included in the server processing information 124 acquired from the control unit 100A of the server 2. For example, when the processor 82 specifies from the processing specification information 126 included in the server processing information 124 that the first recognition processing 116A or the second recognition processing 116B has been executed by the recognition unit 100B, the processor 82 generates information requesting the recognition unit 100B to execute the second recognition processing 116B as the processing request information 115, and includes the information in the metadata 112 of the processing target information 111.

[0103] Here, an example has been given in which the processor 82 generates information requesting the recognition unit 100B to execute the second recognition process 116B as the processing request information 115 when it has determined from the processing specification information 126 that the recognition unit 100B has executed the first recognition process 116A or the second recognition process 116B, but this is merely one example. For example, on the premise that information requesting the recognition unit 100B to execute the first recognition process 116A is generated as the processing request information 115 by the processor 82, the processing request information 115 may be changed by the processor 82 from information requesting the recognition unit 100B to execute the first recognition process 116A to information requesting the recognition unit 100B to execute the second recognition process 116B in accordance with the processing specification information 126. For example, in this case, first, the processor 82 generates information requesting the recognition unit 100B to execute the first recognition process 116A unconditionally as the processing request information 115 every time a medical image 40 is obtained. Then, when the processor 82 determines from the processing specification information 126 included in the server processing information 124 transmitted from the server 2 that the first recognition processing 116A or the second recognition processing 116B has been executed by the recognition unit 100B, it changes the processing request information 115 from information requesting the recognition unit 100B to execute the first recognition processing 116A to information requesting the recognition unit 100B to execute the second recognition processing 116B.

[0104] Fig. 8 is a conceptual diagram showing an example of the processing contents executed by the server 2 when processing target information 111 is transmitted to the server 2 from each of the multiple endoscopic devices 10. As shown in Fig. 8, when processing target information 111 is transmitted to the server 2 from the processor 82 of the endoscopic device 10 used in the first endoscopic examination 5A and the processor 82 of the endoscopic device 10 used in the second endoscopic examination 5B, in the server 2, the control unit 100A acquires the processing target information 111 transmitted from each of the processors 82 of the multiple endoscopic devices 10 (here, as an example, the endoscopic device 10 used in the first endoscopic examination 5A and the endoscopic device 10 used in the second endoscopic examination 5B) via the external I / F 98 (Fig. 4).

[0105] The control unit 100A refers to the processing request information 115 included in the processing target information 111 transmitted from each of the processors 82 of the multiple endoscope devices 10, and determines the order in which the recognition process 116 is executed on the multiple medical images 40 (in the example shown in FIG. 8, two medical images 40) obtained from each of the multiple endoscopic examinations 5 (here, as an example, the first endoscopic examination 5A and the second endoscopic examination 5B) for each medical image 40. The control unit 100A generates order information 128 indicating the order determined for each medical image 40, and includes the generated order information in the metadata 112 associated with the corresponding medical image 40.

[0106] Here, the order in which the recognition process 116 is executed refers to the order in which the recognition process 116 is executed by the recognition unit 100B on the medical images 40 included in the processing target information 111 transmitted from the processor 82 of the endoscopic device 10 used in the first endoscopic examination 5A and the medical images 40 included in the processing target information 111 transmitted from the processor 82 of the endoscopic device 10 used in the second endoscopic examination 5B.

[0107] For example, the first recognition process 116A (i.e., the process for performing screening) is given a higher order than the second recognition process 116B (i.e., the process for performing classification). The reason for this is that the first recognition process 116A must be performed with priority over the second recognition process 116B in order to prevent the lesion 42 from being overlooked (in other words, the lesion 42 is not detected), and the classification result of the lesion 42 is less likely to change than the screening result. The reason for performing the first recognition process 116A with priority over the second recognition process 116B in order to prevent the lesion 42 from being overlooked is that the lesion 42 is more likely to be misrecognized due to being out of frame, moving, or being too small during screening than during classification, and that performing AI processing first on the part that is more likely to be misrecognized can prevent the lesion from being overlooked.

[0108] Fig. 9 is a conceptual diagram showing an example of processing contents by the server 2 using the processing request information 115 and sequence information 128 included in the metadata 112. As shown in Fig. 9, the control unit 100A acquires the sequence information 128 from the metadata 112 associated with each of the multiple medical images 40. Then, the control unit 100A causes the recognition unit 100B to execute the recognition process 116 for each of the multiple medical images 40 in the sequence indicated by the sequence information 128 acquired from each metadata 112 (i.e., the sequence determined based on the multiple processing request information 115). A more specific example will be described below.

[0109] The control unit 100A selects a medical image 40 to be processed by the recognition process 116 from among the multiple medical images 40 in accordance with the order indicated by the order information 128. The control unit 100A outputs the medical image 40 selected from the multiple medical images 40 to the recognition unit 100B. The control unit 100A also refers to the processing request information 115 in the metadata 112 associated with the medical image 40 selected from the multiple medical images 40, determines the first recognition model 110A or the second recognition model 110B as the input destination of the medical image 40 selected from the multiple medical images 40, and instructs the recognition unit 100B of the determined input destination.

[0110] The recognition unit 100B inputs the medical image 40 input from the control unit 100A to the first recognition model 110A or the second recognition model 110B, which is the input destination instructed by the control unit 100A. When the input destination instructed by the control unit 100A as the input destination of the medical image 40 is the first recognition model 110A (i.e., when the medical image 40 is input to the first recognition model 110A), the first recognition result 122A is generated and output by the first recognition model 110A (see FIG. 5). On the other hand, when the input destination instructed by the control unit 100A as the input destination of the medical image 40 is the second recognition model 110B (i.e., when the medical image 40 is input to the second recognition model 110B), the second recognition result 122B is generated and output by the second recognition model 110B (see FIG. 5).

[0111] Fig. 10 is a conceptual diagram showing an example of transmission of server processing information 124 including recognition results 122 obtained by executing the recognition process 116 according to the order indicated by the order information 128. As shown in Fig. 10, the control unit 100A transmits the server processing information 124 including the recognition results 122 obtained by executing the recognition process 116 on medical images 40 selected from the multiple medical images 40 in order from high to low according to the order indicated by the order information 128 to the endoscope device 10 that is the source of the request to execute the recognition process 116, in order from high to low (i.e., in the order in which the recognition results 122 included in the server processing information 124 were obtained). For example, if the order indicated by the order information 128 in the metadata 112 of the medical image 40 transmitted from the endoscopic device 10 used in the first endoscopic examination 5A precedes the order indicated by the order information 128 in the metadata 112 of the medical image 40 transmitted from the endoscopic device 10 used in the second endoscopic examination 5B, the server processing information 124 is transmitted to the endoscopic device 10 used in the first endoscopic examination 5A prior to the endoscopic device 10 used in the second endoscopic examination 5B. That is, the server 2 transmits the server processing information 124 including the recognition result 122 for the medical image 40 on which the recognition process 116 was performed first to the endoscopic device 10 that requested the execution of the recognition process 116 prior to the server processing information 124 including the recognition result 122 for the medical image 40 on which the recognition process 116 was performed later.

[0112] Next, the operation of the portion of the medical support system 1 according to the present disclosure will be described with reference to Fig. 11. The flow of the medical support process shown in Fig. 11 is an example of the "medical support method" according to the present disclosure.

[0113] 11, first, in step ST10, the control unit 100A determines whether or not the plurality of pieces of processing target information 111 transmitted from the plurality of endoscope devices 10 have been received by the external I / F 98. In step ST10, if the plurality of pieces of processing target information 111 transmitted from the plurality of endoscope devices 10 have not been received by the external I / F 98, the determination is negative, and the medical support process proceeds to step ST30. In step ST10, if the plurality of pieces of processing target information 111 transmitted from the plurality of endoscope devices 10 have been received by the external I / F 98, the determination is positive, and the medical support process proceeds to step ST12.

[0114] In step ST12, the control unit 100A acquires processing request information 115 from each of the multiple pieces of processing target information 111 received in step ST10. After the processing in step ST12 is executed, the medical support processing proceeds to step ST14.

[0115] In step ST14, the control unit 100A generates sequence information 128 by referring to the processing request information 115 acquired from each of the plurality of pieces of processing target information 111 received in step ST10. Then, the control unit 100A includes the generated sequence information 128 in the metadata 112 included in each of the plurality of pieces of processing target information 111 received in step ST10 (i.e., associates it with each medical image 40 included in each of the plurality of pieces of processing target information 111 received in step ST10). After the processing of step ST14 is executed, the medical support processing proceeds to step ST16.

[0116] In step ST16, the control unit 100A selects, from among the medical images 40 included in the processing target information 111 and not selected in step ST16, the medical image 40 that is at the forefront of the order indicated by the order information 128 in the metadata 112 (in other words, the highest order). After the processing in step ST16 is executed, the medical support processing proceeds to step ST18.

[0117] In step ST18, the control unit 100A refers to the processing request information 115 in the metadata 112 included in the multiple pieces of processing target information 111, and determines the first recognition model 110A or the second recognition model 110B as the input destination of the medical image 40 selected in step ST16. After the processing of step ST18 is executed, the medical support processing proceeds to step ST20.

[0118] In step ST20, the recognition unit 100B inputs the medical image 40 selected in step ST16 to the first recognition model 110A or the second recognition model 110B determined in step ST18. After the process of step ST20 is executed, the medical support process proceeds to step ST22.

[0119] By executing the process of step ST20, the recognition result 122 is generated by the first recognition model 110A or the second recognition model 110B determined in step ST18. Then, in step ST22, the control unit 100A acquires the recognition result 122. After the process of step ST22 is executed, the medical support process proceeds to step ST24.

[0120] In step ST24, the control unit 100A generates process identification information 126 as information capable of identifying the type of the recognition process 116 corresponding to the recognition result 122 acquired in step ST22. After the process of step ST24 is executed, the medical support process proceeds to step ST26.

[0121] In step ST26, the control unit 100A generates server processing information 124 including the recognition result 122 acquired in step ST22 and the processing specification information 126 generated in step ST24, and transmits the server processing information 124 in order of priority to the endoscope device 10 that requested the execution of the recognition processing 116. Note that the priority corresponds to the order indicated by the order information 128 given to the medical image 40 used to obtain the recognition result 122 included in the server processing information 124. After the processing of step ST26 is executed, the medical support processing proceeds to step ST28.

[0122] In step ST28, the control unit 100A judges whether or not all of the medical images 40 included in the plurality of pieces of processing target information 111 received in step ST10 have been selected in step ST16. In step ST28, if all of the medical images 40 included in the plurality of pieces of processing target information 111 received in step ST10 have not been selected in step ST16, the judgment is negative, and the medical support processing proceeds to step ST16. In step ST28, if all of the medical images 40 included in the plurality of pieces of processing target information 111 received in step ST10 have been selected in step ST16, the judgment is positive, and the medical support processing proceeds to step ST30.

[0123] In step ST30, the control unit 100A determines whether a condition for terminating the medical support process is satisfied. One example of the condition for terminating the medical support process is that an instruction to terminate the medical support process is given to the medical support system 1 (for example, that an instruction to terminate the medical support process is accepted by the accepting device 64 of at least one of the endoscope devices 10).

[0124] In step ST30, if the condition for terminating the medical support process is not satisfied, the determination is negative and the medical support process proceeds to step ST10. In step ST30, if the condition for terminating the medical support process is satisfied, the determination is positive and the medical support process ends.

[0125] As described above, in the medical support system 1, the control unit 100A acquires a plurality of pieces of processing target information 111 (i.e., a plurality of pieces of processing target information 111 each containing different medical images 40) that correspond to a plurality of endoscopic examinations 5. Then, the recognition unit 100B executes recognition processing 116 for each of the medical images 40 included in the plurality of pieces of processing target information 111 in an order determined based on a plurality of pieces of processing request information 115 included in a plurality of pieces of metadata 112 that correspond to the plurality of medical images 40, respectively.

[0126] Therefore, compared to when the recognition process 116 is performed in parallel on all of the multiple medical images 40 corresponding to the multiple endoscopic examinations 5, problems that occur when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to the multiple endoscopic examinations 5 (for example, problems due to a time lag in communication between the server 2 and the endoscopic device 10) can be suppressed.

[0127] Furthermore, in the server 2, server processing information 124 including the recognition result 122 obtained by executing the recognition process 116 is transmitted to the endoscope device 10 that has requested the execution of the recognition process 116. Therefore, the doctor 12 performing the endoscopic examination 5 can be made aware of the recognition result 122.

[0128] Furthermore, in the server 2, the recognition process 116 is executed for each of the multiple medical images 40 in an order determined based on the processing request information 115 included in the metadata 112 of each of the multiple processing target information 111 (i.e., the order indicated by the order information 128), and in the order in which the recognition results 122 are obtained, the server processing information 124 corresponding to the recognition results 122 (in the example shown in FIG. 10, the server processing information 124 including the recognition result 122 and the processing specification information 126) is transmitted to the endoscope device 10 that has requested the execution of the recognition process 116. Therefore, the doctor 12 performing the endoscopic examination 5 with a high priority can be made aware of the recognition result 122 before the doctor 12 performing the endoscopic examination 5 with a low priority.

[0129] In the above embodiment, the metadata 112 includes the processing request information 115. However, this is merely an example. For example, instead of the processing request information 115, processing stage information indicating the stage of the processing being performed by the recognition unit 100B may be included in the metadata 112. The processing stage information classifies the stages of the processing being performed by the recognition unit 100B into a stage during execution of the first recognition process 116A, a stage after execution of the first recognition process 116A and before or during execution of the second recognition process 116B, and a stage after execution of the second recognition process 116B. When the processing stage information is used, for example, the control unit 100A of the server 2 includes the processing stage information in the server processing information 124 and transmits it to the endoscope device 10, and the endoscope device 10 includes the processing stage information included in the server processing information 124 transmitted from the server 2 in the metadata 112. Then, when the processing target information 111, the metadata 112 of which includes processing stage information, is transmitted from the multiple endoscope devices 10 to the server 2, the control unit 100A of the server 2 acquires the processing stage information from the metadata 112 in the processing target information 111, and determines the order by referring to the processing stage information. A stage in which the first recognition process 116A is being executed is after the execution of the first recognition process 116A and is prior to a stage in which the second recognition process 116B is being executed, and a stage in which the first recognition process 116A is after the execution of the first recognition process 116A and is before or during the execution of the second recognition process 116B is prior to a stage in which the second recognition process 116B is after the execution of the second recognition process 116B.

[0130] Furthermore, the processor 82 of the endoscope device 10 may update the processing request information 115 in accordance with processing stage information included in the server processing information 124 transmitted from the server 2. For example, in the case of a stage during execution of the first recognition processing 116A and in the case of a stage after execution of the first recognition processing 116A and before or during execution of the second recognition processing 116B, information requesting the recognition unit 100B to execute the second recognition processing 116B may be used as the processing request information 115, and in the case of a stage after execution of the second recognition processing 116B, information requesting the recognition unit 100B to execute the first recognition processing 116A may be used as the processing request information 115.

[0131] In the server 2, the parts to be observed in the endoscopic examination 5 (e.g., ascending colon, transverse colon, descending colon, sigmoid colon, upper rectum, and lower rectum) may be recognized by the recognition unit 100B, and the order may be determined according to the recognized parts. For example, when a part medically known as a part where a specific lesion such as SSL is likely to occur is recognized by the recognition unit 100B, the control unit 100A determines the order based on the contents of the processing request information 115 and the recognized part. In this case, for example, different weights may be assigned to the processing request information 115 according to the contents, and different weights may be assigned to multiple parts, and the order may be determined according to a total value of the weights (e.g., the larger the total value of the weights, the earlier the order).

[0132] Furthermore, the control unit 100A may make it so that the order in which the recognition process 116 is performed using the WLI method or the LCI method tends to precede the order in which the recognition process 116 is performed using the BLI method, or vice versa. In this case, the weights may be used in the manner described above.

[0133] Alternatively, the order may be determined according to the number of lesions 42 recognized by the recognition unit 100B. For example, the control unit 100A determines that the more lesions 42 recognized by the recognition unit 100B, the earlier the order is. In this case, the weights may be used in the manner described above.

[0134] Furthermore, when the recognition unit 100B recognizes the malignancy of the lesion 42, the order may be determined according to the malignancy of the lesion 42. For example, the control unit 100A determines that the higher the malignancy of the lesion 42, the higher the order. In this case, too, the weights may be used in the manner described above.

[0135] Furthermore, the control unit 100A may differentiate the processing speed when the recognition process 116 is performed using the BLI method from the processing speed when the recognition process 116 is performed using the WLI method or the LCI method. For example, the server 2 may set the processing speed when the recognition process 116 is performed using the BLI method to be higher than the processing speed when the recognition process 116 is performed using the WLI method or the LCI method, or vice versa.

[0136] In the above embodiment, the control unit 100A refers to the processing request information 115 included in the processing target information 111 transmitted from each processor 82 of the multiple endoscope devices 10 to determine the order in which the recognition processes 116 are executed, but this is merely one example. For example, as shown in Fig. 12, the metadata 112 in the processing target information 111 may include subject information 130, which is information about the subject 26 undergoing the endoscopic examination 5, and in this case, the control unit 100A may refer to the subject information 130 included in the metadata 112 to determine the order in which the recognition processes 116 are executed.

[0137] The subject information 130 includes, for example, information indicating the medical history of the subject 26, information indicating the age of the subject 26, information indicating the health condition of the subject 26 (for example, information indicating blood pressure and / or heart rate, etc.), and / or information indicating the gender of the subject 26. The subject information 130 includes information during the examination that becomes known during the implementation period of the endoscopic examination 5 (here, as an example, information indicating the medical history of the subject 26 and information indicating the gender of the subject 26) and pre-examination information that becomes known before the implementation period of the endoscopic examination 5 (here, as an example, information indicating the health condition of the subject 26).

[0138] The control unit 100A may derive the priority from the subject information 130 by using a first arithmetic expression (not shown) or a first table (not shown) capable of deriving the priority from the subject information 130, and determine the order in which the recognition processes 116 are executed based on the derived priority. The order corresponds to the priority derived from the subject information 130, and the order in which the recognition processes 116 are executed is earlier the higher the priority derived from the subject information 130. For example, the order in which the recognition processes 116 are executed on the medical images 40 obtained in the endoscopic examination 5 performed on the subject 26 in poor health is determined by the control unit 100A so that the order in which the recognition processes 116 are executed on the medical images 40 obtained in the endoscopic examination 5 performed on the subject 26 in good health is earlier than the order in which the recognition processes 116 are executed on the medical images 40 obtained in the endoscopic examination 5 performed on the subject 26 in good health. In the example shown in FIG. 12, the subject information 130 is an example of the "subject information" according to the present disclosure.

[0139] By doing this, problems that arise when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to multiple endoscopic examinations 5 can be suppressed with a high degree of accuracy, compared to when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to multiple endoscopic examinations 5 in a predetermined order without taking into account information about the subject 26 undergoing the endoscopic examination 5.

[0140] 12, the control unit 100A determines the order in which the recognition processes 116 are performed by referring to the subject information 130 included in the processing target information 111 transmitted from each processor 82 of the multiple endoscope devices 10, but this is merely one example. For example, as shown in FIG. 13, the metadata 112 in the processing target information 111 may include operator information 132, which is information about the doctor 12 as the operator performing the endoscopic examination 5. In this case, the control unit 100A may determine the order in which the recognition processes 116 are performed by referring to the operator information 132 included in the metadata 112. The operator information 132 is a type of pre-examination information that is known before the endoscopic examination 5 is performed.

[0141] Examples of the technician information 132 include an identifier that can identify the physician 12, information indicating the total number of years that the physician 12 has performed endoscopic examinations 5, information indicating a period during which the physician 12 has not performed endoscopic examinations 5 (i.e., a blank period), information indicating the number of endoscopic examinations 5 performed by the physician 12 within a unit time (e.g., within 24 hours), and / or information indicating the physician 12's field of specialty.

[0142] The control unit 100A may derive the priority from the operator information 132 by using a second arithmetic expression (not shown) or a second table (not shown) capable of deriving the priority from the subject information 130, and may determine the order in which the recognition process 116 is executed based on the derived priority. The order corresponds to the priority derived from the operator information 132, and the order in which the recognition process 116 is executed increases as the priority derived from the operator information 132 increases. For example, the order in which the recognition process 116 is executed for the medical image 40 obtained in the endoscopic examination 5 in which the doctor 12 with little practical experience is involved is determined by the control unit 100A so that the order in which the recognition process 116 is executed for the medical image 40 obtained in the endoscopic examination 5 in which the doctor 12 with plenty of practical experience is involved. In the example shown in FIG. 13, the operator information 132 is an example of the "operator information" and the "pre-examination information" according to the present disclosure.

[0143] By doing this, problems that arise when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to multiple endoscopic examinations 5 can be suppressed with a high degree of accuracy, compared to when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to multiple endoscopic examinations 5 in a predetermined order without taking into account information about the doctor 12 as the technician performing the endoscopic examination 5.

[0144] In the example shown in Fig. 13, the control unit 100A determines the order in which the recognition processes 116 are performed by referring to the operator information 132 included in the processing target information 111 transmitted from each processor 82 of the multiple endoscope devices 10, but this is merely one example. For example, as shown in Fig. 14, the metadata 112 in the processing target information 111 may include modality information 134, which is information about the endoscope device 10 as the modality used in imaging to obtain the medical image 40 in the endoscopic examination. In this case, the control unit 100A may determine the order in which the recognition processes 116 are performed by referring to the modality information 134 included in the metadata 112. The modality information 134 is a type of pre-examination information that is known before the endoscopic examination 5 is performed.

[0145] Examples of the modality information 134 include information indicating the model number of the endoscopic device 10, information indicating the amount of time the endoscopic device 10 has been continuously used in the endoscopic examination 5, information indicating the total period of use of the endoscopic device 10, and / or information indicating an address that can identify the location of the endoscopic device 10 on the network 3.

[0146] The control unit 100A may derive the priority from the modality information 134 by using a third arithmetic expression (not shown) or a third table (not shown) capable of deriving the priority from the modality information 134, and determine the order in which the recognition processes 116 are executed based on the derived priority. The order corresponds to the priority derived from the modality information 134, and the order in which the recognition processes 116 are executed increases as the priority derived from the modality information 134 increases. For example, the order in which the recognition processes 116 are executed on a medical image 40 obtained by imaging using an old endoscope device 10 is determined by the control unit 100A so that the order in which the recognition processes 116 are executed on a medical image 40 obtained by imaging using a new endoscope device 10 is earlier than the order in which the recognition processes 116 are executed on a medical image 40 obtained by imaging using a new endoscope device 10. In the example shown in FIG. 14, the modality information 134 is an example of the "operator information" and the "pre-examination information" according to the present disclosure.

[0147] By doing this, problems that arise when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to multiple endoscopic examinations 5 can be suppressed with a high degree of accuracy, compared to when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to multiple endoscopic examinations 5 in a predetermined order without taking into account information regarding the modality used in the imaging to obtain the medical images 40 in the endoscopic examination 5.

[0148] In the example shown in FIG. 14, the control unit 100A determines the order in which the recognition processes 116 are performed by referring to the modality information 134 included in the processing target information 111 transmitted from each processor 82 of the multiple endoscope devices 10, but this is merely one example. For example, as shown in FIG. 15, the metadata 112 in the processing target information 111 may include a recognition result 122 as an intermediate result of the endoscopic examination 5. In this case, the control unit 100A may determine the order in which the recognition processes 116 are performed by referring to the recognition result 122 included in the metadata 112. The intermediate result of the endoscopic examination 5 refers to, for example, information about a lesion 42 appearing in a medical image 40 obtained during the period in which the endoscopic examination 5 is performed. The intermediate result of the endoscopic examination 5 can also be said to be information during the examination that is identified during the period in which the endoscopic examination 5 is performed. Examples of information about the lesion 42 appearing in the medical image 40 include a screening result and a discrimination result. The recognition result 122 is information identified based on the medical image 40 (ie, information identified by executing the recognition process 116 on the medical image 40).

[0149] The control unit 100A may derive the priority from the recognition result 122 by using a fourth arithmetic expression (not shown) or a fourth table (not shown) capable of deriving the priority from the recognition result 122, and may determine the order in which the recognition process 116 is executed based on the derived priority. The order corresponds to the priority derived from the recognition result 122, and the order in which the recognition process 116 is executed is earlier for the higher priority derived from the recognition result 122. For example, the order in which the recognition process 116 is executed for the medical image 40 associated with the first recognition result 122A (i.e., the screening result) is determined by the control unit 100A so that the order in which the recognition process 116 is executed is earlier than the order in which the recognition process 116 is executed for the medical image 40 associated with the second recognition result 122B (i.e., the discrimination result). This is because, in the endoscopic examination 5, the discrimination result is less likely to change than the screening result, and it is preferable for the doctor 12 to understand the screening result earlier than the discrimination result. Note that this is merely an example, and the order in which the recognition process 116 is performed on the medical image 40 associated with the second recognition result 122B (i.e., the discrimination result) may be determined by the control unit 100A so that the order in which the recognition process 116 is performed on the medical image 40 associated with the first recognition result 122A (i.e., the screening result) precedes the order in which the recognition process 116 is performed on the medical image 40 associated with the first recognition result 122A (i.e., the screening result). In the example shown in FIG. 15, the recognition result 122 is an example of the "intermediate result" and "information during examination" according to the present disclosure. Moreover, the first recognition result 122A (i.e., the screening result) and the second recognition result 122B (i.e., the discrimination result) are an example of the "characteristic region information" according to the present disclosure.

[0150] By determining the order based on the intermediate results of the endoscopic examination 5 in this manner, problems that arise when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to the multiple endoscopic examinations 5 can be suppressed with a high degree of accuracy, compared to when the recognition process 116 is performed on each of the multiple medical images 40 corresponding to the multiple endoscopic examinations 5 in a determined order without taking into account the intermediate results of the endoscopic examination 5 (for example, screening results and differentiation results, which are information regarding the lesions 42 shown in the medical images 40 obtained during the period in which the endoscopic examination 5 is being performed).

[0151] In the above embodiment, the server processing information 124 including the recognition result 122 and the processing specification information 126 associated with each other is illustrated, but this is merely an example. For example, the server processing information 124 may include information in which the medical image 40 used in the recognition process 116 and the recognition result 122 are associated with each other. For example, when the recognition result 122 includes a bounding box BB, as shown in FIG. 16, the server processing information 124 may include information in which the bounding box BB is added to the medical image 40 used in the recognition process 116 as one piece of information included in the recognition result 122. In this way, in the endoscope device 10, the processor 82 (see FIG. 3) receives the server processing information 124 transmitted from the server 2, and obtains information in which the bounding box BB is added to the medical image 40 used in the recognition process 116 from the server processing information 124 received from the server 2, and can display the information on the screen 35 (for example, the first display area 35A).

[0152] Note that, here, an example is given of information in which a bounding box BB is added to the medical image 40 used in the recognition process 116 as one piece of information contained in the recognition result 122, but the server processing information 124 may also include information in which screening results and classification results other than the bounding box BB are associated with the medical image 40 (for example, text 44A, 44B, 44C, and 44D shown in FIG. 6, or images replacing the text 44A, 44B, 44C, and 44D, etc.).

[0153] In the above embodiment, the control unit 100A determines the order by referring only to the processing request information 115, but this is merely an example. For example, the control unit 100A may determine the order by comprehensively judging a plurality of pieces of information among the processing request information 115, the recognition result 122, the object information 130, the operator information 132, the modality information 134, the processing stage information, the information indicating the site, and the information indicating the malignancy of the lesion 42. In this case, for example, the control unit 100A may assign different weights to a plurality of pieces of information among the processing request information 115, the recognition result 122, the object information 130, the operator information 132, the modality information 134, the processing stage information, the information indicating the site, and the information indicating the malignancy of the lesion 42, and determine the order based on a total value of the weights.

[0154] In the above embodiment, the control unit 100A of the server 2 causes the recognition unit 100B to execute the recognition process 116 for each of the multiple medical images 40 in an order determined based on the information included in the processing target information 111. However, this is merely an example. For example, as shown in Fig. 17, a medical support system 150 may be configured to realize the process by the control unit 100A and the process by the recognition unit 100B (i.e., the recognition process 116).

[0155] The medical support system 150 is different from the medical support system 1 described in the above embodiment in that it has a management device 152. The management device 152 is connected to the network 3. The server 2 is also connected to the network 3 via the management device 152. The management device 152 is a device that manages the exchange of various information between the multiple endoscope devices 10 and the server 2. The management device 152 executes the process by the control unit 100A described in the above embodiment, and the server 2 executes the process by the recognition unit 100B described in the above embodiment. That is, the management device 152 causes the server 2 to execute the recognition process 116 for each of the multiple medical images 40 in an order determined based on the information included in the processing target information 111. Then, the management device 152 acquires the recognition result 122 obtained by the server 2 executing the recognition process 116 from the server 2, generates server processing information 124 in the same manner as described in the above embodiment, and transmits it to the multiple endoscope devices 10. The server processing information 124 may be generated by the server 2 instead of the management device 152. In the example shown in FIG. 17, the management device 152 is an example of a "medical support device" according to the present disclosure, and the server 2 is an example of a "processing device" according to the present disclosure.

[0156] In the above embodiment, lesion 42 is given as an example of the "first feature region" and "second feature region" according to the present disclosure, but the present disclosure is not limited thereto, and the present disclosure will be valid even if a resection region, a bleeding region, a marking region, an organ, or a treatment tool (e.g., a hemostatic clip placed inside the body) is applied instead of lesion 42.

[0157] In the above embodiment, the camera 52 mounted on the endoscope 16 is exemplified, but the present disclosure is not limited to this. For example, the present disclosure is also applicable to a fundus camera, a camera mounted on a medical microscope, or the like.

[0158] In the above embodiment, a medical image 40 is shown as being obtained by imaging the inside of the large intestine 28 using a camera 52 mounted on the endoscope 16, but this is merely one example, and the medical image 40 may be a medical image obtained using various modalities such as a radiation imaging device, an MRI, a CT, or an ultrasound examination device (e.g., an ultrasound endoscopic device).

[0159] In the above embodiment, recognition processing 116 using AI in a bounding box method is exemplified, but this is merely one example, and for example, recognition processing using AI in a segmentation method may be performed instead of recognition processing using AI in a bounding box method 116. Also, instead of recognition processing using an AI method, recognition processing using a non-AI method (for example, a template matching method) may be performed, or recognition processing using a combination of a non-AI method and an AI method may be performed.

[0160] In the above embodiment, an example was given of a form in which the medical support processing is performed by the computer 96, but the present disclosure is not limited to this, and at least a portion of the processing included in the medical support processing may be performed by a device provided outside the computer 96.

[0161] In the above embodiment, the server 2 is realized by cloud computing, but this is merely an example. The server 2 may be realized by network computing such as fog computing, edge computing, or grid computing.

[0162] In the above-described embodiment, an example has been described in which the medical support program 108 is stored in the storage 104, but the present disclosure is not limited thereto. For example, the medical support program 108 may be stored in a portable computer-readable non-transitory storage medium such as an SSD or a USB memory. The medical support program 108 stored in the non-transitory storage medium is installed in the computer 96 of the endoscope device 10. The processor 100 executes medical support processing according to the medical support program 108.

[0163] Alternatively, the medical support program 108 may be stored in a storage device such as another computer or server connected to the endoscope device 10 via a network, and the medical support program 108 may be downloaded and installed in the computer 96 in response to a request from the endoscope device 10.

[0164] Note that it is not necessary to store all of the medical support program 108 in a storage device such as another computer or server device connected to the endoscope device 10, or to store all of the medical support program 108 in the storage 104, and a part of the medical support program 108 may be stored.

[0165] As hardware resources for executing the medical support processing, various processors shown below can be used. Examples of the processor include a general-purpose processor such as a CPU that functions as a hardware resource for executing medical support processing by executing software, that is, a program. In addition, examples of the processor include a dedicated electric circuit that is a processor having a circuit configuration designed specifically for executing a specific process such as an FPGA, a PLD, or an ASIC. A memory is built in or connected to any of the processors, and any of the processors executes medical support processing by using the memory.

[0166] The hardware resource for executing the medical support process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource for executing the medical support process may be a single processor.

[0167] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes medical support processing. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute medical support processing on a single IC chip, as typified by SoC. In this way, the medical support processing is realized using one or more of the above various processors as hardware resources.

[0168] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. Also, the above medical support process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processing may be changed, without departing from the spirit of the invention.

[0169] The above description and illustrations are detailed descriptions of the parts related to the present disclosure and are merely an example of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations within the scope of the gist of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the present disclosure.

[0170] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0171] 1,150 Medical Support Systems 2 Server 3. Network 4 Endoscopy Room 4A Endoscopy Room 1 4B 2nd Endoscopy Room 5 Endoscopy 5A First endoscopy 5B Second endoscopy 10 Endoscopic device 12. Doctor 14. Nurse 16 Endoscope 18 Display device 20 Light source device 22 Control device 24 Function expansion device 26 Subject 28 Large intestine 30 light 32 Intestinal wall 34 Wagon 35 screens 35A 1st display area 35B 2nd display area 40 Medical Imaging 42 Lesions 44 Supporting Information 44A, 44B, 44C, 44D Text 46 Control section 48 Insertion section 50 Tip 50A Tip surface 52 Camera 54 Lighting Equipment 54A, 54B Lighting window 56 Treatment opening 58 Treatment tools 60 Treatment tool insertion port 62 Universal Code 64 Reception Device 66, 78, 96 Computer 68, 88, 106 Bus 70, 80, 98 External I / F 72, 82, 100 Processor 74, 84, 102 Memory 76, 86, 104 Storage 100A Control Unit 100B Recognition Unit 108 Medical Support Program 110A First Recognition Model 110B Second Recognition Model 111 Information to be Processed 112 Metadata 114 Device Identification Information 115 Processing Request Information 116 Recognition Process 116A First Recognition Process 116B Second Recognition Process 122 Recognition Result 122A First Recognition Result 122B Second Recognition Result 124 Server Processing Information 126 Process Identification Information 128 Sequence Information 130 Subject Information 132 Operator Information 134 Modality Information 152 Management Device BB Bounding Box

Claims

1. A processor is provided. The processor, Acquire multiple medical images corresponding to multiple medical tests; acquiring a plurality of pieces of medical test information corresponding to the plurality of medical tests, each piece of medical test information being related to the corresponding medical tests; A processing device is caused to execute a recognition process for each of the plurality of medical images in an order determined based on the plurality of pieces of medical examination information. Medical support equipment.

2. The medical test information includes test-in-progress information that is known during the execution period of the corresponding medical test. The medical support device according to claim 1 .

3. The medical test information includes pre-test information that is known prior to the time period during which the corresponding medical test is performed. The medical support device according to claim 1 .

4. The medical examination information includes subject information regarding a subject undergoing the corresponding medical examination. The medical support device according to claim 1 .

5. The medical examination information includes operator information regarding an operator who performs the corresponding medical examination. The medical support device according to claim 1 .

6. said medical images being obtained during a corresponding said medical examination; The medical examination information includes modality information regarding a modality used in imaging to obtain the medical image in the corresponding medical examination. The medical support device according to claim 1 .

7. said medical images being obtained during a corresponding said medical examination; the medical test information includes corresponding intermediate results of the medical test; The intermediate result is information that is identified based on the medical image obtained in the corresponding medical examination. The medical support device according to claim 1 .

8. The intermediate result includes feature region information that is information about a first feature region appearing in the medical image. The medical support device according to claim 7.

9. the first feature region is a lesion; The intermediate result is a screening result of the lesion or a differentiation result of the lesion. The medical support device according to claim 8.

10. The processor transmits medical support information including an execution result obtained by executing the recognition processing to a device used in the medical examination corresponding to the medical image that was the subject of the recognition processing executed to obtain the execution result included in the medical support information. The medical support device according to claim 1 .

11. The medical support information includes information in which the medical image that was the subject of the recognition processing executed to obtain the execution result included in the medical support information is associated with the execution result. The medical support device according to claim 10.

12. The processor transmits the corresponding medical support information to the device in the order in which the execution results are obtained by executing the recognition process on each of the plurality of medical images in the order. The medical support device according to claim 10.

13. The recognition process is a process of generating information related to a second feature region by inputting the medical image to a trained model that receives an image including a first region corresponding to a second feature region and generates information related to the first region. The medical support device according to claim 1 .

14. each of the plurality of medical examinations is an endoscopy examination; Each of the plurality of medical images is an endoscopic image obtained in the endoscopic examination. The medical support device according to claim 1 .

15. A medical support device according to any one of claims 1 to 14; a communication device that transmits the medical image to the medical support device and receives an execution result obtained by executing the recognition process. Medical support system.

16. acquiring a plurality of medical images corresponding to a plurality of medical exams; obtaining a plurality of pieces of medical test information corresponding to the plurality of medical tests, each piece of medical test information being associated with a corresponding one of the medical tests; and and causing a processing device to execute a recognition process for each of the plurality of medical images in an order determined based on the plurality of pieces of medical examination information. Medical assistance methods.

17. acquiring a plurality of medical images corresponding to a plurality of medical exams; obtaining a plurality of pieces of medical test information corresponding to the plurality of medical tests, each piece of medical test information being associated with a corresponding one of the medical tests; and A program for causing a computer to execute a process including causing a processing device to execute recognition processing for each of the plurality of medical images in an order determined based on the plurality of pieces of medical test information.

Images