Control device, medical assistance system, and medical assistance method

Abstract

Provided are a control device, a medical assistance system, and a medical assistance method with which it is possible to prevent, with a simple operation, leakage of execution specifics when inputting an examination report. The control device: acquires examination information; estimates a procedure of the examination information; acquires, on the basis of the estimated procedure and a procedure set information database in which medicine, equipment assumed to be used, and lesion information regarding a lesion treatment site are associated with each procedure, an initial procedure set in which a procedure performed on a patient, the quantity of medicine, and the type of equipment are combined, the initial procedure set being acquired from the procedure set information database; adjusts the quantity of the medicine and / or the type of the equipment in the initial procedure set to that corresponding to the lesion information; and displays the adjusted procedure set on a display unit.

Description

Control Device, Medical Support System, and Medical Support Method

[0001] The present disclosure relates to a control device, a medical support system, and a medical support method.

[0002] Conventionally, in an endoscope system, inspections and treatments may be performed using drugs and equipment to obtain an image of a subject. In this case, a user such as a doctor needs to describe the drugs and equipment used, the details of the procedure, etc. in an inspection report created after the diagnosis.

[0003] Also, in an endoscope system, when performing an inspection based on an image obtained by an endoscope or the like, the relationship between the drugs and equipment used and the region of interest including lesions and the like is important, and a technique of attaching at least one of the drugs and equipment in the image recognized by image recognition and the region of interest in the image recognized by image recognition to the image is known (for example, see Patent Document 1).

[0004] Japanese Patent No. 7091349

[0005] By the way, after the inspection, the user manually performs an implementation input such as an inspection report. This manual implementation input is such that the user manually inputs, and since the input content is complex, there may be a difference between the actual implementation content performed on the patient and the input content, resulting in a possibility of omission. As a result, due to the difference between the implementation content and the input content at the time of accounting, inappropriate accounting may occur, and there was room for improvement.

[0006] However, in the above-mentioned Patent Document 1, simply, the drugs, equipment, and regions of interest recognized by image recognition are merely attached to the image, and no consideration is given to the discrepancy between the implementation content performed by the user on the patient and the input content after the inspection.

[0007] The present disclosure has been made in view of the above, and an object thereof is to provide a control device, a medical support system, and a medical support method that can prevent omission of implementation content during input of an inspection report with a simple operation.

[0008] To solve the above-mentioned problems and achieve the objective, the control device according to this disclosure is a control device equipped with a processor, the processor acquires examination information including examination images of a patient, diagnostic information of lesions recognized in the examination images, equipment information, and lesion information, estimates the procedure of the examination information based on the examination information and a procedure identification information database which associates diagnostic information of lesions and equipment used during examination for each procedure, obtains an initial procedure set from the procedure set information database which combines the procedure performed on the patient, the quantity of drugs, and the type of equipment based on the estimated procedure and a procedure set information database which associates drugs, equipment expected to be used, and lesion information of the treatment site of the lesion for each procedure, adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion information, and displays the adjusted procedure set on the display unit.

[0009] Furthermore, in the control device relating to the present disclosure, the lesion information is lesion size information relating to the size of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion size information.

[0010] Furthermore, in the control device relating to this disclosure, the lesion information is depth information relating to the depth of invasion of the lesion, and the processor adjusts at least one of the quantity of drug and the type of equipment in the initial procedure set to correspond to the depth information.

[0011] Furthermore, in the control device relating to the present disclosure, the lesion information is infiltration information relating to the degree of infiltration of the lesion, and the processor adjusts at least one of the quantity of drug and the type of equipment in the initial procedure set to correspond to the infiltration information.

[0012] Furthermore, in the control device relating to this disclosure, the processor determines whether the inspection information includes the diagnostic information and the equipment information, and if the diagnostic information and the equipment information are included, it estimates the procedure of the inspection information.

[0013] Furthermore, in the control device relating to the present disclosure, the processor displays the initial procedure set and the adjusted procedure set on the display unit, and when it receives a selection signal to apply the adjusted procedure set, it confirms the adjusted procedure set as the procedure set for the patient.

[0014] Furthermore, the control device relating to this disclosure is a medical support device as described in the above disclosure.

[0015] Furthermore, the medical support system relating to this disclosure comprises a control device equipped with a processor and a terminal device capable of receiving input from an operator. The processor acquires examination information including examination images of a patient, diagnostic information of lesions recognized in the examination images, equipment information, and lesion information. Based on the examination information and a procedure identification information database that associates diagnostic information of lesions and equipment used during examination for each procedure, the processor estimates the procedure of the examination information. Based on the estimated procedure and a procedure set information database that associates drugs, equipment expected to be used, and lesion information of the treatment site of the lesion for each procedure, the processor acquires an initial procedure set from the procedure set information database that combines the procedure performed on the patient, the quantity of drugs, and the type of equipment. The processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion information and displays the adjusted procedure set on the display unit of the terminal device.

[0016] Furthermore, in the medical support system relating to this disclosure, the lesion information is lesion size information relating to the size of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion size information.

[0017] Furthermore, in the medical support system relating to this disclosure, the lesion information is depth information relating to the depth of invasion of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the depth information.

[0018] Furthermore, in the medical support system relating to this disclosure, the lesion information is infiltration information relating to the degree of infiltration of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the infiltration information.

[0019] Furthermore, in the medical support system relating to this disclosure, the processor determines whether the examination information includes the diagnostic information and the equipment information, and if the diagnostic information and the equipment information are included, it estimates the procedure of the examination information.

[0020] Furthermore, in the medical support system relating to this disclosure, the processor displays the initial procedure set and the adjusted procedure set on the display unit, and when it receives a selection signal to apply the adjusted procedure set, it confirms the adjusted procedure set as the procedure set for the patient.

[0021] Furthermore, in the medical support system related to this disclosure, the control device is a medical support device as described in the above disclosure.

[0022] Furthermore, the medical support method relating to this disclosure includes the processor acquiring examination information including examination images of a patient, diagnostic information of lesions recognized in the examination images, equipment information, and lesion information; estimating the procedure of the examination information based on the examination information and a procedure identification information database that associates diagnostic information of lesions and equipment used during examination for each procedure; acquiring an initial procedure set from the procedure set information database that combines the procedure performed on the patient, the quantity of medication, and the type of equipment based on the estimated procedure and a procedure set information database that associates medication, equipment expected to be used, and lesion information of the treatment site of the lesion for each procedure; adjusting at least one of the quantity of medication and the type of equipment in the initial procedure set to correspond to the lesion information; and displaying the adjusted procedure set on the display unit.

[0023] Furthermore, in the medical support method relating to this disclosure, the lesion information is lesion size information relating to the size of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion size information.

[0024] Furthermore, in the medical support method relating to this disclosure, the lesion information is depth information relating to the depth of invasion of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the depth information.

[0025] Furthermore, in the medical support method relating to this disclosure, the lesion information is infiltration information relating to the degree of infiltration of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the infiltration information.

[0026] Furthermore, in the medical support method relating to this disclosure, the processor determines whether the examination information includes the diagnostic information and the equipment information, and if the diagnostic information and the equipment information are included, it estimates the procedure of the examination information.

[0027] Furthermore, in the medical support method relating to this disclosure, the processor displays the initial procedure set and the adjusted procedure set on the display unit, and when it receives a selection signal to apply the adjusted procedure set, it confirms the adjusted procedure set as the procedure set for the patient.

[0028] According to this disclosure, the system has the effect of preventing omissions in the details of the inspection when entering the inspection report with simple operations.

[0029] Figure 1 is a diagram showing the overall configuration of a medical support system according to Embodiment 1 of this disclosure. Figure 2 is a block diagram showing the functional configuration of the main parts of an endoscope and control device according to Embodiment 1 of this disclosure. Figure 3 is a block diagram showing the functional configuration of an image diagnostic device according to Embodiment 1 of this disclosure. Figure 4 is a block diagram showing the functional configuration of a medical support device according to Embodiment 1 of this disclosure. Figure 5 is a block diagram showing the functional configuration of a terminal device according to Embodiment 1 of this disclosure. Figure 6 is a flowchart showing an overview of the processing performed by the medical support device according to Embodiment 1 of this disclosure. Figure 7 is a diagram showing an example of examination information acquired from the examination information DB by the acquisition unit of the medical support device according to Embodiment 1 of this disclosure. Figure 8 is a diagram showing a subflowchart of the procedure estimation processing in step S102 of Figure 6. Figure 9 is a diagram showing an example of a procedure identification information table in the procedure identification information recorded in the procedure identification information DB of the medical support device according to Embodiment 1 of this disclosure. Figure 10 is a diagram showing an example of a procedure selection screen displayed on the display unit of the terminal device according to Embodiment 1 of this disclosure. Figure 11 is a diagram showing a subflowchart showing an overview of the procedure set information acquisition processing in step S103 of Figure 6. Figure 12 shows an example of a procedure set information table in the procedure set information recorded in the procedure set information DB of the medical support device according to Embodiment 1 of this disclosure. Figure 13 shows an example of a procedure set confirmation screen displayed on the display unit of the terminal device according to Embodiment 1 of this disclosure. Figure 14 shows a subflowchart of the procedure set information adjustment process in step S104 of Figure 6. Figure 15 shows another example of a procedure set information table in the procedure set information recorded in the procedure set information DB of the medical support device according to Embodiment 1 of this disclosure. Figure 16 shows an example of an adjustment candidate screen displayed on the display unit of the terminal device according to Embodiment 1 of this disclosure. Figure 17 shows an example of an implementation input screen displayed on the display unit of the terminal device according to Embodiment 1 of this disclosure. Figure 18 shows a subflowchart of the procedure set information adjustment process executed by the medical support device according to Embodiment 2 of this disclosure. Figure 19 shows another example of examination information acquired by the acquisition unit of the medical support device according to Embodiment 2 of this disclosure.Figure 20 shows another example of a procedure set information table in the procedure set information recorded in the procedure set information DB of the medical support device according to Embodiment 2 of this disclosure. Figure 21 shows an example of an adjustment candidate screen displayed by the display unit of the terminal device according to Embodiment 2 of this disclosure. Figure 22 shows an example of an implementation input screen displayed by the display unit of the terminal device according to Embodiment 2 of this disclosure. Figure 23 shows a subflowchart of the procedure set information adjustment process executed by the medical support device according to Embodiment 3 of this disclosure.

[0030] The embodiments for implementing this disclosure will be described in detail below with reference to drawings. However, this disclosure is not limited to the embodiments described below. Furthermore, the figures referenced in the following description only provide a schematic representation of the shape, size, and positional relationships to the extent that the content of this disclosure can be understood. In other words, this disclosure is not limited to the shapes, sizes, and positional relationships exemplified in the figures. In addition, the same parts will be denoted by the same reference numerals in the drawings.

[0031] (Embodiment 1) [Overall Configuration of the Medical Support System] Figure 1 is a diagram showing the overall configuration of the medical support system according to Embodiment 1 of the present disclosure. The medical support system 1 shown in Figure 1 continuously images the inside of a subject by inserting the insertion part 21 of the endoscope 2 through the mouth of a subject such as a person or animal into the esophagus or through the anus of the subject into the large intestine, and displays the captured image data in chronological order on the display device 3.

[0032] Furthermore, the medical support system 1 detects (recognizes or estimates) lesion information, including characteristic regions such as organs, body parts, polyps, or cancers, as well as information on the equipment used, drugs, and diagnostic names, based on the image diagnostic device 5 (CAD: Computer Aided Detection) input from the control device 4 and a trained model (inference model) that has been trained using training data obtained by imaging organs, body parts, and lesions of the subject. The system outputs these detection results to the control device 4. The operator, such as a doctor, observes and treats the subject while checking the displayed image on the display device 3 and the detection results (estimated results) from the image diagnostic device 5. Here, the trained model is a deep learning model that is trained using training data in which feature regions such as organs, body parts, and lesions of the subject, lesion size, depth of invasion information, invasiveness, equipment, and drugs are added to image data of subjects such as patients, and outputs organs, body parts, tumors, non-tumors, inflammatory activity, invasive cancer, feature regions, lesion size, depth of invasion information, invasiveness, equipment, and drugs contained in the image data and examination images.

[0033] Furthermore, the medical support system 1 acquires image data input from the control device 4 via the network N100 to the medical support device 6, as well as examination information including the detection results (estimated results) from the image diagnostic device 5, and records the acquired image data and examination information in association. The medical support system 1 then outputs various data in response to requests from terminal devices 7 that can be operated by surgeons, assistants, etc., via the network N100.

[0034] Furthermore, in the medical support system 1, the terminal device 7 generates examination reports, such as examination reports for medical personnel, and these examination reports are output to the medical support device 6 for recording. Subsequently, the user diagnoses the subject and conducts conferences, etc., based on the examination reports and the diagnostic results of living cells obtained by biopsy during the observation of the subject. In addition, the medical support system 1 processes the examination reports using an accounting server (not shown), and bills the patient for medical expenses.

[0035] As shown in Figure 1, the medical support system 1 comprises an endoscope 2, a display device 3, a control device 4, an image diagnostic device 5, a medical support device 6, and a terminal device 7.

[0036] [Endoscope Configuration] The endoscope 2 continuously generates image data (RAW data) by imaging the inside of the subject and sequentially outputs this image data to the control device 4. As shown in Figure 1, the endoscope 2 comprises an insertion unit 21, an operating unit 22, and a universal code 23.

[0037] The insertion portion 21 is flexible in at least a portion and is inserted into the subject. As shown in Figure 1, the insertion portion 21 comprises a tip portion 24 provided at the tip of the insertion portion 21, a bendable portion 25 connected to the base end side (operating portion 22 side) of the tip portion 24 and configured to bendable, and a long, flexible tube portion 26 connected to the base end side of the bendable portion 25 and having flexibility.

[0038] The operating section 22 is connected to the proximal end portion of the insertion section 21. The operating section 22 receives various operations on the endoscope 2. As shown in Figure 1, the operating section 22 is provided with a bending knob 221, an insertion opening 222, and a plurality of operating members 223.

[0039] The bending knob 221 is configured to be rotatable in response to user operation by a user such as a surgeon. By rotating the bending knob 221, a bending mechanism (not shown) made of metal or resin wire disposed within the insertion portion 21 is activated. As a result, the bending portion 25 bends.

[0040] The insertion port 222 communicates with a treatment instrument channel (not shown), which is a conduit extending from the tip of the insertion section 21, and is an insertion port for inserting treatment instruments, etc., into the treatment instrument channel from outside the endoscope 2.

[0041] The plurality of operation members 223 are constituted by buttons or the like that receive various operations by a user such as an operator, and output an operation signal corresponding to the various operations to the control device 4 by passing through the universal code 23. Examples of the various operations include a release operation for instructing a still image by the endoscope 2, an operation for switching the observation mode of the endoscope 2 to a normal light observation mode or a special observation mode, and the like.

[0042] The universal code 23 extends in a direction different from the extending direction of the insertion portion 21 from the operation portion 22, and is a code in which a light guide 231 (see FIG. 2) constituted by an optical fiber or the like, a first signal line 232 (see FIG. 2) for transmitting the above-described image data, a second signal line 233 (see FIG. 2) for transmitting the above-described operation signal, and the like are disposed. And, as shown in FIG. 1, a first connector portion 27 is provided at the proximal end of the universal code 23. The first connector portion 27 is detachably connected to the control device 4.

[0043] The display device 3 is constituted by a display monitor such as a liquid crystal or an organic EL (Electro Luminescence), and displays a display image based on the image data subjected to image processing in the control device 4 and various information regarding the endoscope 2 under the control of the control device 4.

[0044] The control device 4 is realized by using a processor which is a processing device having hardware such as a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or a CPU (Central Processing Unit), and a memory which is a temporary storage area used by the processor. The control device 4 comprehensively controls the operations of each part of the endoscope 2 according to a program recorded in the memory.

[0045] The image diagnostic device 5 (CAD) uses the image data group input from the control device 4 and the learned model pre-trained with learning data to detect lesions including tumors and non-tumors such as organs, parts, polyps, or cancers shown in the image data, calculate inflammation activity evaluation information for evaluating the inflammation activity, and output to the control device 4 the estimation result including the determination result of whether it is invasive cancer, the estimation time of this estimation result, and the examination information in which the diagnostic information, equipment information, lesion size, etc. are associated. The image diagnostic device 5 is realized using a processor which is a processing device having hardware such as a GPU, FPGA, or CPU, and a memory which is a temporary storage area used by the processor.

[0046] The medical support device 6 sequentially records various data input from the control device 4 and the recognition results of the image diagnostic device 5, and transmits data according to requests from the terminal device 7. The medical support device 6 is realized using a processor which is a processing device having hardware such as a GPU, FPGA, or CPU, and a memory which is a temporary storage area used by the processor. Further, the medical support device 6 is configured using an HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0047] The terminal device 7 is configured using a mobile phone, a tablet terminal, etc., and displays various information transmitted from the medical support device 6 on the display unit 71 via the network N100. Further, the terminal device 7 transmits various information input by the user to the medical support device 6. Note that the terminal device 7 may be a laptop personal computer or the like as long as it can be connected to the medical support device 6 via the network N100.

[0048] [Functional Configuration of Main Parts of Medical Support System] Next, the functional configuration of the main parts of the above-described medical support system 1 will be described. FIG. 2 is a block diagram showing the functional configuration of the main parts of the endoscope 2 and the control device 4. Hereinafter, the endoscope 2 and the control device 4 will be described in this order, and then the image diagnostic device 5 and the medical support device 6 will be described.

[0049] [Functional Configuration of the Endoscope] First, the configuration of the endoscope 2 will be described. As shown in Figure 2, the endoscope 2 comprises an illumination optical system 201, an imaging optical system 202, an image sensor 203, an A / D conversion unit 204, a P / S conversion unit 205, an image recording unit 206, and an image control unit 207. Here, the illumination optical system 201, the imaging optical system 202, the image sensor 203, the A / D conversion unit 204, the P / S conversion unit 205, the image recording unit 206, and the image control unit 207 are each located within the tip portion 24.

[0050] The illumination optical system 201 is composed of one or more lenses and the like, and directs illumination light supplied from the light guide 231 toward the subject.

[0051] The imaging optical system 202 is composed of one or more lenses and the like, and forms an image of the subject on the light-receiving surface of the image sensor 203 by collecting reflected light from the subject, light returned from the subject, and light such as fluorescence emitted by the subject. Furthermore, the focal length and angle of view of the imaging optical system 202 can be changed by moving the lens along the optical axis direction O1.

[0052] The image sensor 203 is configured using a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor, in which one of the color filters constituting a Bayer array (RGB) is placed at each of a plurality of pixels arranged in a two-dimensional matrix. Under the control of the imaging control unit 207, the image sensor 203 receives the subject image formed by the imaging optical system 202, performs photoelectric conversion, and generates an captured image (analog signal). In this embodiment, the image sensor 203 may be configured integrally with an image sensor and a TOF (Time Of Flight) sensor that acquires subject distance information (hereinafter referred to as depth map information) using the TOF method. Depth map information is information in which the subject distance from the position of the image sensor 203 (position of the tip 24) to the corresponding position on the observation target corresponding to the pixel position in the captured image is detected for each pixel position. The configuration for generating depth map information is not limited to the TOF sensor described above, but may also include an image sensor including a phase difference sensor. In the following, depth map information and captured images will be collectively referred to as image data. The image sensor 203 outputs the image data to the A / D conversion unit 204.

[0053] The A / D conversion unit 204 is configured using an A / D conversion circuit and the like. Under the control of the imaging control unit 207, the A / D conversion unit 204 performs A / D conversion processing on the analog image data input from the image sensor 203 and outputs it to the P / S conversion unit 205.

[0054] The P / S conversion unit 205 is configured using a P / S conversion circuit or the like, and under the control of the imaging control unit 207, it performs parallel / serial conversion of the digital image data input from the A / D conversion unit 204 and outputs it to the control device 4 via the first signal line 232. Alternatively, instead of the P / S conversion unit 205, an E / O conversion unit that converts image data into an optical signal may be provided, and the image data may be output to the control device 4 using the optical signal. Furthermore, the image data may be transmitted to the control device 4 by wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

[0055] The imaging and recording unit 206 is composed of non-volatile memory and volatile memory, and records various information related to the endoscope 2 (for example, pixel information of the image sensor 203). The imaging and recording unit 206 also records various setting data and control parameters transmitted from the control device 4 via the second signal line 233.

[0056] The imaging control unit 207 is implemented using a TG (Timing Generator), a processor which is a processing unit having hardware such as a CPU, and memory which is a temporary storage area used by the processor. Based on the setting data received from the control unit 4 via the second signal line 233, the imaging control unit 207 controls the operation of the image sensor 203, the A / D conversion unit 204, and the P / S conversion unit 205, respectively.

[0057] [Configuration of the control device] Next, the configuration of the control device 4 will be described. As shown in Figure 2, the control device 4 includes a condensing lens 401, a first light source unit 402, a second light source unit 403, a light source control unit 404, an S / P conversion unit 405, an image processing unit 406, an input unit 407, a recording unit 408, a communication unit 409, and a control unit 410.

[0058] The condensing lens 401 collects the light emitted by the first light source unit 402 and the second light source unit 403, and emits it to the light guide 231.

[0059] The first light source unit 402, under the control of the light source control unit 404, emits visible white light (normal light) to supply the light guide 231 as illumination light. This first light source unit 402 is composed of a collimating lens, a white LED (Light Emitting Diode) lamp, and a drive driver, etc. The first light source unit 402 may also be composed of a red LED lamp, a green LED lamp, and a blue LED lamp that emit light simultaneously to supply visible white light. Alternatively, the first light source unit 402 may be composed of a halogen lamp or a xenon lamp, etc.

[0060] The second light source unit 403, under the control of the light source control unit 404, emits special light having a predetermined wavelength band to supply special light to the light guide 231 as illumination light. Here, the special light is light used for narrow-band imaging (NBI) using narrow-band light including 390-445 nm and 530-550 nm. Of course, in addition to narrow-band light, amber-colored light (600 nm and 630 nm) used for red dichromatic imaging (RDI) may also be used as special light.

[0061] The light source control unit 404 is implemented using a processor having hardware such as an FPGA or CPU, and memory, which is a temporary storage area used by the processor. Based on control data input from the control unit 410, the light source control unit 404 controls the light emission timing and light emission duration of the first light source unit 402 and the second light source unit 403, respectively.

[0062] The S / P conversion unit 405, under the control of the control unit 410, performs serial / parallel conversion on the image data received from the endoscope 2 via the first signal line 232 and outputs it to the image processing unit 406. Note that if the endoscope 2 outputs image data as an optical signal, an O / E conversion unit that converts optical signals to electrical signals may be provided instead of the S / P conversion unit 405. Also, if the endoscope 2 transmits image data via wireless communication, a communication module capable of receiving wireless signals may be provided instead of the S / P conversion unit 405.

[0063] The image processing unit 406 is implemented using a processor with hardware such as a GPU or FPGA, and memory, which is a temporary storage area used by the processor. Under the control of the control unit 410, the image processing unit 406 performs predetermined image processing on the parallel data image data input from the S / P conversion unit 405 and outputs it to the display device 3. Examples of predetermined image processing include demosaicing, white balance processing, gain adjustment processing, gamma correction processing, and format conversion processing.

[0064] The input unit 407 is configured using a mouse, foot switch, keyboard, buttons, switches, and touch panel, and receives user operations from a user such as a surgeon, and outputs an operation signal corresponding to the user operation to the control unit 410.

[0065] The recording unit 408 is configured using recording media such as volatile memory, non-volatile memory, SSDs, HDDs, and memory cards. The recording unit 408 records data including various parameters necessary for the operation of the control device 4 and the endoscope 2. The recording unit 408 also includes a program recording unit 408a for recording various programs for operating the endoscope 2 and the control device 4, and an image data recording unit 408b for recording image data.

[0066] The image data recording unit 408b, under the control of the control unit 410, records the image data group generated by the endoscope 2 continuously imaging multiple observation sites of the subject, still image data, and the estimation results of characteristic regions including organs, body parts, polyps or cancers in the observation site in the examination image based on still image data (captured image data and examination images) estimated by the image diagnostic device 5, as well as the imaging time, in association with patient information, etc.

[0067] The communication unit 409, under the control of the control unit 410, transmits various information to the medical support device 6 via the image diagnostic device 5 and the network N100, and also receives various information from the image diagnostic device 5 and the medical support device 6 and outputs it to the control unit 410. Specifically, under the control of the control unit 410, the communication unit 409 transmits a series of temporally continuous image data processed by the image processing unit 406 to the image diagnostic device 5, and receives the estimation results, including organs, body parts, and characteristic regions estimated by the image diagnostic device 5, and outputs them to the control unit 410. The communication unit 41, under the control of the control unit 410, transmits the image data series, still image data, and examination information, including the estimation results from the image diagnostic device 5, to the medical support device 6. Here, the examination information includes the estimation results estimated by the image diagnostic device 5, examination images, and information including the time the examination images were captured. The communication unit 409 is configured using a communication module or the like.

[0068] The control unit 410 corresponds to a second processor according to this disclosure. This control unit 410 is implemented using a second processor, which is a processing device having hardware such as an FPGA or CPU, and memory, which is a temporary storage area used by the second processor. The control unit 410 comprehensively controls each part that constitutes the endoscope 2 and the control device 4. The control unit 410 includes an imaging control unit 410a, a display control unit 410b, and a communication control unit 410c.

[0069] The imaging control unit 410a controls the imaging of the endoscope 2. The imaging control unit 410a generates a series of temporally continuous image data by causing the endoscope 2 to continuously capture images. In addition, when a release signal is input from the operation unit 22, the imaging control unit 410a causes the image sensor 203 to capture still image data. Furthermore, the imaging control unit 410a acquires image data from the endoscope 2 via the S / P conversion unit 405 or the image processing unit 406 and records it in the image data recording unit 408b.

[0070] The display control unit 410b displays the observation image corresponding to the image data captured by the endoscope 2 on the display device 3.

[0071] The communication control unit 410e causes the communication unit 409 to transmit various information to the medical support device 6 and the image diagnostic device 5 via the network N100, and also causes the communication unit 409 to receive various information from the medical support device 6 and the image diagnostic device 5.

[0072] [Functional Configuration of the Medical Imaging Device] Next, the functional configuration of the medical imaging device 5 will be described. Figure 3 is a block diagram showing the functional configuration of the medical imaging device 5. The medical imaging device 5 shown in Figure 3 comprises a communication unit 51, an input unit 52, a recording unit 53, a display unit 54, and a control unit 55.

[0073] The communication unit 51, under the control of the control unit 55, receives a group of image data generated by the endoscope 2 from the control device 4 via the network N100 and outputs it to the control unit 55. The communication unit 51 also, under the control of the control unit 55, outputs examination information to the control device 4 via the network N100, including the detection results of the detected organs of the subject, and the detection time at which these results were detected. The communication unit 51 is configured using a communication module or the like.

[0074] The input unit 52 is configured using buttons, switches, and a touch panel, and receives various inputs and outputs them to the control unit 55.

[0075] The recording unit 53 is configured using recording media such as volatile memory, non-volatile memory, SSDs, HDDs, and memory cards. The recording unit 53 records data including various parameters necessary for the operation of the medical imaging device 5. The recording unit 53 also includes a program recording unit 531 for recording various programs for operating the medical imaging device 5, and an inference model recording unit 532.

[0076] The inference model recording unit 532 records an inference model used to detect organs, sites, and feature regions based on image data. Here, the inference model takes image data from the endoscope 2 as input and outputs organs, sites, tumors, non-tumors, inflammatory activity, invasive cancer, feature regions, lesion size, depth of invasion information, degree of invasion, equipment, and drugs contained in the image data and examination images. This inference model is a model generated by machine learning using artificial intelligence (AI), for example. Specifically, the inference model is a deep learning model that learns using training data in which the subject's organs, sites, tumors, non-tumors, inflammatory activity, invasive cancer, feature regions, lesion size, depth of invasion information, degree of invasion, equipment, and drugs are added to image data of a subject such as a patient, and outputs organs, sites, tumors, non-tumors, inflammatory activity, invasive cancer, feature regions, lesion size, depth of invasion information, degree of invasion, equipment, and drugs contained in the image data and examination images.

[0077] The display unit 54 displays various information related to the medical imaging device 5 under the control of the control unit 55. The display unit 54 is configured using a liquid crystal display, an organic EL display, or the like.

[0078] The control unit 55 is implemented using a processor having hardware such as an FPGA or CPU, and memory, which is a temporary storage area used by the processor. The control unit 55 comprehensively controls each part that constitutes the medical support device 6. The control unit 55 uses the inference model recorded by the inference model recording unit 532 to recognize and output organs, sites, tumors, non-tumors, inflammatory activity, invasive cancer, characteristic regions, lesion size, lesion diagnosis name, lesion depth information, degree of invasion, instruments used, and drugs from the examination image. The control unit 55 includes an acquisition unit 551, an organ / site estimation unit 552, a lesion estimation unit 533, a procedure estimation unit 554, and an instrument / drug estimation unit 555.

[0079] The acquisition unit 551 acquires image data or still image data that functions as an examination image, sequentially generated by the endoscope 2 from the control device 4 via the communication unit 51.

[0080] The organ / part estimation unit 552 detects the organs, parts, and characteristic regions of the subject contained in the image data and examination images, based on the image data acquired by the acquisition unit 551 and the inference model recorded by the inference model recording unit 532.

[0081] The lesion estimation unit 533 estimates and detects lesion information of the subject and diagnostic information of the lesion included in the image data and examination images, based on the image data acquired by the acquisition unit 551 and the inference model recorded by the inference model recording unit 532. Here, lesion information refers to one or more of the following: lesion size information regarding the size of the lesion, such as a tumor (characteristic region); depth of invasion information regarding the depth of the lesion; and invasion information regarding the degree of invasion of the lesion. Diagnostic information of the lesion refers to information that estimates the type of tumor, such as a malignant tumor.

[0082] The procedure estimation unit 554 estimates and detects the procedure performed on the subject included in the image data and examination images, based on the image data acquired by the acquisition unit 551 and the inference model recorded by the inference model recording unit 532.

[0083] The device / drug estimation unit 555 estimates and detects the type of device, the type of drug, and the quantity of drug for the subject contained in the image data and examination images, based on the image data acquired by the acquisition unit 551 and the inference model recorded by the inference model recording unit 532.

[0084] [Functional Configuration of the Medical Support Device] Next, the functional configuration of the medical support device 6 will be described. Figure 4 is a block diagram showing the functional configuration of the medical support device 6. The medical support device 6 shown in Figure 4 comprises a communication unit 61, a display unit 62, an input unit 63, a database 64 (hereinafter simply referred to as "DB64"), a program recording unit 65, and a control unit 66. In Embodiment 1 of this disclosure, the medical support device 6 functions as a control unit.

[0085] The communication unit 61, under the control of the control unit 66, transmits various information to the control device 4 and the image diagnostic device 5 via the network N100, and receives various information from the control device 4 and the image diagnostic device 5. The communication unit 61 is configured using a communication module.

[0086] The display unit 62 displays various information related to the medical support device 6 under the control of the control unit 66. The display unit 62 is constructed using a liquid crystal display, an organic EL display, or the like.

[0087] The input unit 63 receives various inputs in response to external operations and outputs them to the control unit 66. The input unit 63 is configured using buttons, a keyboard, switches, a touch panel, etc.

[0088] DB64 records various types of information. DB64 is configured using HDDs and SSDs, etc. DB64 includes patient information DB641, examination information DB642, procedure identification information DB643, procedure set information DB644, trained model DB645, procedure set confirmation information DB646, and lesion information DB647.

[0089] The patient information database 641 records, for each patient, the patient's identifying patient ID, date of birth, attending physician, address, and allergy information, all associated with each patient.

[0090] The examination information DB642 records examination information associated with each patient ID. Here, the examination information includes the examination image, diagnostic information of the lesion recognized by the diagnostic imaging device 5 in the examination image, equipment information, and lesion information. Here, the lesion information is at least one of the following: lesion size information regarding the size of the lesion, depth information regarding the depth of invasion of the lesion, and infiltration information regarding the degree of infiltration of the lesion.

[0091] The examination information DB642 records, for each patient ID, the examination image, organ recognition information related to the organs in the examination image obtained by the image diagnostic device 5, and the time of imaging, all associated with each patient ID.

[0092] The procedure-specific information database 643 records diagnostic information of the lesion and the equipment used for each procedure, with these details associated with each procedure.

[0093] The procedure set information DB644 records, for each procedure, the quantity of medication used on the lesion, the type of equipment used on the lesion, and information about the lesion, all linked together.

[0094] The trained model DB645 records an inference model used to estimate lesion information, diagnostic information, and instrument information based on lesion information, diagnostic information, and instrument information associated with examination images. Here, the inference model takes lesion information, diagnostic information, and instrument information as input and outputs procedure, diagnostic information, and instrument information. This inference model is constructed using the deep learning model described above.

[0095] The procedure set confirmation information DB646 records the procedure set performed at the time of entering the examination report, associated with each patient ID.

[0096] The lesion information database 647 records the procedure, lesion information, type of medication, and diagnosis for each lesion, associating them with each other.

[0097] The program recording unit 65 is configured using volatile memory, non-volatile memory, and an SSD, and records various programs for operating the medical support device 6.

[0098] The control unit 66 corresponds to the first processor according to this disclosure. This control unit 66 is implemented using a first processor having hardware such as an FPGA or CPU, and memory which is a temporary storage area used by the first processor. The control unit 66 comprehensively controls each part that constitutes the medical support device 6. The control unit 66 has an acquisition unit 661, a determination unit 662, an estimation unit 663, an adjustment unit 664, a display control unit 665, and a confirmation unit 666.

[0099] The acquisition unit 661 acquires inspection information from the control device 4 via the network N100 and records the acquired inspection information in the inspection information DB 642.

[0100] The determination unit 662 determines whether or not diagnostic information is included in the inspection information acquired by the acquisition unit 661.

[0101] The estimation unit 663 identifies the procedure performed by the user on the patient based on the learned model recorded in the learned model DB 645, the procedure identification information recorded in the procedure identification information DB 643, and the diagnostic information and equipment information included in the examination information acquired by the acquisition unit 661.

[0102] The adjustment unit 664 performs a procedure set information adjustment process to adjust the procedure set information acquired by the acquisition unit 661 based on the learned model recorded in the learned model DB 645 and the lesion information included in the examination information.

[0103] The display control unit 665 displays the adjustment candidates estimated by the adjustment unit 664 on the display unit 71 of the terminal device 7.

[0104] The confirmation unit 666 confirms the procedure set. Specifically, the confirmation unit 666 confirms the procedure set applied in step S405 described above, or the procedure set acquired by the acquisition unit 661, and records it in the procedure set confirmation information DB 646.

[0105] [Functional Configuration of Terminal Device] Next, the functional configuration of the terminal device 7 will be described. Figure 5 is a block diagram showing the functional configuration of the terminal device 7. The terminal device 7 comprises a display unit 71, an input unit 72, an output unit 73, a communication unit 74, a recording unit 75, and a control unit 76.

[0106] The display unit 71, under the control of the control unit 76, displays various information corresponding to various information related to the terminal device 7 and information received from the medical support device 6. The display unit 71 is constructed using a liquid crystal display or an organic EL display, etc.

[0107] The input unit 72 receives various inputs in response to external operations and outputs them to the control unit 76. The input unit 72 is configured using input interfaces such as buttons, switches, and touch panels.

[0108] The output unit 73 outputs various information under the control of the control unit 76. The output unit 73 is configured using, for example, a speaker.

[0109] The communication unit 74, under the control of the control unit 76, transmits various information to the medical support device 6 via the network N100 and receives various information from the medical support device 6. The communication unit 11 is configured using a communication module.

[0110] The recording unit 75 is composed of volatile memory, non-volatile memory, memory cards, and HDDs, SSDs, etc. The recording unit 75 records various programs executed by the terminal device 7 and data being processed.

[0111] The control unit 76 is implemented using a processor having hardware such as an FPGA, GPU, or CPU, and memory, which is a temporary storage area used by the processor. The control unit 76 controls each part that makes up the terminal device 7.

[0112] [Processing by the Medical Support Device] Next, we will explain the processing performed by the medical support device 6. Figure 6 is a flowchart showing an overview of the processing performed by the medical support device 6.

[0113] As shown in Figure 6, the acquisition unit 661 acquires examination information for a patient specified by the user from the examination information DB 642 (step S101).

[0114] Figure 7 shows an example of examination information acquired by the acquisition unit 661 from the examination information DB 642. The examination information D1 shown in Figure 7 includes the examination image taken when the patient was imaged, and the AI ​​judgment information acquired (recognized or detected) by the image diagnostic device 5 during the examination. Furthermore, the AI ​​judgment information includes diagnostic information, equipment information, and lesion size information. Specifically, the AI ​​judgment information contains the diagnostic information "high probability of malignant tumor", equipment information "snare", and lesion size information "2 cm".

[0115] Next, the medical support device 6 performs a procedure estimation process to estimate the procedure according to the contents of the examination information (step S102).

[0116] [Procedure Estimation Process] Figure 8 is a diagram showing the subflowchart of the procedure estimation process in step S102 of Figure 6.

[0117] As shown in Figure 8, first, the determination unit 662 determines whether or not diagnostic information is included in the examination information acquired by the acquisition unit 661 (step S201). Specifically, the determination unit 662 determines whether or not diagnostic information acquired by the image diagnostic device 5 during the examination is included in the examination information D1 (see Figure 7). If the determination unit 662 determines that diagnostic information is included in the examination information acquired by the acquisition unit 661 (step S201: Yes), the medical support device 6 proceeds to step S202, which will be described later. On the other hand, if the determination unit 662 determines that diagnostic information is not included in the examination information acquired by the acquisition unit 661 (step S201: No), the medical support device 6 proceeds to step S206, which will be described later.

[0118] In step S202, the determination unit 662 determines whether or not the examination information acquired by the acquisition unit 661 includes equipment information. Specifically, the determination unit 662 determines whether or not the examination information D1 (see Figure 7) includes equipment information acquired by the diagnostic imaging device 5 during the examination. If the determination unit 662 determines that the examination information acquired by the acquisition unit 661 includes equipment information (step S202: Yes), the medical support device 6 proceeds to step S203, which will be described later. On the other hand, if the determination unit 662 determines that the examination information acquired by the acquisition unit 661 does not include equipment information (step S202: No), the medical support device 6 proceeds to step S206, which will be described later.

[0119] In step S203, the estimation unit 663 identifies the procedure performed by the user on the patient based on the procedure identification information recorded in the procedure identification information DB 643 and the diagnostic information and equipment information included in the examination information acquired by the acquisition unit 661.

[0120] Figure 9 shows an example of a procedure identification information table in the procedure identification information DB 643. As shown in the procedure identification information table T1 in Figure 9, the estimation unit 663 estimates that the procedure is "EMR" and "polypectomy" because the diagnostic information included in the examination information D1 in Figure 7 is "high probability of malignant tumor" and the equipment information is "snare". In this case, the estimation unit 663 refers to the procedure identification information table T1 and estimates the selection reason according to the diagnostic information, equipment information and procedure.

[0121] Returning to Figure 8, we will continue the explanation from step S204 onward. In step S204, the display control unit 665 displays the estimation result estimated by the estimation unit 663 on the display unit 71 of the terminal device 7.

[0122] Figure 10 shows an example of a procedure selection screen displayed on the display unit 71 of the terminal device 7. As shown in Figure 10, the procedure selection screen P1 includes the reason for selecting each procedure name estimated by the estimation unit 663. Furthermore, the procedure selection screen P1 includes an application button B1 for applying a procedure and a procedure selection button B2 for the user to manually select and input a procedure. The user selects the desired procedure by operating the terminal device 7. For example, in the case shown in Figure 10, the user operates the terminal device 7 to select the application button B1 if the desired procedure is displayed on the procedure selection screen P1, and selects the procedure selection button B2 if the desired procedure is not displayed, and manually inputs the procedure.

[0123] Returning to Figure 8, we will continue the explanation from step S205 onwards. In step S205, the determination unit 662 determines whether the user has made a manual selection by pressing the procedure selection button B2. Specifically, the determination unit 662 determines whether it has received a selection signal from the terminal device 7 indicating that the user has made a manual selection by pressing the procedure selection button B2. If a selection signal is received, it determines that a manual selection has been made; on the other hand, if no selection signal is received, it determines that a manual selection has not been made. If the determination unit 662 determines that the user has made a manual selection by pressing the procedure selection button B2 (step S205: Yes), the medical support device 6 proceeds to step S206, which will be described later. Conversely, if the determination unit 662 determines that the user has not made a manual selection without pressing the procedure selection button B2 (step S205: No), the medical support device 6 proceeds to step S208, which will be described later.

[0124] In step S206, the display control unit 665 displays a procedure selection screen on the display unit 71 of the terminal device 7, which allows the selection of multiple procedures.

[0125] Next, the display control unit 665 displays the procedure corresponding to the selection result on the display unit 71 of the terminal device 7 in response to the user's operation from the terminal device 7 (step S207).

[0126] In step S208, the determination unit 662 determines whether the user has selected the apply button B1. If the determination unit 662 determines that the user has selected the apply button B1 (step S208: Yes), the confirmation unit 666 confirms the procedure (step S209). After step S209, the medical support device 6 returns to the main routine in Figure 6. Conversely, if the determination unit 662 determines that the user has not selected the apply button B1 (step S208: No), the medical support device 6 returns to step S204 described above.

[0127] Returning to Figure 6, we will continue the explanation from step S103 onwards. In step S103, the acquisition unit 661 executes a procedure set information acquisition process to acquire procedure set information from the procedure set information DB 644 based on the procedure information estimated by the estimation unit 663.

[0128] [Procedure Set Information Acquisition Process] Figure 11 is a diagram showing the subflowchart of the procedure set information acquisition process in step S103 of Figure 6.

[0129] As shown in Figure 11, the determination unit 662 determines whether the procedure of the examination information estimated by the estimation unit 663 is included in the procedure set information recorded in the procedure set information DB 644 (step S301).

[0130] Figure 12 shows an example of a procedure set information table in the procedure set information DB 644. In the procedure set information table T2 shown in Figure 12, the assumed size of the drug, equipment, and treatment area are associated with each procedure. Specifically, in the procedure set information table T2, the procedure "EMR" is associated with the drug "20 ml of physiological saline x 1", the equipment "high-frequency snare (SD-13U-1 A set)", and the assumed size of the treatment area "1 cm". In the case shown in Figure 12, the determination unit 662 determines that the procedure in the examination information estimated by the estimation unit 663 is "EMR" (see Figure 10), and therefore, procedure set information is present in the procedure set information table T2. If the determination unit 662 determines that the procedure of the examination information estimated by the estimation unit 663 is present in the procedure set information recorded in the procedure set information DB 644 (step S301: Yes), the medical support device 6 proceeds to step S302, which will be described later. Conversely, if the determination unit 662 determines that the procedure of the examination information estimated by the estimation unit 663 is not present in the procedure set information recorded in the procedure set information DB 644 (step S301: No), the medical support device 6 proceeds to step S305, which will be described later.

[0131] In step S302, the acquisition unit 661 acquires procedure set information corresponding to the procedure from the procedure set information DB 644. Specifically, as shown in Figure 12, the acquisition unit 661 acquires procedure set information corresponding to the procedure "EMR", the drug "20 ml of physiological saline x 1", the equipment "high-frequency snare (SD-13U-1 A set)", and the assumed size of the treatment site "1 cm" as procedure set information.

[0132] Next, the display control unit 665 displays the procedure set information acquired by the acquisition unit 661 on the display unit 71 of the terminal device 7 (step S303).

[0133] Figure 13 shows an example of a procedure set confirmation screen displayed on the display unit 71 of the terminal device 7. As shown in Figure 13, the procedure set confirmation screen P2 contains procedure set information acquired by the acquisition unit 661. Furthermore, the procedure set confirmation screen P2 includes an apply button B3 for applying a procedure set and a selection button B4 for the user to manually select and input a procedure set. The user operates the terminal device 7 and selects the apply button B3 if the procedure set is correct, or selects the selection button B4 and manually inputs the procedure set if the desired procedure set is not displayed.

[0134] Returning to Figure 11, we will continue the explanation from step S303 onwards. In step S304, the determination unit 662 determines whether or not the user has made a manual selection by pressing the selection button B4. Specifically, the determination unit 662 determines whether or not it has received a selection signal from the terminal device 7 indicating that the user has selected a manual procedure set by pressing the selection button B4. If a selection signal is received, it determines that a manual selection has been made; on the other hand, if no selection signal is received, it determines that a manual selection has not been made. If the determination unit 662 determines that the user has made a manual selection by pressing the selection button B4 (step S303: Yes), the medical support device 6 proceeds to step S305, which will be described later. Conversely, if the determination unit 662 determines that the user has not made a manual selection without pressing the selection button B4 (step S303: No), the medical support device 6 proceeds to step S307, which will be described later.

[0135] In step S305, the display control unit 665 displays a procedure selection screen on the display unit 71 of the terminal device 7, which allows the selection of multiple procedures.

[0136] Next, the display control unit 665 displays the procedure corresponding to the selection result on the display unit 71 of the terminal device 7 in response to the user's operation from the terminal device 7 (step S306).

[0137] Next, the determination unit 662 determines whether the user has selected the apply button B1 (step S307). If the determination unit 662 determines that the user has selected the apply button B1 (step S307: Yes), the confirmation unit 666 confirms the procedure set (step S308). After step S308, the medical support device 6 returns to the main routine in Figure 6. Conversely, if the determination unit 662 determines that the user has not selected the apply button B1 (step S307: No), the medical support device 6 returns to step S303 described above.

[0138] Returning to Figure 6, we will continue the explanation from step S104 onwards. In step S104, the adjustment unit 664 performs a procedure set information adjustment process, which adjusts the procedure set information acquired by the acquisition unit 661 based on the lesion information included in the examination information.

[0139] [Procedure Set Information Adjustment Process] Figure 14 is a diagram showing the subflowchart of the procedure set information adjustment process in step S104 of Figure 6.

[0140] As shown in Figure 14, the determination unit 662 determines whether or not the examination information contains size information of the lesion (step S401). If the determination unit 662 determines that the examination information contains size information of the lesion (step S401: Yes), the medical support device 6 proceeds to step S402, which will be described later. On the other hand, if the determination unit 662 determines that the examination information does not contain size information of the lesion (step S401: No), the medical support device 6 proceeds to step S407, which will be described later.

[0141] In step S402, the adjustment unit 664 estimates the type of equipment used in the procedure and the quantity of drugs based on the size information of the lesion included in the examination information and the learned model recorded by the learned model DB 645.

[0142] Figure 15 shows another example of a procedure set information table in the procedure set information DB 644. As shown in the procedure set information table T3 in Figure 15, the adjustment unit 664 estimates the drug to be "20 ml of physiological saline x 4" and the equipment to be "high-frequency snare (SD-11U-1 A set)" based on the state of the procedure set information confirmed by the confirmation unit 666, since the lesion size information included in the examination information is "2 cm". Specifically, the estimation unit 663 estimates that the lesion area is approximately four times larger because the lesion size is twice the expected size, and therefore adjusts the amount (quantity) of saline to four times that amount. Furthermore, the adjustment unit 664 initially used a high-frequency snare with a diameter of φ15 as the equipment for when the lesion size was assumed to be 1 cm, but adjusts it by estimating a high-frequency snare with a diameter of φ20 or larger because the lesion size is 2 cm.

[0143] Next, the determination unit 662 determines whether or not adjustment of the procedure set information is necessary, based on the registered procedure set information and the type of equipment and quantity of drugs for the procedure estimated by the adjustment unit 664 (step S403). If the determination unit 662 determines that adjustment of the procedure set information is necessary (step S403: Yes), the medical support device 6 proceeds to step S404, which will be described later. On the other hand, if the determination unit 662 determines that adjustment of the procedure set information is not necessary (step S403: No), the medical support device 6 proceeds to step S407, which will be described later.

[0144] In step S404, the display control unit 665 displays the adjustment candidates estimated by the adjustment unit 664 on the display unit 71 of the terminal device 7. Figure 16 is a diagram showing an example of the adjustment candidate screen displayed on the display unit 71 of the terminal device 7. The adjustment candidate screen P3 shown in Figure 16 includes the contents of the initial procedure set and the contents of the adjustment candidate procedure set estimated by the adjustment unit 664. Furthermore, the adjustment candidate screen P3 includes a change button B21 for changing the procedure and a procedure change button B22 for the user to manually input a change in the procedure. The user operates the terminal device 7 to select the desired procedure.

[0145] Next, the determination unit 662 determines whether the user has applied the procedure set adjusted by the adjustment unit 664 by pressing the change button B21 (step S405). Specifically, the determination unit 662 determines whether it has received a change signal from the terminal device 7 to change to the adjusted procedure set by the user pressing the change button B21. If a change signal is received, the determination unit 662 determines that the procedure set adjusted by the adjustment unit 664 has been applied. On the other hand, if a change signal is not received, the determination unit 662 determines that the procedure set adjusted by the adjustment unit 664 has not been applied. If the determination unit 662 determines that the procedure set adjusted by the adjustment unit 664 has been applied (step S405: Yes), the medical support device 6 proceeds to step S406, which will be described later. Conversely, if the determination unit 662 determines that the procedure set adjusted by the adjustment unit 664 has not been applied (step S405: No), the medical support device 6 proceeds to step S407, which will be described later.

[0146] In step S406, the confirmation unit 666 confirms the procedure set. Specifically, the confirmation unit 666 confirms the procedure set applied in step S405 described above, or the procedure set acquired by the acquisition unit 661, and records it in the procedure set confirmation information DB 646. After step S406, the medical support device 6 returns to the main routine shown in Figure 6.

[0147] In step S407, the display control unit 665 displays a manual change screen on the display unit 71 of the terminal device 7, which allows the equipment and medication to be changed according to the procedure.

[0148] Next, the determination unit 662 determines whether the user has made a manual change by operating the terminal device 7 to change the equipment or medication from the initial procedure set (step S408). If the determination unit 662 determines that the user has made a manual change by operating the terminal device 7 to change the equipment or medication from the initial procedure set (step S408: Yes), the medical support device 6 proceeds to step S409, which will be described later. On the other hand, if the determination unit 662 determines that the user has not made a manual change by operating the terminal device 7 to change the equipment or medication from the initial procedure set (step S408: No), the medical support device 6 returns to the main routine in Figure 6.

[0149] Returning to Figure 6, we will continue the explanation from step S105 onwards. In step S105, the display control unit 665 displays the execution input screen on the display unit 71 of the terminal device 7. Figure 17 is a diagram showing an example of the execution input screen displayed on the display unit 71 of the terminal device 7. As shown in Figure 17, the execution input screen P4 includes a display area H1 in which the procedure set after the procedure set information adjustment process described above is displayed, and a registration button B100 for registering the procedure set. The user can intuitively grasp the contents, quantities, etc. of each procedure, drug, and equipment displayed in the display area H1.

[0150] Next, the determination unit 662 determines whether the user has registered the procedure set displayed in the display area H1 of the procedure input screen P4 via the terminal device 7 (step S106). If the determination unit 662 determines that the user has registered the procedure set displayed on the procedure input screen P4 via the terminal device 7 (step S106: Yes), the medical support device 6 proceeds to step S107, which will be described later. On the other hand, if the determination unit 662 determines that the user has not registered the procedure set displayed on the procedure input screen P〇〇 via the terminal device 7 (step S106: No), the medical support device 6 returns to step S105 as described above.

[0151] In step S107, the confirmation unit 666 registers the procedure set in the procedure set information DB 644. After step S107, the medical support device 6 terminates this process.

[0152] According to Embodiment 1 described above, the adjustment unit 664 estimates the type of equipment used in the procedure and the quantity of drugs based on the size information of the lesion included in the examination information and the learned model recorded by the learned model DB 645. Therefore, it is possible to prevent omissions of the procedure details when inputting the examination report with simple operation.

[0153] Furthermore, according to Embodiment 1, the adjustment unit 664 estimates and adjusts the type of equipment and the quantity of drugs for the procedure based on the initial procedure set information determined by the confirmation unit 666. This prevents users from missing information during the procedure, thus enabling the creation of appropriate billing statements.

[0154] (Embodiment 2) Next, Embodiment 2 will be described. The medical support system according to Embodiment 2 has the same configuration as the medical support system according to Embodiment 1 described above, but the procedure set information adjustment process performed by the medical support device is different. In the following, the procedure set information adjustment process performed by the medical support device according to Embodiment 2 will be described. Note that the same reference numerals are used for components that are the same as those in the medical support system 1 according to Embodiment 1 described above, and detailed descriptions will be omitted.

[0155] [Procedure Set Information Adjustment Processing] Figure 18 is a diagram showing a subflowchart of the procedure set information adjustment processing performed by the medical support device 6 according to Embodiment 2.

[0156] In Figure 18, the determination unit 662 determines whether or not the examination information contains information on the depth of lesion invasion (step S501). Figure 19 shows another example of the examination information acquired by the acquisition unit 661. In Figure 19, the examination information D2 includes the examination image taken when the patient was imaged and the AI ​​determination information acquired (recognized or detected) by the image diagnostic device 5 during the examination. Furthermore, the AI ​​determination information includes diagnostic information, equipment information, and depth of invasion information. Specifically, the AI ​​determination information includes the diagnostic information "high probability of being a malignant tumor", equipment information "snare", and depth of invasion information "up to the lamina propria of the mucosa". If the determination unit 662 determines that the examination information contains information on the depth of lesion invasion (step S501: Yes), the medical support device 6 proceeds to step S502, which will be described later. In contrast, if the determination unit 662 determines that there is no information on the depth of lesion invasion in the examination information (step S501: No), the medical support device 6 proceeds to step S507, which will be described later.

[0157] In step S502, the adjustment unit 664 estimates the instruments and drugs to be used in the procedure based on the lesion depth information included in the examination information and the learned model recorded by the learned model DB 645.

[0158] Figure 20 shows another example of a procedure set information table in the procedure set information DB 644. As shown in the procedure set information table T4 in Figure 20, the adjustment unit 664 estimates the drug to be "20 ml of physiological saline x 4" and the equipment to be "high-frequency snare (SD-11U-1 A set)" based on the state of the procedure set information confirmed by the confirmation unit 666, because the lesion depth information included in the examination information is "up to the lamina propria of the mucosa". Specifically, the estimation unit 663 estimates that the lesion area is approximately four times larger because the lesion size is twice the assumed size, and therefore adjusts the amount of saline to four times the amount. Furthermore, the adjustment unit 664 initially used a high-frequency snare with a diameter of φ15 as the equipment for when the lesion size is assumed to be 1 cm, but adjusts it by estimating a high-frequency snare with a diameter of φ20 or larger because the lesion size is 2 cm.

[0159] Next, the determination unit 662 determines whether or not adjustment of the procedure set information is necessary, based on the registered procedure set information and the procedure equipment and drugs estimated by the adjustment unit 664 (step S503). If the determination unit 662 determines that adjustment of the procedure set information is necessary (step S503: Yes), the medical support device 6 proceeds to step S504, which will be described later. On the other hand, if the determination unit 662 determines that adjustment of the procedure set information is not necessary (step S503: No), the medical support device 6 proceeds to step S507, which will be described later.

[0160] In step S504, the display control unit 665 displays the adjustment candidates estimated by the adjustment unit 664 on the display unit 71 of the terminal device 7. Figure 21 is a diagram showing an example of the adjustment candidate screen displayed on the display unit 71 of the terminal device 7. The adjustment candidate screen P5 shown in Figure 21 includes the contents of the initial procedure set and the contents of the adjustment candidate procedure set estimated by the adjustment unit 664. Furthermore, the adjustment candidate screen P5 includes a change button B31 for changing the procedure and a procedure change button B32 for the user to manually input a change in the procedure. The user operates the terminal device 7 to select the desired procedure.

[0161] Steps S505 to S509 correspond to steps S405 to S409 in Figure 14, respectively, so a detailed explanation is omitted. After that, the medical support device 6 returns to Figure 6 after the procedure set information adjustment process in Figure 18 and proceeds to step S105 described above. In this case, the display control unit 665 displays the execution input screen on the display unit 71 of the terminal device 7. Figure 22 is a diagram showing an example of the execution input screen displayed on the display unit 71 of the terminal device 7. As shown in Figure 22, the execution input screen P6 includes a display area H2 in which the procedure set after the procedure set information adjustment process described above is displayed, and a registration button B100 for registering the procedure set. The user can intuitively grasp the contents, quantities, etc. of each procedure, drug, and equipment displayed in the display area H2.

[0162] According to Embodiment 2 described above, the same effects as Embodiment 1 are achieved, and it is possible to prevent omissions of the implementation details when inputting inspection reports with simple operation.

[0163] (Embodiment 3) Next, Embodiment 3 will be described. The medical support system according to Embodiment 3 has the same configuration as the medical support system according to Embodiment 1 described above, but the procedure set information adjustment process performed by the medical support device is different. In the following, the procedure set information adjustment process performed by the medical support device according to Embodiment 3 will be described. Note that the same reference numerals are used for components that are the same as those in the medical support system 1 according to Embodiment 1 described above, and detailed descriptions will be omitted.

[0164] [Procedure Set Information Adjustment Processing] Figure 23 is a diagram showing a subflowchart of the procedure set information adjustment processing performed by the medical support device 6 according to Embodiment 3.

[0165] In Figure 23, the determination unit 662 determines whether or not there is a degree of lesion infiltration in the examination information (step S601). If the determination unit 662 determines that there is a degree of lesion infiltration in the examination information (step S601: Yes), the medical support device 6 proceeds to step S602, which will be described later. On the other hand, if the determination unit 662 determines that there is no degree of lesion infiltration in the examination information (step S601: No), the medical support device 6 proceeds to step S607, which will be described later. Steps S602 to S609 correspond to steps S402 to S409 in Figure 14 described above, so a detailed explanation is omitted.

[0166] According to Embodiment 3 described above, the same effects as Embodiment 1 are achieved, and it is possible to prevent omissions of the implementation details when inputting inspection reports with simple operation.

[0167] (Other Embodiments) In embodiments 1 to 3 of this disclosure, the diagnostic imaging device 5 recognized (estimated) diagnostic information, equipment information, and lesion information from the examination image (image data). However, the invention is not limited to this, and the functions of the diagnostic imaging device 5 may be provided in the control unit 66 of the medical support device 6.

[0168] Furthermore, in embodiments 1 to 3 of the present disclosure described above, each DB was provided in the medical support device 6, but the invention is not limited to this, and is not limited to any other DB that can be connected via a network.

[0169] Various inventions can be formed by appropriately combining the multiple components disclosed in the medical support system according to Embodiments 1 to 3 of this disclosure described above. For example, some components may be removed from all the components described in the medical support system according to the embodiments of this disclosure described above. Furthermore, the components described in the medical support system according to the embodiments of this disclosure described above may be appropriately combined.

[0170] Furthermore, in the medical support systems according to embodiments 1 to 3 of the present disclosure described above, the terms "part" as used above can be replaced with "means" or "circuits," etc. For example, the control unit can be replaced with control means or control circuit.

[0171] Furthermore, the programs to be executed by the medical support systems according to Embodiments 1 to 3 of this disclosure described above are provided as installable or executable file data recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, DVD (Digital Versatile Disk), USB medium, or flash memory.

[0172] Furthermore, the programs to be executed by the medical support systems according to embodiments 1 to 3 of this disclosure described above may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0173] In this specification, while the flowchart descriptions use expressions such as "first," "then," and "followed by" to indicate the sequence of processes between steps, the order of processes necessary to carry out the present invention is not uniquely determined by these expressions. That is, the order of processes in the flowcharts described herein can be changed within a reasonable range. Furthermore, the program is not limited to simple branching processes; it may also branch by comprehensively evaluating a larger number of decision items.

[0174] Although some embodiments of this application have been described in detail above with reference to the drawings, these are illustrative examples, and the present invention can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure section of the present invention.

[0175] 1 Medical support system 2 Endoscope 3 Display device 4 Control device 5 Diagnostic imaging device 6 Medical support device 7 Terminal device 11, 41, 51, 61, 74 Communication unit 16, 55, 66, 76 Control unit 52, 63, 72 Input unit 53, 75 Recording unit 54, 62, 71 Display unit 64 Database 641 Patient information DB 642 Examination information DB 643 Procedure identification information DB 644 Procedure set information DB 645 Learned model DB 646 Procedure set confirmation information DB 647 Lesion information DB 65 Program recording unit 661 Acquisition unit 662 Judgment unit 663 Estimation unit 664 Adjustment unit 665 Display control unit 666 Confirmation unit

Claims

1. A control device comprising a processor, wherein the processor acquires examination information including examination images of a patient, diagnostic information of lesions recognized in the examination images, equipment information, and lesion information; estimates the procedure of the examination information based on the examination information and a procedure identification information database that associates diagnostic information of lesions and equipment used during the examination for each procedure; acquires an initial procedure set from the procedure set information database that combines the procedure performed on the patient, the quantity of medication, and the type of equipment based on the estimated procedure and a procedure set information database that associates medication, equipment expected to be used, and lesion information of the treatment site of the lesion for each procedure; adjusts at least one of the quantity of medication and the type of equipment in the initial procedure set to correspond to the lesion information; and displays the adjusted procedure set on a display unit.

2. A control device according to claim 1, wherein the lesion information is lesion size information relating to the size of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion size information.

3. A control device according to claim 1, wherein the lesion information is depth information relating to the depth of invasion of the lesion, and the processor adjusts at least one of the quantity of drug and the type of equipment in the initial procedure set to correspond to the depth information.

4. A control device according to claim 1, wherein the lesion information is infiltration information relating to the degree of infiltration of a lesion, and the processor adjusts at least one of the quantity of drug and the type of equipment in the initial procedure set to correspond to the infiltration information.

5. A control device according to claim 1, wherein the processor determines whether the examination information includes the diagnostic information and the equipment information, and if the diagnostic information and the equipment information are included, the control device estimates the procedure of the examination information.

6. A control device according to claim 1, wherein the processor displays the initial procedure set and the adjusted procedure set on the display unit, and when it receives a selection signal to apply the adjusted procedure set, it confirms the adjusted procedure set as the procedure set for the patient.

7. A control device according to claim 1, wherein the control device is a medical support device.

8. A medical support system comprising: a control device equipped with a processor; and a terminal device capable of receiving input from a surgeon, wherein the processor acquires examination information including examination images of a patient, diagnostic information of lesions recognized in the examination images, equipment information, and lesion information; estimates the procedure of the examination information based on the examination information and a procedure identification information database that associates diagnostic information of lesions and equipment used during the examination for each procedure; acquires an initial procedure set from the procedure set information database that combines the procedure performed on the patient, the quantity of drugs, and the type of equipment based on the estimated procedure and a procedure set information database that associates drugs, equipment expected to be used, and lesion information of the treatment site of the lesion for each procedure; adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion information; and displays the adjusted procedure set on the display unit of the terminal device.

9. A medical support system according to claim 8, wherein the lesion information is lesion size information relating to the size of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion size information.

10. A medical support system according to claim 8, wherein the lesion information is depth information relating to the depth of invasion of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the depth information.

11. A medical support system according to claim 8, wherein the lesion information is infiltration information relating to the degree of infiltration of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the infiltration information.

12. A medical support system according to claim 8, wherein the processor determines whether the examination information includes the diagnostic information and the equipment information, and if the diagnostic information and the equipment information are included, estimates the procedure of the examination information.

13. A medical support system according to claim 8, wherein the processor displays the initial procedure set and the adjusted procedure set on the display unit, and when it receives a selection signal to apply the adjusted procedure set, it confirms the adjusted procedure set as the procedure set for the patient.

14. A medical support system according to claim 8, wherein the control device is a medical support device.

15. A medical support method performed by a control device equipped with a processor, wherein the processor acquires examination information including examination images of a patient, diagnostic information of lesions recognized in the examination images, equipment information, and lesion information; estimates the procedure of the examination information based on the examination information and a procedure identification information database that associates diagnostic information of lesions and equipment used during the examination for each procedure; acquires an initial procedure set from the procedure set information database that combines the procedure performed on the patient, the quantity of medication, and the type of equipment based on the estimated procedure and a procedure set information database that associates medication, equipment expected to be used, and lesion information of the treatment site of the lesion for each procedure; adjusts at least one of the quantity of medication and the type of equipment in the initial procedure set to correspond to the lesion information; and displays the adjusted procedure set on a display unit.

16. A medical support method according to claim 15, wherein the lesion information is lesion size information relating to the size of the lesion, and the processor adjusts at least one of the quantity of drugs and the type of equipment in the initial procedure set to correspond to the lesion size information.

17. A medical support method according to claim 15, wherein the lesion information is depth information relating to the depth of invasion of the lesion, and the processor adjusts at least one of the quantity of drug and the type of equipment in the initial procedure set to correspond to the depth information.

18. A medical support method according to claim 15, wherein the lesion information is infiltration information relating to the degree of infiltration of the lesion, and the processor adjusts at least one of the quantity of drug and the type of equipment in the initial procedure set to correspond to the infiltration information.

19. A medical support method according to claim 15, wherein the processor determines whether the examination information includes the diagnostic information and the equipment information, and if the diagnostic information and the equipment information are included, estimates the procedure of the examination information.

20. A medical support method according to claim 15, wherein the processor displays the initial procedure set and the adjusted procedure set on the display unit, and when it receives a selection signal to apply the adjusted procedure set, it confirms the adjusted procedure set as the procedure set for the patient.

Images