Medical support device, medical support system, and operating method of medical support device

Abstract

Provided are a medical assistance device, a medical assistance system, and a program that make it possible to reduce time and effort related to creation of an endoscopy report. This medical assistance device is provided with a processor. The processor: acquires an image data group generated by continuously imaging, with an endoscope, a plurality of observation areas with respect to a subject and examination information including detection result information generated by detecting each of the plurality of observation areas on the basis of the image data group; distinguishes, on the basis of the image data group and the examination information, first image data on images captured during the observation of a predetermined observation area and a second image data group on images captured after the observation of the predetermined observation area; and generates related information, which is related to the movement of the endoscope inside the body of the subject in the second image data group, on the basis of the distinguishment result obtained by distinguishing the first image data and the second image data group, and the examination information.

Description

Medical support device, medical support system and program

[0001] The present disclosure relates to a medical support device, a medical support system, and a program that provide medical support including an endoscopic examination report.

[0002] A conventional technique has been disclosed for generating a tip marker array image in which a plurality of tip markers representing the position and angle of the tip of the insertion section of the endoscope at each predetermined time are arranged in a row based on positional information and angle information of the tip of the insertion section of the endoscope acquired at each predetermined time (see, for example, Patent Document 1). This technique reduces the effort and time required to create an endoscopic report by displaying the tip marker array image at each predetermined time superimposed on a position corresponding to a schematic diagram of the target organ.

[0003] JP 2017-47009 A

[0004] However, when a surgeon such as a doctor prepares an endoscopic report after examining a subject, it is difficult for the surgeon to accurately recall the contents of the examination when preparing the endoscopic report, and it takes a great deal of time and effort to identify the location of the target organ.

[0005] However, the above-mentioned Patent Document 1 does not guarantee the quality of the examination for each part of the target organ, and since the surgeon must determine the organ and the examination content after the examination, it is difficult to reduce the effort and time required to create an endoscopic report.

[0006] The present disclosure has been made in consideration of the above, and aims to provide a medical support device, a medical support system, and a program that can reduce the effort and time required to create an endoscopic report.

[0007] In order to solve the above-mentioned problems and achieve the objective, the medical support device of the present disclosure is a medical support device equipped with a processor, wherein the processor acquires an image data group generated by an endoscope successively capturing images of multiple observation sites on a subject, and examination information including detection result information detecting each of the multiple observation sites based on the image data group, and distinguishes between first image data captured during observation of a predetermined observation site and a second image data group captured after observation of the predetermined observation site based on the image data group and the examination information, and generates associated information related to the movement of the endoscope within the subject in the second image data group based on the examination information and the discrimination result between the first image data and the second image data group.

[0008] In the medical support device according to the present disclosure, in the above disclosure, the related information is time information when the endoscope captured the second image data group.

[0009] In addition, in the medical support device according to the present disclosure, in the above disclosure, the time information includes an insertion time during which the tip of the insertion section of the endoscope is inserted from outside the body of the subject to a predetermined observation site, and an observation time during which the endoscope observes the inside of the subject.

[0010] In addition, in the medical support device according to the present disclosure, in the above disclosure, the examination information further includes speed information that associates insertion speed information regarding the insertion speed of the tip moving inside the subject based on the image data group with a measurement time from a start time when detection of the insertion speed began to an end time when detection of the insertion speed ended.

[0011] Furthermore, in the medical support device according to the present disclosure, in the above disclosure, the examination information includes still image data captured by the endoscope when a release signal is input from an operating member provided on the endoscope and the image capture time when the still image data was captured, the detection result information includes the type of organ or part and the detection time when the type of organ or part was detected, and the processor determines whether the tip portion has reached the final reach site set in the observation order based on observation order information in which a predetermined order of observation of organs and parts is set and the examination information, and generates the related information if it is determined that the tip portion has reached the final reach site.

[0012] In addition, in the medical support device disclosed herein, in the above disclosure, the detection result information includes the type of organ or part and the detection time when the type of organ or part was detected, and the processor distinguishes between the first image data and the second image data group by identifying the arrival time when the tip portion reached the final arrival part based on the detection time.

[0013] In addition, in the medical support device disclosed herein, in the above disclosure, the processor determines the arrival time at which the tip reaches the final arrival site based on the detection result information and the imaging time linked to the still image data.

[0014] In addition, in the medical support device disclosed herein, in the above disclosure, the processor, based on the detection result information, the imaging time linked to the still image data, and the speed information, identifies the measurement time at which the imaging time linked to the still image data is within the judgment time of the final arrival location included in the detection result information and the speed of the tip portion becomes less than a predetermined threshold as the arrival time at which the tip portion reaches the final arrival location.

[0015] In the medical support device according to the present disclosure, in the above disclosure, the processor calculates the insertion time based on the arrival time and an insertion start time of the subject based on the examination information.

[0016] In addition, in the medical support device of the present disclosure, when the processor determines that the tip has not reached the final destination, it generates the related information that corresponds to stenosis information indicating that stenosis has occurred at the most recent observation location included in the detection result information.

[0017] In addition, in the medical support device according to the present disclosure, the processor creates report information including the related information.

[0018] Further, a medical support system according to the present disclosure is a medical support system including an endoscope, a control device for controlling the endoscope, an imaging diagnostic device, and a medical support device, wherein the endoscope includes an insertion section that can be inserted into a subject, and a tip section that is provided at the tip of the insertion section and that sequentially generates image data groups by continuously capturing images of a plurality of observation sites that have been set in advance for the subject, and outputs the image data groups to the control device, and the imaging diagnostic device generates detection result information that detects each of the plurality of observation sites for the image data group using the image data groups and a first inference model that has learned characteristics of organs and sites in advance, and outputs the detection result information to the control device. The medical support device includes a first processor, which acquires the image data group and examination information including the detection result information generated by the image diagnostic device from the control device, distinguishes between first image data captured during observation of a predetermined observation site and a second image data group captured after observation of the predetermined observation site based on the image data group and the examination information, and generates associated information related to the movement of the endoscope within the subject in the second image data group based on the examination information and a discrimination result between the first image data and the second image data group.

[0019] In addition, in the medical support system of the present disclosure, in the above disclosure, the control device is provided with a second processor, and the second processor determines whether or not the tip portion has reached a predetermined final destination location of the subject based on the detection result information, and if it determines that the tip portion has reached the final destination location, outputs that the tip portion has reached the final destination location.

[0020] In addition, in the medical support system according to the present disclosure, in the above disclosure, the endoscope further includes an operating member that accepts input of a release signal that instructs imaging, and the second processor, after determining that the tip portion has reached the final reach site, outputs a warning that the final reach site has not yet been imaged if the imaging diagnostic device detects the next site to be imaged without inputting a release signal from the operating member.

[0021] In addition, in the medical support system according to the present disclosure, in the above disclosure, the second processor generates and records examination information that corresponds the detection result information, still image data captured by the endoscope when a release signal is input from the operating member, and the imaging time at which the still image data was captured.

[0022] Furthermore, in the medical support system according to the present disclosure, in the above disclosure, when the second processor determines that the tip portion has not reached the final destination site and a release signal is input from the operating member, it adds stenosis information to the examination information, indicating that stenosis has occurred in the organ or site currently detected by the imaging diagnostic device, and records the stenosis information.

[0023] Furthermore, in the medical support system according to the present disclosure, in the above disclosure, the imaging diagnostic device uses the image data group and a second inference model that learns the insertion speed of the tip portion using the image data group at the time of insertion of the tip portion into the subject, and outputs to the control device speed information that associates insertion speed information regarding the insertion speed of the tip portion moving inside the subject with a measurement time from a start time when detection of the insertion speed began to an end time when detection of the insertion speed ended.

[0024] In the medical support system according to the present disclosure, in the above disclosure, the related information is time information when the endoscope captured the second image data group.

[0025] In addition, in the medical support system according to the present disclosure, in the above disclosure, the time information includes an insertion time during which the tip of the insertion section of the endoscope is inserted from outside the body of the subject to the predetermined observation site, and an observation time during which the endoscope observes the inside of the subject.

[0026] In addition, a program according to the present disclosure is a program executed by a medical support device having a processor, and causes the processor to acquire an image data group generated by an endoscope successively capturing images of multiple observation sites on a subject, and examination information including detection result information for detecting each of the multiple observation sites based on the image data group; distinguish, based on the image data group and the examination information, between first image data captured during observation of a predetermined observation site and a second image data group captured after observation of the predetermined observation site; and generate associated information relating to the movement of the endoscope within the subject in the second image data group based on the examination information and a discrimination result between the first image data and the second image data group.

[0027] The present disclosure has the effect of reducing the effort and time required to create an endoscopic report.

[0028] FIG. 1 is a diagram showing the overall configuration of a medical support system according to an embodiment. FIG. 2 is a block diagram showing the functional configuration of essential parts of an endoscope and a control device according to an embodiment. FIG. 3 is a block diagram showing the functional configuration of an imaging diagnostic apparatus according to an embodiment. FIG. 4 is a block diagram showing the functional configuration of a medical support device according to an embodiment. FIG. 5 is a flowchart showing an outline of processing executed by a medical support system according to an embodiment. FIG. 6 is a diagram showing an example of an observation order of organs and regions and a list of report items. FIG. 7 is a diagram showing another example of an observation order of organs and regions and a list of report items. FIG. 8 is a flowchart showing an outline of endoscopic imaging processing in FIG. 5. FIG. 9 is a flowchart showing an outline of report creation processing in FIG. 5. FIG. 10 is a diagram schematically showing an outline of when a discrimination unit included in a medical support device according to an embodiment discriminates an image data group into first image data and a second image data group. FIG. 11 is a diagram showing an example of examination information acquired by an acquisition unit included in a medical support device according to an embodiment. FIG. 12 is a diagram schematically explaining a discrimination method used by a discrimination unit included in a medical support device according to an embodiment. FIG. 13 is a diagram showing another example of test information acquired by an acquisition unit included in a medical support device according to an embodiment. FIG. 14 is a diagram schematically explaining a determination method performed by a determination unit included in a medical support device according to an embodiment. FIG. 15 is a diagram showing another example of test information acquired by an acquisition unit included in a medical support device according to an embodiment. FIG. 16 is a diagram schematically explaining a determination method performed by a determination unit included in a medical support device according to an embodiment. FIG. 17 is a diagram schematically explaining a calculation method for insertion time calculated by a calculation unit included in a medical support device according to an embodiment. FIG. 18 is a diagram showing an example of an examination report created by a report creation unit included in a medical support device according to an embodiment. FIG. 19 is a flowchart showing an overview of the unexamined process in FIG. 5.

[0029] Hereinafter, embodiments for carrying out the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the following embodiments. Furthermore, each drawing referred to in the following description merely shows a schematic representation of the shape, size, and positional relationship to the extent that the contents of the present disclosure can be understood. In other words, the present disclosure is not limited to only the shape, size, and positional relationship exemplified in each drawing. Furthermore, in the description of the drawings, the same parts are denoted by the same reference numerals.

[0030] [Overall Configuration of Medical Support System] Fig. 1 is a diagram showing the overall configuration of a medical support system according to one embodiment. The medical support system 1 shown in Fig. 1 continuously captures images of the inside of a subject, such as a human or animal, by inserting an insertion section 21 of an endoscope 2 from the mouth to the esophagus or from the anus to the large intestine of the subject, and displays a group of captured image data in chronological order on a display device 3.

[0031] Furthermore, in the medical support system 1, the image diagnostic device 5 (CAD: Computer Aided Detection) detects (recognizes or estimates) characteristic regions including organs, parts, the insertion speed of the insertion portion 21, and polyps or cancers at multiple observation sites shown in the image data, based on a group of image data input from the control device 4 and an inference model (trained model) trained using training data obtained by previously capturing images of the subject's organs, parts, lesions, etc., and outputs the detection results to the control device 4. An operator such as a doctor observes and treats the subject while checking the display image displayed on the display device 3 and the detection results (estimation results) of the image diagnostic device 5.

[0032] Furthermore, the medical support system 1 acquires image data groups input from the control device 4 via the network N100 to the medical support device 6, and examination information including the detection results (estimated results) of the image diagnostic device 5, and records the acquired image data and examination information in association with each other. The medical support system 1 then outputs various data in response to a request from a laptop computer 7 of an operator or the like via the network N100.

[0033] Furthermore, in the medical support system 1, report information such as an examination report of a subject by an operator or the like is created on the laptop computer 7, and this report information is output and recorded on the medical support device 6. Thereafter, the operator diagnoses the subject and conducts a conference or the like based on the report information and the diagnostic results of living cells obtained by biopsy during observation of the subject.

[0034] As shown in FIG. 1, the medical support system 1 includes an endoscope 2 , a display device 3 , a control device 4 , an image diagnostic device 5 , and a medical support device 6 .

[0035] [Configuration of Endoscope] The endoscope 2 continuously generates image data (RAW data) by capturing images of the inside of the subject, and sequentially outputs this image data to the control device 4. As shown in FIG. 1 , the endoscope 2 includes an insertion section 21, an operation section 22, and a universal cord 23.

[0036] 1, the insertion section 21 includes a distal end portion 24 provided at the distal end of the insertion section 21, a bending portion 25 connected to the proximal end side (the operation section 22 side) of the distal end portion 24 and configured to be bendable, and a flexible tube portion 26 connected to the proximal end side of the bending portion 25 and having flexibility.

[0037] The operation unit 22 is connected to the proximal end portion of the insertion unit 21. The operation unit 22 accepts various operations on the endoscope 2. As shown in FIG. 1 , the operation unit 22 is provided with a bending knob 221, an insertion port 222, and a plurality of operation members 223.

[0038] The bending knob 221 is configured to be rotatable in response to a user operation by a surgeon or other user. The bending knob 221, when rotated, operates a bending mechanism (not shown) such as a metal or resin wire disposed inside the insertion section 21. As a result, the bending section 25 is bent.

[0039] The insertion port 222 is connected to a treatment instrument channel (not shown), which is a duct extending from the tip of the insertion portion 21, and is an insertion port for inserting a treatment instrument or the like into the treatment instrument channel from outside the endoscope 2.

[0040] The multiple operating members 223 are composed of buttons and the like that accept various operations by users such as surgeons, and output operation signals corresponding to the various operations to the control device 4 via the universal cord 23. Examples of the various operations include a release operation that commands a still image to be captured by the endoscope 2, and an operation that switches the observation mode of the endoscope 2 between a normal light observation mode and a special observation mode.

[0041] The universal cord 23 extends from the operation unit 22 in a direction different from the extending direction of the insertion unit 21, and is a cord on which a light guide 231 (see FIG. 2) constituted by an optical fiber or the like, a first signal line 232 (see FIG. 2) for transmitting the image data described above, and a second signal line 233 (see FIG. 2) for transmitting the operation signal described above, etc. A first connector 27 is provided at the base end of the universal cord 23, as shown in FIG. 1. The first connector 27 is detachably connected to the control device 4.

[0042] The display device 3 is composed of a display monitor such as a liquid crystal or organic EL (Electro Luminescence) display monitor, and under the control of the control device 4, displays a display image based on image data that has been image processed in the control device 4, as well as various information related to the endoscope 2.

[0043] The control device 4 is realized using a processor, which is a processing device having hardware such as a GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), or CPU (Central Processing Unit), and a memory, which is a temporary storage area used by the processor. The control device 4 comprehensively controls the operations of each part of the endoscope 2 in accordance with a program recorded in the memory.

[0044] The diagnostic imaging device 5 (CAD) uses the image data group input from the control device 4 and a trained model trained in advance using training data to detect (estimate) characteristic regions (abnormal regions or lesion candidate regions) including organs, parts, the insertion speed of the insertion portion 21, and polyps or cancers, etc., that appear in the image data, and outputs examination information that associates the detection results with the detection times to the control device 4. The diagnostic imaging device 5 is realized using a processor that is a processing device having hardware such as a GPU, FPGA, or CPU, and a memory that is a temporary storage area used by the processor.

[0045] The medical support device 6 sequentially records various data input from the control device 4 and the recognition results of the image diagnostic device 5, and transmits data in response to a request from the laptop computer 7. The medical support device 6 is realized using a processor, which is a processing device having hardware such as a GPU, FPGA, or CPU, and memory, which is a temporary storage area used by the processor. Furthermore, the medical support device 6 is configured using a hard disk drive (HDD), a solid state drive (SSD), etc.

[0046] [Functional Configuration of Main Parts of Medical Support System] Next, a description will be given of the functional configuration of the main parts of the above-mentioned medical support system 1. Fig. 2 is a block diagram showing the functional configuration of the main parts of the endoscope 2 and the control device 4. Below, the endoscope 2 and the control device 4 will be described in that order, followed by the image diagnostic device 5 and the medical support device 6.

[0047] [Functional Configuration of Endoscope] First, a description will be given of the configuration of the endoscope 2. As shown in Fig. 2, the endoscope 2 includes an illumination optical system 201, an imaging optical system 202, an imaging element 203, an A / D conversion unit 204, a P / S conversion unit 205, an imaging and recording unit 206, and an imaging control unit 207. Here, the illumination optical system 201, the imaging optical system 202, the imaging element 203, the A / D conversion unit 204, the P / S conversion unit 205, the imaging and recording unit 206, and the imaging control unit 207 are each disposed within the tip portion 24.

[0048] The illumination optical system 201 is composed of one or more lenses and irradiates the subject with illumination light supplied from a light guide 231 .

[0049] The imaging optical system 202 is composed of one or more lenses, and focuses light such as reflected light from a subject, returned light from the subject, and fluorescent light emitted by the subject, to form an image of the subject on the light receiving surface of the image sensor 203. The imaging optical system 202 is also capable of changing the focal length and angle of view by moving the lens along the optical axis direction O1.

[0050] The image sensor 203 is configured using a charge-coupled device (CCD) or complementary metal oxide semiconductor (CMOS) image sensor, in which one of color filters forming a Bayer array (RGGB) is arranged on each of a plurality of pixels arranged in a two-dimensional matrix. Under the control of the image capture control unit 207, the image sensor 203 receives the subject image formed by the image capture optical system 202 and performs photoelectric conversion to generate a captured image (analog signal). In this embodiment, the image sensor 203 may be configured integrally with an image sensor and a time-of-flight (TOF) sensor that acquires subject distance information (hereinafter referred to as depth map information) using a TOF method. The depth map information is information that detects the subject distance for each pixel position from the position of the image sensor 203 (the position of the tip 24) to a corresponding position on the observation target that corresponds to the pixel position in the captured image. The configuration for generating the depth map information is not limited to the TOF sensor described above, and an image sensor including a phase difference sensor may also be used. Hereinafter, the depth map information and the captured image will be collectively referred to as “image data.” The image sensor 203 outputs the image data to the A / D converter 204.

[0051] The A / D conversion unit 204 is configured using an A / D conversion circuit, etc. Under the control of the imaging control unit 207, the A / D conversion unit 204 performs A / D conversion processing on the analog image data input from the imaging element 203 and outputs the result to the P / S conversion unit 205.

[0052] The P / S conversion unit 205 is configured using a P / S conversion circuit or the like, and, under the control of the imaging control unit 207, performs parallel / serial conversion on the digital image data input from the A / D conversion unit 204 and outputs the data to the control device 4 via a first signal line 232. Note that instead of the P / S conversion unit 205, an E / O conversion unit that converts image data into an optical signal may be provided, and the image data may be output to the control device 4 via the optical signal. Alternatively, the image data may be transmitted to the control device 4 via wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

[0053] The imaging and recording unit 206 is configured with a non-volatile memory or a volatile memory, and records various information (e.g., pixel information of the imaging element 203) related to the endoscope 2. The imaging and recording unit 206 also records various setting data and control parameters transmitted from the control device 4 via a second signal line 233.

[0054] The imaging control unit 207 is realized using a TG (Timing Generator), a processor which is a processing device having hardware such as a CPU, and a memory which is a temporary storage area used by the processor. The imaging control unit 207 controls the operations of the imaging element 203, the A / D conversion unit 204, and the P / S conversion unit 205 based on setting data received from the control device 4 via a second signal line 233.

[0055] [Configuration of Control Device] Next, a description will be given of the configuration of the control device 4. As shown in Fig. 2 , the control device 4 includes a condenser lens 401, a first light source unit 402, a second light source unit 403, a light source control unit 404, an S / P conversion unit 405, an image processing unit 406, an input unit 407, a recording unit 408, a communication unit 409, and a control unit 410.

[0056] The condenser lens 401 condenses the light emitted by each of the first light source unit 402 and the second light source unit 403 and outputs the condensed light to the light guide 231 .

[0057] The first light source unit 402 emits visible white light (normal light) under the control of the light source control unit 404, and supplies the white light as illumination light to the light guide 231. The first light source unit 402 is configured using a collimator lens, a white LED (Light Emitting Diode) lamp, a driver, and the like. Note that the first light source unit 402 may supply visible white light by simultaneously emitting light from a red LED lamp, a green LED lamp, and a blue LED lamp. The first light source unit 402 may also be configured using a halogen lamp, a xenon lamp, or the like.

[0058] The second light source unit 403, under the control of the light source control unit 404, emits special light having a predetermined wavelength band and supplies the special light as illumination light to the light guide 231. Here, the special light is light used for narrow band imaging (NBI) using narrow band light including 390 to 445 nm and 530 to 550 nm. Of course, the special light may be amber light (600 nm and 630 nm) used for red dichromatic imaging (RDI) in addition to narrow band light.

[0059] The light source control unit 404 is realized by using a processor having hardware such as an FPGA or a CPU, and a memory that is a temporary storage area used by the processor. The light source control unit 404 controls the light emission timing and light emission duration of each of the first light source unit 402 and the second light source unit 403 based on control data input from the control unit 410.

[0060] Under the control of the control unit 410, the S / P conversion unit 405 performs serial / parallel conversion on the image data received from the endoscope 2 via the first signal line 232 and outputs the converted data to the image processing unit 406. Note that if the endoscope 2 outputs image data as an optical signal, an O / E conversion unit that converts an optical signal into an electrical signal may be provided instead of the S / P conversion unit 405. Also, if the endoscope 2 transmits image data via wireless communication, a communication module capable of receiving wireless signals may be provided instead of the S / P conversion unit 405.

[0061] The image processing unit 406 is realized using a processor having hardware such as a GPU or FPGA, and a memory that is a temporary storage area used by the processor. Under the control of the control unit 410, the image processing unit 406 performs predetermined image processing on the image data of parallel data input from the S / P conversion unit 405, and outputs the result to the display device 3. Examples of the predetermined image processing include demosaic processing, white balance processing, gain adjustment processing, gamma correction processing, and format conversion processing.

[0062] The input unit 407 is configured using a mouse, foot switch, keyboard, buttons, switches, a touch panel, etc., and accepts user operations by a user such as a surgeon, and outputs an operation signal corresponding to the user operation to the control unit 410.

[0063] The recording unit 408 is configured using a recording medium such as a volatile memory, a non-volatile memory, an SSD (Solid State Drive), an HDD (Hard Disk Drive), a memory card, etc. The recording unit 408 records data including various parameters necessary for the operation of the control device 4 and the endoscope 2. The recording unit 408 also has a program recording unit 408a that records various programs for operating the endoscope 2 and the control device 4, an image data recording unit 408b that records image data, and an examination information recording unit 408c that records examination information.

[0064] Under the control of the control unit 410, the image data recording unit 408b records the image data groups and still image data generated by the endoscope 2 continuously capturing images of multiple observation areas of the subject, in association with patient information, etc.

[0065] The examination information recording unit 408c records examination information. The examination information includes detection result information input from the diagnostic imaging device 5, the detection time of the detection result, still image data, the imaging time at which the still image data was captured, insertion speed information regarding the insertion speed of the distal end portion 24 of the endoscope 2 and speed information correlating the measurement time at which the insertion speed of the distal end portion 24 of the endoscope 2 was measured, and patient information. The still image data is image data captured by the imaging element 203 when imaging is instructed by a release signal input from the operation unit 22, and has a higher resolution than the live view images sequentially generated by the imaging element 203. The insertion speed information is a binary value representing whether the movement of the distal end portion 24 is “fast” or “slow” compared to a predetermined threshold. Of course, the information does not need to be binary, and multiple values ​​may be provided to represent multiple levels, such as “fast,” “normal,” and “slow.”

[0066] Under the control of the control unit 410, the communication unit 409 transmits various types of information to the imaging diagnostic apparatus 5 and the medical support apparatus 6 via the network N100, and receives various types of information from the imaging diagnostic apparatus 5 and the medical support apparatus 6 and outputs it to the control unit 410. Specifically, under the control of the control unit 410, the communication unit 409 transmits a group of temporally consecutive image data that has been subjected to image processing by the image processing unit 406 to the imaging diagnostic apparatus 5, receives detection results including organs, parts, special regions, the insertion speed of the endoscope 2, etc. detected by the imaging diagnostic apparatus 5, and outputs them to the control unit 410. Furthermore, under the control of the control unit 410, the communication unit 41 transmits examination information including the image data group, still image data, and detection results of the imaging diagnostic apparatus 5 to the medical support apparatus 6, and receives patient information on the subject and observation information indicating the observation order of observation locations during the examination of the subject from the medical support apparatus 6. The communication unit 409 is configured using a communication module, etc.

[0067] The control unit 410 corresponds to the second processor according to the present disclosure. This control unit 410 is realized using the second processor, which is a processing device having hardware such as an FPGA or a CPU, and a memory, which is a temporary storage area used by the second processor. The control unit 410 comprehensively controls each unit constituting the endoscope 2 and the control device 4. The control unit 410 has an imaging control unit 410a, an acquisition unit 410b, a determination unit 410c, a display control unit 410d, and a communication control unit 410e.

[0068] The imaging control unit 410a controls imaging by the endoscope 2. The imaging control unit 410a generates a group of temporally continuous image data by causing the endoscope 2 to continuously capture images. Furthermore, when a release signal is input from the operation unit 22, the imaging control unit 410a causes the image sensor 203 to capture still image data.

[0069] The acquisition unit 410b acquires image data from the endoscope 2 via the S / P conversion unit 405 or the image processing unit 406 and records the data in the image data recording unit 408b. The acquisition unit 410b also acquires detection result information from the image diagnostic device 5 via the communication unit 409 and records the information in the examination information recording unit 408c.

[0070] The determination unit 410c determines whether or not the distal end 24 of the endoscope 2 has reached the final arrival site, based on the observation order information of the final arrival site set in the observation order for report creation and the part of the subject detected by the diagnostic imaging device 5. The determination unit 410c determines whether or not the final arrival site has not yet been imaged. Specifically, the determination unit 410c determines whether or not still image data capturing the final arrival site of the organ / site set in the examination information recording unit 408c in the observation order (insertion order) in the report items has been recorded in the image data recording unit 408b.

[0071] When the determination unit 410c determines that the tip 24 of the endoscope 2 has reached the final reach site, the display control unit 410d causes the display device 3 to display information indicating that the tip 24 of the endoscope 2 has reached the final reach site inside the subject. Furthermore, when the determination unit 410c determines that the final reach site has not yet been imaged, the display control unit 410d causes the display device 3 to display a warning to the effect that the final reach site has not yet been imaged.

[0072] The communication control unit 410e causes the communication unit 409 to transmit various information to the medical support device 6 via the network N100, and causes the communication unit 409 to receive various information from the medical support device 6.

[0073] [Functional Configuration of Diagnostic Imaging Apparatus] Next, a description will be given of the functional configuration of the diagnostic imaging apparatus 5. Fig. 3 is a block diagram showing the functional configuration of the diagnostic imaging apparatus 5. The diagnostic imaging apparatus 5 shown in Fig. 3 includes a communication unit 51, an input unit 52, a recording unit 53, a display unit 54, and a control unit 55.

[0074] Under the control of the control unit 55, the communication unit 51 receives, from the control device 4 via the network N100, image data groups generated by the endoscope 2 and organ / region order information indicating the observation order of organs / regions in the subject, who is a patient, and outputs these to the control unit 55. Also, under the control of the control unit 55, the communication unit 51 outputs, via the network N100, to the control device 4, examination information including the detection results of the subject's organs, etc., input from the control unit 55, and the detection times at which these detection results were detected. The communication unit 51 is configured using a communication module, etc.

[0075] The input unit 52 is configured using buttons, switches, a touch panel, etc., and receives various inputs and outputs them to the control unit 55.

[0076] The recording unit 53 is configured using recording media such as volatile memory, non-volatile memory, SSD, HDD, memory card, etc. The recording unit 53 records data including various parameters necessary for the operation of the diagnostic imaging device 5. The recording unit 53 also has a program recording unit 531 that records various programs for operating the diagnostic imaging device 5, an observation order recording unit 532, a first inference model recording unit 533, and a second inference model recording unit 534.

[0077] The observation order recording unit 532 records observation order information, which is input from the medical support device 6 described later, in which the observation order of multiple observation points such as pre-set organs and parts is set when observing a subject using the endoscope 2.

[0078] The first inference model recording unit 533 records a first inference model used to detect organs, parts, and characteristic regions based on image data. Here, the first inference model takes image data from the endoscope 2 as input and outputs organs, parts, and characteristic regions included in the image data. This first inference model is a model generated by machine learning using, for example, artificial intelligence (AI). Specifically, the first inference model is a model such as deep learning that learns using training data in which image data of an object is assigned characteristic regions such as the object's organs, parts, and lesions, and outputs the organs, parts, and characteristic regions included in the image data.

[0079] The second inference model recording unit 534 records a second inference model used to detect the insertion speed of the tip portion 24 of the endoscope 2. Here, the second inference model takes image data of the endoscope 2 as input and outputs insertion speed information related to the insertion speed of the tip portion 24 of the endoscope 2. This second inference model is a model such as deep learning that learns using training data in which the insertion speed and movement distance of the tip portion 24 of the endoscope 2 is added to image data of an image of a subject, and outputs the insertion speed of the tip portion 24 of the endoscope 2 from the image data.

[0080] The display unit 54 displays various information related to the diagnostic imaging apparatus 5 under the control of the control unit 55. The display unit 54 is configured using a liquid crystal display, an organic EL display, or the like.

[0081] The control unit 55 is realized using a processor having hardware such as an FPGA or a CPU, and a memory that is a temporary storage area used by the processor. The control unit 55 comprehensively controls each unit that constitutes the medical support device 6. The control unit 55 has an acquisition unit 551, an organ / site detection unit 552, a detection time acquisition unit 553, an insertion speed detection unit 554, and a measurement time acquisition unit 555.

[0082] The acquisition unit 551 acquires image data sequentially generated by the endoscope 2 from the control device 4 via the communication unit 51 in chronological order.

[0083] The organ / part detection unit 552 detects the organs / parts and characteristic areas of the subject contained in the image data based on the image data sequentially acquired by the acquisition unit 551 and the first inference model recorded by the first inference model recording unit 533.

[0084] The detection time acquisition unit 553 acquires the detection time when the organ / part detection unit 552 detects the organ / part and characteristic region of the subject included in the image data. For example, the detection time acquisition unit 553 acquires, as the detection time, the time measured by a TG (not shown) when the organ / part detection unit 552 detects the organ / part and characteristic region of the subject included in the image data.

[0085] The insertion speed detection unit 554 detects the insertion speed of the tip 24 of the endoscope 2 based on the image data sequentially acquired by the acquisition unit 551 and the second inference model recorded by the second inference model recording unit 534.

[0086] The measurement time acquisition unit 555 acquires the measurement time when the insertion speed of the tip portion 24 of the endoscope 2 is detected by the insertion speed detection unit 554. For example, the detection time acquisition unit 553 acquires, as the measurement time, the time measured by a TG (not shown) when the insertion speed of the tip portion 24 of the endoscope 2 is detected by the insertion speed detection unit 554.

[0087] [Functional Configuration of Medical Support Device] Next, a description will be given of the functional configuration of the medical support device 6. Fig. 4 is a block diagram showing the functional configuration of the medical support device 6. The medical support device 6 shown in Fig. 4 includes a communication unit 61, a display unit 62, an input unit 63, a database 64 (hereinafter simply referred to as "DB 64"), a program recording unit 65, and a control unit 66.

[0088] Under the control of the control unit 66, the communication unit 61 transmits various information to the control device 4 and the diagnostic imaging device 5 via the network N100, and receives various information from the control device 4 and the diagnostic imaging device 5. The communication unit 61 is configured using a communication module.

[0089] The display unit 62 displays various information related to the medical support device 6 under the control of the control unit 66. The display unit 62 is configured using a liquid crystal display, an organic EL display, or the like.

[0090] The input unit 63 receives various inputs in response to external operations and outputs them to the control unit 66. The input unit 63 is configured using buttons, a keyboard, switches, a touch panel, and the like.

[0091] The DB 64 records various types of information. The DB 64 is configured using an HDD, an SSD, etc. The DB 64 includes a patient information DB 641, an endoscope DB 642, an image data DB 643, an examination information DB 644, an observation order information DB 645, a report item DB 646, a related information DB 647, a report information DB 648, and an operator information DB 649.

[0092] The patient information DB 641 records, for each patient, a patient ID for identifying the patient, the patient's date of birth, the patient's doctor, the patient's address, and the patient's allergy information.

[0093] The endoscope DB 642 records, for each endoscope 2, the type of endoscope 2, the endoscope ID, the repair history of the endoscope 2, the maintenance status of the endoscope 2, the usage history, and the usage schedule information.

[0094] The image data DB 643 records a group of image data captured by the endoscope 2 in association with each patient ID or each operator ID.

[0095] The examination information DB 644 records examination information including detection result information of each of a plurality of observation sites detected based on the image data group detected by the image diagnostic apparatus 5 in association with each patient ID or each operator ID.

[0096] The observation order information DB 645 records the order of observation of organs and parts by the endoscope 2 set for each facility or each surgeon ID.

[0097] The report item DB 646 records report items for the endoscope 2 set for each facility or each operator ID.

[0098] The related information DB 647 records related information relating to the movement of the endoscope 2 inside the subject in the second image data group generated by the medical support device 6 described later.

[0099] The report information DB 648 records, for each patient ID or each operator ID, report information generated by the endoscope 2 by the operator (radiography reader) and report information automatically generated by the medical support device 6 in association with each other.

[0100] The surgeon information DB 649 records the name, position, affiliation, and contact information for each surgeon ID in association with each other.

[0101] The program recording unit 65 is configured using a volatile memory, a non-volatile memory, an SSD, etc., and records various programs for operating the medical support device 6.

[0102] The control unit 66 corresponds to the first processor according to the present disclosure. This control unit 66 is realized using a first processor having hardware such as an FPGA or a CPU, and a memory that is a temporary storage area used by the first processor. The control unit 66 comprehensively controls each unit constituting the medical support device 6. The control unit 66 has an acquisition unit 661, a determination unit 662, a calculation unit 664, a generation unit 665, and a creation unit 666.

[0103] The acquisition unit 661 acquires the image data group and the examination information from the control device 4 via the network N100.

[0104] The discrimination unit 662 discriminates between first image data captured during observation of a predetermined observation area and a second image data group captured after observation of the predetermined observation area based on the image data group acquired by the acquisition unit 661 and the examination information.

[0105] The calculation unit 664 calculates the insertion time required for the tip portion 24 of the endoscope 2 to reach the final destination based on the arrival time of the final destination determined by the determination unit 662 and the examination information.

[0106] The generation unit 665 generates related information including the insertion time calculated by the calculation unit 664 until the tip 24 of the endoscope 2 reaches the final destination site, the insertion start time included in the examination information, and the measurement time of the endoscope 2 based on the extracorporeal time detected as extracorporeal by the imaging diagnostic device 5, and records the information in the related information DB 647.

[0107] The creation unit 666 creates report information based on operation information from the input unit 63 or laptop computer 7, the image data group and examination information acquired by the acquisition unit 661, patient information, and related information generated by the generation unit 665, and records the report information in the report information DB 648.

[0108] [Processing of Medical Support System] Next, an outline of the processing executed by the medical support system 1 will be described. FIG.

[0109] 5, first, the control unit 66 determines whether or not the endoscope 2 is connected to the control device 4 via the network N100, thereby connecting the endoscope 2 to the medical support system 1 (step S101). Specifically, the control unit 66 determines whether or not an endoscope ID identifying the endoscope 2 has been input from the control device 4 via the network N100, and determines that the endoscope 2 is connected to the medical support system 1 if the endoscope ID identifying the endoscope 2 has been input from the control device 4.

[0110] If the control unit 66 determines that the endoscope 2 is connected to the control device 4 and thus connected to the medical support system 1 (step S101: Yes), the medical support system 1 proceeds to step S102. On the other hand, if the control unit 66 determines that the endoscope 2 is not connected to the control device 4 and thus not connected to the medical support system 1 (step S101: No), the medical support system 1 ends this process.

[0111] Next, the control unit 66 accepts settings for the observation order of the subject's organs and parts input from the laptop computer 7 or the control device 4, and records them in the observation order information DB 645 and inputs such as the patient's name in the patient information DB 641 (step S102). Specifically, based on the operation contents input from the surgeon's laptop computer 7 or the like, the control unit 66 accepts input of patient information including various information such as the patient's (subject's) name, the subject's patient ID (subject ID), and the patient's (subject's) date of birth, the type of endoscope 2 used to examine the subject, the surgeon ID identifying the attending physician, the type of report items, the contents of the report items, the patient's examination appointment details (examination date and time), the pre-examination consent form, the medical interview results, the consent form, and advance registration, and records this accepted patient information in the patient information DB 641. Furthermore, based on the operation contents input from the surgeon's laptop computer 7 or the like, the control unit 66 records observation order information regarding the observation order when observing organs and parts with the endoscope 2 in the observation order information DB 645.

[0112] Fig. 6 is a diagram showing an example of a list of organ / region observation orders and report items. Fig. 7 is a diagram showing another example of a list of organ / region observation orders and report items.

[0113] 6 and pattern Q2 shown in Fig. 7, the distal end portion 24 of the insertion section 21 (hereinafter simply referred to as "tip portion 24") of the endoscope 2 is inserted from outside the body of the subject into the body, and the order of observation of each organ and each region by the endoscope 2 is set for each operator (each doctor) or each facility. For example, in the organ / region observation order table W1, the order "9" is set for the organ / region "ileum."

[0114] Similarly, in the report item tables W2 and W4, items are set in association with the report type. For example, in the report item table W2, the report type "Gastrointestinal Endoscopy Report" is set with the item "Reached Site." Here, the reached site refers to the organ or site of the observation location within the subject that the distal end portion 24 was able to reach. In the following description, it is assumed that the pattern Q1 shown in FIG. 6 has been set.

[0115] 5 , the description of step S103 and subsequent steps will be continued. In step S103, the control unit 66 causes the display device 3 or the laptop computer 7 to display a consistency check and advice between the patient (i.e., the subject) and the endoscope 2, based on the patient information input in step S102, the past report information linked to the patient information and recorded in the report information DB 648, and the endoscope information recorded in the endoscope DB 642, via the network N100. Specifically, the control unit 66 checks the consistency of the settings of the endoscope 2 used to observe the patient, based on the patient information, the past report information linked to the patient information and recorded in the report information DB 648, and the endoscope information recorded in the endoscope DB 642. For example, the control unit 66 performs a consistency check to determine whether the type of endoscope 2 used to observe the patient is suitable for observation according to the observation order for observing organs and parts, the types of report items, and the examination reservation details set for the endoscope 2. In this case, when the control unit 66 determines that the type or settings of the endoscope 2 are inappropriate for observing the patient, it displays a warning on the display device 3 or laptop computer 7. This allows the surgeon or other person to recognize that the endoscope 2 connected to the control device 4 is inappropriate for observing the patient.

[0116] Furthermore, the control unit 66 displays, as advice on the display device 3 or laptop computer 7, information such as whether or not the current patient has any allergies, the surgeon's findings during the previous observation of the patient, the tumor and the area of ​​the patient that was subject to follow-up observation during the previous observation, based on the patient information, past report information recorded in the report information DB 648 linked to this patient information, and the endoscope information recorded in the endoscope DB 642. This allows the surgeon to understand the details of the handover of information regarding the patient from the previous surgeon or report creator.

[0117] Next, the control unit 66 determines whether or not imaging of the subject with the endoscope 2 has started from the control device 4 or the endoscope 2 via the network N100 (step S104). Specifically, when the control unit 66 receives image data or a startup signal indicating that the endoscope 2 has started from the control device 4 via the network N100, the control unit 66 determines that imaging of the subject with the endoscope 2 has started.

[0118] If the control unit 66 determines that imaging of the subject with the endoscope 2 has started (step S104: Yes), the medical support system 1 proceeds to step S105, which will be described later. On the other hand, if the control unit 66 determines that imaging of the subject with the endoscope 2 has not started (step S104: No), the medical support system 1 proceeds to step S107, which will be described later.

[0119] In step S105, the medical support system 1 executes an endoscopic image capturing process in which the endoscope 2 is inserted into the subject and observed. Details of the endoscopic image capturing process will be described later. After step S105, the medical support system 1 proceeds to step S106.

[0120] Next, the control unit 66 determines whether or not the endoscope 2 has finished capturing images of the subject from the control device 4 or the endoscope 2 via the network N100 (step S106). Specifically, the control unit 66 determines that the endoscope 2 has stopped capturing images of the subject when it receives a stop signal from the control device 4 via the network N100 indicating that the reception of image data has stopped or that the endoscope 2 has stopped. If the control unit 66 determines that the endoscope 2 has stopped capturing images of the subject (step S106: Yes), the medical support system 1 proceeds to step S107, which will be described later. On the other hand, if the control unit 66 determines that the endoscope 2 has not stopped capturing images of the subject (step S106: No), the medical support system 1 returns to step S105.

[0121] In step S107, the control unit 66 determines via the network N100 whether a report creator, such as a doctor or radiologist, has started creating a report using the laptop computer 7. Specifically, the control unit 66 determines whether a request signal has been received from the laptop computer 7 requesting specific patient information and examination information, including image data sets from the endoscope 2, still image data, and detection results from the diagnostic imaging device 5, that are linked to the patient information. If the control unit 66 determines that the report creator has started creating a report using the laptop computer 7 (step S107: Yes), the medical support system 1 proceeds to step S108, which will be described later. On the other hand, if the control unit 66 determines that the report creator has not started creating a report using the laptop computer 7 (step S107: No), the medical support system 1 proceeds to step S113.

[0122] In step S108, the medical support system 1 executes a report creation process for creating a report based on the observation results of the subject. Details of the report creation process will be described later.

[0123] Next, the control unit 66 determines via the network N100 whether the report creator, such as the doctor who interprets the images, has finished creating the report using the laptop computer 7 (step S109). Specifically, the control unit 66 determines whether an end signal indicating the end of report creation has been received from the laptop computer 7. If the end signal has been received, the control unit 66 determines that the report creation has ended. However, if the end signal has not been received, the control unit 66 determines that the report creation has not ended. If the control unit 66 determines that the report creator, such as the doctor who interprets the images, has finished creating the report using the laptop computer 7 (step S109: Yes), the medical support system 1 proceeds to step S110. On the other hand, if the control unit 66 determines that the report creator, such as the doctor who interprets the images, has not finished creating the report using the laptop computer 7 (step S109: No), the medical support system 1 returns to step S106.

[0124] In step S110, the control unit 66 determines, via the network N100, whether the laptop computer 7 or the control device 4 has terminated its connection to the medical support device 6. If the control unit 66 determines that the laptop computer 7 or the control device 4 has terminated its connection to the medical support device 6 (step S110: Yes), the medical support system 1 proceeds to step S115, which will be described later. On the other hand, if the control unit 66 determines that the laptop computer 7 or the control device 4 has not terminated its connection to the medical support device 6 (step S110: No), the medical support system 1 proceeds to step S111, which will be described later.

[0125] In step S111, the control unit 66 determines whether the endoscope 2 connected to the control device 4 has been changed based on the endoscope ID that identifies the endoscope 2 and is input from the control device 4 via the network N100. If the control unit 66 determines that the endoscope 2 connected to the control device 4 has been changed (step S111: Yes), the medical support system 1 proceeds to step S112, which will be described later. On the other hand, if the control unit 66 determines that the endoscope 2 connected to the control device 4 has not been changed (step S111: No), the medical support system 1 returns to step S102.

[0126] In step S112, the control device 4 switches the setting parameters of the endoscope 2 based on the endoscope ID of the connected endoscope 2. Specifically, the control device 4 switches the image processing parameters, such as white balance parameters, based on the endoscope ID of the connected endoscope 2. After step S112, the medical support system 1 returns to step S102.

[0127] In step S113, the control unit 66 determines whether the laptop computer 7 or the control device 4 has terminated the connection to the medical support device 6 via the network N100. If the control unit 66 determines that the laptop computer 7 or the control device 4 has terminated the connection to the medical support device 6 (step S113: Yes), the medical support system 1 proceeds to step S114, which will be described later. On the other hand, if the control unit 66 determines that the laptop computer 7 or the control device 4 has not terminated the connection to the medical support device 6 (step S113: No), the medical support system 1 returns to step S101, which will be described later.

[0128] In step S114, the medical support system 1 executes an unexamined process when the endoscope 2 has not yet been inserted into the subject. Details of the unexamined process will be described later. After step S114, the medical support system 1 proceeds to step S115.

[0129] Next, the control unit 66 determines whether or not to end observation of the subject using the endoscope 2 or report creation using the laptop computer 7 (step S115). If the control unit 66 determines that observation of the subject using the endoscope 2 or report creation using the laptop computer 7 should be ended (step S115: Yes), the medical support system 1 ends this processing. On the other hand, if the control unit 66 determines that observation of the subject using the endoscope 2 or report creation using the laptop computer 7 should not be ended (step S115: No), the medical support system 1 returns to step S102.

[0130] [Outline of Endoscopic Image Capturing Process] Next, details of the endoscopic image capturing process described in step S105 of Fig. 5 will be described. Fig. 8 is a flowchart showing an outline of the endoscopic image capturing process.

[0131] 8 , first, the control unit 410 starts imaging the endoscope 2, recording the image data captured by the endoscope 2, detecting organs and regions by the diagnostic imaging device 5, and detecting the insertion speed of the insertion section 21 of the endoscope 2 by the diagnostic imaging device 5 (step S201). Specifically, the imaging control unit 410a causes the endoscope 2 to start imaging. In this case, the communication control unit 410e outputs image data (live view images) generated by sequentially capturing images by the endoscope 2 to the diagnostic imaging device 5 in chronological order. At this time, the diagnostic imaging device 5 starts detecting organs and regions and characteristic regions contained in the image data based on the image data input from the control device 4 and the first inference model, and outputs this detection result information to the control device 4. Furthermore, the diagnostic imaging device 5 starts detecting the insertion speed of the endoscope 2 and timing the measurement time based on the image data input from the control device 4 and the second inference model, and outputs speed information including insertion speed information regarding the insertion speed and the measurement time to the control device 4. The control unit 410 generates examination information that corresponds the detection result information and speed information input from the imaging diagnostic device 5, and the patient information input from the medical support device 6 or the input unit 407, and starts recording this examination information in the examination information recording unit 408c.

[0132] Next, the determination unit 410c determines whether or not the tip portion 24 of the endoscope 2 has been inserted into the subject's body (step S202) based on the detection result of the diagnostic imaging device 5. If the determination unit 410c determines that the tip portion 24 of the endoscope 2 has been inserted into the subject's body (step S202: Yes), the control device 4 proceeds to step S203, which will be described later. On the other hand, if the determination unit 410c determines that the tip portion 24 of the endoscope 2 has not been inserted into the subject's body (step S202: No), the control device 4 proceeds to step S225, which will be described later.

[0133] In step S203, the control unit 410 starts recording the insertion time when the tip portion 24 of the endoscope 2 is inserted into the subject's body and the measurement time of the subject by the endoscope 2 in the examination information recording unit 408c.

[0134] Next, the determination unit 410c determines whether or not the surgeon has performed an imaging operation on the operation unit 22 (step S204). Specifically, when a release signal is input from the operation unit 22, the determination unit 410c determines that the surgeon has performed an imaging operation on the operation unit 22. When the control unit 410 determines that the surgeon has performed an imaging operation on the operation unit 22 (step S204: Yes), the control device 4 proceeds to step S205, which will be described later. On the other hand, when the determination unit 410c determines that the surgeon has not performed an imaging operation on the operation unit 22 (step S204: No), the control device 4 proceeds to step S206, which will be described later.

[0135] In step S205, the imaging control unit 410a causes the imaging element 203 to capture a still image (still image data). Specifically, the imaging control unit 410a records a still image file in the image data recording unit 408b, which associates the still image data generated by the imaging element 203 with the imaging time at which the imaging element 203 captured the still image data. At this time, the communication control unit 410e outputs the still image file to the medical support device 6 via the network N100. Here, the still image data is an image with a higher resolution (more pixels) than live view image data (e.g., a 4K image). After step S205, the control device 4 proceeds to step S206.

[0136] Next, the determination unit 410c determines whether the diagnostic imaging device 5 has detected a part of the subject (step S206). Specifically, the determination unit 410c determines whether it has received, from the diagnostic imaging device 5, detection result information including organ information indicating that a part of any organ in the subject has been detected and the detection time of the detection result indicating that the organ part has been recognized. If the determination unit 410c determines that the diagnostic imaging device 5 has recognized a part of the subject (step S206: Yes), the control device 4 proceeds to step S208, which will be described later. On the other hand, if the determination unit 410c determines that the diagnostic imaging device 5 has not recognized a part of the subject (step S206: No), the control device 4 proceeds to step S207, which will be described later.

[0137] In step S207, the determination unit 410c determines whether a predetermined time has elapsed since the endoscope 2 was inserted into the subject. If the determination unit 410c determines that the predetermined time has elapsed since the endoscope 2 was inserted into the subject (step S207: Yes), the control device 4 proceeds to step S221, which will be described later. On the other hand, if the determination unit 410c determines that the predetermined time has not elapsed since the endoscope 2 was inserted into the subject (step S207: No), the control device 4 returns to step S204.

[0138] In step S208, the determination unit 410c determines whether or not the distal end portion 24 of the endoscope 2 has reached the final reach portion based on the observation order information of the final reach portion set in the observation order for report creation (see, for example, Part 1Q1 of FIG. 6 ) and the portion of the subject detected by the diagnostic imaging device 5. Specifically, when the final reach portion set in the observation order is the ileum, the determination unit 410c determines that the distal end portion 24 of the endoscope 2 has reached the final reach portion when the diagnostic imaging device 5 detects that the portion detected is the ileum, whereas the determination unit 410c determines that the distal end portion 24 of the endoscope 2 has not reached the final reach portion when the portion detected by the diagnostic imaging device 5 is not the ileum. If the determination unit 410c determines that the distal end portion 24 of the endoscope 2 has reached the final reach portion (step S208: Yes), the control device 4 proceeds to step S209, which will be described later. On the other hand, if the determination unit 410c determines that the tip portion 24 of the endoscope 2 has not reached the final destination (step S208: No), the control device 4 proceeds to step S214, which will be described later.

[0139] In step S209, the display control unit 410d causes the display device 3 to display information indicating that the tip portion 24 of the endoscope 2 has reached the final destination site within the subject. For example, the display control unit 410d notifies the surgeon by causing the display device 3 to display information indicating that the tip portion 24 of the endoscope 2 has reached the "ileum," which is set as the final destination site. This allows the surgeon to understand that the tip portion 24 of the endoscope 2 has reached the final destination site set in the observation order.

[0140] Next, the acquisition unit 410b acquires the detection result information of the organ / site recognized by the diagnostic imaging device 5 and the detection time when the distal end 24 of the endoscope 2 reaches the final destination site, and records them in the examination information recording unit 408c (step S210). In this case, the acquisition unit 410b associates the detection result of detecting the final destination site with the detection time and records them in the examination information recording unit 408c.

[0141] Thereafter, the determination unit 410c determines whether or not the surgeon has performed an imaging operation on the operation unit 22 (step S211). If the determination unit 410c determines that the surgeon has performed an imaging operation on the operation unit 22 (step S211: Yes), the control device 4 proceeds to step S212, which will be described later. On the other hand, if the determination unit 410c determines that the surgeon has not performed an imaging operation on the operation unit 22 (step S211: No), the control device 4 proceeds to step S213, which will be described later.

[0142] In step S212, the imaging control unit 410a causes the imaging element 203 to capture a still image (still image data). In this case, the imaging control unit 410a records, in the image data recording unit 408b, a still image file that associates the still image data generated by the imaging element 203 with the imaging time at which the imaging element 203 captured the still image. At this time, the communication control unit 410e outputs the still image file to the medical support device 6 via the network N100. After step S212, the control device 4 proceeds to step S213.

[0143] In step S213, the determination unit 410c determines whether the diagnostic imaging apparatus 5 has detected the next region within the subject. Specifically, the determination unit 410c determines whether detection result information indicating the detection of the next region within the subject has been received from the diagnostic imaging apparatus 5, and if the detection result information has been received, determines that the next region within the subject has been detected. If the determination unit 410c determines that the diagnostic imaging apparatus 5 has detected the next region within the subject (step S213: Yes), the control device 4 proceeds to step S214, which will be described later. On the other hand, if the determination unit 410c determines that the diagnostic imaging apparatus 5 has not detected the next region within the subject (step S213: No), the control device 4 returns to step S211.

[0144] In step S214, the determination unit 410c determines whether the final arrival site has not yet been imaged. Specifically, the determination unit 410c determines whether still image data capturing the final arrival site of the organ / site set in the observation order (insertion order) of the report item in the examination information recording unit 408c is recorded in the image data recording unit 408b. More specifically, based on the image capture time of the still image data recorded in the image data recording unit 408b, the determination unit 410c determines whether still image data captured within a predetermined time, e.g., within three minutes, from the detection time of the detection result information detecting the ileum set as the final arrival site recorded in the examination information recording unit 408c is recorded in the image data recording unit 408b. If the determination unit 410c determines that the final arrival site has not yet been imaged (step S214: Yes), the control device 4 proceeds to step S215, which will be described later. On the other hand, if the determining unit 410c determines that the final reached part has not yet been imaged (step S214: No), the control device 4 proceeds to step S216, which will be described later.

[0145] In step S215, the display control unit 410d displays a warning that the final reached region has not yet been imaged on the display device 3. This allows the operator to understand that the final reached region of the subject has not yet been imaged.

[0146] Next, the determination unit 410c determines whether the imaging diagnostic apparatus 5 has detected the next part in the subject (step S216). Specifically, the determination unit 410c determines whether detection result information including organ information indicating that the next organ part in the subject has been detected and the detection time of the detection result indicating that the organ part has been recognized has been received from the imaging diagnostic apparatus 5. If the determination unit 410c determines that the imaging diagnostic apparatus 5 has detected the next part in the subject (step S216: Yes), the control device 4 proceeds to step S217, which will be described later. On the other hand, if the determination unit 410c determines that the imaging diagnostic apparatus 5 has not detected the next part in the subject (step S216: No), the control device 4 proceeds to step S221.

[0147] In step S217, the determination unit 410c determines whether or not the surgeon has performed an imaging operation on the operation unit 22. If the control unit 410 determines that the surgeon has performed an imaging operation on the operation unit 22 (step S217: Yes), the control device 4 proceeds to step S218, which will be described later. On the other hand, if the determination unit 410c determines that the surgeon has not performed an imaging operation on the operation unit 22 (step S217: No), the control device 4 proceeds to step S219, which will be described later.

[0148] In step S218, the imaging control unit 410a causes the imaging element 203 to capture a still image (still image data). In this case, the imaging control unit 410a records, in the image data recording unit 408b, a still image file that associates the still image data generated by the imaging element 203 with the imaging time at which the imaging element 203 captured the still image. At this time, the communication control unit 410e outputs the still image file to the medical support device 6 via the network N100. After step S218, the control device 4 proceeds to step S219.

[0149] In step S219, the determination unit 410c determines whether the previous region recognized by the imaging diagnostic device 5 has not yet been imaged. Specifically, based on the imaging time of each still image data piece recorded in the image data recording unit 408b, the determination unit 410c determines whether still image data of the organ / region set in the examination information recording unit 408c in the observation order (insertion order) of the report item, which is captured within a predetermined time, for example, within three minutes, from the detection time of the previous region recognized by the imaging diagnostic device 5, immediately after the imaging diagnostic device 5 recognizes a new region as the distal end portion 24 of the endoscope 2 moves inside the subject, has been recorded in the image data recording unit 408b. For example, if the imaging diagnostic device 5 recognizes the organ / region as the ascending colon and the examination information recording unit 408c records examination information including a detection result in which the imaging diagnostic device 5 recognizes the organ / region as the cecum, but still image data capturing the cecum has not been recorded in the image data recording unit 408b, the determination unit 410c determines whether the previous region recognized by the imaging diagnostic device 5 has not yet been imaged. If the determination unit 410c determines that the previous region recognized by the diagnostic imaging device 5 has not been imaged (step S219: Yes), the control device 4 proceeds to step S220, which will be described later. On the other hand, if the determination unit 410c determines that the previous region recognized by the diagnostic imaging device 5 has not been imaged (step S219: No), the control device 4 proceeds to step S221, which will be described later.

[0150] In step S220, immediately after the imaging diagnostic device 5 recognizes a new region as a result of the distal end 24 of the endoscope 2 moving inside the subject, the display control unit 410d causes the display device 3 to display a warning that the previous region recognized by the imaging diagnostic device 5 has not yet been imaged. This prevents the surgeon from forgetting to image the previous region due to the movement of the distal end 24 of the endoscope 2. After step S220, the control device 4 proceeds to step S221, which will be described later.

[0151] In step S221, the determination unit 410c determines whether or not the tip portion 24 of the endoscope 2 has been removed from outside the subject's body, based on the detection result from the diagnostic imaging device 5. If the determination unit 410c determines that the tip portion 24 of the endoscope 2 has been removed from outside the subject's body (step S221: Yes), the control device 4 proceeds to step S222, which will be described later. On the other hand, if the determination unit 410c determines that the tip portion 24 of the endoscope 2 has not been removed from outside the subject's body (step S221: No), the control device 4 returns to step S204 described above.

[0152] In step S222, the control unit 410 records the external removal time when the tip portion 24 of the endoscope 2 is removed from the subject's body as examination information in the examination information recording unit 408c based on the detection result from the imaging diagnostic device 5.

[0153] Next, the control unit 410 stops the imaging of the endoscope 2, the recording of the image data captured by the endoscope 2, the part recognition by the image diagnostic device 5 and the detection of the insertion speed of the insertion portion 21 of the endoscope 2 by the image diagnostic device 5, the recording of the insertion time, and the recording of the measurement time (step S223).

[0154] Next, the determination unit 410c determines, based on the examination information recorded by the examination information recording unit 408c, whether any of the organs or parts set in the examination information recording unit 408c in the observation order (insertion order) of the report items has not yet been imaged (step S224). If the determination unit 410c determines, based on the examination information recorded by the examination information recording unit 408c, that any of the organs or parts set in the examination information recording unit 408c in the observation order (insertion order) of the report items has not yet been imaged (step S224: Yes), the control device 4 proceeds to step S225, which will be described later. On the other hand, if the determination unit 410c determines, based on the examination information recorded by the examination information recording unit 408c, that any of the organs or parts set in the examination information recording unit 408c in the observation order (insertion order) of the report items has not yet been imaged (step S224: No), the control device 4 terminates this process and returns to the main routine of FIG. 4.

[0155] In step S225, the control unit 410 adds stenosis information corresponding to each site to the examination information and records it in the examination information recording unit 408c. Specifically, if there is an organ / site that has not been imaged among the organs / sites set in the observation order (insertion order) in the report items in the examination information recording unit 408c and the final arrival site has not been reached, the control unit 410 adds stenosis information indicating that stenosis has occurred to the organ / site for which still image data was first captured and records it in the examination information recording unit 408c. After step S225, the control device 4 ends this process and returns to the main routine of FIG. 4.

[0156] In step S226, the determination unit 410c determines whether a predetermined time has elapsed without the endoscope 2 being inserted into the subject. If the determination unit 410c determines that the predetermined time has elapsed without the endoscope 2 being inserted into the subject (step S226: Yes), the control device 4 proceeds to step S227, which will be described later. On the other hand, if the determination unit 410c determines that the predetermined time has not elapsed without the endoscope 2 being inserted into the subject (step S226: No), the control device 4 returns to step S202.

[0157] In step S227, the control unit 410 records non-insertion information indicating that the endoscope 2 has not been inserted into the subject as examination information in the examination information recording unit 408c. Here, the non-insertion information includes cases where the endoscope 2 was not inserted due to the subject's allergy or the like, where an abnormality occurred in the endoscope 2 and it could not be inserted into the subject, and where the current endoscope 2 is not intended for use with the subject. After step S227, the control device 4 ends this processing and returns to the main routine of FIG. 4.

[0158] [Details of Report Creation Processing] Next, details of the report creation processing in step S108 in Fig. 5 will be described. Fig. 9 is a flowchart showing an outline of the report creation processing.

[0159] As shown in FIG. 9, first, the acquisition unit 661 acquires the image data group and the examination information from the control device 4 via the network N100 (step S301).

[0160] Based on the image data group acquired by the acquisition unit 661 and the examination information, the discrimination unit 662 discriminates between first image data captured during observation of a predetermined observation area and a second image data group captured after observation of the predetermined observation area (step S302).

[0161] FIG. 10 is a diagram showing a schematic overview of when the discrimination unit 662 discriminates an image data group into first image data and a second image data group.

[0162] When inserting the tip 24 of the endoscope 2 (lower bowel endoscope) from outside the body into the body of a subject, the surgeon inserts the tip 24 into the ileum, the final destination of the subject, as quickly as possible to reduce pain to the subject. Furthermore, the insertion time of the tip 24 of the endoscope 2 into the subject (patient) varies depending on the surgeon's skill level and the patient's condition. For this reason, as shown in Figure 10, the endoscope 2 captures a large amount of insertion image data group D1 before reaching the examination site, the final destination.

[0163] Next, the surgeon stops inserting the distal end 24 of the endoscope 2 after the distal end 24 of the endoscope 2 reaches the final destination part (ileum) of the subject. In this case, the endoscope 2 generates first image data P1 by capturing an image of the final destination part. At this time, the surgeon may operate the operation unit 22 of the endoscope 2 to cause the endoscope 2 to capture an image of the final destination part and generate still image data.

[0164] The surgeon then observes the inside of the subject while withdrawing the distal end 24 of the endoscope 2. At this time, the endoscope 2 generates a withdrawal image data group P2 (second image data group D2). Therefore, the image data group generated by a conventional endoscope 2 includes an insertion image data group D1 generated by the endoscope 2 while the endoscope 2 is inserted into the subject, and a withdrawal image data group P2 (second image data group D2) generated while the endoscope 2 is withdrawn from the subject, resulting in a large volume of data. Conventionally, surgeons have identified the image data of the final destination region using timing results using a timer or the like until the final destination region of the subject is reached, or have identified the image data of the final destination region from the image data group by interpreting a large volume of image data group after the subject examination. As a result, it has been difficult for surgeons to reduce the effort and time required to create an endoscopic report.

[0165] 10 , the discrimination unit 662 discriminates between first image data P1 captured during observation of a predetermined observation site and a second image data group D2 captured after observation of the predetermined observation site, based on the image data group acquired by the acquisition unit 661 and the examination information. Specifically, the discrimination unit 662 discriminates, from the image data group and the examination information, the timing at which the distal end portion 24 of the endoscope 2 switches from the inserted state to the removed state, based on the image data group and the examination information. For example, as shown in FIG. 10 , the discrimination unit 662 discriminates between first image data P1 (ileum image) of the deep part of the large intestine captured during observation of the predetermined observation site and a second image data group D2 (removal image data group P2) captured after observation of the predetermined observation site, from an image data group including the insertion image data group D1, thereby discriminating the final reached site of the subject reached by the distal end portion 24 of the endoscope 2. In Figure 10, the final destination is the ileum, but if the subject's symptoms, such as stenosis, occur in the ileum, the final destination will be the organ or site immediately before insertion into the ileum.

[0166] The discrimination unit 662 discriminates between the first image data and the second image data group by specifying the arrival time at which the tip portion 24 of the endoscope 2 reaches the final arrival site using one or more of the following conditions 1 to 3. Condition 1 is the organ name detected by the diagnostic imaging device 5, which is linked to the order set in the organ / site order, and the detection time at which this organ was detected. Condition 2 is the imaging time at which still image data linked to the still image data was captured. Condition 3 is the insertion speed (removal speed) of the tip portion 24 of the endoscope 2 detected by the diagnostic imaging device 5, and the measurement time at which this insertion speed was detected.

[0167] (When Conditions 1 to 3 are Used) Fig. 11 is a diagram showing an example of the test information acquired by the acquisition unit 661. Fig. 12 is a diagram for schematically explaining the discrimination method used by the discrimination unit 662.

[0168] The examination information A1 in Figure 11 includes an organ / observation order table H1 that associates the organ names detected by the imaging diagnostic device 5, which are linked to the order set in the organ / site observation order, with the detection time at which the organ was detected, a still image capturing time table H2 that associates still image data with the capturing time at which the still image data was captured, and a speed information table H3 that associates the insertion speed (removal speed) of the tip 24 of the endoscope 2 detected by the imaging diagnostic device 5 with the measurement time at which the insertion speed was detected.

[0169] 12, first, the discrimination unit 662 discriminates the innermost organ in the distal end portion 24 of the endoscope 2 based on the observation order table H1 and the observation order table W1 (see pattern Q1 in FIG. 6) included in the examination information. In the case shown in FIG. 12, the discrimination unit 662 discriminates the innermost organ / site as the "ileum" because the innermost organ / site in the observation order table H1 is the "ileum" and the observation order is "9" and the detection result by the imaging diagnostic device 5 includes the "ileum."

[0170] Next, the discrimination unit 662 determines the arrival time at which the distal end portion 24 of the endoscope 2 reaches the final arrival site based on the imaging time T2 of the still image data captured within the detection time T1 (0:04:00 to 0:05:00) of the "ileum" determined to be the deepest, and the measurement time T3 of the insertion speed. Specifically, the discrimination unit 662 determines the arrival time (0:04:10) at which the distal end portion 24 of the endoscope 2 reaches the final arrival site as the imaging time (0:04:10) of the still image data captured first within the detection time T1 (0:04:00 to 0:05:00) of the "ileum" determined to be the deepest.

[0171] As a result, the discrimination unit 662 discriminates, based on the identified arrival time, between the image data group: first image data captured during observation of the predetermined observation site (first image data of the ileum at the back of the large intestine) and a second image data group captured after observation of the predetermined observation site (second image data group at the time of removal).

[0172] (When conditions 1 and 2 are used) Fig. 13 is a diagram showing another example of the test information acquired by the acquisition unit 661. Fig. 14 is a diagram schematically explaining the determination method used by the determination unit 662. Note that Figs. 13 and 14 explain the determination method when there is no timing to switch the insertion speed.

[0173] The examination information A2 in Figure 13 includes an organ / observation order table H11 that associates the organ names detected by the imaging diagnostic device 5, which are linked to the order set in the organ / site observation order, with the detection time at which the organ was detected, a still image capturing time table H12 that associates still image data with the capturing time at which the still image data was captured, and a speed information table H13 that associates the insertion speed (removal speed) of the tip 24 of the endoscope 2 detected by the imaging diagnostic device 5 with the measurement time at which the insertion speed was detected.

[0174] 14, first, the discrimination unit 662 discriminates the innermost organ in the distal end portion 24 of the endoscope 2 based on the observation order table H11 and the observation order table W1 (see pattern Q1 in FIG. 6). In the case shown in FIG. 14, the discrimination unit 662 discriminates the innermost organ / site as the "ileum" because the innermost organ / site in the observation order table W1 is the "ileum," the observation order is "9," and the detection result by the imaging diagnostic device 5 includes the "ileum."

[0175] Next, as shown in FIG. 14, since there is no measurement time T13 (0:04:05 to 0:05:00) that includes the measurement time when the insertion speed changed from "fast" to "slow" within the detection time T11 (0:04:00 to 0:05:00) of the "ileum" that has been determined to be the deepest, the discrimination unit 662 determines that the arrival time when the tip 24 of the endoscope 2 reached the final arrival site is the imaging time T14 (0:04:10) of the still image data that was captured first, based on the imaging time T12 (0:04:10 to 0:05:00) of the still image data.

[0176] As a result, the discrimination unit 662 can discriminate between the first image data captured during observation of the predetermined observation site (first image data of the ileum at the back of the large intestine) and the second image data group captured after observation of the predetermined observation site (second image data group at the time of removal), based on the identified arrival time.

[0177] (When only condition 1 is used) Fig. 15 is a diagram showing another example of the examination information acquired by the acquisition unit 661. Fig. 16 is a diagram schematically explaining the discrimination method used by the discrimination unit 662. Note that Figs. 15 and 16 explain the discrimination method when the imaging time of the still image data does not fall within the detection time T21 of the "ileum" determined to be the deepest part.

[0178] The examination information A3 in Figure 15 includes an organ / observation order table H21 that associates the organ names detected by the imaging diagnostic device 5, which are linked to the order set in the organ / site observation order, with the detection time at which the organ was detected; a still image capturing time table H22 that associates still image data with the capturing time at which the still image data was captured; and a speed information table H23 that associates the insertion speed (removal speed) of the tip 24 of the endoscope 2 detected by the imaging diagnostic device 5 with the measurement time at which the insertion speed was detected.

[0179] As shown in Figure 16, the discrimination unit 662 determines that the detection time T23 "0:04:00" of the "ileum" determined to be the deepest part is the arrival time when the tip 24 of the endoscope 2 reached the final arrival site, because there is no imaging time of still image data within the detection time T21 of the "ileum" determined to be the deepest part, and there is no measurement time at measurement time T22 when the insertion speed changes from "fast" to "slow."

[0180] This allows the discrimination unit 662 to discriminate between the first image data captured during observation of the predetermined observation site (first image data of the ileum at the back of the large intestine) and the second image data group captured after observation of the predetermined observation site (second image data group at the time of removal), based on the arrival time.

[0181] 11 to 16, the discrimination unit 662 discriminates the "ileum," which is the organ / part with the highest ordinal number in the organ / part order by the imaging diagnostic device 5, as the final destination part, but is not limited thereto. If there is no organ / part with the highest ordinal number in the organ / part order, the discrimination unit 662 may discriminate the detection result of the organ / part with the highest ordinal number in the organ / part order, which is the detection result by the imaging diagnostic device 5 included in the examination information, as the final destination part. For example, if the examination information includes the organ / part with the highest ordinal number in the organ / part order detected by the imaging diagnostic device 5, for example, the cecum "8," the discrimination unit 662 discriminates the cecum as the final destination part.

[0182] Returning to FIG. 9 , the description of step S303 and subsequent steps will be continued. In step S303, the determination unit 663 determines whether the distal end 24 of the endoscope 2 has reached the final destination site (e.g., the ileum) preset for the examination item based on the observation order setting information recorded in the observation order information DB 645 and the examination information. If the determination unit 663 determines that the distal end 24 of the endoscope 2 has reached the final destination site (e.g., the ileum) preset for the examination item (step S303: Yes), the medical support device 6 proceeds to step S304, which will be described later. On the other hand, if the determination unit 663 determines that the distal end 24 of the endoscope 2 has not reached the final destination site (e.g., the ileum) preset for the examination item (step S303: No), the medical support device 6 proceeds to step S306, which will be described later.

[0183] In step S304, the calculation unit 664 calculates the insertion time required for the tip portion 24 of the endoscope 2 to reach the final destination based on the arrival time of the final destination determined by the determination unit 662 and the examination information.

[0184] Fig. 17 is a diagram illustrating a calculation method of the insertion time calculated by the calculation unit 664. As shown in Fig. 17, the calculation unit 664 sets the detection time (0:00:05) at which the organ was detected as "the anus" by the diagnostic imaging device 5, which is included in the examination information, as the insertion start time at which the distal end portion 24 of the endoscope 2 is inserted into the subject, and calculates the insertion time (0:04:05) based on the final arrival time (0:04:10) at which it is determined that the distal end portion 24 has reached the final arrival site.

[0185] 9 , the description of step S305 and subsequent steps will be continued. The generation unit 665 generates related information including the insertion time calculated by the calculation unit 664 until the tip portion 24 of the endoscope 2 reaches the final destination site, the insertion start time included in the examination information, and the measurement time of the endoscope 2 based on the extracorporeal time detected as extracorporeal by the imaging diagnostic device 5. In this case, the generation unit 665 generates the insertion time, the measurement time, and the insertion speed of the tip portion 24 of the endoscope 2 by the imaging diagnostic device 5 as related information related to the movement of the endoscope 2, and records this in the related information DB 647. After step S305, the medical support device 6 proceeds to step S308, which will be described later.

[0186] In step S306, the calculation unit 664 calculates the insertion time required for the tip portion 24 of the endoscope 2 to reach the target site based on the arrival time to the target site determined by the determination unit 662 and the examination information.

[0187] Next, the generation unit 665 generates related information including the insertion time calculated by the calculation unit 664 until the tip portion 24 of the endoscope 2 reaches the target site, the insertion start time included in the examination information, and the observation time (measurement time) of the endoscope 2 based on the extracorporeal time detected as extracorporeal by the diagnostic imaging device 5 (step S307). In this case, the generation unit 665 generates the insertion time, the measurement time, and the insertion speed of the tip portion 24 of the endoscope 2 by the diagnostic imaging device 5 as related information related to the movement of the endoscope 2, and records this in the related information DB 647. After step S307, the medical support device 6 proceeds to step S308, which will be described later.

[0188] Next, the creation unit 666 creates report information based on the operation information from the input unit 63 or laptop computer 7, the image data group and examination information acquired by the acquisition unit 661, the patient information, and the related information generated by the generation unit 665 (step S308). Specifically, the creation unit 666 creates report information (examination report) to which related information regarding the movement of the endoscope 2, including the final arrival site of the tip portion 24 of the endoscope 2, the insertion time of the tip portion 24 of the endoscope 2, and the measurement time by the endoscope 2, is added, and records the report information in the report information DB 648.

[0189] Fig. 18 is a diagram showing an example of an examination report created by the creation unit 666. In Fig. 18, still images captured during observation of the subject and findings based on the interpretation results by the surgeon are input into the examination report R1, but detailed description thereof will be omitted.

[0190] 18 , the creation unit 666 creates an examination report R1 that adds at least the insertion time of the endoscope 2 to the patient information in response to operation information from the input unit 63 or the laptop computer 7. This automatically adds the insertion time, which was previously measured by the surgeon, to the created examination report R1, thereby reducing the effort and time required to create the endoscopic report. While the creation unit 666 adds the final arrival time and the insertion start time to the examination report R1, the invention is not limited to this, and the creation unit 666 may automatically add the final arrival site, or may automatically add still image data captured at a time close to the detection time of each organ.

[0191] Thereafter, the determination unit 663 determines whether any of the organs or regions set in the report items have not yet been imaged, based on the examination information acquired by the acquisition unit 661 (step S309). If the determination unit 663 determines that any of the organs or regions set in the report items have not yet been imaged (step S303: Yes), the medical support device 6 proceeds to step S304, which will be described later. On the other hand, if the determination unit 663 determines that any of the organs or regions set in the report items have not yet been imaged (step S303: No), the medical support device 6 proceeds to step S305, which will be described later.

[0192] In step S310, the creating unit 666 creates stenosis information based on the examination information acquired by the acquiring unit 661, and records the stenosis information in association with the related information generated by the generating unit 665.

[0193] Next, the determination unit 663 determines whether or not there is non-insertion information based on the examination information acquired by the acquisition unit 661 (step S311). If the determination unit 663 determines that there is non-insertion information (step S311: Yes), the medical support device 6 proceeds to step S312, which will be described later. On the other hand, if the determination unit 663 determines that there is no non-insertion information (step S311: No), the medical support device 6 returns to the main routine of FIG. 5.

[0194] In step S312, the creation unit 666 creates the report information by adding handover information indicating that there was a non-insertion in the observation order set in the examination information. After step S312, the medical support device 6 returns to the main routine of FIG. 5.

[0195] [Outline of Uninspected Processing] Next, details of the uninspected processing described in step S114 of Fig. 5 will be described. Fig. 19 is a flowchart showing an outline of the uninspected processing.

[0196] 19 , the creation unit 666 records the setting results of the subject to be observed, the patient name, etc., and the details of why observation was not performed as history in the patient information DB 641 based on the details input from the laptop computer 7 or the control device 4 (step S401). Specifically, the details of why observation was not performed include, for example, discontinuation due to a sudden deterioration in the subject's physical condition, discontinuation due to an allergic reaction to a prior injection, an incompatible type of endoscope 2 for the subject (patient), unmaintained endoscope 2, a malfunction of the endoscope 2, uncleaned endoscope 2, etc.

[0197] Next, the creation unit 666 associates the patient's report information with the details of the endoscope 2 that led to the unobserved portion and records the information in the report information DB 648 (step S402). This allows the surgeon to understand the details of the unobserved portion the next time he or she interprets the patient's report information. After step S402, the medical support device 6 returns to the main routine of FIG. 5.

[0198] According to the embodiment described above, the generation unit 665 generates related information relating to the movement of the endoscope 2 within the subject in the second image data group based on the discrimination result obtained by the discrimination unit 662 between the first image data and the second image data group and the examination information acquired by the acquisition unit 661, thereby reducing the effort and time required to create an endoscopic report.

[0199] Furthermore, according to one embodiment, the related information is time information when the endoscope 2 captured the second image data group, so the surgeon can easily understand the time required to create an endoscopic report, thereby reducing the effort and time required to create an endoscopic report.

[0200] Furthermore, according to one embodiment, the time information includes the insertion time during which the tip 24 of the endoscope 2 is inserted from outside the subject's body to a predetermined observation site, and the observation time during which the endoscope 2 observes the inside of the subject. This allows the surgeon to easily grasp the time required to create an endoscopic report, thereby reducing the effort and time required to create an endoscopic report.

[0201] Furthermore, according to one embodiment, the examination information further includes insertion speed information relating to the insertion speed of the tip 24 of the endoscope 2 as it moves inside the subject based on the image data group, and speed information that correlates the measurement time from the start time when detection of this insertion speed began to the end time when detection of the insertion speed ended, so that an endoscopic report that takes into account the movement of the tip 24 of the endoscope 2 can be created.

[0202] Furthermore, according to one embodiment, when the generation unit 665 determines that the tip 24 of the endoscope 2 has reached the final destination using the discrimination unit 662, it generates related information, making it possible to discriminate the necessary data groups from a large amount of image data groups, thereby reducing the effort and time required to create an endoscopic report.

[0203] Furthermore, according to one embodiment, the discrimination unit 662 determines, based on the detection result information, the imaging time associated with the still image data, and the speed information, that the measurement time at which the imaging time associated with the still image data falls within the determination time of the final arrival location included in the detection result information and the speed of the tip 24 of the endoscope 2 becomes less than a predetermined threshold, as the arrival time of the final arrival location, thereby reducing the effort and time involved in creating an endoscopic report.

[0204] Furthermore, according to one embodiment, the calculation unit 664 calculates the insertion time of the tip 24 of the endoscope 2 based on the arrival time and the insertion start time of the subject based on the examination information, thereby reducing the effort and time required to create an endoscopic report.

[0205] Furthermore, according to one embodiment, when the creation unit 662 determines through the determination unit 663 that the tip 24 of the endoscope 2 has not reached the final destination, the creation unit 662 records stenosis information indicating that stenosis has occurred at the most recently observed location included in the detection result information in association with related information, so that the surgeon can grasp the patient's symptoms in advance when taking over the patient at a conference, etc., or before observing the patient again.

[0206] In one embodiment, the discrimination unit 662 of the medical support device 6 distinguishes between the first image data and the second image data group, but the function of the control unit 66 may be provided in the image diagnostic device 5, and the final arrival site and insertion time of the tip 24 of the endoscope 2 may be determined based on the image data groups sequentially generated by the endoscope 2.

[0207] Various inventions can be formed by appropriately combining multiple components disclosed in the medical support system according to the embodiment of the present disclosure described above. For example, some components may be omitted from all the components described in the medical support system according to the embodiment of the present disclosure described above. Furthermore, the components described in the medical support system according to the embodiment of the present disclosure described above may be appropriately combined.

[0208] Furthermore, in the medical support system according to an embodiment of the present disclosure, the above-described "unit" can be read as "means" or "circuit," etc. For example, a control unit can be read as control means or a control circuit.

[0209] In addition, the program to be executed by the medical support system according to one embodiment of the present disclosure is provided as file data in an installable or executable format recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk (FD), a CD-R, a DVD (Digital Versatile Disk), a USB medium, or a flash memory.

[0210] Furthermore, the program executed by the medical support system according to one embodiment of the present disclosure may be configured to be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network.

[0211] In the description of the flowcharts in this specification, expressions such as "first," "then," and "continue" are used to clearly indicate the order of processing between steps, but the order of processing required to implement the present invention is not uniquely determined by these expressions. In other words, the order of processing in the flowcharts described in this specification can be changed within a consistent range. Furthermore, programs are not limited to those consisting of simple branching processing, and branching can be achieved by comprehensively determining more judgment items.

[0212] Although some of the embodiments of the present application have been described in detail above with reference to the drawings, these are merely examples, and the present invention can be implemented in other forms that have undergone various modifications and improvements based on the knowledge of those skilled in the art, including the aspects described in the disclosure of the present invention.

[0213] REFERENCE SIGNS LIST 1 Medical support system 2 Endoscope 3 Display device 4 Control device 5 Diagnostic imaging device 6 Medical support device 7 Laptop computer 21 Insertion section 22 Operation section 23 Universal cord 24 Tip section 64 Database 65, 531 Program recording section 66, 410 Control section 410a Imaging control section 410b, 661 Acquisition section 410c, 663 Determination section 410d Display control section 410e Communication control section 622 Discrimination section 662 Discrimination section 664 Calculation section 665 Generation section 666 Creation section 641 Patient information DB 642 Endoscope DB 643 Image data DB 644 Examination information DB 645 Observation order information DB 646 Report item DB 647 Related information DB 648 Report information DB 649 Operator information DB

Claims

1. A medical support device including a processor, The processor: Acquiring an image data group generated by an endoscope successively capturing images of a plurality of observation regions of a subject, and examination information including detection result information obtained by detecting each of the plurality of observation regions based on the image data group; Based on observation order information in which a predetermined order of observation of organs and regions is set, the image data group, and the examination information, it is determined whether or not the tip of the insertion section of the endoscope has reached the final arrival region set in the observation order, and when it is determined that the tip has reached the final arrival region, it distinguishes between first image data captured before the final arrival region and a second image data group captured at the final arrival region; generating related information including time information related to the movement of the endoscope within the subject in the second image data group based on a discrimination result obtained by discriminating between the first image data and the second image data group and the examination information; Medical support equipment.

2. The medical support device according to claim 1, The related information is time information when the endoscope captured the second image data group; Medical support equipment.

3. The medical support device according to claim 2, The time information is an insertion time during which the distal end of the insertion section of the endoscope is inserted from outside the body of the subject to a predetermined observation site; an observation time during which the endoscope observes the inside of the subject; Including, Medical support equipment.

4. The medical support device according to claim 3, The inspection information includes: and further including speed information associating insertion speed information regarding an insertion speed of the distal end portion moving inside the subject based on the image data group with a measurement time from a start time when detection of the insertion speed started to an end time when detection of the insertion speed ended. Medical support equipment.

5. The medical support device according to claim 4, The detection result information is A type of organ or part, and a detection time when the type of organ or part was detected; Including, The processor: identifying an arrival time at which the tip portion reaches the final arrival site based on the detection time, and thereby distinguishing between the first image data and the second image data group; Medical support equipment.

6. The medical support device according to claim 5, The inspection information includes: Still image data captured by the endoscope when a release signal is input from an operation member provided in the endoscope, and an imaging time when the still image data is captured; The processor: identifying the arrival time based on the detection result information and an image capturing time associated with the still image data; Medical support equipment.

7. The medical support device according to claim 6, The processor: Based on the detection result information, the imaging time associated with the still image data, and the speed information, the arrival time is identified as a measurement time at which the imaging time associated with the still image data is within a determination time of the final arrival portion included in the detection result information and the speed of the tip portion becomes less than a predetermined threshold. Medical support equipment.

8. The medical support device according to claim 7, The processor: calculating the insertion time based on the arrival time and an insertion start time of the subject based on the examination information; Medical support equipment.

9. The medical support device according to claim 8, The processor: If it is determined that the distal end portion has not reached the final destination, stenosis information indicating that stenosis has occurred at the most recently observed location included in the detection result information is generated and recorded in association with the related information. Medical support equipment.

10. The medical support device according to claim 1, The processor: generating report information including the relevant information; Medical support equipment.

11. An endoscope and a control device for controlling the endoscope; a diagnostic imaging device; A medical support device; A medical support system comprising: The endoscope comprises: an insertion section that can be inserted into a subject; a tip section provided at a tip of the insertion section, which successively captures images of a plurality of predetermined observation sites on the subject to sequentially generate image data groups and output the image data groups to the control device; Equipped with The diagnostic imaging device generating detection result information for detecting each of the plurality of observed regions from the image data group using the image data group and a first inference model that has previously learned the characteristics of organs and regions, and outputting the detection result information to the control device; The medical support device includes: a first processor; The first processor acquiring an image data group from the control device and examination information including detection result information obtained by detecting each of the plurality of observation sites based on the image data group; Based on observation order information in which a predetermined order of observation of organs and regions is set, the image data group, and the examination information, it is determined whether or not the tip of the insertion section of the endoscope has reached the final arrival region set in the observation order, and when it is determined that the tip has reached the final arrival region, it distinguishes between first image data captured before the final arrival region and a second image data group captured at the final arrival region; generating related information including time information related to the movement of the endoscope within the subject in the second image data group based on a discrimination result obtained by discriminating between the first image data and the second image data group and the examination information; Medical support system.

12. The medical support system according to claim 11, The control device a second processor; The second processor determining whether the tip portion has reached a predetermined final destination site of the subject based on the detection result information; when it is determined that the tip portion has reached the final destination portion, outputting information that the tip portion has reached the final destination portion. Medical support system.

13. The medical support system according to claim 12, The endoscope comprises: further comprising an operation member that receives an input of a release signal that instructs image capture; The second processor after determining that the distal end portion has reached the final reach portion, if the imaging diagnostic apparatus detects the next next reach portion without inputting a release signal from the operation member, a warning is output to the effect that the final reach portion has not yet been imaged. Medical support system.

14. The medical support system according to claim 13, The second processor generating and recording examination information that associates the detection result information, still image data captured by the endoscope when a release signal is input from the operation member, and an imaging time when the still image data was captured; Medical support system.

15. The medical support system according to claim 14, The second processor When it is determined that the distal end portion has not reached the final destination site and a release signal is input from the operation member, stenosis information indicating that stenosis has occurred in the organ or site currently being detected by the imaging diagnostic device is added to the examination information and recorded. Medical support system.

16. The medical support system according to claim 11, The diagnostic imaging device outputting, to the control device, speed information that associates insertion speed information regarding the insertion speed of the tip portion moving inside the subject with a measurement time from a start time when detection of the insertion speed begins to a finish time when detection of the insertion speed ends, using the image data group and a second inference model that has learned the insertion speed of the tip portion using the image data group when the tip portion is inserted into the subject; Medical support system.

17. The medical support system according to claim 11, The related information is time information when the endoscope captured the second image data group; Medical support system.

18. The medical support system according to claim 17, The time information is an insertion time during which the distal end of the insertion section of the endoscope is inserted from outside the body of the subject to the predetermined observation site; an observation time during which the endoscope observes the inside of the subject; Including, Medical support system.

19. A method for operating a medical support device including a processor, comprising: the processor: Acquiring an image data group generated by an endoscope successively capturing images of a plurality of observation regions of a subject, and examination information including detection result information obtained by detecting each of the plurality of observation regions based on the image data group; Based on observation order information in which a predetermined order of observation of organs and regions is set, the image data group, and the examination information, it is determined whether or not the tip of the insertion section of the endoscope has reached the final arrival region set in the observation order, and when it is determined that the tip has reached the final arrival region, it distinguishes between first image data captured before the final arrival region and a second image data group captured at the final arrival region; generating related information including time information related to the movement of the endoscope within the subject in the second image data group based on a discrimination result obtained by discriminating between the first image data and the second image data group and the examination information; Including, A method of operating a medical support device.