Medical system and control method of medical system

Abstract

The medical system comprises: an insertion length acquisition device, which acquires the insertion length of an insertion portion of an endoscope inserted into a subject; a communication unit, which receives the information of the insertion length acquired by the insertion length acquisition device; and a recording data generation unit, which associates an image of the subject captured by an imaging element provided on the insertion portion of the endoscope with the information of the insertion length received by the communication unit, and records the information of the insertion length as metadata when recording image data of the captured image.

Description

Technical Field

[0001] The present invention relates to a medical system for recording an endoscopic image and information on the insertion length of an endoscope in association with each other. Background Art

[0002] Endoscopes have been widely used in the medical field. Endoscopes are medical devices with an elongated insertion portion that is flexible and can be inserted into a subject by a surgeon to observe the inside of the subject. Endoscopic images of the subject captured by the endoscope can be displayed on a monitor. However, it is not possible to know how the endoscope insertion portion is inserted into the subject from the endoscopic images.

[0003] Therefore, as a device capable of knowing the insertion state of an endoscope when the endoscope is inserted, an endoscope shape observation device has been developed, which includes: a receiving antenna composed of a plurality of transmitting coils assembled in an insertion portion and a plurality of induction coils arranged on a coil block; and a monitor that displays the insertion shape of the insertion portion. For example, various endoscope shape observation devices disclosed in Japanese Patent Laid-Open No. 2005-312770 and the like have been proposed.

[0004] However, for diagnosis or case recording, medical images obtained by endoscopes are recorded on various media. In recent years, with the increase in the capacity of recording media, dynamic images from endoscopes have also begun to be recorded. For the purpose of recording cases, medical images can be used as backups of evidence images or as educational materials. In addition, by comparing the dynamic images of past cases of the same patient with the current dynamic images, it is sometimes effective during observation.

[0005] However, even if the entire period of the case is recorded, depending on the purpose, sometimes only images of a part of the operation or examination period are needed, and sometimes editing work is performed after recording the case. For example, sometimes you only want to record dynamic images of a certain part of the body cavity. However, it is not easy to know which position in the body cavity the image is from the endoscopic image, and the editing work takes a long time. In addition, even if the dynamic images of past cases are compared during observation, it is difficult to determine which position in the body cavity the dynamic images of past cases are from, and sometimes it takes time to reproduce the necessary scenes.

[0006] An object of the present invention is to provide a medical system capable of easily identifying which position in a body cavity an endoscopic image is an image of by recording information on the insertion length of an endoscope in association with an endoscopic image. Summary of the invention

[0007] Means for solving problems

[0008] A medical system according to one embodiment of the present invention comprises: an insertion length acquisition device, which acquires the insertion length of an insertion portion of an endoscope inserted into a subject; a communication unit, which receives the information of the insertion length acquired by the insertion length acquisition device; and a recording data generation unit, which associates an image of the subject captured by an imaging element provided on the insertion portion of the endoscope with the information of the insertion length received by the communication unit, and records the information of the insertion length as metadata when recording image data of the captured image. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a block diagram showing a medical system according to a first embodiment of the present invention.

[0010] Figure 2 It is an explanatory diagram showing an example of the configuration of a medical system in an operating room.

[0011] Figure 3 It is an explanatory diagram showing a situation of endoscopic examination.

[0012] Figure 4 It is an explanatory diagram showing a display example of a reproduced image.

[0013] Figure 5A It is an explanatory diagram showing a display example of a reproduced image.

[0014] Figure 5B It is an explanatory diagram showing a display example of a reproduced image.

[0015] Figure 6 It is a block diagram showing the second embodiment of the present invention.

[0016] Figure 7 This is a flowchart showing an example of the operation flow adopted in the second embodiment.

[0017] Figure 8 This is an explanatory diagram for explaining the operation of the second embodiment.

[0018] Fig. 9 This is a block diagram showing a third embodiment of the present invention.

[0019] Fig.10 This is an explanatory diagram for explaining a storage dynamic image generated by the image generating unit 26 .

[0020] Fig.11 It is a block diagram showing a fourth embodiment of the present invention.

[0021] Fig.12 is shown by Figure 8 This is a diagram illustrating changes in the insertion length at each recording time of an endoscopic image using the same method.

[0022] Fig.13 It is an explanatory diagram showing a display example of an image displayed on the monitor 5 .

[0023] Fig.14 It is an explanatory diagram for explaining a modified example. DETAILED DESCRIPTION

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0025] (First embodiment)

[0026] Figure 1 : is a block diagram showing a medical system according to a first embodiment of the present invention. Figure 2 : is an explanatory diagram showing an example of the configuration of a medical system in an operating room. Figure 3 It is an explanatory diagram showing a situation of endoscopic examination.

[0027] This embodiment uses an endoscope shape observation device for observing the insertion shape of an endoscope to associate information indicating the position of the observation image in the body cavity, such as insertion length information indicating the length of the insertion portion of the endoscope inserted into the body cavity (hereinafter referred to as the insertion length), with the endoscope image and record it. Thus, when editing the endoscope image or when referring to past cases during observation, the observation position represented by the endoscope image can be easily identified.

[0028] exist Figure 2 and Figure 3 In the present invention, the medical system 1 is configured to include an endoscope device 2, an endoscope shape observation device 3, and an image recording device 10. The endoscope device 2 includes an endoscope 4, a light source device 8, a video processor 9, and a monitor 5. The endoscope 4 includes an elongated and flexible insertion portion 4b inserted into a body cavity of a subject P as a subject, an operation portion 4a connected to the base end of the insertion portion 4b and provided with various operators, and a cable 4c for connecting the operation portion 4a and the video processor 9.

[0029] exist Figure 2 , an example is shown in which the endoscope shape observation device 3, the image recording device 10, the light source device 8, and the video processor 9 are placed on a medical cart 40. In addition, the monitor 5 is mounted on a movable arm provided on the medical cart 40. The endoscope 4 can be hung on a hook of the medical cart 40.

[0030] Figure 3 FIG. 4 shows a state where the insertion portion 4b is inserted into the large intestine from the anus of the subject P lying on the examination bed 6. Figure 34 shows a situation in which a surgeon O holds an operation portion 4a and an insertion portion 4b of an endoscope 4 connected to a video processor 9 on a medical cart 40 via a cable 4c.

[0031] The light source device 8 generates illumination light for illuminating the subject. The illumination light from the light source device 8 is guided to the front end of the insertion portion 4b by a light guide (not shown) inserted through the insertion portion 4b of the endoscope 4, and irradiates the subject from the front end of the insertion portion 4b. An imaging element (not shown) is arranged at the front end of the insertion portion 4b, and the reflected light (return light) from the subject reflected by the subject is imaged as an optical image of the subject on the light receiving surface of the imaging element. The imaging element is driven and controlled by the video processor 9, so that the optical image of the subject is converted into an image signal and output to the video processor 9. The video processor 9 has an image signal processing unit (not shown), which receives the image signal from the imaging element, performs signal processing, and outputs the endoscopic image after the signal processing to the monitor 5. In this way, the endoscopic image of the subject is displayed on the display screen of the monitor 5.

[0032] The insertion portion 4b has a bending portion at the distal end thereof, and the bending portion is driven to bend by a bending knob 4d provided on the operation portion 4a. The operator can push the insertion portion 4b into the body cavity while operating the bending knob 4d to bend the bending portion.

[0033] The insertion state of the insertion portion 4b is observed by a known endoscope shape observation device 3. A plurality of transmission coils for detecting the insertion state are arranged on the insertion portion 4b. For example, a plurality of transmission coils are arranged at predetermined intervals in the axial direction of the insertion portion 4b. In addition, for example, a probe having a plurality of transmission coils installed at predetermined intervals may be inserted into the treatment instrument insertion channel and the front end or rear end of the probe may be fixed.

[0034] On the other hand, a receiving antenna 7 is arranged near the bed 6. The receiving antenna 7 is connected to the endoscope shape observation device 3 via a cable 7a. The receiving antenna 7 has a plurality of coil blocks not shown in the figure, and is arranged, for example, on the side of the bed 6. Each coil block of the receiving antenna 7 is composed of, for example, three induction coils wound in three directions in a manner perpendicular to their respective coil surfaces, and in the entire receiving antenna 7, for example, four coil blocks, that is, 12 induction coils are arranged. Each induction coil detects a signal proportional to the intensity of the axial component of the magnetic field perpendicular to the coil surface of each induction coil. For example, the coil block receives the generated magnetic field and converts it into a voltage signal, and outputs the voltage signal as a detection result. The operating state of the transmitting coil and the receiving antenna 7 arranged in the insertion part 4b is controlled by the endoscope shape observation device 3.

[0035] like Figure 1As shown in FIG. 1 , a control unit 31 is provided in the endoscope shape observation device 3. The control unit 31 is composed of a processor such as a CPU, and can control each component according to a program stored in a memory not shown in the figure, or can realize part or all of the functions by a hardware electronic circuit. The control unit 31 controls the entire endoscope shape observation device 3.

[0036] The control unit 31 supplies a sine wave signal for driving the plurality of transmitting coils provided in the insertion portion 4b to each transmitting coil (not shown). Each transmitting coil in the insertion portion 4b radiates electromagnetic waves accompanied by a magnetic field to the surroundings by applying a high-frequency sine wave. In addition, the endoscope shape observation device 3 can sequentially drive each transmitting coil at an appropriate time interval, for example, at a few milliseconds.

[0037] The receiving antenna 7 receives the magnetic field generated by the transmitting coil through the induction coil and converts it into a voltage signal. The receiving antenna 7 provides the voltage signal as a detection result to the receiving unit 33 of the endoscope shape observation device 3. The receiving unit 33 is provided with the signal from the receiving antenna 7, and outputs it to the position detection unit 34 after performing predetermined signal processing such as amplification processing.

[0038] The position detection unit 34 is composed of, for example, a DSP, and performs frequency extraction processing (Fourier transform: FFT) on the input digital data, separates and extracts the magnetic field detection information of the frequency component corresponding to the high-frequency sine wave of each transmitting coil, and calculates the spatial position coordinates of each transmitting coil based on each digital data of the separated magnetic field detection information. The calculation result of the position coordinates by the position detection unit 34 is provided to the insertion shape image generation unit 35. The insertion shape image generation unit 35 connects the position coordinates of each transmitting coil and generates a linear image as an insertion shape image.

[0039] The insertion shape image generation unit 35 supplies the generated insertion shape image to the control unit 31. The control unit 31 converts the insertion shape image generated by the insertion shape image generation unit 35 into display data for display on the monitor 5 and supplies it to the transmission unit 36. The transmission unit 36 ​​transmits the input display data to the video processor 9. The video processor 9 synthesizes the endoscopic image from the endoscope 4 and the insertion shape image from the endoscopic shape observation device 3 provided as display data, and supplies the synthesized image to the monitor 5. The monitor 5 can be composed of, for example, an LCD, and simultaneously displays the endoscopic image and the insertion shape image based on the relative positional relationship between the transmission coil and the receiving antenna 7 according to the output of the video processor 9.

[0040] In addition, the insertion shape image generated by the insertion shape image generating unit 35 is generated using a coordinate system (hereinafter referred to as a measurement coordinate system) based on the position of the receiving antenna 7. The control unit 31 performs a coordinate transformation for displaying the insertion shape image at a predetermined position on the display screen of the monitor 5. That is, the control unit 31 performs a coordinate transformation on the input image to transform the measurement coordinate system into the display coordinate system. Through this coordinate transformation, the control unit 31 can display the insertion shape image at a predetermined position on the display screen of the monitor 5 in a predetermined direction and size. In addition, the display position, direction, and size of the insertion shape image can be changed by the operation of the surgeon.

[0041] An insertion length calculation unit 37 is provided in the endoscope shape observation device 3 as an insertion length acquisition device. The insertion length calculation unit 37 calculates the length of the insertion unit 4b inserted into the body cavity. When it is assumed that after the insertion unit 4b is inserted into the subject P, a specified transmitting coil in the transmitting coil is located at the anus position (insertion position) of the subject P, the insertion unit 4b is inserted into the body cavity from the position of the specified coil to the front end of the insertion unit 4b. The position of each transmitting coil inserted into the insertion unit 4b from the front end of the insertion unit 4b is known. Therefore, by detecting the coil located at the anus position, the length (insertion length) of the insertion unit 4b inserted into the body cavity can be obtained.

[0042] In addition, various methods for detecting the anus position are considered, such as a method of positioning the anus of the subject P at a predetermined position relative to the receiving antenna 7, a method of placing a marker having a built-in transmitting coil near the anus and receiving the magnetic field generated by the high-frequency sine wave applied to the marker via the receiving antenna 7, and a method of detecting the position of a coil disposed at the front end of the insertion portion 4b when the insertion portion 4b starts to be inserted into the anus.

[0043] The insertion length calculation unit 37 is provided with the information of the anus position coordinates and the information of the position coordinates of each coil currently inserted in the insertion unit 4b from the position detection unit 34. The insertion length calculation unit 37 detects the coil near the anus position coordinates among the coils, and calculates the length from the position of the coil to the front end of the insertion unit 4b as the insertion length. The insertion length calculation unit 37 sequentially updates the insertion length according to the output of the position detection unit 34, and provides the insertion length information to the communication unit 38. The communication unit 38 sends the input insertion length information to the image recording device 10.

[0044] Alternatively, the insertion length information from the insertion length calculation unit 37 may be provided to the control unit 31, and the insertion length information may be transmitted from the transmission unit 36 ​​to the video processor 9 together with the insertion shape image. In this case, the video processor 9 may generate an image including the endoscope image under observation, the insertion shape image, and the mark (indication) of the insertion length.

[0045] The image recording device 10 has a control unit 11. The control unit 11 is composed of a processor such as a CPU, and can control each unit by operating according to a program stored in a memory not shown in the figure, or can realize part or all of the functions by a hardware electronic circuit. The image recording device 10 is provided with an endoscopic image from the video processor 9. The image holding unit 12 of the image recording device 10 holds the endoscopic image from the video processor 9 and is controlled by the control unit 11 to output the held endoscopic image to the recording data generating unit 15. In addition, the image output from the video processor 9 is sometimes only an endoscopic image, and sometimes a composite image of an endoscopic image and an insertion shape image. In the following, in order to simplify the description, it is sometimes assumed that the endoscopic image includes both.

[0046] The image recording device 10 is provided with a communication unit 19. The communication unit 19 communicates with the communication unit 38 of the endoscope shape observation device 3, receives the insertion length information from the endoscope shape observation device 3, and provides it to the insertion length information storage unit 14. The insertion length information storage unit 14 stores the insertion length information, and is controlled by the control unit 11 to output the stored insertion length information to the recording data generation unit 15. In addition, the time information generation unit 13 of the image recording device 10 is controlled by the control unit 11 to generate time information, and outputs the generated time information to the recording data generation unit 15.

[0047] The recording data generating unit 15 is provided with the endoscopic image from the image holding unit 12, the time information from the time information generating unit 13, and the insertion length information from the insertion length information holding unit 14. The time from the start of recording of the endoscopic image is determined based on the time information. The recording data generating unit 15 associates the insertion length information at each recording time of the endoscopic image with the endoscopic image and records it as metadata. That is, the recording data generating unit 15 uses the time information to associate the endoscopic image with the insertion length information and records it in the recording unit 16.

[0048] For example, the recording data generating unit 15 may generate an image file composed of image data based on the endoscopic image and metadata based on time information and insertion length information. The recording data generating unit 15 is controlled by the control unit 11 to provide the generated image file to the recording unit 16 for recording.

[0049] Various recording media such as a hard disk and an IC memory can be used as the recording unit 16. The recording unit 16 is not limited to being built in the image recording device 10, and various disk media, USB memories, or various memories such as a file server on a network can also be used.

[0050] The playback unit 17 can read and play back the image file recorded in the recording unit 16 . The playback unit 17 reads the played back endoscopic image and metadata associated with the endoscopic image from the recording unit 16 , and outputs the read metadata to the display control unit 18 .

[0051] The display control unit 18 is supplied with the endoscopic image and the insertion length information included in the metadata from the reproduction unit 17, and can supply an image including a mark based on the insertion length information (insertion length mark) and the endoscopic image to the monitor 5 for display. In addition, the display control unit 18 is controlled by the control unit 11 so that the endoscopic image including the insertion length mark can be displayed on the display screen of the monitor 5 in various display modes.

[0052] In addition, the monitor 5 can also synthesize and display the image from the video processor 9 and the image from the image recording device 10. For example, the monitor 5 can also display the image from the video processor 9 and the image from the image recording device 10 on a dual screen.

[0053] Next, refer to Figure 4 , Figure 5A and Figure 5B The operation of the embodiment configured as above will be described. Figure 4 , Figure 5A and Figure 5B It is an explanatory diagram showing a display example of a reproduced image.

[0054] Currently, if Figure 3 As shown, the surgeon inserts the insertion section 4b into the large intestine from the anus of the subject P lying on the examination bed 6 in the side-lying position. The endoscope shape observation device 3 obtains the three-dimensional position coordinates of the multiple transmitting coils built into the insertion section 4b at a predetermined time interval. That is, the control unit 31 of the endoscope shape observation device 3 provides high-frequency signals to the multiple transmitting coils of the insertion section 4b at a predetermined timing. The transmitting coils provided with the high-frequency signals generate electromagnetic waves accompanied by a magnetic field. The magnetic field is received in each coil block of the receiving antenna 7, and the detection result corresponding to the magnetic field strength is taken into the position detection unit 34 via the receiving unit 33 of the endoscope shape observation device 3.

[0055] The position detection unit 34 is provided with information on the drive timing of each transmitting coil from the control unit 31, and obtains the three-dimensional position coordinates of each transmitting coil according to the detection results of the coil block according to a known position estimation algorithm. The position coordinates are provided to the insertion shape image generation unit 35, and the insertion shape image generation unit 35 generates an insertion shape image based on the position coordinates. Each transmitting coil is arranged at a known position at a predetermined interval along the shape of the insertion part 4b. That is, the position of each transmitting coil represents a discrete position of the insertion part 4b. The insertion shape image generation unit 35 generates an insertion shape image corresponding to the approximate shape of the insertion part 4b by interpolating the discrete positions. In addition, the insertion shape image is obtained in the measurement coordinate system.

[0056] The insertion shape image generating unit 35 provides the generated insertion shape image to the control unit 31. The control unit 31 generates display data based on the insertion shape image and outputs it to the transmitting unit 36. The transmitting unit 36 ​​transmits the display data to the video processor 9. The endoscope 4 provides the video processor 9 with an endoscopic image of the body cavity captured by the imaging element disposed at the front end of the insertion unit 4b, and the video processor 9 outputs a composite image synthesized by the endoscopic image and the insertion shape image to the monitor 5. In this way, the composite image of the endoscopic image and the insertion shape image can be displayed on the display screen of the monitor 5.

[0057] The insertion length calculation unit 37 detects the coil at the anus position based on the output of the position detection unit 34, and calculates the length from the position of the coil to the front end of the insertion unit 4b as the insertion length. The insertion length calculation unit 37 provides the information of the calculated insertion length to the communication unit 38, and the communication unit 38 sends the insertion length information to the communication unit 19 of the image recording device 10.

[0058] The control unit 11 of the image recording device 10 provides the insertion length information received via the communication unit 19 to the insertion length information holding unit 14 for storage. In addition, the insertion length information is updated successively. In addition, the control unit 11 stores the endoscopic image output from the video processor 9 in the image holding unit 12. The control unit 11 provides the endoscopic image held by the image holding unit 12, the moment information from the moment information generating unit 13, and the insertion length information held by the insertion length information holding unit 14 to the recording data generating unit 15. The recording data generating unit 15 uses the moment information to record the endoscopic image and the insertion length information in the recording unit 16 in association with each other. For example, the recording data generating unit 15 generates an image file consisting of the image data of the endoscopic image, the moment information, and the metadata of the insertion length information, and provides it to the recording unit 16 for recording.

[0059] When the operator or the like operates the operation unit (not shown) to instruct the reproduction of the image file recorded in the recording unit 16, the control unit 11 instructs the reproduction unit 17 to reproduce. The reproduction unit 17 reads the image file recorded in the recording unit 16 and outputs it to the display control unit 18. The display control unit 18 is provided with the endoscopic image and the insertion length information included in the metadata from the reproduction unit 17, and provides the monitor 5 with an image including the insertion length mark based on the insertion length information and the endoscopic image. In this way, the endoscopic image including the insertion length mark is displayed on the display screen of the monitor 5.

[0060] The insertion length information indicates the insertion length of the insertion portion 4b of the endoscope 4 for which the endoscopic image corresponding to the insertion length information is obtained. That is, the insertion length information associated with the endoscopic image at a predetermined recording time indicates the insertion position, i.e., the observation position, at the predetermined time. Therefore, by simultaneously displaying the endoscopic image and the insertion length mark, the operator can easily recognize at which insertion position (observation position) the endoscopic image is captured.

[0061] Figure 4 , Figure 5A and Figure 5B An example of an image displayed on the display screen 5a of the monitor 5 is shown. Figure 4 , Figure 5A and Figure 5B In the display example of , the endoscopic image provided to the image holding unit 12 is a composite image of the endoscopic image and the insertion shape image. In addition, in each of these figures, various indicators related to the insertion length are indicators based on the reproduced insertion length information.

[0062] exist Figure 4 In the example of FIG. 5 , an endoscopic image display area 41, an insertion shape image display area 42, and a status information display area 43 are provided on the display screen 5a by the display control unit 18. An endoscopic image 41a is displayed in the endoscopic image display area 41, and an insertion shape image 42a, a mark 42b indicating that the subject P is in a supine position, and a bar mark 42c indicating the insertion length are displayed in the insertion shape image display area 42. In addition, a mark 43a indicating the date and the recording time and an insertion length mark 43b are displayed in the status information display area 43. It can be seen from the insertion length mark 43b that the endoscopic image 41a is an image observed at a position 150 cm after the anus is inserted.

[0063] exist Figure 5AIn the example of , an endoscopic image display area 51 and a status information display area 53 are provided on the display screen 5a by the display control unit 18. Furthermore, in the endoscopic image display area 51, an insertion shape image display area 52 is provided as a sub-screen. An endoscopic image 51a is displayed in the endoscopic image display area 51, and an insertion shape image 52a, a mark 52b indicating that the subject P is in a supine position, and a bar mark 52c indicating the insertion length are displayed in the insertion shape image display area 52 of the sub-screen. In addition, a mark 53a indicating the date and the recording time and an insertion length mark 53b are displayed in the status information display area 53. It can be seen from the insertion length mark 53b that the endoscopic image 51a is an image observed at a position 150 cm after insertion from the anus.

[0064] Figure 5B It shows that Figure 5A The insertion shape image display area 55 is displayed in an example in which the insertion shape image display area 52 in FIG. is displayed in an overlapping manner on the endoscope image display area 51. For example, the operator can make such a pop-up display by operating an operation unit (not shown). The insertion shape image display area 55 displays an enlarged insertion shape image 55a, a mark 55b indicating that the subject P is in the supine position, and a bar mark 55c indicating the insertion length.

[0065] Thus, in this embodiment, the insertion length information is recorded as metadata in association with the endoscopic image. Thus, when reproducing, by displaying the insertion length mark based on the insertion length information together with the endoscopic image, the operator can easily recognize which observation part the displayed endoscopic image is while watching the display on the monitor. In addition, since the insertion length information is recorded as metadata, the image portion of the desired observation position in the endoscopic image recorded in the form of a dynamic image can be easily retrieved using the insertion length information.

[0066] (Second embodiment)

[0067] Figure 6 FIG. 2 is a block diagram showing a second embodiment of the present invention. Figure 6 In, with Figure 1 The same components are denoted by the same reference numerals and their description is omitted. This embodiment differs from the first embodiment in that an image recording device 20 having a chapter generating unit 11a provided in a control unit 11 is used. This embodiment is an embodiment in which chapter marks are added to endoscopic images (moving images) to be recorded.

[0068] The chapter generating unit 11a of the control unit 11 can add chapter marks at a predetermined timing to the image file recorded by controlling the recorded data generating unit 15. For example, the chapter generating unit 11a can control to add chapter marks to the image file at a timing when the operator operates an operating unit (not shown), or can add chapter marks corresponding to the insertion length information by reproducing the recorded image file and reading the insertion length information.

[0069] For example, in the middle of recording an endoscopic image (dynamic image), the chapter generating unit 11a can control the recording data generating unit 15 by reading the insertion length information (not shown) provided from the insertion length information retaining unit 14 to add a chapter mark to the image portion of the endoscopic image corresponding to the specified insertion length.

[0070] For example, the chapter generating unit 11a may add a chapter mark for each predetermined insertion length, or when the tip of the insertion unit 4b reaches the anus, that is, when the insertion of the insertion unit 4b starts and when the removal is completed.

[0071] Furthermore, the chapter generating unit 11a may add a chapter mark at the time when the insertion unit 4b inserted from the anus reaches the cecum, that is, at the time when the insertion length becomes the maximum. For example, the chapter generating unit 11a is supplied with an image file (not shown) read from the recording unit 16 from the reproducing unit 17, and can control the recording data generating unit 15 so as to add a chapter mark to the image portion of the endoscopic image corresponding to the insertion length information indicating the maximum insertion length.

[0072] Next, refer to Figure 7 and Figure 8 The operation of the embodiment configured in this way will be described. Figure 7 is a flowchart showing an example of the operation flow adopted in the second embodiment. Figure 8 It is an explanatory diagram for explaining the operation of the embodiment.

[0073] The chapter generating unit 11a Figure 7 In step S1, the image data recorded in the recording unit 16 is read sequentially in time order from metadata at the start of recording, for example, and the insertion length obtained based on the insertion length information is confirmed (step S2).

[0074] Figure 8The horizontal axis represents the recording time of the endoscopic image, and represents the change in the insertion length based on the insertion length information recorded in association with each recording time. Before the insertion portion 4b is inserted, that is, when the front end portion of the insertion portion 4b is located outside the body cavity, the insertion length is not detected, and the insertion length is 0 at the start of insertion. In step S3, the chapter generation unit 11a determines whether the insertion lengths confirmed sequentially from the start of recording are 0. When the insertion starts and the insertion length is 0, the chapter generation unit 11a transfers the processing to step S4, and stores the time corresponding to the timing of the insertion length being 0 as the chapter time of the insertion length being 0 in a memory not shown. In step S5, the chapter generation unit 11a determines whether the storage of the chapter time of the insertion length being 0 is the second time. At the start of insertion, the first storage of the chapter time is performed, so the chapter generation unit 11a returns the processing to step S1 to read the next insertion length information.

[0075] The insertion length gradually increases by inserting the insertion unit 4b. When the chapter generating unit 11a determines in step S3 that the insertion length is not 0, the processing is transferred to step S6 to determine whether the insertion length confirmed this time is greater than the insertion length confirmed until the last time, that is, to determine whether an insertion length greater than the maximum value of the confirmed insertion length is confirmed. When the insertion is performed, the insertion length gradually increases, so the chapter generating unit 11a transfers the processing to step S7, updates the time (chapter time) of obtaining the maximum insertion length and stores it in a memory not shown, and then returns the processing to step S1.

[0076] After that, the same operation is repeated, and the maximum value of the insertion length is updated during the insertion, and the chapter time is updated and stored in step S7. In addition, even if the insertion portion 4b is inserted and removed before the front end portion of the insertion portion 4b reaches the cecum, the insertion length may be shortened, but the insertion length becomes the maximum when the front end portion of the insertion portion 4b finally reaches the cecum. Figure 8 In the example, there are two times when the insertion length is the largest, and it can be seen that once the front end of the insertion part 4b reaches the cecum, the insertion part 4b is pulled out and inserted again to reach the cecum. In step S7, the time when the cecum is finally reached is stored as the chapter time.

[0077] When the operator starts to pull out the insertion portion 4b, the insertion length gradually becomes a smaller value, and when the front end of the insertion portion 4b reaches the anus, the insertion length becomes 0. Therefore, the chapter generation unit 11a records the chapter time when the insertion length is 0 for the second time in step S4. When the chapter generation unit 11a performs the storage of step S4 twice, in the next step S8, the chapter mark is added corresponding to the chapter time stored in steps S4 and S7. That is, the chapter generation unit 11a controls the recording data generation unit 15 to record the chapter mark as metadata of the recording position corresponding to the chapter time in the image data recorded by the recording unit 16.

[0078] exist Figure 8 In the example, chapter 01 is added corresponding to the moment when the front end of the insertion portion 4b reaches the anus from outside the body cavity and the insertion begins. In addition, chapter M is added corresponding to the moment when the front end of the insertion portion 4b reaches the cecum and the insertion interval ends and the withdrawal interval begins. Furthermore, the front end of the insertion portion 4b is withdrawn from the anus, and chapter 02 is added corresponding to the moment when the withdrawal interval ends. That is, the chapter generating unit 11a can distinguish each medical scene, such as outside the body cavity, insertion scene, withdrawal scene, etc., by detecting 0, the final maximum value, and 0 of the insertion length, and can specify the division of these scenes by chapter marks. Therefore, by using chapter marks, it is possible to perform the beginning of the reproduced image, edit each scene, etc.

[0079] Thus, in this embodiment, chapter marks can be automatically added to image data according to the insertion length. Therefore, by using the chapter marks, the endoscopic image at the observation position of the specified insertion length can be easily retrieved. For example, by using a playback device having a function of jumping to the image part of the chapter mark, the endoscopic image at the specified observation position can be retrieved. In addition, even when editing, by using chapter marks, the editing work can be facilitated.

[0080] (Third embodiment)

[0081] Fig. 9 : is a block diagram showing a third embodiment of the present invention. Fig. 9 In, with Figure 6 The same components are denoted by the same reference numerals and their description is omitted. This embodiment can automate the editing of recorded endoscopic dynamic images.

[0082] The present embodiment is different from the second embodiment in that an image recording device 25 is used in place of the image recording device 20 to which an image generating unit 26 is added. The image generating unit 26 is controlled by the control unit 11 to generate a dynamic image for storage by editing the main image, for example, by adjusting the image quality of the main image, using the image recorded by the recording unit 16 as the main image. The dynamic image for storage generated by the image generating unit 26 is recorded in the recording unit 16.

[0083] Next, refer to Fig.10 The operation of the embodiment configured in this way will be described. Fig.10 This is an explanatory diagram for explaining a storage dynamic image generated by the image generating unit 26 . Fig.10 The following paragraph shows that Figure 8 The change in the insertion length at each recording time of the endoscopic image is similarly performed, and the upper part shows the recording state of the dynamic image recorded by the recording unit 16 .

[0084] Fig.10 The top section shows the period during which the main dynamic image is recorded. That is, the endoscope 4 starts recording from the state in which the front end of the insertion portion 4b is located outside the body cavity, and shoots during the entire period from the insertion section 4b, the withdrawal section, and the state in which the front end of the insertion portion 4b is withdrawn and located outside the body cavity. The video processor 9 provides the endoscopic images of the entire period to the image recording device 25, and the recording data generation unit 15 records the dynamic images of the entire period as the main dynamic image in the recording unit 16. In addition, in subsequent editing, the video processor 9 generates a predetermined high-quality dynamic image as the main dynamic image, so that dynamic images of various qualities can be generated. Fig.10 The grids showing the narrow intervals during the recording period of the main moving image indicate that high-quality recording is performed.

[0085] However, when considering the purpose of the recorded dynamic images, it is considered that the image portion outside the body cavity is unnecessary. In addition, in large intestine examination, the more important image portion is the extraction period from the arrival of the cecum to the end of the extraction. Therefore, it is considered to generate a dynamic image for storage that takes into account the image quality of these periods by using chapter marks.

[0086] The chapter generating unit 11a adds chapter marks of chapter 01, chapter M and chapter 02 to the recorded images, where chapter 01 indicates the image portion at the beginning of endoscope insertion, chapter M indicates the image portion at the moment when the insertion portion 4b reaches the cecum, and chapter 02 indicates the image portion at the end of endoscope removal.

[0087] The control unit 11 controls the image generation unit 26 based on the information of the chapter marker, and generates two storage dynamic images I and II based on the main dynamic image. That is, the image generation unit 26 deletes the image portion from the start of recording to chapter 01 and the image portion from chapter 02 to the end of recording in the main dynamic image, and generates the storage dynamic image I in which the image portion from chapter 01 to chapter M is converted into a low-quality image. Fig.10 The saving dynamic image I indicates the recording period, and the relatively coarse grid indicates that low-quality recording is performed. The insertion scene is not a very important scene, and by recording it at low quality, the required capacity can be reduced.

[0088] Furthermore, the image generating unit 26 generates a storage moving image II by deleting the image portion from the start of recording to chapter M and the image portion from chapter 02 to the end of recording in the main moving image. Fig.10 The saved moving image II shows the recording period, indicating that only the extraction section is recorded with relatively high image quality. That is, the extraction scene that is more important in the inspection is saved with high image quality, making it easier to observe.

[0089] In addition, the example in which the control unit 11 determines the medical scene based on the information of the chapter marker and sets the image quality of the dynamic image for storage is described, but the medical scene may be determined based on the insertion length information and the image quality of the dynamic image for storage may be set. For example, the control unit 11 may control the image generation unit 26 to record the image with high image quality from the time when the maximum insertion length is finally obtained to the time when the insertion length is 0 and the extraction is completed.

[0090] Thus, in this embodiment, for example, a main dynamic image recorded with high image quality during the entire period of endoscopic examination can be automatically generated into a dynamic image for storage that has been edited according to the insertion length. For example, it is possible to perform editing processing such as deleting the image portion captured outside the body cavity and recording only the important inspection portion with high image quality. In this way, complicated editing operations can be automated and dynamic images suitable for the purpose can be simply generated.

[0091] (Fourth embodiment)

[0092] Fig.11 : is a block diagram showing a fourth embodiment of the present invention. Fig.11 In, with Figure 6 The same components are denoted by the same reference numerals and their description is omitted. This embodiment facilitates comparison with past medical records of the same patient during observation.

[0093] The present embodiment is different from the second embodiment in that an image recording device 27 having a reproduction position determination unit 11b added to the control unit 11 is used instead of the image recording device 20. Insertion length information is provided from the insertion length information storage unit 14 to the reproduction position determination unit 11b. The reproduction position determination unit 11b provides the insertion length information from the insertion length information storage unit 14 as information for specifying a reproduction position to the reproduction unit 17. The reproduction unit 17 reads the image data from the recording unit 16 based on the information from the reproduction position determination unit 11b. That is, the reproduction unit 17 refers to the metadata in the image file recorded in the recording unit 16, reads the image data of the image portion corresponding to the insertion length information, and reproduces the insertion length information, which is consistent with the insertion length information specified by the reproduction position determination unit 11b or has a value within a specified threshold.

[0094] Furthermore, when the image recorded by the recording unit 16 includes image portions of the insertion period and the extraction period, at least two image portions have the same insertion length. In this case, considering that treatment is performed during extraction, the reproduction unit 17 preferably reproduces image data with the same or similar insertion length in the extraction period in consideration of observation.

[0095] The reproduction unit 17 outputs the reproduced image data and metadata to the display control unit 18. The display control unit 18 generates a composite image based on the input image data and metadata, and outputs it to the monitor 5. In addition, the monitor 5 is provided with an endoscopic image from the video processor 9. In addition, the video processor 9 may include an insertion length mark in the endoscopic image to be output.

[0096] Next, refer to Fig.12 and Fig.13 The operation of the embodiment configured in this way will be described. Fig.12 is shown by Figure 8 The same method is used to illustrate the change in insertion length at each recording time of the endoscope image. The upper and lower sections show examples of past case dynamic images and current observation dynamic images of the same subject, respectively. Fig.13 It is an explanatory diagram showing a display example of an image displayed on the monitor 5 .

[0097] Now, it is assumed that a past observation of the subject P is being performed. For the subject P being observed, it is assumed that a past case dynamic image is recorded in the recording unit 16. That is, the image data with time information and insertion length information as metadata is recorded as a case dynamic image in the recording unit 16. In addition, chapter marks may or may not be added to the case dynamic image.

[0098] The operator starts capturing endoscopic images using the endoscope 4, and inserts the insertion portion 4b from the anus of the subject P. The imaging signal from the endoscope 4 is supplied to the video processor 9. In addition, the insertion length calculation unit 37 of the endoscopic shape observation device 3 calculates the insertion length of the insertion portion 4b. The information of the insertion length is supplied to the video processor 9, and the video processor 9 outputs an endoscopic image including an insertion length mark to the monitor 5. In addition, the insertion length information from the endoscopic shape observation device 3 is supplied to the insertion length information storage unit 14 via the communication unit 19 of the image recording device 27.

[0099] In the present embodiment, the reproduction position determination unit 11b of the control unit 11 reads the insertion length information from the insertion length information storage unit 14, and outputs the read insertion length information to the reproduction unit 17 as information for specifying the reproduction position. The reproduction unit 17 reads the insertion length information of the recording unit 16, reads the image data of the image portion that is consistent with or similar to the insertion length information specified by the reproduction position determination unit 11b, and reproduces it. The reproduction unit 17 outputs the read image data and the associated metadata to the display control unit 18. The display control unit 18 adds an insertion length mark to the endoscopic image reproduced by the reproduction unit 17 and outputs it to the monitor 5.

[0100] The insertion length information stored in the insertion length information storage unit 14 changes according to the insertion and removal operation of the insertion unit 4b by the surgeon. According to the change of the insertion length, the image portion of the recording unit 16 reproduced by the reproduction unit 17 also changes, and the image portion of the past observation position corresponding to the observation position (insertion position) in the current observation is read from the recording unit 16 and reproduced. That is, the past image from the display control unit 18 and the current image from the video processor 9 are the past image and the current observation image at the same observation position of the same subject P. The monitor 5 displays these past images and the current observation image on the display screen 5a at the same time.

[0101] Fig.12 The lower part of shows the change of the insertion length corresponding to the endoscope image currently being observed. It can be considered that Fig.12 The change in insertion length during the current observation shown in the lower paragraph is Fig.12 The insertion length changes when recording the past case dynamic images shown in the upper section are different, and the inspection time is also different. However, in this embodiment, regardless of the image recording time, the current observation image and the past case image are displayed simultaneously according to the insertion length. Fig.12 The insertion length shown by the dotted line in the lower section of the image 61a and Fig.12 The image 62a of the same insertion length indicated by the dotted line in the upper stage is the past and present images at the same observation position of the same subject.

[0102] As the surgeon inserts and removes the insertion portion 4b, the current observation position changes and the image changes, and the past image output from the image recording device 27 also changes to the image at the observation position corresponding to the current observation position. Therefore, the surgeon can always judge the current state based on the observed image while comparing it with the past image at the same observation position of the same subject, which is very useful for post-observation.

[0103] Fig.13 The display of the monitor 5 in this case is shown. A display area 61 of the current observation image is provided on the left side of the display screen 5a of the monitor 5, and a display area 62 of the past case image is provided on the right side. The display area 61 displays the endoscopic image 61a currently under observation and the status information mark 61b. The status information mark 61b shows that the endoscopic image 61a was taken at 14:04 on March 14, 2018, and the observation position is the position where the insertion length is 100 cm.

[0104] In addition, an endoscopic image 62a as a past case image and a status information mark 62b are displayed in the display area 62. The status information mark 62b shows that the endoscopic image 62a was taken at 15:03 on February 14, 2015, and the observation position is the position where the insertion length is 100 cm. In addition, it is shown that a chapter mark C is set in the image portion corresponding to the insertion length. In addition, the display of the status information marks 61b and 62b can be turned off.

[0105] Thus, in this embodiment, the reproduction position of the recorded past case images can be set according to the insertion length at the time of current observation, so that the past case images at the same observation position as the current observation position can be displayed simultaneously with the current observation image. The surgeon can compare the current and past images at the same observation position of the same subject without specifying complicated reproduction positions, which is extremely useful for post-observation, etc.

[0106] In addition, the insertion length information can also be associated with the current observation image and recorded as metadata in the recording unit 16. Therefore, in the above description, an example of determining the reproduction position of the past case image based on the current observation position is described, but the reproduction position of the currently recorded observation image can also be determined based on the observation position (insertion position) of the past observation image.

[0107] (Variation Example)

[0108] However, in the above description, the reproduction position determination unit 11b specifies the reproduction position of the recording unit 16 based on the insertion length information of the current observation image. However, the current observation position to be observed during the observation may be specified based on the past case image.

[0109] The playback position determination unit 11b of the control unit 11 may be configured to switch between a first mode in which the playback position of the past case record is specified based on the insertion length information corresponding to the image currently being observed and a second mode in which the playback position is specified based on the insertion length information corresponding to the chapter mark in the past case record. For example, it is assumed that during the past observation, the chapter generation unit 11a sets the chapter mark at the site (insertion position) where treatment was performed based on the operation signal based on the operation of the operator. In this case, the second mode may also be set during the past observation.

[0110] Fig.14 is shown by Figure 8 The same method is used to illustrate the change in insertion length at each recording time of the endoscope image. Fig.14 In the example, in addition to the chapter mark Amin indicating the insertion length of 0 at the beginning of insertion, the chapter mark Amin' indicating the insertion length of 0 at the end of withdrawal, and the chapter mark Bmax' when the cecum reaches the maximum insertion length, the chapter mark Bmax is automatically assigned when the cecum reaches the maximum insertion length for the first time. Fig.14 In the example of FIG. 1 , a manually set chapter mark C is set at the observation position (insertion position) where the operator has discovered a polyp.

[0111] The chapter generating unit 11a associates the chapter marker with the image data and records the associated chapter markers as metadata in the recording unit 16. The reproduction position determining unit 11b reads the chapter markers from the recording unit 16 and outputs the chapter markers as information for specifying a reproduction position to the reproduction unit 17. The reproduction unit 17 reproduces the image portion and metadata corresponding to the specified chapter marker and outputs the metadata to the display control unit 18.

[0112] In this way, the display control unit 18 can output the endoscopic image including the insertion length mark at the designated chapter marker position to the monitor 5 .

[0113] On the display screen 5a of the monitor 5, for example, Fig.13The endoscopic image at the position of the chapter mark C is displayed as in the display area 62 of the display example. An insertion length mark indicating that the insertion length is 100 cm is displayed in the display area 62, and the operator can refer to the insertion length mark and insert and remove the insertion portion 4b so that the insertion length of the endoscopic image in the display area 61 is 100 cm. In this way, the parts that have been treated in the past are specified by the chapter marks, so that the past case images and the current observation images of the parts that have been treated can be easily compared.

[0114] Thus, in this modification, by adding a chapter mark at an arbitrary position of the recorded image, a desired position of a past case image can be reproduced. Since the insertion length mark is displayed according to the insertion length information at the reproduction position set by the chapter mark, the surgeon can easily recognize how much the insertion portion is inserted or withdrawn to observe the observation position such as the treated part, which is extremely useful in the process observation, etc.

[0115] In the above modification, an example of specifying the playback position by a chapter marker is described, but the playback position may be set by directly specifying the insertion length. In this case, the playback position can be specified regardless of the presence or absence of a chapter marker.

[0116] The present invention is not directly limited to the above-mentioned embodiments, and the structural elements can be deformed and concretized in the implementation stage without departing from the scope of the main purpose. In addition, various inventions can be formed by appropriate combinations of multiple structural elements disclosed in the above-mentioned embodiments. For example, several structural elements of all the structural elements shown in the embodiments can be deleted. In addition, the structural elements of different embodiments can be appropriately combined.

[0117] This application is filed based on Japanese Patent Application No. 2018-075400 filed in Japan on April 10, 2018, and the above-mentioned disclosed contents are cited in the specification and claims of this application.

Claims

1. A medical system, characterized in that: have: An endoscope having an insertion portion to be inserted into a subject, wherein an imaging element for outputting an image signal of the subject is arranged on the insertion portion; A video processor, which processes the image signal to generate image data; an insertion length obtaining device for obtaining information on the insertion length when the insertion portion of the endoscope is inserted into the subject; an image recording device including a recording unit for recording the image data as a dynamic image; a reproducing unit that reproduces the image data recorded by the recording unit; as well as a display control section that displays a display image based on the information of the insertion length together with the reproduction output of the reproduction section on a monitor, The image recording device includes an image generating unit that edits the dynamic image based on the information of the insertion length supplied from the insertion length obtaining unit to generate a dynamic image for storage; and a chapter generating unit that adds chapter marks to the dynamic image based on the information of the insertion length supplied from the insertion length obtaining unit. The image recording device records the dynamic image during the entire shooting period as a main dynamic image in the recording unit, and the image generation unit edits the main dynamic image recorded in the recording unit to generate the dynamic image for storage. The image generation unit determines the medical scene based on the information of the chapter marker and deletes the useless part of the main dynamic image. The monitor can simultaneously display a past case dynamic image from the display control unit and a current observation dynamic image from the video processor at the same observation position of the subject.

2. The medical system according to claim 1, characterized in that The chapter generating unit adds the chapter marker to a corresponding portion of the moving image when the insertion length reaches a predetermined length.

3. The medical system according to claim 2, characterized in that: The chapter generating unit adds the chapter marker to a corresponding portion of the moving image at least when the insertion of the insertion unit starts, when the withdrawal of the insertion unit ends, and when the maximum insertion length of the insertion unit is reached.

4. The medical system according to claim 1, characterized in that: The portion of the main moving image that is deleted by the image generating unit is a portion captured outside the body cavity.

5. The medical system according to claim 1, characterized in that: The image generation unit generates a high-quality storage moving image and a low-quality storage moving image using a portion of the main moving image that has not been deleted.

6. The medical system according to claim 5, characterized in that: The image generating unit generates the low-quality storage image using a portion of the main dynamic image captured during the insertion scene of the endoscope, and generates the high-quality storage image using a portion of the main dynamic image captured during the removal scene of the endoscope.

7. An image processing method, wherein an endoscope dynamic image generated by processing a received image signal of a subject is associated with information of an acquired insertion length and recorded in a recording unit, The image processing method is characterized in that reading the endoscope dynamic image recorded in the recording unit, deleting useless parts according to the information of the insertion length, The remaining portion is stored in the recording unit as a dynamic image for storage, The image data recorded by the recording unit is reproduced in the monitoring device, and an image based on the information of the insertion length is simultaneously displayed at the same time. The monitoring device can simultaneously display a past case dynamic image and a current observation dynamic image at the same observation position of the subject.

8. The image processing method according to claim 7, characterized in that: The useless portion of the endoscopic dynamic image is the portion captured outside the body cavity.

9. The image processing method according to claim 7, characterized in that: In the image processing method, a portion of the dynamic image for storage captured during the insertion scene of the endoscope is generated as a low-quality storage image, and a portion of the dynamic image for storage captured during the removal scene of the endoscope is generated as a high-quality storage image.

Images