Puncture system, information processing method, and computer program

Abstract

To provide a puncture system or the like that is capable of presenting an image of a puncture instrument with improved visibility.　This puncture system comprises: a puncture instrument provided with a first light source and a light-emitting unit that emits light from the first light source; a second light source; an imaging unit that captures images of a living body and the puncture instrument; and a control unit. The first light source can emit light in a state in which the second light source does not emit light. The control unit acquires images consecutively captured by the imaging unit, classifies the acquired images into a first image that includes the light-emitting unit imaged in a state in which the first light source emits light and the second light source does not emit light and a second image that includes a living blood vessel imaged in a state in which the second light source emits light, and displays the classified first image and second image.

Description

Puncture system, information processing method, and computer program

[0001] The present invention relates to a puncture system, an information processing method, and a computer program.

[0002] In medical settings, puncture procedures are performed on subjects for blood sampling, injections, transfusions, placement of intravascular catheters, etc. Medical personnel insert a puncture device such as a needle or catheter from the surface of the subject's body to a target location in a blood vessel. It is not easy to properly position the puncture device at the target location. Therefore, technologies to assist in such puncture procedures have been proposed.

[0003] For example, US Pat. No. 6,299,649 discloses a device for visually identifying a patient's blood vessels using a catheter device that may include a cannula and a light-emitting optical fiber.

[0004] Patent No. 6734392

[0005] It is conceivable to use a puncture instrument having a light-emitting mechanism such as that described in Patent Document 1 in combination with a blood vessel visualization device capable of displaying a blood vessel visualization image that visualizes blood vessels. The blood vessel visualization device is equipped with a light source, and captures a blood vessel visualization image by irradiating a living body with light from the light source. The blood vessel visualization image displays the living body and the captured puncture instrument. If the puncture instrument is made to emit light while light is being irradiated from the light source of the blood vessel visualization device, the light emitted from the puncture instrument will be weaker than the light irradiated for blood vessel visualization, resulting in a problem in that the puncture instrument cannot be seen in the image.

[0006] An object of the present disclosure is to provide a puncture system and the like that can present an image with improved visibility of a puncture device.

[0007] (1) A puncture system according to one aspect of the present disclosure comprises a puncture device having a first light source and a light-emitting unit that emits light from the first light source, a second light source, an imaging unit that images a living body and the puncture device, and a control unit, wherein the first light source is capable of emitting light when the second light source is not emitting light, and the control unit acquires images continuously captured by the imaging unit, separates the acquired images into a first image including the light-emitting unit that is captured when the first light source is emitting light and the second light source is not emitting light, and a second image including the blood vessels of the living body that is captured when the second light source is emitting light, and displays the separated first image and second image.

[0008] (2) In the puncture system described in (1) above, the first image and the second image may be synthesized and displayed.

[0009] (3) In the puncture system described in (1) or (2) above, the most recent first image and the most recent second image among the images acquired successively may be synthesized and displayed.

[0010] (4) In the puncture system according to any one of (1) to (3) above, the first image and the second image may be displayed alternately.

[0011] (5) In the puncture system described in any one of (1) to (4) above, the first image and the second image may be distinguished based on the light emission patterns of the first light source and the second light source and the frame rate of the imaging unit.

[0012] (6) In the puncture system described in any one of (1) to (5) above, a subject to be photographed in an image may be detected, and the first image and the second image may be separated based on the detected subject to be photographed.

[0013] (7) In the puncture system described in (6) above, an image in which an object representing the subject of photography is detected whose brightness and size are equal to or greater than a predetermined value may be distinguished from the first image, and an image in which an object representing the subject of photography is detected whose at least one of the straightness, length, thickness, and extension direction is equal to or greater than a predetermined value may be distinguished from the second image.

[0014] (8) In the puncture system described in any one of (1) to (7) above, a wavelength in an image may be detected, and the first image and the second image may be separated based on the detected wavelength.

[0015] (9) In the puncture system described in any one of (1) to (8) above, if the latest first image or second image is not acquired according to the frame rate of the imaging unit, an interpolated image may be generated.

[0016] (10) In the puncture system according to any one of (1) to (9) above, the light emitting unit may be provided at the tip of the puncture device.

[0017] (11) In the puncture system according to any one of (1) to (10) above, the first image and the second image may be displayed on the surface of the living body. The puncture system according to any one of (1) to (10) above may further include a display unit that displays the first image and the second image.

[0018] (12) An information processing method according to one aspect of the present disclosure is a puncture system including a puncture device having a first light source and a light-emitting unit that emits light from the first light source, a second light source, an imaging unit that images a living body and the puncture device, and a computer, the computer acquiring images continuously captured by the imaging unit, separating the acquired images into a first image including the light-emitting unit that was captured when the first light source was emitting light and the second light source was not emitting light, and a second image including the blood vessels of the living body that was captured when the second light source was emitting light, and executing a process to display the separated first image and second image.

[0019] (13) A computer program according to one aspect of the present disclosure causes a computer of a puncture system including a first light source and a light-emitting unit that emits light from the first light source, a second light source, an imaging unit that images a living body and the puncture device, and a computer to acquire images continuously captured by the imaging unit, separate the acquired images into a first image including the light-emitting unit that is captured when the first light source is emitting light and the second light source is not emitting light, and a second image including the blood vessels of the living body that is captured when the second light source is emitting light, and to execute a process of displaying the separated first image and second image.

[0020] According to the present disclosure, an image with improved visibility of the puncture device can be presented.

[0021] FIG. 1 is a schematic diagram of a puncture system. FIG. 2 is an explanatory diagram illustrating an example of the configuration of an indwelling needle. FIG. 3 is a cross-sectional diagram illustrating an example of the configuration of an indwelling needle. FIG. 4 is an enlarged cross-sectional diagram of the tip side of an indwelling needle. FIG. 5 is a block diagram illustrating an example of the configuration of a processing device. FIG. 6 is a schematic diagram illustrating an example of an indwelling needle image and a blood vessel image. FIG. 7 is a schematic diagram illustrating an example of an indwelling needle image and a blood vessel image. FIG. 8 is a flowchart illustrating an example of a processing procedure executed by a processing device. FIG. 9 is a flowchart illustrating a first example of detailed procedures for image classification processing. FIG. 10 is a flowchart illustrating a second example of detailed procedures for image classification processing. FIG. 11 is a flowchart illustrating a third example of detailed procedures for image classification processing.

[0022] The present disclosure will be specifically described with reference to the drawings showing embodiments thereof.

[0023] FIG. 1 is a schematic diagram of a puncture system 100. The puncture system 100 includes an indwelling needle 1 capable of puncturing a blood vessel in a living body (a puncture target site) 3, and a blood vessel visualization device 2. The indwelling needle 1 is an example of a puncture instrument. The puncture system 100 is a system that supports a surgeon, such as a doctor, in performing a puncture operation by presenting images showing the state of the indwelling needle 1 and the blood vessel. The surgeon is an example of a user of this system. The following description will be given taking as an example a case where the puncture target is a human (subject) and the puncture target site is part of the subject's forearm.

[0024] The blood vessel visualization device 2 includes a processing device 20, a second light source 21, a camera 22, and a display device 23. The blood vessel visualization device 2 of this embodiment functions as a blood vessel visualization device that presents a blood vessel image that visualizes blood vessels, and also functions as an indwelling needle detection device that detects the light-emitting unit 17 of the indwelling needle 1. The blood vessel visualization device 2 is installed in a facility such as a medical institution or testing institution where a puncture operation is performed.

[0025] Fig. 2 is an explanatory diagram illustrating an example of the configuration of the indwelling needle 1, Fig. 3 is a cross-sectional view illustrating an example of the configuration of the indwelling needle 1, and Fig. 4 is an enlarged cross-sectional view of the tip end side of the indwelling needle 1. In the following, various configurations and the like will be described with the direction along the axis C of the indwelling needle 1 being the axial direction, the left side in Figs. 2 to 4 being the "tip end side," and the right side being the "base end side."

[0026] The indwelling needle 1 is, for example, an indwelling needle for a peripheral artery or vein. As shown in Figures 1 to 4, the indwelling needle 1 comprises a catheter 11, a catheter hub 12, an inner needle (needle) 13, an inner needle hub 14, a blood exposure prevention unit 15, a first light source unit 16, and a light emitting unit 17. The catheter hub 12 is attached to the proximal end of the catheter 11. The inner needle hub 14 is attached to the proximal end of the inner needle 13, and the inner needle hub 14 is located at the proximal end of the catheter hub 12. The blood exposure prevention unit 15 is attached to the proximal end of the inner needle hub 14, and the first light source unit 16 is attached to the proximal end of the blood exposure prevention unit 15.

[0027] The indwelling needle 1 is a catheter assembly. In the initial state (assembled state) of the indwelling needle 1, the inner needle 13 is inserted through the lumen of the catheter 11 and the catheter hub 12. In the initial state of the indwelling needle 1, the inner needle 13 protrudes from the tip opening of the catheter 11. When the tip of the inner needle 13 is inserted into a blood vessel, the catheter 11 is also inserted into the same blood vessel.

[0028] The catheter 11 constitutes the outer needle and is a flexible tubular member. The catheter 11 reaches close to the tip of the inner needle 13. The material constituting the catheter 11 is not particularly limited, but a transparent resin material, particularly a soft resin material, is preferred, and examples thereof include fluorine-based resins such as ethylene-tetrafluoroethylene copolymer (ETFE), polytetrafluoroethylene (PTFE), and perfluoroalkoxy fluorine resin (PFA), olefin-based resins such as polyethylene and polypropylene, and mixtures thereof.

[0029] The catheter hub 12 is formed into a hollow (cylindrical) shape. The material of which the catheter hub 12 is made is not particularly limited, but examples thereof include thermoplastic resins such as polypropylene, polycarbonate, polyamide, polysulfone, polyarylate, methacrylate-butylene-styrene copolymer, polyurethane, acrylic resin, and ABS resin.

[0030] A light-emitting unit 17 is provided at the tip of the catheter 11. The light-emitting unit 17 emits light using light that is irradiated from the first light source unit 16 and propagates through a light path formed inside the indwelling needle 1. The position of the tip of the indwelling needle 1 can be recognized by detecting light L1 emitted from the light-emitting unit 17. The light-emitting unit 17 may be provided around the entire outer circumferential surface at the tip of the catheter 11, or may be provided only along a portion of the circumferential direction. When a blade surface 133, described below, is faced upward, the light-emitting unit 17 is preferably formed in an area that includes at least a position that faces upward.

[0031] The light-emitting unit 17 contains, for example, a near-infrared fluorescent dye that emits near-infrared fluorescence of a specific wavelength when irradiated with excitation light, and is formed by coating or mixing the near-infrared fluorescent dye onto the tip of the catheter 11. The wavelength of the light L1 emitted by the light-emitting unit 17 can be, for example, 700 to 2500 nm, preferably 700 to 1400 nm, and more preferably 780 to 940 nm. The wavelength range of the light-emitting unit 17 may include the visible light range. It is preferable that the excitation wavelength range of the light-emitting unit 17 includes the wavelength range of the light of the first light source unit 16, but does not include the wavelength range of the light of the second light source 21.

[0032] The configuration and detection method of the light-emitting unit 17 are not limited as long as it emits light using light emitted from the first light source unit 16 and the light emission can be detected by the camera 22. The light-emitting unit 17 may include, for example, a light-emitting body that emits light of a specific wavelength, an upconversion (UC) phosphor, a reflector that reflects the irradiated light of the specific wavelength, etc. The light-emitting unit 17 may have a reflective structure that reflects light by applying surface treatment such as embossing or unevenness.

[0033] The light emitting unit 17 is not limited to being provided at the tip of the catheter 11, but may be provided at any appropriate position including at least the periphery of the tip of the indwelling needle 1. The light emitting unit 17 may be provided, for example, at the tip of the inner needle 13. Multiple light emitting units 17 may be provided.

[0034] As shown in Figures 3 and 4, a light-guiding material 18 is provided inside the indwelling needle 1 to guide light emitted from the first light source unit 16 to the light-emitting unit 17. Examples of the light-guiding material 18 include an optical fiber, an acrylic rod, a glass rod, and a light-emitting tube. The light-guiding material 18 may be provided continuously or intermittently in the optical path of light from the first light source unit 16. In the example shown in Figures 3 and 4, the light-guiding material 18 is an elongated optical fiber and is disposed in the internal space of the indwelling needle 1 between the vicinity of the tip of the catheter 11 and the blood-exposure prevention unit 15. The light-guiding material 18 may be hollow, allowing blood to flow through the light-guiding material 18. In this case, the hollow portion of the light-guiding material 18 is in communication with the blood-exposure prevention unit 15.

[0035] The light guide material 18 may be formed at least near the tip of the indwelling needle 1 so that the light guide material 18 functions as the light emitting section 17. The indwelling needle 1 does not necessarily have to include the light guide material 18.

[0036] The inner needle 13 is a tubular member having enough rigidity to puncture a living body. The inner needle 13 has a lumen 131 extending along its axial direction. The lumen 131 communicates with the blood exposure prevention part 15. The above-mentioned light-guiding material 18 is inserted into the lumen 131. The lumen 131 functions as a blood introduction channel through which blood can flow, and can also function as an optical path for light passing through the inner needle 13. A blade surface 133 inclined relative to the axial direction of the inner needle 13 is formed at the tip of the inner needle 13.

[0037] Examples of metal materials that can be used to form the inner needle 13 include stainless steel, aluminum, aluminum alloys, titanium, and titanium alloys.

[0038] When the light-guiding material 18 is formed to be hollow, the cross-sectional area of ​​the hollow portion of the light-guiding material 18 is preferably larger than the cross-sectional area of ​​the region surrounded by the inner periphery of the inner needle 13 and the outer periphery of the light-guiding material 18. In this case, blood flows preferentially through the hollow portion of the light-guiding material 18, thereby increasing the efficiency of light propagation by the light-guiding material 18. The ratio of the cross-sectional area of ​​the hollow portion of the light-guiding material 18 to the cross-sectional area of ​​the region surrounded by the inner periphery of the inner needle 13 and the outer periphery of the light-guiding material 18 (cross-sectional area of ​​the hollow portion of the light-guiding material 18 / cross-sectional area of ​​the region surrounded by the inner periphery of the inner needle 13 and the outer periphery of the light-guiding material 18) is preferably 1.2 or greater, and more preferably 1.5 or greater.

[0039] A reflecting portion 132 that reflects light from the first light source 16 may be provided on the inner circumferential surface of the inner needle 13. The reflecting portion 132 is formed, for example, by coating the inner circumferential surface with a light-reflecting material. Examples of light-reflecting materials include silver, aluminum, chromium, and nickel. The reflecting portion 132 may be provided over the entire inner circumferential surface of the inner needle 13, or may be provided only on a portion of the inner circumferential surface.

[0040] The inner needle hub 14 may be formed in a hollow, tapered shape. The internal space of the inner needle hub 14 can function as an optical path for light passing through the inner needle hub 14. It is preferable that the base end (terminal end) of the inner needle 13 does not protrude further toward the base end than the tapered tip of the inner needle hub 14 (it is not convex toward the base end). Examples of materials constituting the inner needle hub 14 include the same materials as those constituting the catheter hub 12 described above. A light-guiding member that guides light from the first light source unit 16 may be provided by coating or the like on the wall surface that forms the internal space of the inner needle hub 14.

[0041] The outer peripheral surface of the inner needle hub 14 is preferably provided with a first light leakage prevention part 141 that prevents light that is irradiated from the first light source part 16 and passes through the inner needle hub 14 from leaking to the outside. As the first light leakage prevention part 141, for example, the outer peripheral surface may be colored with a light-opaque material such as carbon black, or the outer peripheral surface may be coated with a metal mirror, a multilayer film filter, or the like as a light-opaque material.

[0042] A blood exposure prevention part 15 is provided at the proximal end of the inner needle hub 14. The blood exposure prevention part 15 is connected to the inner needle hub 14 in a liquid-tight manner via a first connecting part 142 provided at the proximal end of the inner needle hub 14.

[0043] The blood-exposure prevention unit 15 is intended to prevent blood exposure by blocking the inflow of blood into the first light source unit 16. At least a portion of the blood-exposure prevention unit 15 is formed to be transparent or translucent. The transparent or translucent region can function as an optical path for light passing through the blood-exposure prevention unit 15.

[0044] The outer peripheral surface of the blood-exposure prevention part 15 is preferably provided with a second light-leakage prevention part 151 that prevents light that is irradiated from the first light source part 16 and passes through the blood-exposure prevention part 15 from leaking to the outside. The second light-leakage prevention part 151 can be formed in the same manner as the first light-leakage prevention part 141 of the inner needle hub 14 described above. The second light-leakage prevention part 151 may be formed in an area that covers at least the transparent or translucent region of the blood-exposure prevention part 15.

[0045] A fitting portion 152 into which the light-guiding material 18 is fitted is provided at the tip of the blood exposure prevention portion 15. The light-guiding material 18 is inserted, for example, from the inner needle hub 14 side into the tip of the inner needle 13. By joining the inner needle hub 14 and the blood exposure prevention portion 15 together, the light-guiding material 18 is fixed within the indwelling needle 1.

[0046] A second connecting portion 153 is provided at the base end of the blood exposure prevention portion 15. The blood exposure prevention portion 15 and the first light source portion 16 are liquid-tightly connected via the second connecting portion 153. In order to increase the amount of light propagating to the light-guiding material 18, a hollow path may be formed between the base end (terminal end) of the light-guiding material 18 and the second connecting portion 153 in the blood exposure prevention portion 15. The blood exposure prevention portion 15 may also be provided with an air vent hole. If an air vent hole is provided in the blood exposure prevention portion 15, it is preferable to provide a filter or the like at the air vent outlet (proximal end side). The installation position of the blood exposure prevention portion 15 is not particularly limited. The blood exposure prevention portion 15 may be provided within the inner needle hub 14. Alternatively, the blood exposure prevention portion 15 may be omitted. In this case, the inner needle hub 14 and the first light source portion 16 may be configured to be connectable.

[0047] The first light source unit 16 includes a first light source 161 and a housing 162 that accommodates the first light source 161. The housing 162 is formed of, for example, an opaque material, metal, or other material, and prevents leakage of light emitted by the first light source 161 to the outside. The housing 162 may be configured to prevent leakage of light to the outside by providing an opaque member, such as an opaque pigment (for example, carbon black), a metal mirror, or a multilayer filter, on its outer circumferential surface.

[0048] A protrusion 163 is provided at the tip of housing 162. By inserting protrusion 163 into second connecting portion 153, blood-exposure prevention unit 15 and first light source unit 16 are detachably connected. First light source unit 16 also includes a power source (e.g., a battery, a wireless power supply, a solar cell, etc.) (not shown) that supplies power to first light source 161. The method of attaching first light source unit 16 is not particularly limited, and it may be configured to be attached to blood-exposure prevention unit 15 by, for example, a screw, a fitting member, a magnet, etc.

[0049] The first light source 161 includes, for example, an LED (Light Emitting Diode), a laser irradiation unit (for example, a laser diode), a lamp (for example, a halogen lamp), or the like. The first light source 161 emits light of a specific wavelength that is biotransparent. As described above, the light emitted by the first light source 161 is irradiated to the light-emitting unit 17 via the light-guiding material 18. It is preferable that the light emitted by the first light source 161 has a narrow viewing angle, i.e., high directivity. A lens, a reflector, or the like may be provided on the optical path of the light emitted from the first light source 161 to enhance the directivity of the light.

[0050] The wavelength of the light emitted by the first light source 161 includes, for example, the visible light region and the near-infrared light region. The emission wavelength of the first light source 161 can be, for example, 600 nm to 2500 nm, preferably 700 nm to 1400 nm, and more preferably 780 nm to 940 nm. Light of such wavelengths is easily absorbed by hemoglobin in blood. The first light source may include multiple irradiation units and be capable of irradiating light of multiple wavelengths, such as visible light of 600 nm and near-infrared light of 850 nm.

[0051] The indwelling needle 1 is not limited to the above example, and may have any appropriate configuration as long as it is a puncture device that is inserted into a blood vessel of a living body 3. The indwelling needle 1 may be a dialysis needle, a peripherally inserted central venous catheter (PICC), a midline catheter, a central venous catheter (CVC), etc. The puncture device may not have a catheter, but may include a puncture needle (inner needle) and a syringe, etc., like a blood collection device.

[0052] 1 , the blood vessel visualization device 2 includes a bottom surface 24, a top surface 25, and a support 26 connecting the bottom surface 24 and the top surface 25. The bottom surface 24 and the top surface 25 are both rectangular and extend horizontally, and are arranged perpendicular to the support 26. The bottom surface 24 and the top surface 25 overlap when viewed vertically. When in use, the blood vessel visualization device 2 is placed on a horizontal table such as a treatment table.

[0053] A recess 27 is formed on the upper surface of the bottom surface portion 24. The recess 27 is shaped so that the arm of the subject, which is the living body 3, can be placed on it, and the posture and position of the arm are fixed by the subject placing their arm on this recess 27 during puncture. Note that the bottom surface portion 24 does not necessarily have to be provided with the recess 27.

[0054] A second light source 21, a camera 22, and a display device 23 are provided on the underside of the top surface 25 above the bottom surface 24. The second light source 21, the camera 22, and the display device 23 are arranged, for example, in a line in the longitudinal direction. The camera 22 is disposed so as to be located above the region to be punctured. The camera 22 is fixed to the top surface 25 so that the imaging area includes the area around the blood vessel that is the region to be punctured and the indwelling needle 1 that will be inserted into the blood vessel. The camera 22 is preferably configured to be fixed to the top surface 25 in a state where its position and angle can be adjusted, so that the camera 22 is positioned directly above the region to be punctured in accordance with the running state of the blood vessel and the movement of the indwelling needle 1.

[0055] The second light source 21 includes, for example, an LED, a laser irradiation unit, a lamp, etc. The second light source 21 irradiates the living body 3 with light L2, which is near-infrared light. The emission wavelength of the second light source 21 can be, for example, 700 nm to 2500 nm, preferably 700 nm to 1400 nm, and more preferably 780 nm to 940 nm. Light in such a wavelength range has high biological permeability and is easily absorbed by hemoglobin. The second light source 21 may include multiple irradiation units and be able to irradiate light L2 with multiple wavelengths. The wavelength of the light L2 from the second light source 21 and the wavelength of the light L1 from the light-emitting unit 17 may be the same or different.

[0056] The camera 22 is an imaging device having an imaging element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor. The camera 22 captures images at a predetermined frame rate. The camera 22 receives light L1 from the light-emitting unit 17 and light L2 from the second light source 21. Specifically, the camera 22 receives light L1 emitted from the light-emitting unit 17 of the indwelling needle 1 and reflected or transmitted light of light L2 irradiated from the second light source 21 to obtain an imaging signal, and generates image data based on the obtained imaging signal.

[0057] The spectral sensitivity wavelength range of the camera 22 is not particularly limited and can be set appropriately as long as it corresponds to the wavelengths of the light L1 from the light-emitting unit 17 and the light L2 from the second light source 21. The spectral sensitivity wavelength range of the camera 22 can be, for example, 600 to 2500 nm.

[0058] From the viewpoint of facilitating the system configuration, it is preferable that one camera 22 be capable of detecting light L1 from the light-emitting unit 17 and light L2 from the second light source 21, but it is also possible to provide a camera 22 for detecting the light-emitting unit 17 having a spectral sensitivity wavelength corresponding to the light L1 from the light-emitting unit 17, and a camera 22 for detecting blood vessels having a spectral sensitivity wavelength corresponding to the light L2 from the second light source 21. In this case, the spectral sensitivity wavelength ranges of the camera 22 for detecting the light-emitting unit 17 and the camera 22 for detecting blood vessels may be the same or partially overlapping, or may be different from each other, and are preferably different.

[0059] The installation positions of the second light source 21 and the camera 22 are not particularly limited as long as they are positions that allow detection of blood vessels and the light-emitting unit 17. For example, the second light source 21 and the camera 22 may be installed on opposing sides with the detection object sandwiched between them, or a combination of a second light source 21 and a camera 22 on the same side of the detection object and a second light source 21 and a camera 22 that face each other with the detection object sandwiched between them may be used.

[0060] The display device 23 is an output device for presenting a blood vessel image including the blood vessels of the living body 3 and an indwelling needle image including the light-emitting unit 17 of the indwelling needle 1. The display device 23 of this embodiment is equipped with a projector and projects an image onto the body surface of the living body 3. The display device 23 is preferably configured to be fixed to the top surface 25 in a state in which its position and angle can be adjusted, so that the display device 23 is positioned above the site to be punctured in accordance with the running state of the blood vessels and the movement of the indwelling needle 1. The display device 23 may be, for example, a display device such as goggles, glasses, or a head-mounted display that can be worn by the surgeon, a liquid crystal display, an organic EL (Electro Luminescence) display, or the like.

[0061] The configuration of the blood vessel visualization device 2 is not limited to the example shown in Fig. 1. Although Fig. 1 shows a fixed blood vessel visualization device 2, the blood vessel visualization device 2 may be portable or configured to be holdable by the surgeon. The blood vessel visualization device 2 may also be configured as an AR (Augmented Reality) device having, for example, a goggle shape or a head-mounted display shape.

[0062] The following describes a procedure for puncturing a blood vessel using the puncture system 100. As shown in Fig. 1, the forearm of a living body 3 to be punctured is placed on the bottom surface 24, light L2 is irradiated onto the living body 3 from the second light source 21, and the indwelling needle 1 is inserted from the surface of the living body 3 (the skin surface of the subject) toward the blood vessel.

[0063] The light L2 emitted by the second light source 21 is absorbed by hemoglobin in the blood flowing through the blood vessels of the living body 3, and is reflected by biological tissues other than the blood vessels (for example, skin tissue and muscle tissue), and the reflected light L2 (reflected light) is received by the camera 22. As a result, a blood vessel image generated based on the light L2 received by the camera 22 is displayed. The blood vessel image includes biological tissues and blood vessels.

[0064] Furthermore, light L1 emitted by the light-emitting unit 17 in response to light from the first light source 161 of the indwelling needle 1 passes through the biological tissue while being scattered, and the transmitted light L1 (transmitted light) is received by the camera 22. As a result, an indwelling needle image generated based on the light L1 received by the camera 22 is displayed. The indwelling needle image includes the light-emitting unit 17 provided at the tip of the indwelling needle 1. Details of the blood vessel image and the indwelling needle image will be described later.

[0065] When the tip of the indwelling needle 1 is inserted into a blood vessel, the light L1 emitted by the light-emitting unit 17 is absorbed by hemoglobin in the blood and does not reach the camera 22, or even if it does reach the camera 22, the intensity is greatly reduced. As a result, the brightness of the light-emitting unit 17 is greatly reduced in the image of the indwelling needle. This allows the surgeon to know that the tip of the indwelling needle 1 has been inserted into the blood vessel (the blood vessel has been secured by the indwelling needle 1).

[0066] After inserting the indwelling needle 1, the surgeon removes the inner needle 13, the inner needle hub 14, and the first light source unit 16, leaving behind the catheter 11 and the catheter hub 12, thereby leaving the catheter 11 and the catheter hub 12 in the blood vessel.

[0067] 5 is a block diagram showing an example of the configuration of the processing device 20. The processing device 20 is a computer and includes a control unit 201, a storage unit 202, and an input / output unit 203. Each unit is connected via a bus. The processing device 20 may be a multi-computer consisting of multiple computers, or may be a virtual machine virtually constructed by software.

[0068] The control unit 201 includes one or more arithmetic processing devices such as a central processing unit (CPU), a micro-processing unit (MPU), a graphics processing unit (GPU), etc. The control unit 201 controls each component and executes processing using built-in memories such as a read-only memory (ROM) or a random access memory (RAM), a clock, a counter, etc. The functional units of the control unit 201 may be realized by software, or some or all of them may be realized by hardware such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0069] The storage unit 202 includes a non-volatile memory such as a hard disk, a flash memory, or an SSD (Solid State Drive). The storage unit 202 may be an external storage device connected to the processing device 20. The storage unit 202 stores various computer programs and data referenced by the control unit 201. The storage unit 202 stores a program 2P for causing a computer to execute processing related to the output of indwelling needle images and blood vessel images.

[0070] A computer program (program product) including program 2P may be provided by a non-transitory recording medium 2A on which the computer program is readably recorded. The storage unit 202 stores the computer program read from the recording medium 2A by a reading device (not shown). The recording medium 2A may be, for example, a magnetic disk, an optical disk, or a semiconductor memory. Alternatively, the computer program may be downloaded from an external server connected to a communications network and stored in the storage unit 202. Program 2P may be a single computer program or may be composed of multiple computer programs, and may be executed on a single computer or on multiple computers interconnected by a communications network.

[0071] The input / output unit 203 includes an input / output interface for connecting an external device. The input / output unit 203 is connected to the first light source 161, the second light source 21, the camera 22, and the display device 23. The control unit 201 receives image data output from the camera 22 via the input / output unit 203. The control unit 201 also outputs various signals to the first light source 161, the second light source 21, and the display device 23 via the input / output unit 203.

[0072] The configuration of processing device 20 is not limited to the above example, and may include, for example, an operation unit for accepting user operations, a communication unit for communicating with external devices via a network such as the Internet, etc. Processing device 20 may be configured as a server computer or the like located at a location remote from the facility, such as a medical institution, where the puncture procedure is performed. In this case, processing device 20 may receive image data obtained by camera 22 via communication, and may also transmit information corresponding to the received image data to display device 23 via communication, for display.

[0073] The configuration of the blood vessel visualization device 2 is not limited to the example shown in Fig. 1 , and may be any appropriate configuration as long as it can function as a blood vessel visualization device and an indwelling needle detection device. In this embodiment, an example has been described in which the blood vessel visualization device 2 functions as an indwelling needle detection device and a blood vessel visualization device, but the indwelling needle detection device and the blood vessel visualization device may be provided as separate devices. The processing device 20 may be provided as a separate device from the blood vessel visualization device 2.

[0074] The processing device 20 of this embodiment separates multiple images (image frames) continuously captured by the camera 22 at a predetermined frame rate into an indwelling needle image including an indwelling needle image showing the area corresponding to the light-emitting portion 17 of the indwelling needle 1, and a blood vessel image including a blood vessel visualization image showing the area corresponding to the blood vessel.

[0075] The indwelling needle image is an image captured with the first light source 161 emitting light and the second light source 21 not emitting light. The blood vessel image is an image captured with at least the second light source 21 emitting light. When capturing a blood vessel image, the first light source 161 may be in either an emitting state or a non-emitting state, and is preferably in a non-emitting state. When capturing a blood vessel image with the first light source 161 emitting light, the camera 22 preferably has an automatic brightness adjustment function.

[0076] 6A and 6B are schematic diagrams showing examples of an indwelling needle image and a blood vessel image. Fig. 6A shows the indwelling needle image, and Fig. 6B shows the blood vessel image.

[0077] 6A, the brightness of pixels corresponding to the indwelling needle image is higher than the brightness of pixels not corresponding to the indwelling needle image. By capturing an image of the light-emitting unit 17 and the living body 3 in a non-light-emitting state in which the second light source 21 is not emitting light, only the light-emitting unit 17 can be made to emit light effectively, thereby increasing the brightness difference between the light-emitting unit 17 and other areas in the indwelling needle image.

[0078] As shown in Figure 6B, the brightness of pixels corresponding to the visualized blood vessel image in the blood vessel image is lower than the brightness of pixels corresponding to biological tissue other than blood vessels. The blood vessel image may further include the indwelling needle 1. When the second light source 21 is emitting light, light L2 from the second light source 21 is stronger than light L1 from the light-emitting unit 17 of the indwelling needle 1, and therefore the brightness of pixels corresponding to the light-emitting unit 17 in the blood vessel image is significantly lower than in the indwelling needle image. Note that if the first light source 161 is emitting light when the blood vessel image is captured and the above-mentioned automatic brightness adjustment function is not provided, the brightness of the pixels corresponding to the light-emitting unit 17 in the blood vessel image and the light-emitting unit 17 in the indwelling needle image may be the same.

[0079] A method for distinguishing between an indwelling needle image and a blood vessel image will be described below.

[0080] As a first example, the processing device 20 separates the images output from the camera 22 into an indwelling needle image and a blood vessel image based on the light emission patterns of the first light source 161 and the second light source 21 and the frame rate of the camera 22.

[0081] The first light source 161 and the second light source 21 are controlled to emit light (turn on) alternately for each frame in synchronization with a predetermined frame rate preset in the camera 22. The turning on and off of the first light source 161 and the second light source 21 is preferably controlled by the processing device 20, but may also be controlled by a control mechanism (not shown) built into the first light source 161 and the second light source 21. When the light emission of the first light source 161 and the second light source 21 is controlled by an internal mechanism, the memory unit 202 of the processing device 20 stores in advance light emission patterns (e.g., lighting intervals and lighting times) to be set for each of the first light source 161 and the second light source 21.

[0082] For example, at time t1, an image F1 is acquired which is captured with the first light source 161 emitting light (turned on) and the second light source 21 not emitting light (turned off), and at time t2, an image F2 is acquired which is captured with the first light source 161 not emitting light and the second light source 21 emitting light. At time t3, an image F3 is acquired which is captured with the first light source 161 emitting light and the second light source 21 not emitting light, and at time t4, an image F4 is acquired which is captured with the first light source 161 not emitting light and the second light source 21 emitting light.

[0083] The processing device 20 identifies the image type based on the capture time of the frame image and the light emission patterns of the first light source 161 and the second light source 21. If the processing device 20 identifies that the first light source 161 was emitting light and the second light source 21 was not emitting light at the time the acquired image was captured, the processing device 20 identifies the image type as an indwelling needle image and classifies the acquired image as an indwelling needle image. If the processing device 20 identifies that the first light source 161 was not emitting light and the second light source 21 was emitting light at the time the image was captured, the processing device 20 identifies the image type as a blood vessel image and classifies the acquired image as a blood vessel image. In the above example, images F1 and F3 are classified as indwelling needle images, and images F2 and F4 are classified as blood vessel images.

[0084] The processing device 20 displays the separated indwelling needle images and blood vessel images on the display device 23. The processing device 20 may alternately display the obtained indwelling needle images and blood vessel images on the display device 23, or may display a composite image in which the indwelling needle images and blood vessel images are combined. That is, the processing device 20 may sequentially display images F1, F2, F3, and F4, or may sequentially display a composite image in which images F1 and F2 are combined, and a composite image in which images F3 and F4 are combined.

[0085] 7 is a schematic diagram showing an example of a composite image of an indwelling needle image and a blood vessel image. As shown in Fig. 7, in the composite image, the brightness of pixels corresponding to the indwelling needle image is relatively high, and the brightness of pixels corresponding to the blood vessel visualization image is lower than the brightness of pixels corresponding to biological tissue other than blood vessels. The composite image may be an image obtained by extracting the indwelling needle image from the indwelling needle image and the blood vessel visualization image from the blood vessel image, and then combining the extracted indwelling needle image and blood vessel visualization image.

[0086] As shown in Fig. 6B, the brightness of pixels corresponding to the visualized blood vessel images in the blood vessel image is lower than the brightness of pixels corresponding to biological tissue other than blood vessels. Note that Figs. 6 and 7 are merely examples, and are not intended to limit the display content, brightness, etc. of the images to the examples.

[0087] At least one of the indwelling needle image and the blood vessel image displayed on the display device 23 may be an image obtained by subjecting the image acquired from the camera 22 to predetermined image processing to improve the visibility of the indwelling needle image and the visualized blood vessel image. Examples of image processing include contrast enhancement, brightness smoothing, edge enhancement, gamma correction, and binarization. The image processing content may be set according to the type of image, i.e., the indwelling needle image or the blood vessel image.

[0088] It should be noted that the light emission timing of the first light source 161 and the second light source 21 do not have to be completely synchronized with the frame rate of the camera 22. As long as the lighting of either the first light source 161 or the second light source 21 is synchronized for a certain number of frames, the lighting of either light source does not necessarily have to be assigned to every frame.

[0089] For example, the following light emission pattern may be used: At time t1, an image F1 is acquired in a state where the first light source 161 is emitting light and the second light source 21 is not emitting light, and at time t2, an image F2 is acquired in a state where the first light source 161 is not emitting light and the second light source 21 is emitting light. At time t3, an image F3 is acquired in a state where the first light source 161 is emitting light and the second light source 21 is not emitting light, and at time t4, an image F5 is acquired in a state where the first light source 161 and the second light source 21 are not emitting light.

[0090] In the above case, the processing device 20 classifies images F1, F2, and F3, which are obtained when the imaging timing of the camera 22 coincides with the light emission timing of either the first light source 161 or the second light source 21, into either an indwelling needle image or a blood vessel image based on the light emission pattern. Image F5, which is obtained when the imaging timing and light emission timing do not coincide, is classified as an "other image" that does not correspond to either an indwelling needle image or a blood vessel image. It is assumed that the "other image" does not contain an image that should be presented. The processing device 20 displays images F1, F2, and F3 classified as either an indwelling needle image or a blood vessel image on the display device 23, but does not display image F5 classified as an "other image" on the display device 23. Note that the processing device 20 may be configured so that the user can select in advance whether or not to display the "other image" on the display device 23.

[0091] If the timing of light emission of the first light source 161 and the second light source 21 is not completely synchronized with the frame rate of the camera 22, and therefore the latest image of an image type corresponding to the frame rate cannot be properly acquired, the processing device 20 may generate an interpolated image in place of the missing latest image. For example, the interpolated image may be the most recent image captured while the first light source 161 or the second light source 21, which was in a non-emitting state at the time the latest image was missing, was emitting light. For example, at the above-mentioned time t4, a blood vessel image captured while the second light source 21 was emitting light should be acquired, but the most recent blood vessel image is missing because the second light source 21 was in a non-emitting state. In this case, the processing device 20 displays, as an interpolated image at time t4, the most recent blood vessel image (image F2) captured while the second light source 21 was emitting light, instead of image F5 captured without emitting light at the time t4. Alternatively, instead of combining image F3 and image F5, a composite image may be generated and displayed by combining image F3 and image F2 as an interpolated image at time t4.

[0092] The interpolated image is not limited to one that uses the immediately preceding image of the same image type, but may be generated based on multiple frames obtained immediately before the imaging in a non-light-emitting state. For example, the processing device 20 calculates the amount of movement of the indwelling needle image based on the indwelling needle image detected from each of the multiple frames, and predicts the position of the indwelling needle image at the time of imaging in a non-light-emitting state from the calculated amount of movement of the indwelling needle image. The processing device 20 generates an interpolated image in which the indwelling needle image is positioned according to the prediction result. Note that the interpolated image may be generated using a machine learning model.

[0093] The configuration may be such that only one of the light emission of the first light source 161 and the second light source 21 is controlled by the processing device 20. For example, assume a case where the light emission of the first light source 161 is controlled by the processing device 20 and the light emission of the second light source 21 is controlled by an internal control mechanism. The processing device 20 detects the light emission interval of the second light source 21 in advance under a test environment, and adjusts the light emission of the first light source 161 in consideration of the frame rate of the camera 22 so that the first light source 161 emits light during the light emission interval of the second light source 21 according to the detection result. The processing device 20 can classify images output from the camera 22 based on the detected light emission pattern of the second light source 21, the light emission pattern of the first light source 161 controlled by the processing device 20 itself, and the frame rate of the camera 22.

[0094] As a second example of an image classification method, an indwelling needle image and a blood vessel image are classified by detecting an imaging target from images captured at each frame rate. The processing device 20 detects an imaging target contained in the image by analyzing the image acquired by the camera 22, and identifies the image type based on the detection result. The detection of an imaging target in an image is performed, for example, based on whether or not the feature amount of an object representing the imaging target satisfies predetermined requirements. The memory unit 202 of the processing device 20 pre-stores the types of feature amounts used to detect blood vessels and the light-emitting unit 17, as well as threshold values ​​for each feature amount.

[0095] The processing device 20 detects the light-emitting unit 17 in the image by detecting from the image, for example, an object whose brightness and size (range) are equal to or greater than a preset threshold. If the light-emitting unit 17 is detected from the image, the processing device 20 identifies the image type as an indwelling needle image and classifies the acquired image as an indwelling needle image.

[0096] The processing device 20 also detects blood vessels in the image by detecting from the image an object whose feature amount, including at least one of straightness, length, thickness, and extension direction, is equal to or greater than a predetermined threshold. When blood vessels are detected from the image, the processing device 20 identifies the image type as a blood vessel image and classifies the acquired image as a blood vessel image.

[0097] As described in the first example, the processing device 20 may alternately display the obtained vascular image and indwelling needle image on the display device 23, or may display a composite image of the vascular image and the indwelling needle image.

[0098] If neither blood vessels nor the light-emitting unit 17 are detected in the image acquired from the camera 22, the processing device 20 determines that the image type is an "other image" and classifies the acquired image as an "other image." The "other image" may or may not be displayed on the display device 23, as in the first example.

[0099] When an image is captured with both the first light source 161 and the second light source 21 turned on, both blood vessels and the light-emitting unit 17 can be detected from the image. When both blood vessels and the light-emitting unit 17 are detected from the image acquired from the camera 22, the processing device 20 identifies the image type as a second blood vessel image and classifies the acquired image as a second blood vessel image.

[0100] If the second vascular image is identified, the processing device 20 performs a removal process to remove the light-emitting element 17 from the second vascular image, and the image after the removal process can be used as the vascular image. The removal process can be performed using an indwelling needle image acquired immediately before the capture of the second vascular image or a previously acquired reference indwelling needle image. Pixels corresponding to the indwelling needle image in the immediately preceding indwelling needle image or reference indwelling needle image are detected, and the detected pixels are removed (subtracted from the brightness value) from the second vascular image, which includes both blood vessels and the light-emitting element 17, thereby removing the light-emitting element 17 from the second vascular image. After the removal process is performed, an image restoration process may be performed. Image restoration refers to recreating the removed pixel by filling in the removed pixel with the color or texture of the surrounding pixel. Examples of image restoration methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. Removal of the light-emitting element 17 and image restoration may be performed using machine learning techniques.

[0101] The second blood vessel image may be used as it is without undergoing the above-described removal process. Like other images, the second blood vessel image may not be displayed on the display device 23, or the user may be able to select whether or not to display it.

[0102] For example, when both the first light source 161 and the second light source 21 blink as the light emission pattern, any of four types of images including a blood vessel image, an indwelling needle image, a second blood vessel image, and other images can be acquired. When the first light source 161 is always on and the second light source 21 blinks, any of two types of images including an indwelling needle image and a second blood vessel image can be acquired.

[0103] As a third example of a method for classifying images, an indwelling needle image and a blood vessel image are distinguished by detecting the wavelength of the image captured at each frame rate. The processing device 20 analyzes the image acquired by the camera 22 to detect specific wavelengths contained in the image and identifies the image type based on the detection result.

[0104] When detecting the wavelength of an image, the wavelength of light L1 emitted by the light-emitting unit 17 and the wavelength of light L2 emitted by the second light source 21 are different, and it is necessary to use a camera 22 that can capture images by dispersing light into wavelengths. Examples of such a camera 22 include a hyperspectral camera, a prism spectroscopic camera, and a camera with a spectroscopic filter such as a bandpass filter. The memory unit 202 of the processing device 20 stores wavelength ranges corresponding to the light L1 and the light L2 in advance.

[0105] When a wavelength corresponding to light L1 from light-emitting unit 17 is detected from the image, processing device 20 determines that the image type is an indwelling needle image and classifies the acquired image as an indwelling needle image. When a wavelength corresponding to light L2 from second light source 21 is detected from the image, processing device 20 determines that the image type is a blood vessel image and classifies the acquired image as a blood vessel image.

[0106] The image types may further include "other images" and "second blood vessel images." If wavelengths corresponding to the light L1 from the light-emitting unit 17 and the light L2 from the second light source 21 are detected from the image, the processing device 20 classifies the acquired image as a "second blood vessel image." If neither the wavelengths corresponding to the light L1 from the light-emitting unit 17 nor the light L2 from the second light source 21 are detected from the image, the processing device 20 classifies the acquired image as a "other image."

[0107] 8 is a flowchart showing an example of a processing procedure executed by the processing device 20. The processing in the flowchart below is executed by the control unit 201 in accordance with a program 1P stored in the storage unit 202 of the processing device 20.

[0108] The control unit 201 of the processing device 20 synchronizes the start time and starts image capture by the camera 22 and emission of light from the first light source 161 and the second light source 21 according to a predetermined emission pattern (step S11). If the image capture by the camera 22 and the emission of light from the first light source 161 and the second light source 21 are not controlled by the control unit 201, step S11 may be omitted.

[0109] The control unit 201 acquires an image captured by the camera 22 (step S12). The control unit 201 temporarily stores the acquired image in the storage unit 202. As a result, a plurality of images continuously output from the camera 22 are stored in the storage unit 202 in chronological order.

[0110] The control unit 201 classifies the acquired images into blood vessel images and indwelling needle images by identifying the type of the acquired images (step S13). The image type may further include second blood vessel images and other images.

[0111] 9 is a flowchart showing a first example of a detailed procedure for image classification processing. The processing procedure shown in the flowchart of FIG. 9 corresponds to the details of step S13 in the flowchart of FIG.

[0112] The control unit 201 acquires the light emission patterns of the first light source 161 and the second light source 21 at the time of capturing the acquired image (step S21). The control unit 201 identifies the image type of the acquired image based on the acquired light emission patterns of the first light source 161 and the second light source 21 and the frame rate set in the camera 22 (step S22). The control unit 201 returns the process to step S14 in the flowchart of FIG. 8.

[0113] 10 is a flowchart showing a second example of the detailed procedure of the image classification process. The processing procedure shown in the flowchart of FIG. 10 corresponds to the details of step S13 in the flowchart of FIG.

[0114] The control unit 201 detects an imaging target included in the image based on the feature amount of the object included in the acquired image (step S31). The control unit 201 identifies the image type of the acquired image based on the correspondence between image types and imaging targets stored in advance and the detected imaging target (step S32). The control unit 201 returns the process to step S14 in the flowchart of FIG. 8.

[0115] 11 is a flowchart showing a third example of detailed image classification processing procedures. The processing procedures shown in the flowchart of FIG. 11 correspond to details of step S13 in the flowchart of FIG.

[0116] The control unit 201 detects the wavelengths contained in the acquired image (step S41). The control unit 201 identifies the image type of the acquired image based on the pre-stored correspondence between image types and wavelengths and the detected wavelengths (step S42). The control unit 201 returns the process to step S14 in the flowchart of FIG. 8.

[0117] 8, the control unit 201 determines whether the latest image of the type that should be acquired at the time of imaging has been acquired based on the image classification result (step S14). The control unit 201 specifies the type of image that should be acquired at the time of imaging, for example, based on a state in which blood vessel images and indwelling needle images are alternately acquired in accordance with the frame rate of the camera 22.

[0118] If it is determined that the latest image of the type that should be acquired has not been acquired at the time of image capture (S14: NO), the control unit 201 generates an interpolated image (step S15). The control unit 201 generates the interpolated image by, for example, reading out an image of the same type that was acquired immediately before the time of image capture from the storage unit 202. The control unit 201 proceeds to step S16.

[0119] If it is determined that the latest image of the type that should be acquired at the time of image capture has been acquired (S14: YES), the control unit 201 skips the process of generating an interpolated image and proceeds to step S16.

[0120] The control unit 201 performs various image processing on the acquired image or the generated interpolated image according to the classification result of the acquired image (step S16). Examples of image processing include contrast enhancement, brightness smoothing, edge enhancement, gamma correction, and binarization. Note that steps S14 to S16 may be omitted. Furthermore, image processing may be performed before the generation of the interpolated image.

[0121] The control unit 201 generates a composite image by combining the latest vascular image or interpolated image after image processing with the indwelling needle image (step S17). The composite image may be an image obtained by combining the latest second vascular image with the indwelling needle image. The control unit 201 displays the generated composite image on the display device 23 (step S18). Note that the control unit 201 may also display the acquired image on the display device 23 without generating a composite image. By sequentially outputting the vascular image and the indwelling needle image to the display device 23, the vascular image and the indwelling needle image can be displayed alternately on the display device 23.

[0122] The control unit 201 determines whether or not to end the process (step S19). For example, if it is determined not to end the process because a predetermined end operation has not been received (S19: NO), the control unit 201 returns the process to step S12 and repeats acquiring a new image. For example, if it is determined to end the process because a predetermined end operation has been received (S19: YES), the control unit 201 ends the series of processes.

[0123] According to this embodiment, the visibility of the light-emitting unit of the puncture instrument in an image can be improved by emitting light from the first light source and the light-emitting unit of the puncture instrument while the second light source of the blood vessel visualization device is not emitting light. By separately acquiring an indwelling needle image in which the first light source is emitting light and the second light source is not emitting light, and an image of a blood vessel in which at least the second light source is emitting light, it becomes easy to capture the puncture instrument and the blood vessel separately.

[0124] By photographing blood vessels when the puncture instrument is not emitting light, it is possible to prevent light from the light-emitting unit from reflecting off the surface of the body, making the blood vessels difficult to see, thereby improving the visibility of the blood vessel visualization image.

[0125] By combining and displaying an image of the indwelling needle and an image of the blood vessel, the positional relationship between the blood vessel and the puncture instrument can be accurately grasped. By alternately displaying an image of the indwelling needle and an image of the blood vessel, the time lag can be reduced and images can be presented in more real time.

[0126] By alternately emitting light from the first light source and the second light source in synchronization with the frame rate of the camera, light emission loss from the light source can be reduced, and images of the indwelling needle and blood vessel can be acquired efficiently and reliably. By classifying the images based on the subject or wavelength in the images, it is possible to properly classify the images even when the first light source and the second light source are not controlled by a processing device.

[0127] By providing a light-emitting unit at the tip of the puncture instrument, the position of the tip of the puncture instrument can be clearly grasped, and it can be reliably grasped that the blood vessel has been secured by the puncture instrument. By providing a light leakage prevention unit on the outer periphery of the puncture instrument between the first light source unit and the light-emitting unit, it is possible to prevent light from the puncture instrument from leaking outside and the leaked light from being reflected on the surface of the living body, making it difficult to see the blood vessel.

[0128] By projecting the indwelling needle image and blood vessel image onto the surface of the living body, the images can be viewed without moving the line of sight from the target to be punctured, thereby improving convenience for the surgeon.

[0129] When an image containing both blood vessels and light-emitting areas is acquired, a removal process can be performed to remove the light-emitting areas, thereby generating a blood vessel image that is not affected by the light-emitting areas, thereby improving the visibility of the blood vessel visualization image.

[0130] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The technical features described in each embodiment can be combined with each other, and the scope of the present invention is intended to include all modifications within the scope of the claims and equivalents thereto. The sequences shown in each embodiment are not limited, and within the scope of no contradiction, each processing step may be executed in a different order, or multiple processes may be executed in parallel. The entity that performs each process is not limited, and within the scope of no contradiction, the process of each device may be executed by another device.

[0131] The matters described in each embodiment can be combined with each other. Furthermore, the independent claims and dependent claims described in the claims can be combined with each other in any and all combinations, regardless of the reference format. Furthermore, the claims use a format in which a claim references two or more other claims (multiple claim format), but this is not limited to this. A multiple claim (multi-multi claim) that references at least one other multiple claim may also be used.

[0132] REFERENCE SIGNS LIST 100 Puncture system 1 Indwelling needle 11 Catheter 16 First light source unit 161 First light source 17 Light emitting unit 2 Blood vessel visualization device 20 Processing device 201 Control unit 202 Storage unit 2P Program 2A Recording medium 21 Second light source 22 Camera 23 Display device 3 Living body

Claims

1. A puncture system comprising: a puncture device equipped with a first light source and a light-emitting unit that emits light from the first light source; a second light source; an imaging unit that images a living body and the puncture device; and a control unit, wherein the first light source is capable of emitting light when the second light source is not emitting light; the control unit acquires images continuously captured by the imaging unit, separates the acquired images into a first image including the light-emitting unit that is captured when the first light source is emitting light and the second light source is not emitting light, and a second image including the blood vessels of the living body that is captured when the second light source is emitting light, and displays the separated first image and second image.

2. The puncture system according to claim 1, wherein the first image and the second image are synthesized and displayed.

3. The puncture system according to claim 1 or 2, wherein the most recent first image and second image are synthesized and displayed from among successively acquired images.

4. The puncture system according to claim 1, wherein the first image and the second image are displayed alternately.

5. The puncture system according to claim 1 or 2, wherein the first image and the second image are distinguished based on the light emission patterns of the first light source and the second light source and the frame rate of the imaging unit.

6. The puncture system according to claim 1 or claim 2, wherein a subject in an image is detected, and the first image and the second image are separated based on the detected subject.

7. The puncture system according to claim 6, wherein an image in which an object representing the subject is detected with brightness and size equal to or greater than a predetermined value is distinguished from the first image, and an image in which an object representing the subject is detected with at least one of straightness, length, thickness, and extension direction equal to or greater than a predetermined value is distinguished from the second image.

8. The puncture system according to claim 1 or 2, further comprising: detecting wavelengths in an image; and distinguishing the first image from the second image based on the detected wavelengths.

9. The puncture system according to claim 1 or claim 2, wherein an interpolated image is generated if the latest first image or second image is not acquired according to the frame rate of the imaging unit.

10. The puncture system according to claim 1 or 2, wherein the light emitting unit is provided at the tip of the puncture device.

11. The puncture system according to claim 1 or 2, wherein the first image and the second image are displayed on the surface of the living body.

12. An information processing method for a puncture system comprising a puncture device equipped with a first light source and a light-emitting unit that emits light from the first light source, a second light source, an imaging unit that images a living body and the puncture device, and a computer, the computer acquiring images continuously captured by the imaging unit, separating the acquired images into a first image including the light-emitting unit that was captured with the first light source emitting light and the second light source not emitting light, and a second image including the blood vessels of the living body that was captured with the second light source emitting light, and displaying the separated first image and second image.

13. A computer program for causing a computer of a puncture system comprising a first light source and a puncture device equipped with a light-emitting unit that emits light from the first light source, a second light source, an imaging unit that images a living body and the puncture device, and a computer to execute the following process: acquire images continuously captured by the imaging unit; separate the acquired images into a first image including the light-emitting unit that is captured with the first light source emitting light but the second light source not emitting light, and a second image including the blood vessels of the living body that is captured with the second light source emitting light; and display the separated first image and second image.

Images