Computer program, blood vessel visualization device, blood vessel visualization system, and blood vessel visualization method

Abstract

Provided are a computer program, a blood vessel visualization device, a blood vessel visualization system, and a blood vessel visualization method with which it is possible to provide assistance so as to enable easy puncture without operating a device button.　The computer program causes a computer to execute processing for: detecting vascular information on blood vessels included in a prescribed part of a subject on the basis of an image obtained by imaging the prescribed part; generating a projection image of the blood vessels on the basis of the detected vascular information; and when projecting the generated projection image, projecting a virtual operation part for operating the projection image onto a region that is included in the projection image and that is not the blood vessel region.

Description

Computer program, blood vessel visualization device, blood vessel visualization system, and blood vessel visualization method

[0001] The present disclosure relates to a computer program, a blood vessel visualization device, a blood vessel visualization system, and a blood vessel visualization method.

[0002] Medical professionals perform venipuncture (venipuncture) in a variety of situations, including blood sampling, intravenous drips, and dialysis. However, when the patient's veins are thin, it can be difficult to find a suitable vein for puncture.

[0003] Patent Document 1 discloses a blood vessel visualization device that irradiates the inside of a person's forearm with near-infrared light, displays an image of the blood vessels on a monitor, and allows the needle of a syringe to be inserted into a blood vessel by targeting a blood vessel suitable for puncture.

[0004] Japanese Patent Application Laid-Open No. 2004-237051

[0005] However, blood vessel visualization devices such as those described in Patent Document 1 require the operation of device buttons located away from the puncture site. Operating the device buttons is difficult for medical professionals because both hands are required to perform the puncture. Furthermore, operating the device buttons during the puncture raises concerns about infection via the device buttons.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide a computer program, a blood vessel visualization device, a blood vessel visualization system, and a blood vessel visualization method that can assist in easily performing puncture without operating any device buttons.

[0007] (1) A computer program according to the present disclosure causes a computer to execute the following process: detect vascular information of blood vessels contained in a specific part of a subject based on an image of the specific part; generate a projection image of the blood vessels based on the detected vascular information; and, when projecting the generated projection image, project a virtual operation unit for operating the projection image onto an area excluding the vascular area contained in the projection image.

[0008] Here, an embodiment of the present disclosure is as follows: (2) The computer program of (1) above causes a computer to execute a process of detecting a projectable area of ​​the virtual operation unit and projecting the virtual operation unit onto the detected projectable area excluding the blood vessel area.

[0009] (3) The computer program according to (1) or (2) causes a computer to execute a process of detecting an operation on the virtual operation unit and changing the display of the projection image in accordance with the detected operation.

[0010] (4) Any one of the computer programs (1) to (3) above causes a computer to execute a process of identifying a recommended projection area for the virtual operation panel and projecting the virtual operation panel into the identified recommended projection area.

[0011] (5) The computer program of (4) causes a computer to execute a process of detecting the range of movement of the arm, including the fingers, of the puncture operator at the specified site, and excluding the detected range of movement to identify the recommended projection area.

[0012] (6) In the computer program of (5) above, the recommended projection area is an area excluding the movement range and including an area closer to the puncture operator.

[0013] (7) Any one of the computer programs (1) to (6) above causes a computer to execute the following process: calculate vascular parameters including at least one of the length, thickness, depth, straightness, and direction of the blood vessel based on the detected vascular information; identify an optimal vascular region for puncturing among the vascular regions included in the projection image based on the calculated vascular parameters; and project the virtual operation unit onto an area excluding the optimal vascular region for puncturing.

[0014] (8) Any one of the computer programs (1) to (7) above causes a computer to execute the following process: calculate vascular parameters including at least one of the length, thickness, depth, straightness, and direction of the blood vessel based on the detected vascular information; calculate a projection avoidance priority of the vascular region included in the projection image by weighting with the calculated vascular parameters; and select a projection region onto which the virtual operation unit is projected based on the calculated priority.

[0015] (9) Any one of the computer programs (1) to (8) above causes a computer to execute a process of detecting the position of a vascular access device to be inserted into the specified site based on an image of the specified site, and projecting the virtual operating unit onto an area other than the detected position.

[0016] (10) Any one of the computer programs (1) to (9) above causes a computer to execute a process of selecting an object to be operated in the virtual operation unit according to the priority of multiple operated objects included in the virtual operation unit, and projecting the selected object to be operated.

[0017] (11) The blood vessel visualization device according to the present disclosure includes a control unit that detects vascular information of blood vessels contained in a specific area of ​​a subject based on an image of the specific area, generates a projection image of the blood vessels based on the detected vascular information, and when projecting the generated projection image, projects a virtual operation unit for operating the projection image in an area excluding the vascular area contained in the projection image.

[0018] (12) A vascular visualization system according to the present disclosure includes the aforementioned vascular visualization apparatus and a vascular access device that is inserted into a predetermined site of a subject.

[0019] (13) The vascular visualization method according to the present disclosure detects vascular information of blood vessels contained in a specific area of ​​a subject based on an image of the specific area, generates a projection image of the blood vessels based on the detected vascular information, and when projecting the generated projection image, projects a virtual operation unit for operating the projection image in an area excluding the vascular area contained in the projection image.

[0020] According to the present disclosure, it is possible to assist in easily performing puncture without operating any device buttons.

[0021] FIG. 1 is a diagram showing an example of the configuration of a blood vessel visualization device of this embodiment. FIG. 2 is a diagram showing a first example of an implementation method of a blood vessel visualization device. FIG. 3 is a diagram showing a second example of an implementation method of a blood vessel visualization device. FIG. 4 is a diagram showing an example of blood vessel information (blood vessel image). FIG. 5 is a diagram showing an example of blood vessel parameters. FIG. 6 is a diagram showing an example of a projection image of a blood vessel. FIG. 7 is a diagram showing an example of a projection area of ​​a virtual operation unit. FIG. 8 is a diagram showing an example of a method for calculating a puncture blood vessel priority and a projection-avoidance blood vessel priority. FIG. 9 is a diagram showing an example of a virtual operation unit. FIG. 10 is a diagram showing an example of a virtual operation unit. FIG. 11 is a diagram showing an example of a virtual operation unit. FIG. 12 is a diagram showing an example of a virtual operation unit during projection. FIG. 13 is a diagram showing an example of an indirect operation on the virtual operation unit. FIG. 14 is a diagram showing an example of the configuration of a blood vessel visualization system equipped with a vascular access device. FIG. 15 is a diagram showing an example of a processing procedure by the blood vessel visualization device.

[0022] An embodiment of the present disclosure will now be described. Fig. 1 is a diagram showing an example of the configuration of a blood vessel visualization device 50 according to this embodiment. The blood vessel visualization device 50 includes a control unit 51 that controls the entire device, a communication unit 52, a light source unit 53, a camera unit 54, a projection unit 55, a sensor unit 56, a memory 57, an image processing unit 58, and a storage unit 59.

[0023] The control unit 51 is configured by incorporating a required number of CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), etc. The control unit 51 may also be configured by combining DSPs (Digital Signal Processors), FPGAs (Field-Programmable Gate Arrays), etc.

[0024] The communication unit 52 may include, for example, a communication module, and may have a communication function (either wireless communication or wired communication, or both) with an external device (not shown).

[0025] Light source unit 53 includes, for example, one or more LED light sources that emit near-infrared light of 700 to 2500 nm. Light source unit 53 includes, for example, one or more LED light sources that emit visible light of 380 to 770 nm. The light intensity of light source unit 53 is adjustable under the control of control unit 51. Light source unit 53 irradiates a predetermined site (site to be punctured) of the subject (person to be punctured) with the above-mentioned light.

[0026] The camera unit 54 can capture near-infrared light of, for example, 700 to 2500 nm. The camera unit 54 can also capture visible light of, for example, 380 to 770 nm. The camera unit 54 can also capture far-infrared light of, for example, 2500 nm or longer. The camera unit 54 may be configured as a single (integrated) camera capable of capturing near-infrared light, visible light, and far-infrared light, or as separate cameras. If the cameras are a single unit, a filter or the like may be used to switch between near-infrared light, visible light, and far-infrared light. If the cameras are separate, it is preferable to use a half mirror or the like so that the cameras can capture images coaxially.

[0027] The camera unit 54 is configured to photograph a predetermined part of the subject from a position appropriately separated from the predetermined part. In this case, the light source unit 53 may be located near the camera unit 54 on the same side of the predetermined part as the camera unit 54, or on the opposite side of the predetermined part from the camera unit 54 (i.e., so that the predetermined part is sandwiched between the camera unit 54 and the light source unit 53), or the light source unit 53 may be located on both sides of the predetermined part so that the predetermined part is sandwiched between them.

[0028] The projection unit 55 includes a projector that can project a projected image of blood vessels and a virtual operation unit (digital object) for operating the projected image onto a desired location such as human skin or the surface of a desk. The projector that projects the projected image and the projector that projects the virtual operation unit may be the same or different. The projected image and the virtual operation unit will be described in detail later.

[0029] Furthermore, the camera unit 54 captures an image of the operation of the puncture operator (healthcare worker) on the virtual operation unit. The control unit 51 can identify the content of the operation by the puncture operator based on the image captured by the camera unit 54. For example, it can identify which operation item on the virtual operation unit the operation is for based on the movement of the puncture operator's finger or hand, the relative position of the puncture operator to the operated object in the virtual operation unit, etc. Furthermore, a learning model (one learning model of the learning model unit 61) that has been trained to output expected operation content when an image of a finger or hand including the virtual operation unit is input may be used.

[0030] The sensor unit 56 detects the movement of the fingers or hands of a puncture operator such as a medical professional, and detects an operation on the virtual operation unit. The sensor unit 56 includes at least one of an acoustic sensor, a vibration sensor, a depth sensor, an acceleration sensor, and an electric potential sensor, for example.

[0031] Acoustic sensors and vibration sensors can detect vibrations based on the movement of the arm or palm (e.g., vibrations due to contact or vibrations when flicking), and can determine the position and method of flicking (i.e., the distinction between the operations of the puncture operator) based on the shape and strength of the detected vibration waveform.

[0032] The depth sensor works by having the subject wear a ring with a depth sensor on their wrist, shining infrared light onto the target area (e.g., a finger, arm, etc.), measuring the time from the time of irradiation until the reflected light returns, and determining the position (coordinates) of the target area (e.g., a finger) and any changes in the target area (e.g., an arm) based on the measured time.

[0033] For example, the acceleration sensor can be used by having the puncture operator wear a ring with an acceleration sensor, detecting the acceleration of the puncture operator's finger movements, and calculating the finger movements (magnitude, number of movements, direction of movement, etc.) based on the detected acceleration to determine the finger movements.

[0034] The potential sensor can be configured, for example, by having the puncture operator wear a ring with a potential sensor, detecting the potential difference when the skin is touched, and determining whether or not contact has occurred, the location of the contact, etc., based on the detected potential difference.

[0035] The memory 57 can be configured with a semiconductor memory such as an SRAM (Static Random Access Memory), a DRAM (Dynamic Random Access Memory), or a flash memory.

[0036] The image processing unit 58 detects a blood vessel image of blood vessels contained in a predetermined region of the subject based on an image obtained by photographing the predetermined region of the subject with the camera unit 54, and detects blood vessel information about the blood vessels. The image processing unit 58 also detects a projectable area of ​​the virtual operation unit based on an image obtained by photographing the predetermined region of the subject with the camera unit 54. The image processing unit 58 also detects direct or indirect operation of the virtual operation unit by the puncture operator. A direct operation of the virtual operation unit means an operation by directly touching the projected virtual operation unit, and an indirect operation means an operation by indirectly operating the virtual operation unit by operating an operation object other than the virtual operation unit.

[0037] The storage unit 59 can be configured, for example, with a hard disk or semiconductor memory, and can store a computer program (program product) 60, a learning model unit 61, and required information.

[0038] The computer program 60 is a computer program (application) that runs on the blood vessel visualization device 50, and is deployed in the memory 57 and executed by the control unit 51. In other words, processing by the control unit 51 is also processing by the computer program 60. The computer program 60 may be recorded on a recording medium M (for example, an optically readable disk storage medium such as a CD-ROM) and read by a recording medium reading unit (not shown) and stored in the storage unit 59, or it may be downloaded from an external device via the communication unit 52 and installed. Details of the learning model unit 61 will be described later.

[0039] Next, a method for implementing the blood vessel visualization device 50 will be described.

[0040] FIG. 2 is a schematic diagram illustrating a first example of a method for implementing the blood vessel visualization device 50. Hereinafter, the arm (forearm) will be used as an example of the predetermined site of the subject. However, the predetermined site is not limited to the arm and may be another site punctured for blood sampling, intravenous drip, dialysis, etc. The blood vessel visualization device 50 is fixed to a support such as a stand so as to be positioned at an appropriate distance from the subject's arm. The blood vessel visualization device 50 may be movably attached to the support so that the light source unit 53, camera unit 54, and projection unit 55 are oriented in optimal directions. The sensor unit 56 is, for example, a depth sensor. However, in the case of other sensors, the sensor unit 56 may be provided within the blood vessel visualization device 50 or may be attached to the finger of the puncture operator. Furthermore, if the movement of the puncture operator's finger or hand is detected based on an image captured by the camera unit 54, the sensor unit 56 may not be provided.

[0041] A projection image 20 of blood vessels is projected onto the arm of the subject by the projection unit 55. In the example of Fig. 2, two blood vessel regions 21 and 22 are displayed in the projection image 20.

[0042] Fig. 3 is a diagram schematically illustrating a second example of a method for implementing the blood vessel visualization device 50. In the second example shown in Fig. 3, a wearing belt 70 is worn on the subject's upper arm, and the blood vessel visualization device 50 is implemented on the wearing belt 70. A light source unit 53, a camera unit 54, and a projection unit 55 are attached to appropriate positions on the wearing belt 70 so that they face in optimal directions. As in the first example, a blood vessel projection image 20 is projected onto the subject's arm by the projection unit 55, and two blood vessel regions 21 and 22 are displayed in the projection image 20.

[0043] Next, a method for detecting blood vessel information of the subject's arm based on an image obtained by photographing the subject's arm with the camera unit 54 will be described.

[0044] 4 is a diagram showing an example of blood vessel information (blood vessel image). The control unit 51 irradiates near-infrared light from the light source unit 53 toward the subject's arm, and acquires an image by photographing the arm irradiated with the near-infrared light with the camera unit 54. Hemoglobin contained in blood has the property of absorbing near-infrared light, and by utilizing this property, parts of the image corresponding to blood vessels appear dark, and parts other than blood vessels appear brighter than the parts corresponding to blood vessels.

[0045] The image processing unit 58 performs edge detection and binarization on the obtained image to distinguish between blood vessels and non-blood vessels, thereby obtaining blood vessel information (blood vessel image). In the example of Fig. 4, blood vessel information (blood vessel image) 10 includes blood vessels 11, 12, 13, 14, 15, 16, 17, and 18.

[0046] As described above, the control unit 51 can detect blood vessel information of blood vessels contained in a predetermined part of a subject based on an image of the predetermined part.

[0047] The detection of vascular information is not limited to edge detection or binarization processing in the image processing unit 58. For example, a learning model (one of the learning models in the learning model unit 61) that has been trained to output the vascular shape when an image of a predetermined part of the subject is input may be used. Also, a thermal camera that irradiates far-infrared light from the light source unit 53 and detects the temperature of the skin surface irradiated with the far-infrared light with the camera unit 54 may be used in combination.

[0048] Next, a method for calculating blood vessel parameters will be described.

[0049] 5 is a diagram showing an example of vascular parameters. The vascular parameters can be used when selecting a blood vessel to be punctured (e.g., selecting an optimal blood vessel for puncture or determining the priority of blood vessels to be punctured), and can also be used when selecting a projection region of the virtual operation unit while avoiding blood vessel regions included in a projection image of the blood vessel. The vascular parameters include at least one of the length, thickness, depth, straightness, and running direction of the blood vessel.

[0050] The length of a blood vessel is the distance from one end to the other. Generally, the longer and more linear a blood vessel, the easier it is to puncture it. The thickness of a blood vessel is the distance (the width of the blood vessel) perpendicular to the length of the blood vessel. Generally, the thicker the blood vessel, the easier it is to puncture it. The depth of a blood vessel is the distance from the skin surface to the blood vessel. When light is irradiated from the light source unit 53, the depth of the blood vessel can be calculated by measuring the ultrasound generated by the blood vessel that selectively absorbs the light energy. Generally, the shallower a blood vessel is, the easier it is to puncture it. The linearity of a blood vessel can be calculated by thinning the blood vessel image to generate an approximate line and using the coefficient of determination (R2), which indicates how closely the generated approximate line matches a straight line. The closer the coefficient of determination is to 1, the higher the linearity and the easier it is to puncture it. The running direction of a blood vessel is the angle between the direction of the approximate line of the blood vessel and a specific direction. The specific direction can be, for example, the direction of the puncture needle when the puncture operator punctures the blood vessel. The puncture needle includes both an inner needle and an outer needle. The inner needle is made of metal and is used to insert into a blood vessel. The outer needle is inserted into the blood vessel using, for example, a soft plastic catheter, and then removed, leaving only the outer needle in the blood vessel.

[0051] 5, for the blood vessel 11, the length, thickness, depth, straightness, and running direction are represented by L11, T11, D11, S11, and R11 for convenience. The same applies to the other blood vessels 12 to 18.

[0052] As described above, the control unit 51 can calculate blood vessel parameters including at least one of the length, thickness, depth, straightness, and running direction of the blood vessel based on the detected blood vessel information.

[0053] FIG. 6 is a diagram showing an example of a projection image 20 of a blood vessel. The values ​​of the blood vessel parameters (length, thickness, depth, straightness, and running direction of the blood vessel) of each of the blood vessels 11 to 18 shown in FIG. 4 are compared with the respective blood vessel parameter thresholds to select a blood vessel suitable for puncture. In the example of FIG. 6, blood vessels 11 and 12 are selected from the blood vessels 11 to 18, and a projection image 20 is generated based on the blood vessel information of the blood vessels 11 and 12. Blood vessel regions 21 and 22 in the projection image 20 correspond to the blood vessels 11 and 12 in the blood vessel information 10. The blood vessel regions 21 and 22 can be generated by simplifying the shapes of the actual blood vessels 11 and 12. For example, the outer diameter shape of the blood vessel can be displayed as a straight line or the like within the blood vessel, but not beyond the outer diameter shape.

[0054] As described above, the control unit 51 can generate a projection image of the blood vessels based on the detected blood vessel information.

[0055] Next, the projection of the virtual control panel will be described.

[0056] First, the control unit 51 detects the projectable area of ​​the virtual operation unit and can project the virtual operation unit onto the detected projectable area, excluding the blood vessel area of ​​the projected image. Positioning the virtual operation unit within the area excluding the blood vessel area prevents interference with puncture. The projectable area of ​​the virtual operation unit can be determined, for example, by the following methods: (1) The projectable area is detected by performing image processing such as edge detection on the image captured by the camera unit 54. In this case, light may be emitted from the light source unit 53 or the projection unit 55. (2) Far-infrared light may be emitted from the light source unit 53 and captured by the camera unit 54 (using a thermal camera in combination). (3) Another method for detecting moving objects may be to use a learning model (one learning model within the learning model unit 61) that is trained to output the projectable area when an image of the arm or its surroundings is input. For example, markers for defining the projectable area may be attached to multiple required locations on the arm, and the area surrounded by the markers may be detected to determine the projectable area.

[0057] Fig. 7 is a diagram showing an example of the projection area 30 of the virtual operation unit. In Fig. 7, a projection image including blood vessel areas 21 and 22 is projected onto the skin of the arm and displayed. The puncture operator can insert the puncture needle 100 by referring to the blood vessel area in the projected image. In the example of Fig. 7, the blood vessel area 21 is selected as the blood vessel to be punctured.

[0058] When projecting the generated projection image, the control unit 51 can project a virtual operation unit for operating the projection image onto an area (projection area 30) excluding the vascular areas 21 and 22 included in the projection image. In the example of FIG. 7 , the projection area 30 of the virtual operation unit is divided into two projection areas 31 and 32 and projected. There may be one or more projection areas. The virtual operation unit may also be projected within the vascular area. For example, if a specific vascular area can be visually confirmed on the arm of the person to be punctured, the puncture operator can manually or semi-automatically select the vascular projection image to be projected by directly touching the specific vascular area to be visualized. Furthermore, using the virtual operation unit can prevent infection.

[0059] The control unit 51 can identify a recommended projection area for the virtual operation unit and project the virtual operation unit into the identified recommended projection area. The recommended projection area is an area that does not overlap at least a blood vessel area in the projected image. This allows the virtual operation unit to be projected in an optimal position that does not interfere with puncture.

[0060] The recommended projection area is an area that does not overlap the blood vessel area of ​​the projection image and is an area where the puncture operator's fingers or hands are not present. The control unit 51 can detect the movement range of the puncture operator's arm, including his / her fingers, at a predetermined site and identify the recommended projection area by excluding the detected movement range. This allows the virtual operation unit to be projected at an optimal position that does not interfere with the puncture. Furthermore, puncture can be performed with minimal discomfort (for example, the projection image does not appear on the puncture operator's arm, including his / her fingers). The recommended projection area is an area where the puncture operator's fingers or hands are not present and is closer to the puncture operator (an area closer to the puncture operator). By positioning the recommended projection area closer to the puncture operator and projecting the virtual operation unit, the puncture operator can easily operate the virtual operation unit and operate the buttons more smoothly.

[0061] It is possible to select a blood vessel to be punctured and a blood vessel onto which the projection of the virtual operation unit is to be avoided based on the blood vessel parameters calculated based on the above-mentioned blood vessel information.

[0062] Equation 1 in Figure 8 shows an example of a method for calculating the puncture vessel priority and the projection-avoidance vessel priority. The weighting coefficients for weighting the values ​​of the vessel parameters, namely, the length, thickness, depth, straightness, and running direction of the blood vessel, are w1, w2, w3, w4, and w5. w1 to w5 can also be 0. The puncture vessel priority W1 can be calculated using the following formula: W1 = (length) x w1 + (thickness) x w2 + (depth) x w3 + (straightness) x w4 + (running direction) x w5. The puncture vessel priority W1 of a blood vessel can be calculated from the blood vessel information of the blood vessel included in the predetermined portion. The blood vessel with the highest puncture vessel priority W1 value can be determined as the optimal blood vessel for puncture.

[0063] The control unit 51 can identify the optimum vascular region for puncture from among the vascular regions included in the projection image based on the vascular parameters, and project the virtual operation unit onto the region excluding the optimum vascular region for puncture. By displaying the blood vessels suitable for puncture, the blood vessels can be evaluated quickly and accurately.

[0064] In addition, training data including the value of a vascular parameter and the puncture success rate or success or failure of puncture for a blood vessel having the vascular parameter of that value may be collected, and a learning model (one learning model of the learning model unit 61) may be used that is trained to output the puncture success rate or success or failure of puncture when the value of the vascular parameter is input using the collected training data.

[0065] 8, the projection-avoidance vessel priority W2 can be calculated by the formula W2=K / W1, where K is a constant. The lower (smaller) the puncture vessel priority W1 is, the higher (larger) the projection-avoidance vessel priority W2 is.

[0066] The control unit 51 calculates the projection avoidance priority of the blood vessel region included in the projection image by weighting with the calculated blood vessel parameters, and can select the projection region onto which the virtual operation unit is to be projected based on the calculated priority. By weighting, even if the projection range is narrow, the virtual operation unit can be projected so as not to interfere with blood vessels suitable for puncture.

[0067] 9A to 9C are diagrams showing examples of a virtual operation unit. The items that can be operated using the virtual operation unit include, for example, the contrast and brightness of the projected image of blood vessels, the number of blood vessels to be displayed, visualization image ON / OFF, device image ON / OFF, and blood vessel color temperature, but the items that can be operated are not limited to the example shown in FIG. 9. In FIG. 9A, the operation can be performed by tapping the operation button for the operation item on the virtual operation unit. In FIG. 9B, the operation can be performed by sliding the slider bar for the operation item on the virtual operation unit. In FIG. 9C, the operation can be performed by rotating the ring button for the operation item on the virtual operation unit. The visualization image OFF operation button is used to stop the projection of the projected image (to prevent it from being displayed). The device image ON operation button is used to turn on the display of the tip position of the puncture needle of the vascular access device. Vascular access devices include, for example, peripheral arterial and venous indwelling needles, dialysis indwelling needles, PICCs, midlines, and CV catheters. Furthermore, when the device image is ON, the size, color, etc. of the area display (for example, surrounding the position with a circular mark) for indicating the tip position of the puncture needle of the vascular access device may be changed.

[0068] Although not shown in Fig. 9, a button (ON / OFF button) for selecting whether to project only the optimum blood vessels for puncture or all blood vessels may be projected. Also, operation buttons for adjusting the edge and brightness of blood vessels may be projected.

[0069] Fig. 10 is a diagram showing an example of a virtual operation unit during projection. In the example of Fig. 10, the projection area is divided into two areas. Of the two projection areas, the operation items of the virtual operation unit with a higher priority can be placed in the projection area closer to the puncture operator. In the example of Fig. 10, "Visualized image OFF" and "Device image ON" are projected from the operation items of the virtual operation unit into the projection area closer to the puncture operator, and the remaining operation items of the virtual operation unit (contrast, brightness, number of blood vessels) are projected into the projection area farther from the puncture operator.

[0070] 10, the virtual operation unit is projected onto the surface of the subject's arm, but this is not limiting. For example, the virtual operation unit may be projected onto the surface of a table or desk on which the subject's arm is placed, at a position close to the puncture operator.

[0071] The puncture operator can change the display of the projected image by operating the operation items on the virtual operation unit shown in Figures 9 and 10. For example, operating "Visualization Image OFF" stops the projection of the projected image of blood vessels. Operating "Device Image ON" displays the position of the vascular access device in the projected image. Operating "Contrast," "Brightness," and "Number of Blood Vessels" changes the contrast of the projected image, the brightness of the projected image, and the number of blood vessels in the projected image, respectively.

[0072] As described above, the control unit 51 can detect a direct operation on the virtual operation unit and change the display of the projection image in accordance with the detected operation. A direct operation refers to a case where an operation is performed directly on the virtual operation unit.

[0073] Furthermore, the control unit 51 can select an object to be operated on the virtual operation unit in accordance with the priority of the multiple objects to be operated included in the virtual operation unit, and project the selected object to be operated. Even if the projection range is narrow, the virtual operation unit can be projected while maintaining ease of use.

[0074] FIG. 11 is a diagram showing an example of indirect operation of a virtual operation unit. In the example of FIG. 11 , a virtual operation unit 41 is projected onto the subject's upper arm. An indirect operation area 45 is also projected onto the subject's arm. When the puncture operator slides his / her finger on the indirect operation area 45, a pointer 46 on the virtual operation unit 41 moves in conjunction with the slide, allowing the puncture operator to select a desired operation item. The puncture operator can indirectly operate an operation item on the virtual operation unit 41 by tapping his / her finger on the indirect operation area 45. Note that, although the indirect operation area 45 is projected onto the surface of the subject's arm in the example of FIG. 11 , this is not a limitation. For example, the indirect operation area 45 may be projected onto the surface of a table or desk on which the subject's arm is placed, at a position close to the puncture operator.

[0075] As described above, the control unit 51 can detect indirect operations on the virtual operation unit and change the display of the projection image in accordance with the detected operations. An indirect operation refers to indirectly operating the virtual operation unit via an operation unit separate from the virtual operation unit. By changing the display of the projection image, the virtual operation unit can be projected in an optimal state for puncturing.

[0076] Figure 12 is a diagram showing an example of the configuration of a blood vessel visualization system equipped with a vascular access device. A light source unit emitting near-infrared light and a light receiving unit are arranged to sandwich the visualization target site. Near-infrared light is irradiated from the light source unit, and the light that passes through the visualization target site is received by the light receiving unit. A projection image of the blood vessel is generated and projected, and the position of the tip of the puncture needle of the vascular access device is detected and the detected position of the tip of the puncture needle is displayed on the projection image. In this case, when the light-emitting unit at the tip of the puncture needle reaches the blood vessel, the light from the light-emitting unit goes out, allowing visual confirmation that the puncture needle has secured the blood vessel.

[0077] Vascular access devices include, for example, peripheral arterial and venous indwelling needles, dialysis indwelling needles, PICCs, midlines, and CV catheters, and the tip of the catheter or the tip of the inner needle is configured to emit, fluoresce, or reflect near-infrared light.

[0078] The light source unit and the camera unit 54 may be disposed on the same side of the region to be visualized, and the reflected light of the near-infrared light irradiated from the light source unit may be detected. Also, the camera unit 54 may be provided in addition to the light receiving unit.

[0079] In the above-described blood vessel visualization system, the control unit 51 detects the position of the vascular access device to be inserted into a predetermined site based on an image of the predetermined site, and can project a virtual operation unit onto an area other than the detected position. By detecting the position of the vascular access device, the virtual operation unit can be projected without interfering with the puncture.

[0080] Furthermore, by combining the vascular visualization device 50 of this embodiment with a semi-automatic puncture robot, the medical professional can evaluate the puncture vessel with the assistance of the vascular visualization device 50 and make the final decision on whether to perform the puncture, which is expected to enable quick button operations and smooth puncture.

[0081] FIG. 13 is a diagram showing an example of a processing procedure performed by the blood vessel visualization device 50. For convenience, the following description will be given with the control unit 51 acting as the main actor in the processing. The control unit 51 irradiates a predetermined part of the subject with predetermined light and photographs the predetermined part (S11). The predetermined light is, for example, near-infrared light. The predetermined light may also include visible light and far-infrared light. The control unit 51 acquires blood vessel information about blood vessels contained in the predetermined part (S12), and generates a projection image of the blood vessels based on the acquired blood vessel information (S13). For blood vessel information, see FIG. 4. For the projection image, see FIG. 6.

[0082] The control unit 51 detects the projectable area of ​​the virtual operation unit (S14), projects a projection image of the blood vessel, and projects the virtual operation unit onto the projectable area excluding the blood vessel area included in the projection image (S15). Note that if a suitable blood vessel for puncture has been selected, the virtual operation unit may be projected onto a blood vessel area other than the suitable blood vessel area. Also, if the priority of the blood vessel for puncture has been calculated and multiple blood vessels as candidates for puncture have been selected according to the priority, the virtual operation unit may be projected onto a blood vessel area other than the selected suitable blood vessel area.

[0083] The control unit 51 determines whether or not an operation (direct operation or indirect operation) has been performed on the virtual operation unit (S16), and if an operation has been performed (YES in S16), changes the display of the projection image in accordance with the operation (S17), and determines whether or not to end the processing (S18). If no operation has been performed on the virtual operation unit (NO in S16), the control unit 51 performs the processing of step S18. If the processing is not to be ended (NO in S18), the control unit 51 continues the processing from step S16 onwards, and if the processing is to be ended (YES in S18), the control unit 51 ends the processing.

[0084] According to this embodiment, an operable virtual operation unit (operation buttons, icons, bars, etc. displayed by projection) can be placed in an appropriate position close to the puncture site that does not interfere with the puncture (for example, on the surface of the skin at the specified site, or on the surface of a stand or table on which the specified site is placed), eliminating the need to operate device buttons located far from the puncture site, and since no actual device buttons are required, there is no risk of infection via device buttons. This makes it possible to assist in easy puncture without operating device buttons.

[0085] 10 Blood vessel information 11, 12, 13, 14, 15, 16, 17, 18 Blood vessel 20 Projected image 21, 22 Blood vessel region 50 Blood vessel visualization device 51 Control unit 52 Communication unit 53 Light source unit 54 Camera unit 55 Projection unit 56 Sensor unit 57 Memory 58 Image processing unit 59 Storage unit 60 Computer program 61 Learning model unit 100 Puncture needle

Claims

1. A computer program that causes a computer to execute the following processes: detect vascular information of blood vessels contained in a specified part of a subject based on an image of the specified part, generate a projection image of the blood vessels based on the detected vascular information, and when projecting the generated projection image, project a virtual operation unit for operating the projection image onto an area excluding the blood vessel area contained in the projection image.

2. The computer program according to claim 1, which causes a computer to execute the process of: detecting a projectable area of ​​the virtual operation unit; and projecting the virtual operation unit onto the detected projectable area excluding the blood vessel area.

3. The computer program according to claim 1, which causes a computer to execute a process of detecting an operation on the virtual operation unit, and changing the display of the projection image in accordance with the detected operation.

4. A computer program according to any one of claims 1 to 3, which causes a computer to execute the process of: identifying a recommended projection area for the virtual operation unit; and projecting the virtual operation unit into the identified recommended projection area.

5. The computer program according to claim 4, which causes a computer to execute a process of detecting a movement range of the arm, including the fingers, of a puncture operator at the specified site, and specifying the recommended projection area by excluding the detected movement range.

6. The computer program according to claim 5, wherein the recommended projection area is an area excluding the movement range and including an area closer to the puncture operator.

7. A computer program as claimed in any one of claims 1 to 3, which causes a computer to execute the following processes: calculating vascular parameters including at least one of the length, thickness, depth, straightness and direction of the blood vessel based on detected vascular information; identifying an optimum vascular region for puncturing from among the vascular regions included in the projection image based on the calculated vascular parameters; and projecting the virtual operation unit onto an area excluding the optimum vascular region for puncturing.

8. A computer program according to any one of claims 1 to 3, which causes a computer to execute the following processes: calculating vascular parameters including at least one of the length, thickness, depth, straightness and direction of the blood vessel based on the detected blood vessel information; calculating a projection avoidance priority of the blood vessel region included in the projection image by weighting with the calculated vascular parameters; and selecting a projection region into which the virtual operation unit is projected based on the calculated priority.

9. A computer program as claimed in any one of claims 1 to 3, which causes a computer to execute a process of detecting a position of a vascular access device to be inserted into the specified site based on an image of the specified site, and projecting the virtual operation unit onto an area other than the detected position.

10. A computer program as claimed in any one of claims 1 to 3, which causes a computer to execute a process of selecting an object to be operated of the virtual operation unit in accordance with the priority of a plurality of objects to be operated included in the virtual operation unit, and projecting the selected object to be operated.

11. A blood vessel visualization device comprising a control unit which detects blood vessel information of blood vessels contained in a specific part of a subject based on an image of the specific part, generates a projection image of the blood vessels based on the detected blood vessel information, and when projecting the generated projection image, projects a virtual operation unit for operating the projection image in an area excluding the blood vessel area included in the projection image.

12. A vascular visualization system comprising the vascular visualization apparatus according to claim 11 and a vascular access device that is inserted into a predetermined site of a subject.

13. A blood vessel visualization method comprising: detecting vascular information of blood vessels contained in a specified part of a subject based on an image of the specified part; generating a projection image of the blood vessels based on the detected vascular information; and, when projecting the generated projection image, projecting a virtual operation unit for operating the projection image in an area excluding the blood vessel area contained in the projection image.

Images