Safety device and program

Abstract

To prevent a patient from unconsciously removing a puncture needle or a tube during dialysis treatment, nasogastric tube treatment, etc.SOLUTION: A safety device 10 comprises a terminal device 30 and a camera 40. The camera 40 takes an image of a patient 20 and transmits the captured image data to the terminal device 30. A control device 31 in the terminal device 30, on the basis of sample images stored in a memory 37, identifies a target such as a puncture needle appearing in the captured image. The control device 31, when detecting that the targeted image is blurred, a part of the targeted image is missing, or a focal distance is changed, determines that the patient has put out his / her hand to the target, and alerts a doctor or nurse by image, voice, etc.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This invention relates to a safety device used in artificial dialysis, nasogastric tubes, and the like.

Background Art

[0002] In artificial dialysis, nasogastric tubes, etc., typically when a patient has dementia, the patient may accidentally remove a puncture needle or a (nutrition) tube. When the puncture needle is removed during artificial dialysis, blood leaks, which is dangerous for the patient. To avoid such risks, a moisture sensor as a safety device in artificial dialysis has been proposed (see, for example, Patent Document 1). This moisture sensor detects the blood flowing through the arm when the puncture needle comes off the patient's arm. When the moisture sensor determines that the puncture needle has come off the arm based on the detection of blood, it gives an alarm by making a buzzer sound.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the safety device (moisture sensor) disclosed in Patent Document 1, since it detects the blood flowing through the patient's arm, it cannot give an alarm before the puncture needle comes off the patient's arm. That is, with the safety device disclosed in Patent Document 1, it is impossible to prevent the patient from accidentally removing the puncture needle from the arm. The same applies to the tube in a nasogastric tube.

[0005] This invention has been made under such circumstances, and its object is to provide a safety device that can prevent a patient from removing a puncture needle or a tube in artificial dialysis, nasogastric tubes, etc.

Means for Solving the Problems

[0006] (1) The safety device according to the present invention includes a camera that captures an imaging region including a patient's arm or nose and outputs imaging image data, a control device to which the imaging image data is input, and a notification device communicable with the control device. The control device executes a target identification process of identifying a target image indicating a predetermined target reflected in the imaging image indicated by the imaging image data, a determination process of determining whether a foreign object has approached the target based on changes in two different target images identified at predetermined time intervals, and a notification process of outputting notification information to the notification device based on a determination in the determination process that a foreign object has approached the target.

[0007] The camera captures an imaging region including a patient's arm punctured by a puncture needle or a patient's nose into which a (nutritional) tube is inserted, and outputs imaging image data. The imaging image data is input to the control device. The control device identifies an image of a target (target image) reflected in the imaging image indicated by the input imaging image data. The target is, for example, a puncture needle, a tube, a gauze on which the puncture needle or tube is placed, a tape for stopping the puncture needle or tube, an arm, a nose, a marker, or the like. When a patient unconsciously extends a hand toward a puncture needle stabbed in the arm or a tube connected to the nose, the extended hand hides the target image or the focus is shifted and the target image blurs. The control device determines whether a foreign object (the patient's hand) has approached the target based on changes in two different target images identified at predetermined time intervals. That is, the control device determines whether the patient has unconsciously extended a hand toward the puncture needle or the tube. When the control device determines that the patient is unconsciously extending a hand toward the puncture needle or the tube and trying to remove the puncture needle or the tube, it outputs notification information to the notification device to give a notification. Therefore, the safety device according to the present invention can prevent a patient from accidentally removing a puncture needle or a tube from the arm or the nose. Note that the notification device is, for example, a speaker, a display, or a mobile communication terminal possessed by a doctor (nurse).

[0008] (2) The camera may have an autofocus function. The change between two different target images identified at a predetermined time interval is a change in the contour or pattern width of the target image due to the autofocus function. Based on the change in the width being equal to or greater than a first threshold value, the determination process determines that a foreign object has approached the target, and based on the change in the width being less than the first threshold value, determines that no foreign object has approached the target.

[0009] When the patient reaches out their hand towards the puncture needle or tube and the hand intrudes between the camera and the target, the camera focuses on the hand by means of the autofocus function. Then, the puncture needle or other target goes out of focus and the target image becomes blurred. That is, the width of the contour or pattern of the target image widens. When the change in the width of the contour or pattern of the target image is equal to or greater than a first threshold value, the control device determines that the patient is unconsciously reaching out their hand towards the puncture needle or tube and trying to remove the puncture needle or tube.

[0010] (3) The change between two different target images identified at a predetermined time interval may be a change in the coincidence rate of the two target images. In the target identification process, based on the coincidence rate being less than a second threshold value, the determination process determines that a foreign object has approached the target, and based on the coincidence rate being equal to or greater than the second threshold value, determines that no foreign object has approached the target.

[0011] When the patient unconsciously reaches out their hand towards the puncture needle in the arm or the tube in the nose and the hand intrudes between the target and the camera, part or all of the target image fails to appear. When the coincidence rate of two different target images identified at a predetermined time interval is less than a second threshold value, the control device determines that the patient has unconsciously reached out their hand towards the puncture needle or tube.

[0012] (4) The camera may have an autofocus function and output a focal length together with the captured image data. The change between two different target images identified at a predetermined time interval is the change in the two focal lengths corresponding to the target images. The determination process determines that a foreign object has approached the target based on the change in the focal length being equal to or greater than a third threshold value, and determines that no foreign object has approached the target based on the change in the focal length being less than the third threshold value.

[0013] When a patient unconsciously extends their hand towards a puncture needle on the arm or a tube in the nose, and the hand enters between the target, which is the puncture needle, tube, or gauze, and the camera, the autofocus function of the camera activates and focuses on the hand, causing the focal length to change. When the change in the focal length is equal to or greater than the third threshold value, the control device determines that the patient has unconsciously extended their hand towards the puncture needle or tube.

[0014] (5) The control device may include a memory that stores sample image data indicating at least one of an arm, a nose, a tape, a puncture needle, a tube, a gauze, or a marker. The target is at least one of a patient's arm, nose, tape, puncture needle, tube, gauze, or marker. The target identification process includes a first process of identifying an object reflected in the captured image, and a second process of identifying the target based on the coincidence rate between the identified object and a sample image indicated by the sample image data.

[0015] According to the above configuration, the target can be reliably identified using the sample image.

[0016] (6) The control device may further include a memory that stores first sample image data indicating an arm or a nose, and second sample image data indicating at least one of a tape, a puncture needle, a tube, a gauze, or a marker. The target is at least one of a tape, a puncture needle, a tube, a gauze, or a marker. The target identification process includes a first process of identifying an object reflected in the captured image, a third process of identifying a patient's arm or nose based on a coincidence rate between the identified object and a first sample image indicated by the first sample image data, and a second process of identifying the target based on a coincidence rate between the object located above the identified patient's arm or below the nose and a second sample image indicated by the second sample image data.

[0017] The control device identifies a patient's arm or nose reflected in the captured image. The control device identifies the target based on the object reflected above the identified arm or below the nose and the second sample image. Therefore, it is possible to prevent an object reflected in an area other than the arm or an area other than below the nose from being erroneously identified as the target.

[0018] (7) The control device may further include a touch panel. The target identification process includes a display process of acquiring captured image data input from the camera, identifying an object reflected in the captured image data, and displaying on the touch panel a captured image overlaid with an icon indicating the identified object, a designation reception process of receiving a designation of the icon through the touch panel, and a process of identifying as the target the object indicated by the designated icon.

[0019] The control device can surely identify a predetermined object (such as a patient's arm, nose, puncture needle, tube, tape, gauze, marker, etc.) as the target by receiving a designation from a doctor or a nurse.

[0020] (8) The safety device according to the present invention may further include a camera mounting member having a fixing portion for fixing the camera and a mounting portion to be mounted on a patient's arm or head.

[0021] Since the camera is attached to the patient's arm or head, even if the patient moves their arm or head, or even if the patient moves, the relative position between the camera and the target (such as a puncture needle, tube, gauze, etc.) does not change. Therefore, the safety device according to the present invention can surely image the target by the camera even if the patient moves their arm or head, or even if the patient moves.

[0022] (9) The safety device according to the present invention includes a marker attached to the patient's hand or arm, a camera that images the patient, a control device into which imaging image data output by the camera is input, and a notification device that can communicate with the control device. The control device executes a target position specifying process for specifying a target position that is the position of a predetermined target reflected in the imaging image indicated by the imaging image data, a marker position specifying process for specifying a marker position that is the position of the marker reflected in the imaging image, an approach determination process for determining whether or not the separation distance between the target position and the marker position is less than a fourth threshold value, and a notification process for outputting notification information to the notification device based on the determination that the separation distance is less than the fourth threshold value.

[0023] When the patient unconsciously extends their hand towards a puncture needle or tube, the distance (separation distance) between the target and the marker becomes shorter. When the separation distance becomes less than the fourth threshold value, notification is made. Therefore, the safety device according to the present invention can prevent the puncture needle or tube from coming off the patient's arm or nose.

[0024] (10) Two of the cameras may be arranged at a distance from each other. The control device executes the target position specifying process and the marker position specifying process for each of the two imaging images output by each camera, and in the approach determination process, calculates the separation distance for each of the two imaging image data, and executes the notification process based on the determination that both of the two separation distances are less than the fourth threshold value.

[0025] Two cameras arranged separately image the patient from different directions. The imaging image data output by each camera shows the imaging images of the patient from different directions. In each imaging image, the separation distance between the target position and the marker position is calculated respectively and compared with a fourth threshold value. When both of the two separation distances are less than the fourth threshold value, a notification process is executed. That is, when the marker (the patient's arm) approaches the target (such as a puncture needle) in both of the two imaging images, the notification process is executed. Therefore, it is possible to more accurately detect whether the patient has unconsciously extended their hand towards the target than in the case of using a single camera.

[0026] (11) The above camera may be a stereo camera having two adjacent cameras. The above target position and the above marker position are positions in a three-dimensional space. The above separation distance is the distance between the above target position and the above marker position in a three-dimensional space.

[0027] A stereo camera outputs two pieces of imaging image data. An object shown in one imaging image and an object shown in the other imaging image produce a parallax according to the distance from the camera to the object. Based on the position of the object (target and marker) in the imaging image and the parallax, the positions of the target and the marker in a three-dimensional space are calculated. The said position is shown by, for example, the direction and distance (three-dimensional polar coordinates) of the target or marker with respect to the camera. Thus, a stereo camera functions as a device for detecting the position of an object in a three-dimensional space within the imaging range. The control device calculates the separation distance between the target (such as a puncture needle) and the marker (the patient's arm), and compares the separation distance with a fourth threshold value. Therefore, it is possible to more accurately detect whether the patient has unconsciously extended their hand towards the target than in the case of using a normal camera.

[0028] (12) The program according to the present invention is installed in a terminal device having a communication interface and a computer. The program includes an imaging image data acquisition process for acquiring imaging image data input from a camera that images an imaging area including a patient's arm or nose at predetermined time intervals, a target identification process for identifying a target image indicating a predetermined target reflected in the imaging image indicated by the imaging image data, and a determination process for determining whether a foreign object has approached the target based on changes in two different target images identified at predetermined time intervals, and a notification process for outputting notification information based on the determination in the determination process that a foreign object has approached the target, and causes the computer to execute them.

[0029] The present invention can also be regarded as a program installed in a terminal device.

[0030] (13) The program according to the present invention is installed in a terminal device having a communication interface and a computer. The program according to the present invention includes an imaging image data acquisition process for acquiring imaging image data input from a camera that images a patient with a marker attached to the arm at predetermined time intervals, a target position identification process for identifying a target position that is the position of a predetermined target reflected in the imaging image indicated by the imaging image data, a marker position identification process for identifying a marker position that is the position of the marker, an approach determination process for determining whether the separation distance between the target position and the marker position is less than a fourth threshold value, and a notification process for outputting notification information based on the determination that the separation distance is less than the fourth threshold value, and causes the computer to execute them.

[0031] The present invention can also be regarded as a program installed in a terminal device.

[0032] (14) The safety device according to the present invention includes a camera that images a patient and outputs imaging image data, a control device to which the imaging image data is input, a notification device that can communicate with the control device, a first marker attached to one arm of the patient, and a second marker attached to the other arm of the patient. The control device executes a first marker position specifying process for specifying the position of the first marker reflected in the imaging image shown by the imaging image data, a second marker position specifying process for specifying the position of the second marker reflected in the imaging image, a relative distance calculating process for calculating the relative distance between the first marker and the second marker based on the positions of the first marker and the second marker, a proximity determination process for determining whether the relative distance is less than a fifth threshold value, and a notification process for outputting notification information to the notification device based on the determination in the proximity determination process that the relative distance is less than the fifth threshold value.

[0033] The camera images the patient and outputs imaging image data. The imaging image data is input to the control device. The control device specifies the positions of the first marker and the second marker reflected in the imaging image shown by the input imaging image data, and calculates the relative positions of the first marker and the second marker. When the patient unconsciously extends the other arm (hand) to a puncture needle stabbed in the arm, the relative distance becomes shorter. Based on the relative distance becoming less than the fifth threshold value, the control device determines that the patient is unconsciously trying to remove the puncture needle by extending the hand, and outputs notification information to the notification device to give a notification. Therefore, the safety device according to the present invention can prevent the patient from removing the puncture needle from the arm. Note that the notification device is, for example, a speaker, a display, or a mobile communication terminal possessed by a doctor (nurse).

[0034] (15) The safety device according to the present invention includes a camera that images a patient and outputs imaging image data, a control device to which the imaging image data is input, a notification device that can communicate with the control device, a first marker attached to the head and neck of the patient, and second markers attached to both arms of the patient, respectively. The control device executes a first marker position specifying process for specifying the position of the first marker reflected in the imaging image shown by the imaging image data, a second marker position specifying process for specifying the positions of the two second markers reflected in the imaging image, respectively, a relative distance calculation process for calculating a first relative distance between the first marker and one of the second markers and a second relative distance between the first marker and the other second marker based on the position of the first marker and the positions of the second markers, a proximity determination process for determining whether at least one of the first relative distance or the second relative distance is less than a fifth threshold value, and a notification process for outputting notification information to the notification device based on a determination in the proximity determination process that at least one of the first relative distance or the second relative distance is less than the fifth threshold value.

[0035] The camera images the patient and outputs imaging image data. The imaging image data is input to the control device. The control device specifies the positions of the first marker and the second markers reflected in the imaging image shown by the input imaging image data, and calculates a first relative position between the first marker and one of the second markers and a second relative distance between the first marker and the other second marker, respectively. When the patient unconsciously extends an arm (hand) to a tube inserted into the nose, the first relative distance or the second relative distance becomes shorter. Based on the fact that at least one of the first relative distance or the second relative distance is less than the fifth threshold value, the control device determines that the patient is unconsciously extending a hand to the tube and trying to remove the tube, and outputs notification information to the notification device to give a notification. Therefore, the safety device according to the present invention can prevent the patient from removing the tube from the nose. Note that the notification device is, for example, a speaker, a display, or a mobile communication terminal possessed by a doctor (nurse).

Advantages of the Invention

[0036] The safety device according to the present invention can prevent a patient from removing a puncture needle or a tube in artificial dialysis, a nasogastric tube, etc.

Brief Description of Drawings

[0037]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Mode for Carrying Out the Invention

[0038] Hereinafter, preferred embodiments of the present invention will be described with appropriate reference to the drawings. It should be noted that this embodiment is merely one aspect of the safety device according to the present invention, and it goes without saying that the embodiment may be changed without changing the gist of the present invention.

[0039] [First Embodiment]

[0040] FIG. 1 shows a state in which a safety device 10 according to the first embodiment of the present invention is being used for a patient 20 undergoing hemodialysis. FIG. 2 is an enlarged view of part II in FIG. 1, showing an enlarged view of the arm 21 of the patient 20. FIG. 3 is a functional block diagram of the safety device 10. FIG. 4 is a flowchart of a safety confirmation process executed by the safety device 10.

[0041] As shown in FIG. 1, the safety device 10 is applied to a patient 20 undergoing hemodialysis. This safety device 10 gives a warning notification by a warning screen or a warning voice when the patient tries to remove the puncture needle 62 (see FIG. 2) from the arm 21 unconsciously. Further, this safety device 10 can also be applied to a patient 20 undergoing nasogastric intubation. In this case, a warning notification is given when the patient tries to remove the (nutrition) tube from the nose unconsciously. Hereinafter, the description will be made on the premise that the safety device 10 is used for a patient 20 undergoing hemodialysis.

[0042] The safety device 10 includes a terminal device 30, a camera 40, and a mobile communication terminal 51 (see FIG. 3). The terminal device 30 and the camera 40 are installed near the bed 25 on which the patient 20 lies or a chair (not shown) on which the patient sits. In FIG. 1, the terminal device 30 and the camera 40 are placed on a stand 26. The stand 26 is typically a storage box, a desk, or a table placed beside the bed 25. The mobile communication terminal 51 is a smartphone carried by a doctor or a nurse for work, a tablet installed in the hospital's supervision room, a personal computer, or the like.

[0043] As shown in FIG. 3, the camera 40 is a so-called digital camera and has an optical system 41 and a plurality of imaging elements 42. The optical system 41 consists of a plurality of lenses. The optical system 41 guides the light incident from the outside to the imaging element 42. The imaging element 42 is a CCD, a CMOS, or the like. Each imaging element 42 outputs pixel data corresponding to the incident light respectively. The pixel data includes pixel position data indicating the pixel position and color data indicating color, brightness, or luminance, etc. The pixel position data may be determined by the arrangement order in a plurality of pixel data. The plurality of pixel data forms imaging image data. The imaging image data may be data indicating a color image or data indicating a monochrome image. Also, the imaging image data may be data indicating a still image or data (frames) indicating a moving image.

[0044] The camera 40 further includes an autofocus mechanism 46. The autofocus mechanism 46 is, for example, a motor that changes the position of the lens of the optical system 41 and changes the focal length of the optical system 41. That is, the camera 40 has an autofocus function.

[0045] Camera 40 further includes a communication interface 47. The communication interface 47 is an interface for performing wireless communication such as Wi-Fi (registered trademark) and Bluetooth (registered trademark) (so-called wireless LAN (registered trademark)), or wired communication using a communication cable such as a LAN (registered trademark) cable. Camera 40 transmits the captured image data generated by imaging to the terminal device 30 through the communication interface 47.

[0046] Camera 40 further includes a memory 44 in which a driver program 45 is stored, and a CPU 43 which is a central processing unit that executes the driver program 45. The driver program 45 executes generation processing of captured image data by the imaging element 42, transmission processing of the captured image data through the communication interface 47, and change processing of the focal length using the autofocus mechanism 46.

[0047] As the camera 40, for example, an existing (commercially available) digital camera is used.

[0048] The terminal device 30 is a terminal capable of communicating with other devices, such as a personal computer, a smartphone, or a tablet.

[0049] The terminal device 30 includes a control device 31, a communication interface 32, a speaker 33, a display device 34, and an input interface 35.

[0050] The communication interface 32 is an interface for performing wireless communication such as Wi-Fi (registered trademark) and Bluetooth (registered trademark) (so-called wireless LAN (registered trademark)), wired communication using a communication cable such as a LAN (registered trademark) cable, mobile communication with a base station of a mobile communication network, etc. The communication interface 32 may be a single communication interface or may include a plurality of communication interfaces.

[0051] The speaker 33 outputs sound corresponding to the input voice data.

[0052] The display device 34 is a display, a touch panel, a monitor, etc., and displays a screen according to the input screen data. When the display device 34 is a touch panel, the display device 34 includes a touch sensor that is the input interface 35, and outputs input data input through the touch sensor. The speaker 33, or the display device 34, or the speaker 33 and the display device 34 correspond to the "notification device" described in the claims.

[0053] The input interface 35 is a touch sensor of a touch panel, a microphone for voice input, a keyboard, a mouse, etc. The input interface 35 receives input from users such as doctors and nurses.

[0054] Although not shown in FIG. 3, the communication interface 32, the speaker 33, the display device 34, and the input interface 35 are communicably connected to the control device 31 by signal lines, patterns on a circuit board, etc. The control device 31 transmits commands such as imaging instructions to the camera 40 through the communication interface 32 and receives imaging image data transmitted by the camera 40. Further, the control device 31 inputs voice data to the speaker 33 to output voice from the speaker 33. Further, the control device 31 inputs screen data to the display device 34 to display a screen on the display device 34. Further, the control device 31 receives instructions from the user through the input interface 35.

[0055] The control device 31 is realized by electronic components such as a CPU 36, an IC, a capacitor, a resistor, and a coil mounted on a circuit board.

[0056] The control device 31 has a CPU 36 which is a central processing unit and a memory 37.

[0057] The CPU 36 executes the OS 38 and the control program 39 stored in the memory 37. Specifically, the CPU 36 executes the instructions described in the OS 38 and the control program 39. The CPU 36 pseudo-parallelly executes the OS 38 and the control program 39, for example, by multitasking. The CPU 36 corresponds to the "computer" described in the claims.

[0058] The memory 37 stores the OS 38 which is an operating system and the control program 39 which is an application program installed in the terminal device 30. Also, the memory 37 stores data necessary for the execution of the control program 39, such as sample image data, threshold values, and warning notification data (not shown). The sample image data, threshold values, and warning notification data are stored in the memory 37 at the time of installation of the control program 39. Alternatively, the sample image data, threshold values, and warning notification data may be part of the control program 39. The control program 39 may be a single program or a program consisting of a plurality of module programs. The plurality of module programs are, for example, a UI module that receives user input, a communication module that performs data input / output through the OS 38, an image processing module that performs image processing, and a main body control module. The control program 39 is created, for example, by diverting existing communication modules and image processing modules and further using an existing SDK (software development kit). The control program 39 corresponds to the "program" described in the claims.

[0059] The sample image data is, for example, image data obtained by imaging an arm or a nose, image data obtained by imaging a puncture needle or a tube, or image data obtained by imaging a tape or a gauze or a marker. The sample image data is used by the control program 39 to identify the arm 21, nose, puncture needle 62, tube 63, tape 64, gauze 61, or marker (not shown) of the patient 20 shown in FIG. 2.

[0060] The mobile communication terminal 51 shown in FIG. 3 is a smartphone (registered trademark) held by a doctor or a nurse, or a tablet or a personal computer installed in a monitoring room. The mobile communication terminal 51 includes a communication interface, a display (or a touch panel), and a speaker. The mobile communication terminal 51 receives the warning notification data transmitted by the terminal device 30, displays the warning screen indicated by the warning notification data on the display, or outputs the warning sound indicated by the warning notification data from the speaker. The mobile communication terminal 51 corresponds to the "notification device" described in the claims. Note that when the warning screen is displayed on the display device 34 of the terminal device 30, or when the warning sound is output from the speaker 33 of the terminal device 30, the mobile communication terminal 51 may not be used.

[0061] FIGS. 1 and 3 show a marker 50 attached to the arm 22 of the patient 20. The marker 50 will be described in the second embodiment.

[0062] FIG. 4 is a flowchart of the safety confirmation process that the control program 39 causes the CPU 36 to execute.

[0063] Hereinafter, with reference to FIGS. 1 to 4, the safety confirmation process will be described. Note that the safety confirmation process that the control program 39 causes the CPU 36 to execute is also a process executed by the CPU 36, the control device 31, or the terminal device 30.

[0064] As shown in FIGS. 1 and 2, after a doctor or a nurse places a gauze 61 or the like on the arm 21 of the patient 20, the puncture needle 62 is inserted into the arm 21, and the tube 63 extending from the puncture needle 62 is fixed to the arm 21 using a tape 64. The user, who is a doctor or a nurse, adjusts the orientation of the camera 40 so that the lens of the camera 40 faces the arm 21 of the patient 20, and then operates the terminal device 30 to start the control program 39. The activated control program 39 inputs input screen data indicating a predetermined input screen to the display device 34. The user inputs an execution instruction for the safety confirmation process to the terminal device 30 through the input interface 35 according to the input screen displayed on the display device 34. Also, the doctor or the nurse operates the hemodialysis device 65 to start hemodialysis.

[0065] Based on receiving the execution instruction for the safety confirmation process, the control program 39 executes the safety confirmation process shown in FIG. 4. First, the control program 39 transmits an imaging instruction to the camera 40 through the communication interface 32 (S11). The imaging instruction is, for example, a command.

[0066] The driver program 45 of the camera 40 drives the autofocus mechanism 46 to adjust the focal length, and then performs imaging at a predetermined time interval. Then, the driver program 45 transmits the captured image data at a predetermined time interval to the terminal device 30 through the communication interface 47. The captured image data is data generated by imaging an imaging area including the arm 21 of the patient 20. That is, the captured image data is data indicating a captured image that at least shows the arm 21 of the patient 20. The predetermined time interval is, for example, from several hundred milliseconds to several seconds. Note that the driver program 45 sequentially adjusts the focal length at a predetermined time interval.

[0067] The control program 39 of the terminal device 30 receives and acquires the captured image data transmitted by the camera 40 through the communication interface 32 (S12). The process of step S12 corresponds to the "captured image data acquisition process" described in the claims.

[0068] The control program 39 executes image processing on the acquired captured image data (S13). The image processing is, for example, binarization processing. By executing the binarization processing, the amount of data processed by the CPU 36 is reduced.

[0069] The control program 39 acquires boundary line data based on the processed image data, which is the captured image data on which image processing has been performed (S14). Specifically, the control program 39 identifies, as boundaries, the pixels between which the difference in color, brightness, or luminance between pixels changes by a threshold value or more, and acquires the pixel position information of the pixels indicating the boundaries as the boundary line data. The region surrounded by the boundary line indicated by the boundary line data indicates, for example, the arm 21 of the patient 20, the gauze 61 placed on the arm 21, the tape 64 attached to the arm 21, the puncture needle 62 stuck into the arm 21, and the tube 63 extending from the puncture needle 62. That is, the control program 39 detects a plurality of objects (detected objects) such as the arm 21 as the boundary line data. The boundary line data indicates the shape, size, and position of the detected object. Note that an existing module having a function of acquiring boundary line data, that is, detecting an object reflected in the captured image, may be used for the control program 39. The process of step S14 corresponds to the "first process" described in the claims.

[0070] Also, the control program 39 reads and acquires the first sample image data stored in the memory 37 from the memory 37 (S15). The first sample image data is boundary line data indicating the outline of the arm extracted from the captured image data of a human arm. That is, the first sample image data is data indicating the shape of a human arm. Note that the first sample image data may be a plurality of data generated by capturing the arm at various angles.

[0071] The control program 39 determines whether there is a detected object among the plurality of detected objects detected in step S14 whose shape approximates the shape of the arm shown in the first sample image data (S16). That is, in step S16, it is determined whether the arm 21 of the patient 20 has been detected. Whether the shape of the detected object approximates the shape of the arm shown in the first sample image data is determined, for example, by whether the coincidence rate of the shapes is equal to or higher than a predetermined threshold value. For example, an existing authentication module is used in the control program 39.

[0072] If the patient 20 unconsciously extends the arm 22 that the puncture needle 62 is not inserted into toward the puncture needle 62 and a part or all of the arm 21 is covered by the arm 22, the control program 39 determines in step S16 that the arm 21 of the patient 20 cannot be detected (S16: No). If the patient 20 does not extend the arm 22 toward the arm 21, the control program 39 determines in step S16 that the arm 21 of the patient 20 has been detected (S16: Yes). The process of step S16 corresponds to the "third process" described in the claims.

[0073] When the control program 39 determines that the arm 21 of the patient 20 has been detected (S16: Yes), it acquires boundary line data approximating the first sample image data as first specific data and stores it in the memory 37 (S17). On the other hand, when the control program 39 determines that the arm 21 of the patient 20 cannot be detected (S16: No), it reads and acquires the first specific data stored in the memory 37 in the past from the memory 37 (S18). That is, when the arm 21 is covered by the arm 22 of the patient 20, the past first specific data is used. Note that the first specific data read from the memory 37 in step S18 may be the newest data, the oldest data, or data between the two among the plurality of first specific data stored in the memory 37. That is, the first specific data used may be the immediately preceding data, the data immediately after the start of hemodialysis, or data between the two.

[0074] The control program 39 specifies and acquires, as second specific data, the boundary line data indicating the detected object on the arm 21 of the patient 20 among the plurality of detected objects detected in step S14 (S19). Specifically, the control program 39 specifies a plurality of detected objects (second detected objects) within the range of the boundary line indicated by the first specific data acquired in step S17 or step S18 among the plurality of detected objects detected in step S14. The second detected objects are the gauze 61 placed on the arm 21 of the patient 20, the tape 64 attached to the arm 21, the puncture needle 62 stuck into the arm 21, the tube 63 extending from the puncture needle 62, and the like. Since the second detected object is specified after the arm 21 of the patient 20 is specified, it is possible to prevent an object other than the arm 21 from being erroneously specified as the second detected object. That is, misrecognition in the control program 39 is prevented.

[0075] The control program 39 determines whether or not the second specific data could be acquired in step S19 (S20). When the patient 20 unconsciously extends the arm 22 to the arm 21 and the target such as the puncture needle 62 is covered by the arm 22, the control program 39 determines that the second specific data could not be acquired (S20: No). When the patient 20 does not extend the arm 22 to the arm 21, the control program 39 determines that the second specific data could be acquired (S20: Yes).

[0076] When the control program 39 determines that the second specific data could be acquired (S20: Yes), it reads and acquires the second sample image data stored in the memory 37 from the memory 37 (S21). The second sample image data is boundary line data indicating the contour of at least one of the gauze 61, the puncture needle 62, the tube 63, and the tape 64. The gauze 61, the puncture needle 62, the tube 63, and the tape 64 correspond to the "target" described in the claims. The target may be any one of the gauze 61, the puncture needle 62, the tube 63, and the tape 64, or may be a plurality of them.

[0077] Based on the second specific data acquired in step S19 and the second sample image data read from the memory 37 in step S21, the control program 39 determines whether the target has been detected (S22). That is, in step S22, it is determined whether the patient 20 has unconsciously extended the arm 22 to the puncture needle 62 and the target has been covered by the arm 22. Specifically, the control program 39 calculates the coincidence rate between the shape indicated by the second specific data and the shape indicated by the second sample image data. If the calculated coincidence rate is equal to or higher than the threshold value stored in the memory 37, it is determined that the second specific data indicates the data of the target, and it is determined that the target has been detected. The process of step S22 corresponds to the "second process" described in the claims.

[0078] When the control program 39 determines that the target has been detected (S22: Yes), that is, when it is determined that the target such as the gauze 61 or the puncture needle 62 is not covered by the arm 22 of the patient 20, the processes from step S23 to step S26 for determining whether the target is blurred are executed. More specifically, when the patient 20 unconsciously extends the arm 22 to the puncture needle 62, although the target is not covered by the arm 22, the camera 40 changes the focus from the arm 21 to the arm 22 by the autofocus function. Then, the image of the target on the arm 21 becomes blurred. In steps S23 to S26, it is determined whether the focal length has been changed and the target image has become blurred.

[0079] First, the control program 39 stores the boundary line data indicating the target detected in step S22 as target image data in the memory 37 (S23). The control program 39 determines the boundary line width based on the target image data and stores it in the memory 37 in association with the target image data (S24). For example, the control program 39 determines the separation distance between two pixels whose color, luminance, or brightness changes by a threshold value or more as the boundary line width. The process of step S23 corresponds to the "target identification process" described in the claims. The image indicated by the target image data corresponds to the "target image" described in the claims. The width of the boundary line corresponds to the "width of the contour of the target image" described in the claims.

[0080] Next, the control program 39 reads and acquires the boundary line width stored in the memory 37 in the past from the memory 37 using it as an initial value (S25). The boundary line width read from the memory 37 may be the latest data, the oldest data, or data in between among the plurality of boundary line widths stored in the memory 37. That is, the boundary line width used may be the immediately previous data, the data immediately after the start of hemodialysis, or data in between.

[0081] The control program 39 determines whether the difference (change) between the boundary line width determined in step S24 and the boundary line width (initial value) read from the memory 37 in step S25 is equal to or greater than the first threshold value stored in the memory 37 (S26). That is, in step S26, it is determined whether the target image has become blurred. When the control program 39 determines that the difference is equal to or greater than the first threshold value (S26: Yes), that is, when it determines that the target image has become blurred, it outputs the warning notification data stored in the memory 37 to at least one of the display device 34, the speaker 33, and the communication interface 32 (S27). The warning notification data is screen data indicating a warning screen and voice data indicating a warning sound. The display device 34 displays the warning screen indicated by the input warning notification data. The speaker 33 outputs the warning sound indicated by the input warning notification data. The communication interface 32 transmits the warning notification data to the mobile communication terminal 51 held by a doctor or a nurse. The change in the boundary line width corresponds to "the change in the width of the contour or pattern of the target image by the autofocus function" described in the claims. The process of step S26 corresponds to the "determination process" described in the claims. The arm 22 of the patient 20 corresponds to the "foreign object" described in the claims. The warning notification data corresponds to the "notification information" described in the claims. The process of step S27 corresponds to the "notification process" described in the claims. Note that an application program for receiving the warning notification data transmitted by the terminal device 30, displaying a warning screen, and outputting a warning sound is pre-installed in the mobile communication terminal 51.

[0082] If the control program 39 determines in step S20 that the second specific data cannot be acquired (S20: No), it outputs the warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27). That is, when the entire arm 21 is covered by the arm 22 of the patient 20, a warning screen is displayed on the display device 34, or a warning sound is output from the speaker 33, or a warning screen is displayed and a warning sound is output on the mobile communication terminal 51 held by a doctor or a nurse.

[0083] Also, when the control program 39 determines in step S22 that the target cannot be detected (S22: No), it outputs the warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27). That is, when the target such as the puncture needle 62 on the arm 21 is covered by the arm 22 of the patient 20, a warning screen is displayed on the display device 34, a warning sound is output from the speaker 33, or a warning screen is displayed and a warning sound is output on the mobile communication terminal 51 held by a doctor or a nurse.

[0084] Thus, when the entire arm 21 of the patient 20 is covered by the other arm 22 of the patient 20 (S20: No), when the entire arm 21 is not covered by the arm 22 of the patient 20 but the target such as the puncture needle 62 is covered (S22: No), and when the target is not covered by the arm 22 of the patient 20 but the arm 22 of the patient 20 approaches the target such as the puncture needle 62 and the target image becomes blurred (S26: Yes), a warning screen is displayed on the display device 34, or a warning sound is output from the speaker 33, or a warning screen is displayed on the mobile communication terminal 51 while a warning sound is output (S27).

[0085] In addition, when the patient 20 unconsciously brings the arm 22 close to the puncture needle 62, it is detected in step S26 that the target image has become blurred before the entire or part of the arm 21 is covered by the arm 22 (S26: Yes). Therefore, the determination processes in steps S20 and S22 are auxiliary processes for improving safety.

[0086] After executing the process of step S27, the control program 39 determines whether the user has input a reset instruction to the terminal device 30 using the input interface 35 (S28). For example, a doctor or a nurse (user) who has visually recognized the warning screen or heard the warning sound inputs a reset instruction to the terminal device 30 using the input interface 35 after prompting the patient 20.

[0087] When the control program 39 determines that no reset instruction has been input (S28: No), it repeatedly executes the output of warning notification data (S27). Alternatively, the control program 39 maintains the display of a warning screen on the display device 34 and the output of warning sounds by the speaker 33. When the control program 39 determines that a reset instruction has been input (S28: Yes), it executes the processes after step S12 again.

[0088] In step S26, when the control program 39 determines that the difference is not equal to or greater than the first threshold value (S26: No), that is, when it determines that the target image is not blurred, it determines whether the user has input an end instruction to the terminal device 30 using the input interface 35 (S29). For example, after the hemodialysis is completed, a doctor or a nurse (user) inputs an end instruction to the terminal device 30 using the input interface 35.

[0089] When the control program 39 determines that no end instruction has been input (S29: No), it executes the processes after step S12 again. That is, until the user inputs an end instruction to the terminal device 30, it is repeatedly determined whether the patient 20 has unconsciously extended the arm 22 to the arm 21. The repetition period is, for example, in the interval from several hundred milliseconds to several seconds. The said period corresponds to the "predetermined time interval" described in the claims.

[0090] When the control program 39 determines that an end instruction has been input (S29: Yes), it ends the safety confirmation process (end).

[0091] [Operational Effects of the First Embodiment]

[0092] The safety device 10 according to the first embodiment detects whether the patient 20 has extended the arm 22 to the arm 21 by whether the target image is blurred, and gives a warning to a doctor or a nurse. Therefore, the safety device 10 according to the first embodiment can prevent the patient 20 from inadvertently removing the puncture needle 62 or the tube 63 in hemodialysis, nasogastric tube, etc.

[0093] After identifying the arm 21 of the patient 20 in the captured image (S17), the safety device 10 according to the first embodiment identifies an object on the arm 21 as a target (S23). Accordingly, it is possible to prevent an object located at a position other than on the arm 21 from being erroneously targeted. That is, it is possible to prevent an object other than the puncture needle 62 or the like from being misrecognized as a target.

[0094] [Modification Example 1]

[0095] In the first embodiment, an example in which it is determined whether the patient 20 is trying to remove the puncture needle 62 inadvertently based on whether the image of the target (target image), such as the puncture needle 62, is blurred has been described. In this modification example, an example in which it is determined whether the patient 20 is trying to remove the puncture needle 62 inadvertently based on whether part or all of the target image is missing will be described.

[0096] The configurations and processes other than the configurations and processes described below are the same as the configurations and processes described in the first embodiment. The same reference numerals and step numbers as those in the first embodiment are used for the same configurations and processes as those in the first embodiment. Note that in this modification example, since it is determined whether the target image is missing, the camera 40 may or may not have the autofocus mechanism 46.

[0097] FIG. 5 is a flowchart of the safety confirmation process that the control program 39 causes the CPU 36 to execute. Instead of the safety confirmation process shown in FIG. 4, the control program 39 executes the safety confirmation process shown in FIG. 5.

[0098] The control program 39 executes the processes from step S11 to step S23. That is, the control program 39 identifies the arm 21 of the patient 20 from the captured image captured by the camera 40 (S17), and identifies the target on the arm 21 (S23). Further, when the control program 39 cannot identify the arm 21 of the patient 20 (S20: No) or cannot identify the target (S22: No), it outputs warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27).

[0099] After the control program 39 detects the target (S22: Yes) and stores the target image data in the memory 37 (S23), it reads out the past target image data stored in the memory 37 from the memory 37 (S31). The target image data read out from the memory 37 may be the latest data, the oldest data, or data between the two among the plurality of target image data stored in the memory 37. That is, the target image data to be used may be the immediately preceding data, the data immediately after the start of hemodialysis, or data between the two.

[0100] The control program 39 calculates the coincidence rate between the target image indicated by the current target image data acquired in step S23 and the target image indicated by the target image data read out from the memory 37 in step S31 (S32). The control program 39 determines whether the calculated coincidence rate is less than the second threshold value stored in the memory 37 (S33). For example, when the patient 20 extends the arm 22 to the puncture needle 62 and a part of the target (such as the puncture needle 62) is covered by the arm 22, the above coincidence rate decreases. The change (decrease) in the coincidence rate due to the missing target image corresponds to the "change in the target image" described in the claims. The process of step S33 corresponds to the "determination process" described in the claims.

[0101] In step S22, the identification of the target using the second sample image data is performed based on the coincidence rate between the boundary line data and the second sample image data and a threshold value. The threshold value is set low so that even an object with some parts missing can be recognized as a target. Then, the boundary line data (target image data) identified as the target is compared with past target image data instead of the second sample image data.

[0102] When the control program 39 determines that the coincidence rate is less than the second threshold value (S33: Yes), it outputs warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27).

[0103] After executing the process of step S27, the control program 39 executes the process of step S28 in the same manner as in the first embodiment. Also, when the control program 39 determines in step S33 that the coincidence rate is not less than the second threshold value (S33: No), it executes the process of step S29 in the same manner as in the first embodiment.

[0104] [Operation and Effect of Modification 1]

[0105] In this modification, based on the fact that the image of the target reflected in the captured image is missing, it is determined whether the patient 20 is trying to remove the puncture needle 62. Therefore, even when the camera 40 does not have an autofocus function, a warning notification can be given to the doctor or nurse before the patient 20 removes the puncture needle 62.

[0106] [Modification 2]

[0107] In the first embodiment, an example was described in which it is determined whether the patient 20 is inadvertently trying to remove the puncture needle 62 based on whether the image (target image) of the target such as the puncture needle 62 is blurred. In this modification, an example is described in which it is determined whether the patient 20 is inadvertently trying to remove the puncture needle 62 based on the focal length output by the camera 40.

[0108] Configurations and processes other than those described below are the same as those described in the first embodiment. The same reference numerals and step numbers as those in the first embodiment are used for the same configurations and processes in the first embodiment.

[0109] FIG. 6 is a flowchart of the safety confirmation process that the control program 39 causes the CPU 36 to execute. Instead of the safety confirmation process shown in FIG. 4, the control program 39 executes the safety confirmation process shown in FIG. 6.

[0110] The control program 39 transmits an imaging instruction to the camera 40 (S11) and receives the captured image data and the focal length transmitted by the camera 40. Thereafter, the control program 39 executes the processes from step S13 to step S22. That is, the control program 39 identifies the arm 21 of the patient 20 from the captured image captured by the camera 40 (S17) and identifies the target on the arm 21 (S22: Yes). Also, when the control program 39 cannot identify the arm 21 of the patient 20 (S20: No) or cannot identify the target (S22: No), it outputs warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27).

[0111] When the control program 39 determines that the target has been detected in step S22 (S22: Yes), it stores the boundary line data as target image data together with the focal length received in step S41 in the memory 37 (S42). Also, the control program 39 reads out the past focal length stored in the memory 37 from the memory 37 (S43). The focal length read out from the memory 37 may be the latest focal length, the oldest focal length, or a focal length between the two among the plurality of focal lengths stored in the memory 37. That is, the read focal length may be the immediately previous focal length, the focal length immediately after the start of hemodialysis, or a focal length between the two.

[0112] The control program 39 calculates, as the amount of change in the focal length, the absolute value of the difference between the focal length acquired in step S41 and the past focal length read from the memory 37 in step S43 (S44). The control program 39 determines whether or not the calculated amount of change in the focal length is equal to or greater than a third threshold value stored in the memory 37 (S45). That is, in step S45, it is determined whether or not the focus of the camera 40 has moved from a target such as the puncture needle 62 on the arm 21 of the patient 20 to another object. When the control program 39 determines that the amount of change in the focal length is equal to or greater than the third threshold value (S45: Yes), it outputs warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27). The change in the focal length when the focus of the camera 40 changes from the arm 21 to the arm 22 corresponds to the "change in two different target images specified at a predetermined time interval" described in the claims. The process of step S45 corresponds to the "determination process" described in the claims.

[0113] After executing the process of step S27, the control program 39 executes the process of step S28 in the same manner as in the first embodiment. Further, when the control program 39 determines in step S45 that the amount of change in the focal length is not equal to or greater than the third threshold value (S45: No), it executes the process of step S29 in the same manner as in the first embodiment.

[0114] [Operation and Effect of Modification 2]

[0115] In this modification, since it is determined whether or not the patient 20 is trying to remove the puncture needle 62 unconsciously based on the amount of change in the focal length, the processing load on the CPU 36 is reduced as compared with the case of determining whether or not the patient 20 is trying to remove the puncture needle 62 unconsciously based on whether or not the target image has become blurred.

[0116] Note that there may be a plurality of past focal lengths read in step S43, and the "change amount of the focal length" calculated in step S44 may be the change amount per unit time, that is, the slope of the time change of the focal length. In that case, in step S45, it is determined whether the focal length has changed rapidly during a short time (unit time).

[0117] [Modification Example 3]

[0118] In the above-described first embodiment, an example in which the target is specified after identifying the arm 21 of the patient 20 has been described. In this modification example, an example in which the target is directly specified will be described.

[0119] The configurations and processes other than the configurations and processes described below are the same as the configurations and processes described in the first embodiment. The same reference numerals and step numbers as those in the first embodiment are used for the same configurations and processes as those in the first embodiment.

[0120] FIG. 7 is a flowchart of the safety confirmation process that the control program 39 causes the CPU 36 to execute. Instead of the safety confirmation process shown in FIG. 4, the control program 39 executes the safety confirmation process shown in FIG. 7.

[0121] After executing the processes from step S11 to step S14, the control program 39 reads out the second sample image data indicating the target from the memory 37 (S21). The control program 39 determines whether the target has been detected based on the boundary line data indicating the detected object specified in step S14 and the target indicated by the second sample image data acquired in step S21 (S22). Specifically, the control program 39 determines whether the coincidence rate between the boundary line data specified in step S14 and the second sample image data is equal to or greater than the threshold value stored in the memory 37.

[0122] When the control program 39 determines that the target cannot be detected (S22: No), it determines whether the detected object specified in step S14 approximates a part of the target indicated by the second sample image data (S51). When the control program 39 determines that the detected object does not approximate a part of the target indicated by the second sample image data (S51: No), it outputs warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27). When the control program 39 determines that the detected object approximates a part of the target indicated by the second sample image data (S51: Yes), it stores the boundary line data indicating the detected object in the memory 37 as target image data (S52). That is, in step S52, the target image data indicating the target image with a part missing is stored in the memory 37.

[0123] When the control program 39 determines that the target has been detected in step S22 (S22: Yes), it stores the boundary line data indicating the detected object in the memory 37 as target image data (S23). After executing the process of step S23 or the process of step S52, the control program 39 executes the processes from step S24 to step S29 in the same manner as in the first embodiment. That is, based on whether the target image indicated by the acquired target image data (S52, S23) is blurred, it is determined whether the patient 20 is trying to extend the arm 22 to the arm 21 and remove the puncture needle 62 from the arm 21, and a warning notification is given to the doctor or nurse.

[0124] [Operation and Effect of Modification 3]

[0125] In this modification, since the target such as the puncture needle 62 is directly detected without detecting the arm 21 of the patient 20, the processing load on the CPU 36 is reduced compared to the case of the first embodiment.

[0126] [Modification 4]

[0127] In the above-described first embodiment, an example in which the control program 39 identifies a target reflected in the captured image using the second sample image data has been described. In this modification example, an example in which a doctor or a nurse designates a target on the captured image will be described.

[0128] Configurations and processes other than those described below are the same as the configurations and processes described in the first embodiment. The same reference numerals and step numbers as those in the first embodiment are used for the same configurations and processes as those in the first embodiment.

[0129] In this modification example, the display device 34 is a touch panel having a touch sensor as the input interface 35.

[0130] FIG. 8 is a flowchart of the safety confirmation process that the control program 39 causes the CPU 36 to execute. Instead of the safety confirmation process shown in FIG. 4, the control program 39 executes the safety confirmation process shown in FIG. 8.

[0131] The control program 39 executes the processes from step S11 to step S14. That is, the control program 39 acquires captured image data from the camera 40 (S11, S12), and acquires boundary line data indicating an object (detected object) reflected in the captured image (S13, S14). The control program 39 outputs icon data indicating an icon superimposed on the detected object or arranged in the vicinity of the detected object and the captured image data to the display device 34 (S61). The display device 34 displays a captured image having a plurality of icons each indicating a detected object. The icon is, for example, a frame such as a square or a circle, or an arrow. The process of step S61 corresponds to the "display process" described in the claims.

[0132] A user who is a doctor or a nurse touches an icon indicating a target such as the puncture needle 62 in the captured image displayed on the display device 34 which is a touch panel. That is, the user designates the target on the touch panel. Note that the number of icons designated by the user may be one or a plurality.

[0133] The control program 39 receives the user's icon designation (S62). The process of step S62 corresponds to the "designation reception process" described in the claims.

[0134] The control program 39 stores, in the memory 37 as target image data, the boundary line data indicating the detected object corresponding to the icon designated by the user (S63). The process of step S63 corresponds to the "specifying as target process" described in the claims.

[0135] Thereafter, the control program 39 executes the processes from step S24 to step S29 in the same manner as in the first embodiment.

[0136] [Operation and effect of Modification 4]

[0137] In this modification, since a doctor or a nurse designates the target, the processing load on the CPU 36 is reduced compared to the case where the control program 39 specifies the target.

[0138] Note that, in the same manner as in the first embodiment, after the control program 39 specifies the arm 21 of the patient 20 shown in the captured image, an icon indicating the object shown on the arm 21 may be superimposed on the captured image. In that case, it is possible to prevent the icon from indicating an object that is not on the arm 21.

[0139] [Modification 5]

[0140] In the above-described first embodiment, as shown in FIG. 1, an example in which the camera 40 is placed on a stand 26 placed beside the bed 25 has been described. In this modification, an example in which the camera 40 is attached to the patient 20 will be described.

[0141] The configurations and processes other than the configurations and processes described below are the same as the configurations and processes described in the first embodiment. The same reference numerals and step numbers as those in the first embodiment are used for the same configurations and processes as in the first embodiment.

[0142] FIG. 9 is a perspective view of the camera mounting member 70.

[0143] The safety device 10 includes a camera mounting member 70 in addition to the terminal device 30 and the camera 40. The camera mounting member 70 is a member for attaching the camera 40 to the patient 20.

[0144] The camera mounting member 70 includes a fixing portion 71 for fixing the camera 40 and a mounting portion 72 to be worn on the arm 21 of the patient 20.

[0145] The fixing portion 71 is rod-shaped. The camera 40 is attached to one end (upper end) of the fixing portion 71. The camera 40 is supported by the fixing portion 71 so that the orientation of the lens can be changed with respect to the fixing portion 71. Specifically, a spherical body is provided at one end of the fixing portion 71, and a spherical concave portion into which the spherical body fits is provided at the bottom of the housing of the camera 40. However, the camera 40 may be fixed to the camera mounting member 70.

[0146] The mounting portion 72 is annular (ring-shaped) with a gap. The mounting portion 72 is, for example, a synthetic resin molded product and has elasticity. The mounting portion 72 is worn on the arm 21 of the patient 20 or removed from the arm 21 of the patient 20 by elastic deformation. The mounting portion 72 is connected to the other end (lower end) of the rod-shaped fixing portion 71.

[0147] After a doctor or a nurse wears the camera mounting member 70 to which the camera 40 is fixed on the upper arm of the arm 21 of the patient 20, the lens of the camera 40 is directed toward the puncture needle 62. That is, the camera 40 is attached to the patient 20 with the arm 21 of the patient 20 including the puncture needle 62 as an imaging range.

[0148] When the camera 40 is attached to the upper arm of the patient 20, the distance from the camera 40 to the puncture needle 62 always remains constant even if the patient 20 moves the arm 21. Also, even if the patient 20 moves or the patient 20 moves the arm 21, the puncture needle 62 etc. do not go out of the imaging range of the camera 40. That is, the camera 40 can always image the arm 21 of the patient 20 including the puncture needle 62 etc. regardless of the movement of the patient 20.

[0149] [Operation and effect of Modification 5]

[0150] In this modification, regardless of the movement of the patient 20, an imaging image of the puncture needle 62 etc. can always be obtained.

[0151] In the case of a nasogastric tube instead of hemodialysis, the mounting part 72 is mounted on the head of the patient 20 instead of the arm 21 of the patient 20.

[0152] [Second Embodiment]

[0153] In the second embodiment of the present invention, a marker 50 is attached to the arm 22 of the patient 20. The arm 22 is the arm on the opposite side of the arm 21 where the puncture needle 62 is inserted. In this embodiment, an example in which a warning notification is given based on the marker 50 approaching the target on the arm 21 is explained.

[0154] The configurations and processes other than the configurations and processes described below are the same as the configurations and processes described in the first embodiment. The same reference numerals and step numbers as those in the first embodiment are used for the same configurations and processes as those in the first embodiment.

[0155] In addition to the terminal device 30 and the camera 40, the safety device 10 includes the marker 50 shown in FIGS. 1 and 3. The marker 50 is a paper or plastic seal having an adhesive surface on one side. However, the marker 50 may not have an adhesive surface. In that case, the marker 50 is attached to the back of the hand, wrist, etc. on the arm 22 of the patient 20 with tape. Also, two markers 50 may be attached to both sides of the arm 22, that is, the back side and the palm side. Thereby, the camera 40 can surely image the marker 50.

[0156] The marker 50 is patterned to facilitate recognition in the control program 39. The pattern is a grid pattern or the like. However, if the control program 39 can recognize (identify) the marker 50 by the outer shape of the marker 50, the marker 50 may be plain. Also, instead of the pattern, a code such as a barcode or a QR code (registered trademark) may be attached to the marker 50. In that case, the control program 39 has a code reading program (off-the-shelf program) as a module program.

[0157] The camera 40 is arranged so that not only the arm 21 of the patient 20 but also the arm 22 of the patient 20 is within the imaging range. Specifically, the entire patient 20 is reflected in the captured image.

[0158] FIG. 10 is a flowchart of the safety confirmation process executed by the control program 39 in the present embodiment. The control program 39 (CPU 36, control device 31, terminal device 30) executes the safety confirmation process shown in FIG. 10 instead of the safety confirmation process shown in FIG. 4.

[0159] The control program 39 executes the processes from step S11 to step S23. That is, the control program 39 stores target image data indicating a target such as the puncture needle 62 on the arm 21 of the patient 20 in the memory 37 based on the captured image data output by the camera 40 (S23). The process of step S23 corresponds to the "target position specifying process" described in the claims.

[0160] Next, the control program 39 reads the third sample image data from the memory 37 (S71). The third sample image data is the image data indicating the marker 50. Based on the boundary line data (the first detected object) acquired in step S14 and the third sample image data, the control program 39 identifies the boundary line data indicating the marker 50, and stores the identified boundary line data in the memory 37 as marker image data (S72). Specifically, the control program 39 stores, in the memory 37 as marker image data, the boundary line data with a matching rate equal to or higher than a threshold with respect to the shape, pattern, etc. of the marker indicated by the third sample image data. The process of step S72 corresponds to the "marker position identification process" described in the claims.

[0161] Based on the target image data acquired in step S23 and the marker image data acquired in step S73, the control program 39 calculates the separation distance between the target and the marker 50 (S73). The separation distance is calculated, for example, as the pixel distance between the pixel indicating the target and the pixel indicating the marker 50.

[0162] Note that the separation distance may be calculated based on the target image data, the marker image data acquired in step S73, and the past marker image data stored in the memory 37. The past marker image data is the data immediately before, the data immediately after the start of hemodialysis, or the data between the two. For example, when the arm 22 (hand) of the patient 20 approaches the arm 21 (camera 40), the size of the marker 50 shown in the captured image becomes larger. The control program 39 calculates the distance by which the arm 22 (hand) of the patient 20 has approached the camera 40 based on the size of the marker 50 indicated by the marker image data acquired in step S73 and the size of the marker 50 indicated by the past marker image data, and calculates the separation distance between the target and the marker 50 based on the calculated distance, the target image data, and the marker image data acquired in step S73.

[0163] The control program 39 determines whether the separation distance calculated in step S73 is less than a fourth threshold value stored in the memory 37 (S74). That is, in step 74, it is determined whether the patient 20 has unconsciously extended the arm 22 (hand) to the puncture needle 62. The process of step S74 corresponds to the "approach determination process" described in the claims.

[0164] When the control program 39 determines that the separation distance is less than the fourth threshold value (S74: Yes), it outputs warning notification data to at least one of the display device 34, the speaker 33, and the communication interface 32 (mobile communication terminal 51) (S27). Thereafter, the control program 39 executes the process of step S28 in the same manner as in the first embodiment.

[0165] When the control program 39 determines that the separation distance is not less than the fourth threshold value (S74: No), it executes the process of step S29 in the same manner as in the first embodiment.

[0166] [Operation and Effect of Second Embodiment]

[0167] In this embodiment, when the patient 20 unconsciously extends the arm 22 (hand) to the puncture needle 62 and the separation distance between the target such as the puncture needle 62 and the marker 50 becomes short, a warning notification is issued. Therefore, the safety device 10 according to this embodiment can prevent the puncture needle 62 from being removed from the arm 21 of the patient 20.

[0168] When the safety device 10 is used for a nasogastric tube, the markers 50 are respectively attached to both arms of the patient 20, and the separation distance is calculated for each marker 50.

[0169] In addition, the configurations and processes described in Modifications 3 to 5 above may be adopted in the second embodiment.

[0170] [Modification 1]

[0171] In the second embodiment, an example in which one camera 40 is used was described. In this modified example, an example in which two cameras 40 are used will be described.

[0172] The configurations and processes other than those described below are the same as the configurations and processes described in the second embodiment. The same reference numerals and step numbers as those in the second embodiment are used for the same configurations and processes in the second embodiment.

[0173] FIG. 11 is an explanatory diagram for explaining the use of the safety device 10 according to this modified example.

[0174] As shown in FIG. 11, the safety device 10 includes a terminal device 30, two cameras 40, and a marker 50.

[0175] One of the two cameras 40 is arranged on the side of the patient 20 lying on the bed 25, and the other camera 40 is arranged near the head or feet of the patient 20 lying on the bed 25. However, one camera 40 may be arranged on the side of the bed 25, and the other camera 40 may be arranged above the bed 25 (above the patient 20). In the example shown in FIG. 11, the other camera 40 is arranged near the head of the patient 20.

[0176] Similar to the second embodiment, the control program 39 executes the safety confirmation process shown in FIG. 10. In step S11, the control program 39 transmits imaging instructions to the two cameras 40 respectively. In step S12, the control program 39 acquires the imaging image data transmitted by the two cameras 40 respectively. In steps S13 to S23, after identifying the arms 21 of the patient 20 shown in each imaging image data respectively, the control program 39 identifies the objects on the arm 21 as targets respectively using the first sample image data, and stores two boundary line data indicating the targets in the memory 37 respectively. That is, a target is identified for each of the two imaging images. Next, the control program 39 executes the processes of steps S71 and S72, and stores the marker image data indicating the position of the marker 50 pasted on the arm 22 of the patient 20 in the memory 37 for each imaging image data (for each of the two cameras 40). That is, the marker 50 (the arm 22 of the patient 20) is identified for each of the two imaging images.

[0177] In step S73, the control program 39 calculates the separation distance between the target and the marker 50 for each of the two imaging image data. In step S75, the control program 39 determines whether both of the two separation distances are less than the fourth threshold value. That is, even if it is determined in the imaging image of one camera 40 that the other arm 22 of the patient 20 approaches the target on the arm 21 of the patient 20 (less than the fourth threshold value), if in the imaging image of the other camera 40 the target on the arm 21 of the patient 20 and the other arm 22 of the patient are separated (equal to or greater than the fourth threshold value), in step S74, it is determined that the separation distance is equal to or greater than the fourth threshold value (S74: No).

[0178] If the control program 39 determines in step S74 that both of the two separation distances are less than the fourth threshold value (S74: Yes), it executes the processes of steps S27 and S28. If the control program 39 determines in step S74 that both of the two separation distances are not less than the fourth threshold value (S74: No), it executes the process of step S29.

[0179] [Operation and Effect of Modification Example 1]

[0180] In this modification example, by using two cameras 40, it is possible to more accurately detect whether or not the patient 20 has unconsciously extended the arm 22 to the puncture needle 62 or the like.

[0181] [Modification Example 2]

[0182] In this modification example, an example in which the camera 40 is a stereo camera is described. The camera 40 has two adjacent cameras. The camera 40 transmits by associating two imaging image data respectively generated by each camera.

[0183] The control program 39 executes the safety confirmation process shown in FIG. 10 in the same manner as in the second embodiment. After executing the process of step S11, in step S12, the control program 39 acquires two pieces of imaging image data transmitted by the camera 40. In steps S13 to S23, the control program 39 identifies the arm 21 of the patient 20 for each of the two pieces of imaging image data, then identifies an object on the arm 21 as a target using the first sample image data, and stores two boundary line data indicating the target in the memory 37 respectively. Also, in step S23, the control program 39 calculates a parallax based on the two boundary line data indicating the target. The control program 39 calculates the direction and distance at which the target is located based on the parallax and the boundary line data, and stores them in the memory 37. The direction and the distance function as coordinate data (spherical coordinate data or three-dimensional polar coordinate data) indicating the position of the target three-dimensionally.

[0184] Next, the control program 39 executes the processes of steps S71 and S72, and stores marker image data indicating the position of the marker 50 attached to the arm 22 of the patient 20 in the memory 37 for every two pieces of imaging image data. Further, in step S72, the control program 39 calculates a parallax based on the two pieces of marker image data. Based on the said suggestion and the marker image data, the control program 39 calculates the direction and distance in which the marker 50 is located, and stores them in the memory 37. The said direction and distance function as coordinate data (spherical coordinate data or three-dimensional polar coordinate data) indicating the position of the marker 50 three-dimensionally.

[0185] In step S73, the control program 39 calculates the separation distance between the target and the marker 50 based on the direction and distance respectively stored in the memory 37 in steps S23 and S72. The separation distance calculated in step S73 is the distance in three-dimensional space.

[0186] In step S75, the control program 39 determines whether or not the said separation distance is less than a fourth threshold value. That is, the control program 39 determines whether or not the arm 22 of the patient 20 has approached the target in three-dimensional space, rather than whether or not the arm 22 of the patient 20 has approached the target in the imaging image.

[0187] If the control program 39 determines in step S74 that the said separation distance is less than the fourth threshold value (S74: Yes), it executes the processes of steps S27 and S28. If the control program 39 determines in step S74 that both of the said separation distances are not less than the fourth threshold value (S74: No), it executes the process of step S29.

[0188] [Operation and effect of Modification 2]

[0189] In this modification, by using the stereo camera (camera 40), it is possible to more accurately detect whether or not the patient 20 has unconsciously extended the arm 22 toward the puncture needle 62.

[0190] [Third Embodiment]

[0191] FIG. 12 is a diagram for explaining the use of the safety device 10 according to the third embodiment.

[0192] Configurations and processes other than those described below are the same as the configurations and processes described in the first embodiment or the second embodiment. The same reference numerals and step numbers as those in the first embodiment or the second embodiment are used for the same configurations and processes in the first embodiment or the second embodiment.

[0193] Hereinafter, the safety device 10 used for artificial dialysis will be described.

[0194] As shown in FIG. 12, the safety device 10 includes a terminal device 30, a camera 40, a mobile communication terminal 51 (see FIG. 3), and a first marker 81 and a second marker 82. The first marker 81 and the second marker 82 are stickers made of paper or plastic having an adhesive surface on one side. However, the first marker 81 and the second marker 82 may not have an adhesive surface. In that case, the first marker 81 and the second marker 82 are attached to the patient 20 by tape.

[0195] The first marker 81 and the second marker 82 are each provided with a pattern in order to facilitate recognition in the control program 39 (see FIG. 3). The pattern is a grid pattern or the like. However, if the control program 39 can recognize (discriminate) the first marker 81 and the second marker 82 by their outer shapes, the first marker 81 and the second marker 82 may be plain. Also, instead of the pattern, codes such as barcodes or QR codes (registered trademarks) may be attached to the first marker 81 and the second marker 82. In that case, the control program 39 has a code reading program (off-the-shelf program) as a module program.

[0196] The first marker 81 and the second marker 82 can adopt various shapes such as circular, rectangular, and elliptical.

[0197] The pattern attached to the first marker 81 is different from the pattern attached to the second marker 82. Alternatively, the outer shape of the first marker 81 is different from the outer shape of the second marker 82. The control program 39 identifies the first marker 81 and the second marker 82 based on the outer shape or the pattern.

[0198] The first marker 81 is attached to the arm 21 of the patient 20. The arm 21 is the arm on the patient 20's left or right arm where the puncture needle 62 is inserted. The second marker 82 is attached to the arm 22 of the patient 20. The arm 22 is the arm on the patient 20's left or right arm where the puncture needle 62 is not inserted.

[0199] Two second markers 82 may be attached to both sides of the arm 22, that is, the back of the hand side and the palm side. Thereby, the camera 40 can surely image the second marker 82. The arm 21 corresponds to the "one arm" described in the claims. The arm 22 corresponds to the "other arm" described in the claims.

[0200] The camera 40 is arranged so that not only the arm 21 of the patient 20 but also the arm 22 of the patient 20 is within the imaging range, specifically, so that the entire patient 20 is reflected in the captured image.

[0201] The memory 37 (see FIG. 3) of the terminal device 30 stores in advance the fourth sample image data generated by imaging the first marker 81 and the fifth sample image data generated by imaging the second marker 82.

[0202] FIG. 13 is a flowchart of the safety confirmation process according to the third embodiment.

[0203] The control program 39 (see FIG. 3) causes the CPU 36 (see FIG. 3) to execute the safety confirmation process shown in FIG. 13. The process that the control program 39 causes the CPU 36 to execute is also a process executed by the CPU 36 or the control device 31 (see FIG. 3) or the terminal device 30 (see FIG. 3).

[0204] The control program 39 executes the processes from step S11 to S14 in the same manner as in the first embodiment. That is, the control program 39 causes the camera 40 to perform imaging (S11), acquires imaging image data (S12), performs image processing on the acquired imaging image data (S13), and acquires boundary line data indicating a detected object reflected in the imaging image (S14).

[0205] Also, the control program 39 reads the fourth sample image data and the fifth sample image data from the memory 37 (S81). Based on the boundary line data (detected object) acquired in step S14 and the fourth sample image data, the control program 39 specifies the boundary line data indicating the first marker 81, and stores the position of the specified boundary line data as the first marker position in the memory 37 (S82). Specifically, the control program 39 specifies, as the boundary line data indicating the first marker 81, the boundary line data whose coincidence rate with the shape, pattern, etc. of the first marker 81 indicated by the fourth sample image data is equal to or higher than a threshold value. The first marker position is a position in the imaging image, and for example, is the pixel position (pixel number) of the boundary line data indicating the first marker 81. The process of step S82 corresponds to the "first marker position specifying process" described in the claims.

[0206] Similarly, based on the boundary line data (detected object) acquired in step S14 and the fifth sample image data, the control program 39 specifies the boundary line data indicating the second marker 82, and stores the position of the specified boundary line data as the second marker position in the memory 37 (S83). The process of step S83 corresponds to the "second marker position specifying process" described in the claims.

[0207] The control program 39 calculates the relative distance on the imaging image between the specified first marker position and the second marker position (S84). The process of step S84 corresponds to the "relative distance calculating process" described in the claims.

[0208] The control program 39 determines whether the calculated relative distance is less than the fifth threshold value stored in the memory 37 (S85). That is, in step S85, it is determined whether the patient 20 has unconsciously extended the arm 22 (hand) to the puncture needle 62. The process of step S85 corresponds to the "approach determination process" described in the claims.

[0209] When the control program 39 determines that the relative distance is less than the fifth threshold value (S85: Yes), that is, when it determines that the patient 20 has unconsciously extended the arm 22 (hand) to the puncture needle 62, it outputs the warning notification data stored in the memory 37 to at least one of the display device 34, the speaker 33, and the communication interface 32 (see FIG. 3) (S27). The process of step S27 corresponds to the "notification process" described in the claims.

[0210] Thereafter, the control program 39 executes the process of step S28 in the same manner as in the first embodiment. Also, when the control program 39 determines that the relative distance is greater than or equal to the fifth threshold value (S85: No), that is, when it determines that the patient 20 has not unconsciously extended the arm 22 (hand) to the puncture needle 62, it executes the process of step S29 in the same manner as in the first embodiment.

[0211] [Operation and Effect of the Third Embodiment]

[0212] The first marker 81 is attached to the arm 21 of the patient 20, and the second marker 82 is attached to the arm 22. Identifying the position of the first marker 81 and identifying the position of the second marker 82 are easier and more reliable than identifying the position of the arm 21 or the arm 22 of the patient 20 itself. Therefore, the safety device 10 of the present embodiment can easily and reliably determine whether the patient 20 has unconsciously extended the arm 22 (hand) to the puncture needle 62. As a result, the safety device 10 can prevent the patient 20 from unconsciously removing the puncture needle 62 stuck in the arm 21 during hemodialysis or intravenous drip.

[0213] [Modification Example 1 of the Third Embodiment]

[0214] In the third embodiment, the case where the safety device 10 is used for hemodialysis was described. In this modification, an example in which the safety device 10 is used for a nasogastric tube will be described.

[0215] FIG. 14 is a diagram for explaining the use of the safety device 10 according to this modification.

[0216] The configurations and processes other than those described below are the same as the configurations and processes described in the first to third embodiments. The same reference numerals and step numbers as those in the first to third embodiments are used for the same configurations and processes in the first to third embodiments.

[0217] As shown in FIG. 14, in addition to the terminal device 30, the camera 40, and the mobile communication terminal 51 (see FIG. 3), the safety device 10 includes a first marker 81 and a plurality of second markers 82.

[0218] The first marker 81 is attached to the head and neck of the patient 20. The head and neck means a part including the face, head, and neck of the patient 20. That is, the first marker 81 is attached in the vicinity of the tube 23 inserted into the nose of the patient 20.

[0219] The second markers 82 are respectively attached to both arms 21 and 22 of the patient 20. The second markers 82 may be respectively attached to both sides of the arm 21 and both sides of the arm 22. Thereby, the camera 40 can surely image the second markers 82 on each of the arms 21 and 22. Both sides of the arms 21 and 22 are the back of the hand side and the palm side.

[0220] The pattern attached to the first marker 81 and the pattern attached to the second marker 82 are different. Alternatively, the outer shape of the first marker 81 and the outer shape of the second marker 82 are different. The control program 39 identifies the first marker 81 and the second marker 82 based on the outer shape or the pattern.

[0221] In addition, the pattern attached to the second marker 82 attached to the arm 21 is different from the pattern attached to the second marker 82 attached to the arm 22. Alternatively, the outer shape of the second marker 82 attached to the arm 21 is different from the outer shape of the second marker 82 attached to the arm 22. The control program 39 identifies the second marker 82 attached to the arm 21 and the second marker 82 attached to the arm 22 based on the outer shape or the pattern.

[0222] The memory 37 of the terminal device 30 stores in advance the fourth sample image data, the fifth sample image data, and the sixth sample image data. The fourth sample image data is image data generated by imaging the first marker 81 attached to 20 head and neck parts of the patient. The fifth sample image data is image data generated by imaging the second marker 82 attached to the arm 21 of the patient 20. The sixth sample image data is image data generated by imaging the second marker 82 attached to the arm 22 of the patient 20.

[0223] The camera 40 is arranged such that the head and neck parts and both arms 21 and 22 of the patient 20 are within the imaging range, specifically, such that the entire patient 20 is shown in the captured image.

[0224] FIG. 15 is a flowchart of the safety confirmation process according to this modified example.

[0225] The control program 39 (see FIG. 3) causes the CPU 36 (see FIG. 3) to execute the safety confirmation process shown in FIG. 15 instead of the safety confirmation process shown in FIG. 13. The process that the control program 39 causes the CPU 36 to execute is also a process executed by the CPU 36 or the control device 31 (see FIG. 3) or the terminal device 30 (see FIG. 3).

[0226] The control program 39 executes the processes from step S11 to S14 in the same manner as in the third embodiment. That is, the control program 39 causes the camera 40 to perform imaging (S11) to acquire imaging image data (S12), performs image processing on the acquired imaging image data (S13), and acquires boundary line data indicating a detected object shown in the imaging image (S14).

[0227] Further, the control program 39 reads out the fourth sample image data, the fifth sample image data, and the sixth sample image data from the memory 37 (S91). Based on the boundary line data (detected object) acquired in step S14 and the fourth sample image data, the control program 39 specifies the boundary line data indicating the first marker 81, and stores the position of the specified boundary line data as the first marker position in the memory 37 (S82). Specifically, the control program 39 specifies, as the boundary line data indicating the first marker 81, the boundary line data with a matching rate with the shape, pattern, etc. of the first marker 81 indicated by the fourth sample image data being equal to or higher than the threshold value. The first marker position is a position in the captured image, for example, the pixel position (pixel number) of the boundary line data indicating the first marker 81. The process of step S82 corresponds to the "first marker position specifying process" described in the claims.

[0228] Similarly, based on the boundary line data (detected object) acquired in step S14 and the fifth sample image data, the control program 39 specifies the boundary line data indicating the second marker 82 attached to the arm 21 of the patient 20, and stores the position of the specified boundary line data as one of the second marker positions in the memory 37 (S92). Similarly, based on the boundary line data (detected object) acquired in step S14 and the sixth sample image data, the control program 39 specifies the boundary line data indicating the second marker 82 attached to the arm 22 of the patient 20, and stores the position of the specified boundary line data as the other second marker position in the memory 37 (S92). The process of step S92 corresponds to the "second marker position specifying process" described in the claims.

[0229] The control program 39 calculates the first relative distance on the captured image between the specified first marker position and the second marker position on the arm 21 (S93). Further, the control program 39 calculates the second relative distance on the captured image between the specified first marker position and the second marker position on the arm 22 (S93). The process of step S93 corresponds to the "relative distance calculating process" described in the claims.

[0230] The control program 39 determines whether at least one of the calculated first relative distance or second relative distance is less than the fifth threshold value stored in the memory 37 (S94). That is, in step S94, it is determined whether the patient 20 has unconsciously extended the arm 21 or arm 22 to the tube 23 inserted into the nose. The process of step S94 corresponds to the "approach determination process" described in the claims.

[0231] When the control program 39 determines that at least one of the first relative distance or second relative distance is less than the fifth threshold value (S94: Yes), that is, when it is determined that the patient 20 has unconsciously extended the arm 21 or arm 22 to the nasal tube 23, it outputs the warning notification data stored in the memory 37 to at least one of the display device 34, the speaker 33, and the communication interface 32 (S27). Thereafter, the control program 39 executes the process of step S28 in the same manner as in the third embodiment. Also, when the control program 39 determines that the first relative distance and the second relative distance are equal to or greater than the fifth threshold value (S94: No), that is, when it is determined that the patient 20 has not unconsciously extended the arm 21 or arm 22 to the nasal tube 23, it executes the process of step S29 in the same manner as in the third embodiment.

[0232] [Function and effect of Modification 1]

[0233] The safety device 10 can prevent the patient 20 from unconsciously removing the tube 23 inserted into the nose in the nasogastric tube.

[0234] [Other modifications]

[0235] In the above-described first embodiment and second embodiment, an example in which the safety device 10 is used for hemodialysis has been described. However, the safety device 10 may be used for a nasogastric tube. In that case, as the first sample image data, data showing a human face including the area below the nose is used instead of data showing a human arm. Also, as the second sample image, image data showing gauze, a tube, tape, the nose, ears, or eyes is used. Further, the gauze, tube, tape, nose, ears, or eyes are used as targets for determining whether the image is blurred, whether the target image is missing, or whether the focal length has changed.

[0236] Modification example 2 or modification example 3 of the first embodiment and modification example 4 or modification example 5 of the first embodiment may be combined. Also, modification example 2 or modification example 3 of the first embodiment and modification example 4 or modification example 5 of the first embodiment may be combined and used in the second embodiment.

[0237] In the above-described first embodiment, an example in which the gauze 61, puncture needle 62, tube 63, or tape 64 on the arm 21 of the patient 20 is used as a target has been described. However, a marker (not shown) attached to the arm 21 of the patient 20 may be used as a target.

[0238] In the above-described first embodiment, an example in which it is determined whether the arm 22 of the patient 20 has approached a target such as the puncture needle 62 based on a change in the width of the boundary line (contour) of the target image has been described. However, whether the arm 22 of the patient 20 has approached a target such as the puncture needle 62 may be determined based on a change in the width of the pattern of the target.

[0239] In the above-described first and second embodiments, an example in which the terminal device 30 and the camera 40 are separate devices has been described. However, the terminal device 30 and the camera 40 may be the same device. In that case, the CPU 43 and the memory 44 of the camera 40 become the CPU 36 and the memory 37 of the terminal device 30, and the driver program 45 is stored in the memory 37. The terminal device 30 is, for example, a smartphone (registered trademark) equipped with a camera 40, a tablet, or a notebook computer. The control program 39 and the driver program 45 stored in the memory 37 correspond to the "program" described in the claims.

[0240] The safety device 10 according to the first and second embodiments may be used for other procedures using a needle or a tube, such as intravenous drip, in addition to artificial dialysis and nasogastric tube.

[0241] In the above-described first and second embodiments, the display device 34, the speaker 33, and the mobile communication terminal 51 have been described as notification devices. However, the notification device may be a lamp or an LED such as red.

Explanation of reference numerals

[0242] 10 ··· Safety device 20 ··· Patient 21 ··· Arm 22 ··· Arm 23 ··· Tube 25 ··· Bed 26 ··· Stand 30 ··· Terminal device 31 ··· Control device 32 ··· Communication interface 33 ··· Speaker 34 ··· Display device 35 ··· Input interface 36 ··· CPU 37 ··· Memory 39 ··· Control program 40 ··· Camera 41 ··· Optical system 42 ··· Image sensor 46... Auto-focus mechanism 50... Marker 51... Mobile communication terminal 61... Gauze 62... Puncture needle 63... Tube 64... Tape 65... Artificial dialysis device 70... Camera mounting member 71... Fixed part 72... Mounting part 81... First marker 82... Second marker

Claims

1. A camera having an autofocus function, capturing an image of an imaging area including a patient's arm or nose, and outputting captured image data; a control device to which the captured image data is input; An alarm device capable of communicating with the control device, The control device includes: a target identification process for identifying a target image showing a predetermined target that appears in a captured image represented by the captured image data; a determination process for determining whether or not a foreign object has approached the target based on a change in the outline or width of a pattern of the target image caused by the autofocus function, the change being a change in two different target images identified at a predetermined time interval; a notification process for outputting notification information to the notification device based on the determination that a foreign object has approached the target in the determination process; The above determination process is as follows: A safety device that determines that a foreign object has approached the target based on the change in width being equal to or greater than a first threshold, and determines that a foreign object has not approached the target based on the change in width being less than the first threshold.

2. A camera having an autofocus function, capturing an image of an imaging area including a patient's arm or nose, and outputting captured image data and a focal length; a control device to which the captured image data is input; An alarm device capable of communicating with the control device, The control device includes: a target identification process for identifying a target image showing a predetermined target that appears in a captured image represented by the captured image data; a determination process for determining whether or not a foreign object has approached the target based on changes in two different target images identified at a predetermined time interval, the changes being changes in the two focal lengths corresponding to the target images; a notification process for outputting notification information to the notification device based on the determination that a foreign object has approached the target in the determination process; The above determination process is as follows: A safety device that determines that a foreign object has approached the target based on the change in focal length being equal to or greater than a third threshold, and determines that a foreign object has not approached the target based on the change in focal length being less than the third threshold.

3. the control device includes a memory configured to store sample image data representing at least one of an arm, a nose, a tape, a puncture needle, a tube, gauze, or a marker; the target is at least one of a patient's arm, a nose, a tape, a puncture needle, a tube, gauze, or a marker; The above target identification process is a first process for identifying an object appearing in the captured image; The safety device according to claim 1 , further comprising a second process for identifying the target based on a matching rate between the identified object and a sample image represented by the sample image data.

4. the control device includes a memory configured to store sample image data representing at least one of an arm, a nose, a tape, a puncture needle, a tube, gauze, or a marker; the target is at least one of a patient's arm, a nose, a tape, a puncture needle, a tube, gauze, or a marker; The above target identification process is a first process for identifying an object appearing in the captured image; The safety device according to claim 2 , further comprising a second process for identifying the target based on a matching rate between the identified object and a sample image represented by the sample image data.

5. the control device includes a memory that stores first sample image data representing an arm or a nose, and second sample image data representing at least one of a tape, a puncture needle, a tube, gauze, or a marker; the target is at least one of a tape, a needle, a tube, gauze, or a marker; The above target identification process is a first process for identifying an object appearing in the captured image; a third process of identifying an arm or a nose of a patient based on a matching rate between the identified object and a first sample image represented by the first sample image data; The safety device of claim 1 or 3, further comprising a second process for identifying the target based on a matching rate between the identified object located on the patient's arm or under the nose and a second sample image represented by the second sample image data.

6. the control device includes a memory that stores first sample image data representing an arm or a nose, and second sample image data representing at least one of a tape, a puncture needle, a tube, gauze, or a marker; the target is at least one of a tape, a needle, a tube, gauze, or a marker; The above target identification process is a first process for identifying an object appearing in the captured image; a third process of identifying an arm or a nose of a patient based on a matching rate between the identified object and a first sample image represented by the first sample image data; The safety device of claim 2 or 4, further comprising a second process for identifying the target based on a matching rate between the identified object located on the patient's arm or under the nose and a second sample image represented by the second sample image data.

7. A camera that captures an image of an imaging area including a patient's arm or nose and outputs captured image data; a control device having a touch panel and into which the captured image data is input; An alarm device capable of communicating with the control device, The control device includes: a target identification process for identifying a target image showing a predetermined target that appears in a captured image represented by the captured image data; a determination process for determining whether or not a foreign object has approached the target based on changes in the two different target images identified at a predetermined time interval; a notification process for outputting notification information to the notification device based on the determination that a foreign object has approached the target in the determination process; The above target identification process is a display process of acquiring the captured image data input from the camera, identifying an object appearing in the captured image data, and displaying the captured image on the touch panel with an icon representing the identified object superimposed thereon; a designation receiving process for receiving designation of the icon via the touch panel; and a process of identifying an object indicated by the designated icon as the target.

8. The safety device according to claim 1 or 3, further comprising a camera mounting member having a fixing portion for fixing the camera, and a mounting portion for mounting on an arm or head of a patient.

9. 5. The safety device according to claim 2, further comprising a camera mounting member having a fixing portion for fixing the camera, and a mounting portion for mounting on an arm or head of a patient.

10. A program implemented in a terminal device having a communication interface and a computer, an image data acquisition process for acquiring image data and a focal length input from a camera having an autofocus function that captures an image of an imaging area including a patient's arm or nose at a predetermined time interval; a target identification process for identifying a target image showing a predetermined target that appears in a captured image represented by the captured image data; a determination process for determining whether or not a foreign object has approached the target based on changes in two different target images identified at a predetermined time interval, the changes being changes in the two focal lengths corresponding to the target images; a notification process for outputting notification information based on the determination that a foreign object has approached the target in the determination process, The above determination process is as follows: A program that determines that a foreign object has approached the target based on the change in focal length being equal to or greater than a third threshold, and determines that a foreign object has not approached the target based on the change in focal length being less than the third threshold.

11. A program implemented in a terminal device having a communication interface and a computer, an image data acquisition process for acquiring image data and a focal length input from a camera having an autofocus function that captures an image of an imaging area including a patient's arm or nose at a predetermined time interval; a target identification process for identifying a target image showing a predetermined target that appears in a captured image represented by the captured image data; a determination process for determining whether or not a foreign object has approached the target based on changes in two different target images identified at a predetermined time interval, the changes being changes in the two focal lengths corresponding to the target images; a notification process for outputting notification information based on the determination that a foreign object has approached the target in the determination process, The above determination process is as follows: A program that determines that a foreign object has approached the target based on the change in focal length being equal to or greater than a third threshold, and determines that a foreign object has not approached the target based on the change in focal length being less than the third threshold.

Images