Computer program, dropping speed estimation device, dropping speed estimation method, and background component

Abstract

To provide a computer program, a dropping speed estimation device, a dropping speed estimation method, and a background component that can reduce the work of checking infusion drops.SOLUTION: A computer program runs on a terminal device, and causes a computer to execute the processing of: acquiring multiple frame images in a time series which are obtained by imaging the drip part of a drip tube with an imaging unit of the terminal device; detecting the falling of drops at the drip part on the basis of the acquired multiple frame images; and estimating the drip speed on the basis of the detected falling of drops.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a computer program, a drop velocity estimation device, a drop velocity estimation method, and a background member. [Background technology]

[0002] Nurses' rounds include checking whether the drip rate of IV fluid is normal. When checking the drip rate, nurses visually count the number of drips while checking their wristwatches. This task depends on the nurse's individual skills and experience, and is time-consuming as it also requires verification that the drip rate check was performed accurately.

[0003] Patent Document 1 discloses a measuring device that can be detachably fitted to the side wall of a drip tube, has a light-emitting unit and a light-receiving unit, detects when the light emitted from the light-emitting unit is blocked by the drip liquid dripping from the duct of the drip tube, and measures the drip rate based on the number of detections per unit time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2014-140467 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, a measuring device like that in Patent Document 1 requires that one measuring device be attached to one drip tube, and in medical institutions that use many drip tubes for many patients, many measuring devices must be prepared, which increases the introduction cost. For this reason, the current situation is that the drip check work relies on the manual work of nurses.

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a computer program, a drip rate estimation device, a drip rate estimation method, and a background member that can reduce the workload of checking dripping of an IV. [Means for solving the problem]

[0007] (1) The computer program of the present invention is a computer program that runs on a terminal device and causes a computer to execute the following processes: acquiring a plurality of time-series frame images obtained by photographing the drip portion of a drip tube with an imaging unit of the terminal device, detecting the falling of droplets at the drip portion based on the acquired plurality of frame images, and estimating the drip speed based on the detected drop of droplets.

[0008] Here, an embodiment of the present invention is (2) The computer program of (1) above causes a computer to execute a process of calculating a first difference in pixel values ​​of the dripping portion of each of adjacent first and second frame images among the plurality of frame images, calculating a second difference in pixel values ​​of the dripping portion of each of adjacent second and third frame images among the plurality of frame images, calculating a difference between the first difference and the second difference across the plurality of frame images, and detecting the falling of droplets based on the calculated differences.

[0009] (3) The computer program of (1) or (2) above causes a computer to execute a process of calculating a drop falling interval across the plurality of frame images based on the detected drop falling, storing the calculated drop interval data, and estimating a dropping speed based on a statistical value obtained by statistically processing the stored drop interval data.

[0010] (4) The computer program of (2) above causes a computer to execute a process of identifying a cap region of the drip tube based on the acquired multiple frame images, estimating a center of gravity position of the identified cap region, and aligning the estimated center of gravity position to a predetermined position on each frame image.

[0011] (5) The computer program of (2) above, when an image including a cap region of an IV tube is input, causes a computer to execute a process of inputting the acquired multiple frame images into a learning model that outputs the center of gravity position of the cap region to acquire the center of gravity position, and aligning the acquired center of gravity position to a predetermined position on each frame image.

[0012] (6) The computer program of (4) or (5) above causes a computer to execute a process of calculating a difference between the first difference and the second difference across a plurality of frame images in which the center of gravity position of the cap region is aligned at the specified position, and detecting the falling of a drop based on the calculated difference.

[0013] (7) Any one of the computer programs (1) to (6) above causes a computer to execute a process of acquiring a plurality of frame images in a time series in which the drip portion of the drip tube is photographed with a background member provided outside the drip tube at a position corresponding to the drip portion of the drip tube.

[0014] (8) Any one of the computer programs (1) to (7) above causes a computer to execute a process of outputting an estimated drip rate in association with a patient ID of a patient receiving an intravenous drip through the drip tube, a medical staff ID of a medical staff member operating the drip tube, and an estimated date and time of the drip rate.

[0015] (9) Any one of the computer programs described above in (1) to (8) causes a computer to execute a process of displaying the estimated drip rate on a display unit of the terminal device.

[0016] (10) The computer program of (9) above causes a computer to execute a process for changing a display mode of the display unit when a falling drop is detected.

[0017] (11) The computer program according to (9) or (10) causes a computer to execute a process of identifying a reliability of a drop detection result and displaying the reliability together with an estimated drop speed on the display unit.

[0018] (12) The drip speed estimation device of the present invention includes a control unit, which acquires a plurality of frame images in a time series obtained by photographing the drip portion of a drip tube with an imaging unit, detects the falling of droplets at the drip portion based on the acquired plurality of frame images, and estimates the drip speed based on the detected falling of droplets.

[0019] (13) The drip speed estimation method of the present invention acquires a plurality of frame images in a time series obtained by photographing the drip portion of a drip tube with an imaging unit, detects the falling of droplets at the drip portion based on the acquired plurality of frame images, and estimates the drip speed based on the detected falling of droplets.

[0020] (14) The background member of the present invention is a background member that can be attached to and detached from an infusion tube, has a shape larger than the shape of the side side of the infusion tube, has a side portion facing the side side of the infusion tube and an upper surface portion facing the upper surface side of the infusion tube and bent in an L shape relative to the side portion, and the upper surface portion has an insertion hole for inserting a tube of the infusion tube, and all or a part of the side portion is colored in the opposite color to the color of the cap region of the infusion tube. Effect of the Invention

[0021] According to the present invention, the work of checking the drip of an intravenous drip can be reduced. [Brief description of the drawings]

[0022] [Figure 1] FIG. 13 is a diagram showing an example of an intravenous drip confirmation operation according to the present embodiment. [Diagram 2] FIG. 4 is a diagram illustrating an example of a configuration of a background member. [Diagram 3] FIG. 1 illustrates an example of a configuration of a dropping speed estimation device. [Figure 4] FIG. 13 is a diagram showing an example of a drip portion of a drip tube. [Diagram 5] FIG. 11 is a diagram illustrating an example of a position alignment process. [Figure 6] FIG. 13 is a diagram showing an example of estimation of a center of gravity position using a learning model. [Figure 7] FIG. 13 is a diagram illustrating an example of detection of falling droplets. [Figure 8] 10A and 10B are diagrams illustrating an example of a drop falling detection result. [Figure 9] FIG. 13 is a diagram illustrating an example of a method for estimating a dropping speed. [Figure 10] FIG. 4 is a diagram showing an example of a display screen on a display unit. [Figure 11] FIG. 11 is a diagram illustrating an example of a processing procedure performed by the dropping speed estimation device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, an embodiment of the present invention will be described. Fig. 1 is a diagram showing an example of an IV drip confirmation operation according to this embodiment. A medical worker such as a nurse can carry a drip rate estimation device 50 as a terminal device. The drip rate estimation device 50 is not a special dedicated device, but a mobile terminal such as a smartphone, tablet terminal, or PC carried by a medical worker, with a computer program 58 (described later) having a drip rate estimation function installed.

[0024] The drip tube 20 is a transparent tube, allowing the dripping to be confirmed. A cap part 21 is fixed to the upper part of the drip tube 20 (upstream side of the drip), and a tube 23 is connected to the infusion agent from the cap part 21. A tube 24 is connected to the lower part of the drip tube 20 (downstream side of the drip), and is connected to a cremen for adjusting the flow rate. The drip tube 20 is filled with the infusion liquid 25 up to an appropriate position to make it easy to count the number of drops. A background member 10 is provided on the background side of the drip tube 20 (opposite the drip rate estimation device 50 across the drip tube 20 and close to the side wall of the drip tube 20).

[0025] FIG. 2 is a diagram showing an example of the configuration of the background member 10. The background member 10 is a rectangular plate-like member, and can be bent at the location indicated by the dashed line in the figure along the short side direction of the background member 10. The background member 10 is a member that can be attached to and detached from the drip tube 20, and has a shape larger than the shape of the side side (side wall) of the drip tube 20. For example, the length of the short side of the background member can be about 3 to 4 cm, and the length of the long side can be about 9 to 10 cm. The background member 10 has a side portion 11 facing the side side (side wall) of the drip tube 20 and an upper surface portion 12 facing the upper surface side of the drip tube 20 and bent in an L-shape with respect to the side portion 11. The length of the long side of the side portion 11 can be about 7 to 8 cm. The side portion 11 and the upper surface portion 12 may be formed in advance in an L-shaped bent state, or may be formed so as to be bent in an L-shape.

[0026] The top surface portion 12 has an insertion hole 13 through which the tube 23 of the drip tube 20 is inserted. All or a part of the surface of the side surface portion 11 facing the drip tube 20 is colored in the opposite color to the color of the cap portion (cap region) 21 of the drip tube 20. For example, when the color of the cap portion 21 is green, the color of the side surface portion 11 of the background member 10 can be gray, but is not limited to gray. The color of the side surface portion 11 of the background member may be the color of the cap portion 21, i.e., a color that makes it easy to detect the outline of the cap portion 21.

[0027] 3 is a diagram showing an example of the configuration of a dripping velocity estimation device 50. The dripping velocity estimation device 50 includes a control unit 51 that controls the entire device, a communication unit 52, a memory 53, an imaging unit 54, a display unit 55, an operation unit 56, and a storage unit 57.

[0028] The control unit 51 is configured by incorporating a required number of central processing units (CPUs), micro-processing units (MPUs), graphics processing units (GPUs), general-purpose computing on graphics processing units (GPGPUs), etc. The control unit 51 may also be configured by combining digital signal processors (DSPs), field-programmable gate arrays (FPGAs), etc.

[0029] The communication unit 52 may include, for example, a communication module, and may have a communication function with an external device (not shown).

[0030] The memory 53 can be configured with a semiconductor memory such as a static random access memory (SRAM), a dynamic random access memory (DRAM), or a flash memory.

[0031] The imaging unit 54 is, for example, a camera, and is capable of capturing video at 240 frames per second (240 fps). Note that the frame rate is not limited to 240 fps.

[0032] The display unit 55 can be configured with a liquid crystal display, an organic EL display, or the like.

[0033] The operation unit 56 is configured with, for example, a touch panel or the like, and can accept operations on the information displayed on the display unit 55.

[0034] The storage unit 57 can be configured, for example, with a semiconductor memory or the like, and can store a computer program (program product) 58, a learning model 59, and required information.

[0035] The computer program 58 is a program that runs on a terminal device such as a mobile terminal such as a smartphone, a tablet terminal, or a PC, has a drip speed estimation function, and can be downloaded and installed from an external device via the communication unit 52. The computer program 58 is deployed in the memory 53 and executed by the control unit 51. That is, the processing by the control unit 51 is also the processing by the computer program 58. The computer program 58 may be stored in the storage unit 57 by reading the computer program 58 recorded on a recording medium (for example, an optically readable disk storage medium such as a CD-ROM) by a recording medium reading unit. Details of the learning model 59 will be described later.

[0036] When a medical professional performs a drip check operation, the medical professional may capture the drip tube 20 including the cap part 21 and the dripping part for a required time (for example, 10 to 20 seconds) with the imaging unit 54 of the drip rate estimation device 50. That is, the medical professional may capture a video so that the cap part 21 and the dripping part of the drip tube 20 are captured.

[0037] FIG. 4 is a diagram showing an example of the dripping part of the drip tube 20. The dripping part includes a conduit protruding below the cap part 21 of the drip tube 20, and can be the area shown by the dashed line in FIG. 4. The range of the dripping part may be a range in which the dripping from the conduit of the cap part 21 can be photographed, and the dripping falling from the conduit can be photographed. When a medical professional photographs the drip tube 20 including the dripping part with the imaging unit 54 of the drip speed estimation device 50, the control unit 51 can cut out (extract) a frame image of the drip part required for the drip speed estimation process.

[0038] When the control unit 51 (computer program 58) acquires a plurality of frame images in a time series from the imaging unit 54, the control unit 51 performs pre-processing (e.g., cutting out the image of the dripping part, resizing the image, etc.). Note that the cutting out of the image of the dripping part is not essential. After the pre-processing is completed, the control unit 51 performs a drip speed estimation process. The drip speed estimation process repeats each of the processes of (1) alignment process (color detection process of the cap area, center of gravity position estimation process of the cap area, and image stabilization process), (2) drop falling detection process, and (3) drip speed estimation process. Note that the (1) alignment process uses a plurality of frame images in a time series before the image of the dripping part is cut out. When the frame rate is 240 fps, the (1) alignment process and the (2) drop falling detection process are performed on a frame-by-frame basis (i.e., 240 times per second). Also, the (3) drip speed estimation process does not need to be performed on a frame-by-frame basis, and can be performed, for example, once per second. The following describes in detail (1) the alignment process, (2) the drop falling detection process, and (3) the drop speed estimation process.

[0039] Fig. 5 is a diagram showing an example of the alignment process. Fig. 5 illustrates the alignment process in frame (n) and the alignment process in frame (n+1). The process flow is the same in frame (n) and frame (n+1). First, the case of frame (n) will be described.

[0040] The control unit 51 acquires an image of a frame (n) of the drip tube 20 including the cap portion 21 and the dripping portion. In the example of FIG. 5, for convenience, the drip tube 20 on the frame image is shown in a state in which the upper part of the drip tube 20 is slightly tilted to the left. This represents camera shake at the time of shooting. The control unit 51 performs image processing, such as color detection, on the acquired image to identify the cap area. For example, if the color of the cap portion 21 is green and the color other than the cap portion 21 is a color other than green, the cap area can be identified by detecting the green area.

[0041] The control unit 51 estimates the center of gravity of the identified cap region. The center of gravity may be estimated by using a known method as appropriate. For example, a plurality of line segments having a long length among the line segments intersecting two opposing boundaries of the identified cap region are drawn, and the average values ​​of the positions (x coordinate, y coordinate) where the line segments intersect are set as the x coordinate and y coordinate of the center of gravity.

[0042] The control unit 51 aligns the estimated center of gravity position to a predetermined position on the frame image. In the example of Fig. 5, by aligning the center of gravity position to a predetermined position, the frame image in which the upper part of the drip tube 20 is slightly tilted to the left is in a state in which the drip tube 20 is straightened in the frame image after image stabilization.

[0043] Similarly, the control unit 51 acquires an image of frame (n+1) of the drip tube 20 including the cap portion 21 and the dripping portion. In the example of FIG. 5, for convenience, the drip tube 20 on the frame image is shown in a state in which the upper part of the drip tube 20 is slightly tilted to the right. This represents camera shake at the time of shooting. The control unit 51 performs image processing for color detection on the acquired image to identify the cap area.

[0044] The control unit 51 estimates the position of the center of gravity of the identified cap region. The estimation of the position of the center of gravity is similar to that in the case of frame (n).

[0045] The control unit 51 aligns the estimated center of gravity position to a predetermined position on the frame image. In the example of Fig. 5, by aligning the center of gravity position to a predetermined position, the frame image in which the upper part of the drip tube 20 is slightly tilted to the right is in a state in which the drip tube 20 is straightened in the frame image after image stabilization.

[0046] As described above, the control unit 51 can identify the cap area of ​​the drip tube 20 based on the multiple acquired frame images, estimate the center of gravity position of the identified cap area, and align the estimated center of gravity position to a predetermined position on each frame image.

[0047] With the above-mentioned configuration, when a medical professional such as a nurse photographs the drip tube 20 while holding the drip speed estimation device 50, the photographing time is 10 to 20 seconds, and even if camera shake occurs during photographing, the position and posture of the drip tube 20 can be photographed in a consistent state in all frames, thereby improving the estimation accuracy of the drip speed.

[0048] In addition, the control unit 51 can acquire a plurality of frame images in a time series in which the dripping portion is photographed with the background member 10 provided at a position outside the drip tube 20 corresponding to the dripping portion of the drip tube 20. This can improve the accuracy of identifying the cap area.

[0049] The center of gravity position can be estimated using a learning model 59.

[0050] FIG. 6 is a diagram showing an example of estimation of the center of gravity position by the learning model 59. The learning model 59 can use algorithms such as CNN (Convolutional Neural Networks), YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector). The learning model 59 is generated so that when an image including the cap region of the drip tube 20 is input, the learning model 59 outputs the center of gravity position of the cap region.

[0051] The control unit 51 can input the acquired multiple frame images to the learning model 59, acquire the center of gravity positions output by the learning model 59, and align the acquired center of gravity positions to predetermined positions on each frame image. The process of aligning the center of gravity positions to predetermined positions is the same as in the case of FIG.

[0052] The training data and evaluation data for learning (generating) the learning model 59 can include a plurality of images (still images) of the drip tube 20 to which the background member 10 is attached, for example, the right side surface, the left side surface, and the conduit of the cap part 21 taken from diagonally below, and teacher data in which the center of gravity position is annotated from the plurality of images. In addition, the captured images may be augmented (for example, rotated, moved, or changed in magnification of the image, etc.) to increase the amount of image data. The imaging direction of the drip tube 20 is not limited to three directions, but may be one or two directions, and the drip tube 20 may be photographed from multiple angles. This allows the collection of training data and evaluation data for learning the learning model 59, assuming a scene in which it will be used in an actual hospital ward.

[0053] Next, the process of detecting falling droplets will be described.

[0054] FIG. 7 is a diagram showing an example of detection of a falling drop. FIG. 7 shows the drip tube 20 in frames (n), ..., (n+5). In the figure, the rectangular area surrounded by the dashed line is the dripping portion. Droplet falling detection is performed by analyzing the differences in frame images in time series and searching for differences that can be determined as a falling drop. That is, the control unit 51 detects a falling drop based on the difference between the most recent frame difference and the most recent frame difference.

[0055] As shown in FIG. 7, the difference between the frame image of frame (n) and the frame image of frame (n+1) is calculated and set as the difference Δ(n+1). The area on the frame image for calculating the difference may be calculated using, for example, a cut-out image of the dripping portion (area indicated by the dashed line in the figure) cut out from the captured frame image. Similarly, the difference between the frame image of frame (n+1) and the frame image of frame (n+2) is calculated and set as the difference Δ(n+2), the difference between the frame image of frame (n+2) and the frame image of frame (n+3) is calculated and set as the difference Δ(n+3), the difference between the frame image of frame (n+3) and the frame image of frame (n+4) is calculated and set as the difference Δ(n+4), and the difference between the frame image of frame (n+4) and the frame image of frame (n+5) is calculated and set as the difference Δ(n+5).

[0056] In the example of FIG. 7, the difference Δ(n+1) is medium, the difference Δ(n+2) is medium, the difference Δ(n+3) is large, the difference Δ(n+4) is small, and the difference Δ(n+5) is small.

[0057] That is, the control unit 51 calculates a first difference in pixel values ​​of the dropping portion of each of the adjacent first and second frame images among the multiple frame images, and calculates a second difference in pixel values ​​of the dropping portion of each of the adjacent second and third frame images among the multiple frame images. If the first and second frame images are the frame images of frames (n) and (n+1), respectively, the first difference is the difference Δ(n+1). If the second and third frame images are the frame images of frames (n+1) and (n+2), respectively, the second difference is the difference Δ(n+2). The same applies below.

[0058] Next, the difference between the difference Δ(n+1) and the difference Δ(n+2) is set as the difference ΔΔ(n+2). Similarly, the difference between the difference Δ(n+2) and the difference Δ(n+3) is set as the difference ΔΔ(n+3), the difference between the difference Δ(n+3) and the difference Δ(n+4) is set as the difference ΔΔ(n+4), and the difference between the difference Δ(n+4) and the difference Δ(n+5) is set as the difference ΔΔ(n+5). If the calculated difference ΔΔ is, for example, equal to or greater than a predetermined threshold, it is determined that a droplet has fallen, thereby detecting the falling of the droplet.

[0059] 7, the difference ΔΔ(n+2) is small, the difference ΔΔ(n+3) is large, the difference ΔΔ(n+4) is large, and the difference ΔΔ(n+5) is small. In this case, frame (n+3) or frame (n+4) can be determined as the time when the drop fell.

[0060] That is, the control unit 51 can calculate the difference ΔΔ between the first difference Δ and the second difference Δ across a plurality of frame images, and detect the falling of a drop based on the calculated difference ΔΔ.

[0061] In addition, the control unit 51 calculates the difference ΔΔ between the first difference and the second difference across multiple frame images in which the center of gravity of the cap region is aligned to a predetermined position, and can detect the falling of the droplet based on the calculated difference ΔΔ. As a result, when a medical professional photographs the drip tube 20 while holding the drip speed estimation device 50, even if camera shake occurs during shooting, the position and posture of the drip tube 20 can be photographed in a consistent state in all frames, so that the falling of the droplet can be detected with high accuracy.

[0062] FIG. 8 is a diagram showing an example of a drop falling detection result. In FIG. 8, the vertical axis indicates the difference ΔΔ, and the horizontal axis indicates time. The line graph in FIG. 8 shows the difference ΔΔ at the time of sampling. In FIG. 8, the difference ΔΔ becomes equal to or exceeds a predetermined threshold (however, on the + side, equal to or exceeds the + threshold, and on the - side, equal to or less than the - threshold) at the timing shown by the dashed line, and indicates the timing at which the drop falling is detected. The method of determining the reliability of the drop falling detection result is not limited to a simple threshold determination, but can also apply, for example, pattern matching that takes into account the preceding and following waveforms, machine learning, and the like. The sampling period may be 240 fps, or the number of samples may be thinned out to make it smaller than 240 fps.

[0063] Next, the dropping speed estimation process will be described.

[0064] FIG. 9 is a diagram showing an example of a method for estimating a drop speed. In FIG. 9, symbols D1, D2, ..., D8, ... indicate the drop detection time points. The drop interval between the drop drop time points Dj and Dk is represented by Djk. In FIG. 9, the drop interval D12 is the time interval between the drop time points D1 and D2. The same is true for the other drop intervals D23, D34, D45, D56, D67, D78, .... The drop speed V can be calculated by the formula V = (D12 + D23 + D34 + D45 + D56 + D67 + D78 + ...) / (total number of drop intervals within the required time). That is, the drop speed V can be calculated by the average value of the drop intervals detected during the required time (for example, 10 to 20 seconds).

[0065] The drop speed V is not limited to the average value of the drop intervals, and may be estimated based on other statistical values, such as the median value of the drop intervals detected during the required time.

[0066] As described above, the control unit 51 can calculate the intervals between drops across a plurality of frame images based on the detected drops, and estimate the dropping speed based on the statistics of the calculated intervals between drops.

[0067] According to this embodiment, the control unit 51 can acquire a plurality of frame images in a time series obtained by photographing the dripping portion of the drip tube 20 with the imaging unit 54 of the dripping speed estimation device (terminal device) 50, detect the falling of the drops at the dripping portion based on the acquired plurality of frame images, and estimate the dripping speed based on the detected falling of the drops. In addition, the control unit 51 can display the estimated dripping speed on the display unit 55 of the dripping speed estimation device 50.

[0068] This eliminates the need for medical staff such as nurses to visually count the number of drops while checking their wristwatches, and allows medical staff to check the drip rate simply by photographing the drip tube 20 for the required time with a portable terminal device, thereby reducing the work of checking the drip rate.

[0069] FIG. 10 is a diagram showing an example of a display screen on the display unit 55. In FIG. 10A, when a medical professional starts photographing the drip tube 20 with the imaging unit 54 of the drip rate estimation device 50, the control unit 51 displays characters such as "Drip measurement in progress" on the display unit 55. This allows the medical professional to confirm that the measurement of the drip rate has started correctly. In the process of photographing the required time, when a drop is detected, the control unit 51 switches to display characters such as "Drip detection" and changes the display mode of all or part of the display screen displayed on the display unit 55 to attract the attention of the medical professional. In the example of FIG. 10A, the characters "Drip detection" on the display screen are displayed in a manner such that they flash or the screen color around the characters is repeatedly changed.

[0070] That is, when the control unit 51 detects the drop of the droplet, the control unit 51 may change the display mode of the display unit 55. The time for changing the display mode and displaying the droplet may be, for example, about one second from the drop drop point, but is not limited to this. This allows the medical staff to easily confirm the drop drop.

[0071] As shown in FIG. 10B, when the control unit 51 estimates the drip rate, the estimated drip rate may be displayed (output) in association with the patient ID of the patient receiving the drip from the drip tube 20, the nurse ID (medical worker ID) of the nurse (medical worker) operating the drip tube 20, and the estimated date and time of the drip rate. The patient ID may be read in advance from the patient's chart or name tag. The nurse ID may be read in advance from a name tag worn by the nurse. The control unit 51 may also specify the reliability of the drop detection result and display the reliability on the display unit 55 together with the estimated drip rate. This allows the confirmation of who performed the drip confirmation work for which patient and when, and what the results were, in association with each other, and allows the confirmation to be recorded. The information exemplified in FIG. 10B may not only be displayed on the display unit 55, but may also be recorded in an electronic medical record or the like via an in-hospital communication network.

[0072] 11 is a diagram showing an example of a processing procedure by the dropping speed estimation device 50. For convenience, the following description will be given with the control unit 51 acting as the subject of the processing. The control unit 51 acquires a frame image (S11), and performs preprocessing on the acquired frame image (S12). The preprocessing includes, for example, changing the image size, cutting out the image of the dropping part, and the like.

[0073] The control unit 51 estimates the position of the center of gravity of the cap area of ​​the frame image (S13), aligns the position of the center of gravity to a predetermined position, and performs camera shake correction (S14). The control unit 51 calculates the difference Δ in the dripping part between the frame images after camera shake correction (S15), and calculates the difference ΔΔ between the differences Δ between the frame images (S16).

[0074] The control unit 51 identifies the time when the droplet falls depending on whether the calculated difference ΔΔ exceeds a predetermined threshold range (+Th to -Th) (S17), and judges whether it is the timing to estimate the drip speed (S18). The timing to estimate the drip speed can be determined in advance, for example, every second. If it is not the timing to estimate the drip speed (NO in S18), the control unit 51 continues the process from step S11 onwards.

[0075] If it is time to estimate the dropping speed (YES in S18), the control unit 51 estimates the dropping speed based on the interval between falling drops (S19), outputs the estimated dropping speed (S20), and ends the process. [Explanation of symbols]

[0076] 10 Background material 11 Side part 12 Top part 13 Insertion hole 20 IV tube 21 Cap section 23, 24 Tube 50 Dripping speed estimation device 51 Control section 52 Communications Department 53 Memory 54 Imaging unit 55 Display section 56 Operation section 57 Memory section 58 Computer Programs 59 Learning Model

Claims

1. A computer program that operates on a terminal device, A plurality of frame images in a time series obtained by photographing the dripping portion of the drip tube with the imaging unit of the terminal device are acquired, Detecting the falling of the droplet at the dripping portion based on the acquired plurality of frame images; Estimating a drop velocity based on the detected drop fall; A computer program that causes a computer to carry out processing.

2. calculating a first difference between pixel values ​​of the dropping portion of each of a first frame image and a second frame image adjacent to each other among the plurality of frame images; calculating a second difference between pixel values ​​of the dropping portion of each of the second and third frame images adjacent to each other among the plurality of frame images; Calculating a difference between the first difference and the second difference across the plurality of frame images; Detecting the falling of a drop based on the calculated difference.

2. A computer program product according to claim 1, which causes a computer to carry out a process.

3. Calculating intervals between drops across the plurality of frame images based on the detected drops; The calculated fall interval data is stored. A drop speed is estimated based on a statistical value obtained by statistically processing the stored drop interval data.

3. A computer program product according to claim 2, which causes a computer to carry out a process.

4. Identifying a cap region of the drip tube based on the acquired plurality of frame images; A center of gravity position of the identified cap region is estimated, Align the estimated center of gravity position to a predetermined position on each frame image.

3. A computer program product according to claim 2, which causes a computer to carry out a process.

5. When an image including a cap region of a drip tube is input, the acquired multiple frame images are input to a learning model that outputs the center of gravity position of the cap region to obtain the center of gravity position, The obtained center of gravity position is aligned to a predetermined position on each frame image.

3. A computer program product according to claim 2, which causes a computer to carry out a process.

6. calculating a difference between the first difference and the second difference across a plurality of frame images in which the center of gravity of the cap region is aligned at the predetermined position; Detecting the falling of a drop based on the calculated difference.

6. A computer program product according to claim 4 or 5, which causes a computer to execute a process.

7. A plurality of frame images in a time series in which the dripping portion is photographed in a state where a background member is provided outside the drip tube at a position corresponding to the dripping portion of the drip tube, is acquired. A computer program product according to any one of claims 1 to 5, which causes a computer to carry out a process.

8. The estimated drip rate is output in association with the patient ID of the patient receiving the drip from the drip tube, the medical worker ID of the medical worker operating the drip tube, and the estimated date and time of the drip rate. A computer program product according to any one of claims 1 to 5, which causes a computer to carry out a process.

9. The estimated drip rate is displayed on a display unit of the terminal device. A computer program product according to any one of claims 1 to 5, which causes a computer to carry out a process.

10. When a drop is detected, the display mode of the display unit is changed.

10. A computer program product according to claim 9, which causes a computer to carry out a process.

11. determining a confidence level of the drop detection result; and displaying the reliability together with the estimated drip rate on the display unit.

10. A computer program product according to claim 9, which causes a computer to carry out a process.

12. A control unit is provided, The control unit is A plurality of frame images are acquired in a time series by photographing the dripping portion of the drip tube with an imaging unit, Detecting the falling of the droplet at the dripping portion based on the acquired plurality of frame images; Estimating a drop velocity based on the detected drop fall; Dripping rate estimation device.

13. A plurality of frame images are acquired in a time series by photographing the dripping portion of the drip tube with an imaging unit, Detecting the falling of the droplet at the dripping portion based on the acquired plurality of frame images; Estimating a drop velocity based on the detected drop fall; Dripping rate estimation method.

14. A background member that is detachable from the drip tube, The drip tube has a shape larger than the shape of the side surface thereof, A side surface portion facing the side surface side of the drip tube and an upper surface portion facing the upper surface side of the drip tube and bent in an L-shape with respect to the side surface portion, The upper surface portion has an insertion hole through which the tube of the drip tube is inserted, All or a part of the side surface is colored in a color opposite to the color of the cap region of the drip tube. Background material.

Images