Radiation imaging apparatus having camera and image processing method

Abstract

To provide a radiation imaging apparatus that allows a doctor to recognize anesthesia termination without interrupting an operation of an endoscope or the like.SOLUTION: A radiation detection apparatus includes a camera to image a subject. From images taken by a camera, images of multiple portions of a subject are extracted and a movement amount is calculated for each portion. The calculated movement amount for each portion is weighted by a predetermined weight for each of the multiple portions, and if the movement amount after weighting exceeds a predetermined threshold, an operator is notified of this information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a radiation imaging apparatus that irradiates a subject with radiation and obtains an image. [Background technology]

[0002] Interventional radiology (IVR) is a known technique in which an endoscope or catheter is inserted into a patient to perform various procedures on the patient while irradiating the patient with radiation from a radiation imaging device to obtain fluoroscopic images. In IVR, when a doctor wants to check the patient's condition, he or she stops manipulating the catheter or endoscope and visually checks the patient's face, feet, or hands. Generally, a monitor that displays images captured by the endoscope and a monitor that displays fluoroscopic images are placed in front of the doctor, and the subject is placed to the side of the doctor. For this reason, in order to check the patient's condition, it is necessary to move one's gaze from the monitor to the patient.

[0003] Patent Document 1 discloses a system for a doctor or technician operating an X-ray CT scanner in a console room next to a gantry room to check the condition of the patient while taking images of the patient's condition using the X-ray CT scanner installed in the gantry room. In this system, the patient is photographed using the X-ray CT scanner or a camera installed in its vicinity, and the photographed images are displayed on a part of the operation screen in the console room. Specifically, a mode in which multiple cameras are used to photograph multiple parts of the patient, and multiple display areas are provided on the operation screen to display multiple images, or a mode in which one image is displayed is disclosed (see Figs. 3(a) and (b) of Patent Document 1). It also discloses detecting the edge of the image of the patient in the image, detecting the presence or absence of body movement from changes in the width direction and thickness direction, and displaying the detected edge together with the image. This allows the operator to check the condition of the patient from the image displayed on a part of the operation screen while operating the operation console, and to know the presence or absence of body movement. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2004-208954 A Summary of the Invention [Problem to be solved by the invention]

[0005] One of the reasons why doctors visually check on a patient during an IVR procedure is to check whether the anesthesia is wearing off, but in order for a doctor to directly check on a patient's condition, they need to temporarily stop operating the endoscope or other device and move their gaze from the monitor to the patient.

[0006] In addition, in the technology of Patent Document 1, images of multiple parts of a patient taken by multiple cameras are displayed on the operation screen of the operation console, so the area on the operation screen for displaying images taken by the endoscope and the area for displaying fluoroscopic images are small. On the other hand, if the part of the patient displayed is limited to one, it becomes difficult for a doctor to determine whether the anesthesia is wearing off based on only that part.

[0007] An object of the present invention is to provide a radiation imaging apparatus that allows a doctor to recognize the wearoff of anesthesia without interrupting the operation of an endoscope or the like. [Means for solving the problem]

[0008] In order to achieve the above object, the present invention provides a radiation imaging device having a tabletop on which a subject is placed, a radiation source which irradiates radiation onto the subject, a radiation detector which detects radiation irradiated from the radiation source and transmitted through the subject, a camera which images the subject, and a motion amount calculation unit which processes the images captured by the camera to calculate the amount of motion of the subject. The motion amount calculation unit extracts images of a plurality of predetermined parts of the subject from the images captured by the camera, and calculates the amount of motion for each part. The motion amount calculation unit weights the calculated amount of motion for each part with a weight which has been predetermined for each of the plurality of parts, and if the weighted amount of motion exceeds a predetermined threshold, notifies the operator of this fact. Effect of the Invention

[0009] According to the present invention, it is possible to provide a radiation imaging apparatus that allows a doctor to recognize the wearoff of anesthesia without interrupting the operation of an endoscope or the like. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a front view of a radiation irradiation device 1 according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a functional block diagram of the radiation irradiation device 1 according to the first embodiment. [Diagram 3] 4 is a flowchart showing the operation of the radiation irradiation apparatus 1 of the first embodiment. [Figure 4] 3A and 3B are diagrams showing examples of images on a display monitor of the radiation irradiation apparatus 1 of the first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of a display screen of the radiation irradiation apparatus 1 of the first embodiment. [Figure 6] 10 is a flowchart showing the operation of a radiation irradiation apparatus according to a second embodiment. [Figure 7] 11 is a flowchart showing the operation of a radiation irradiation device according to a third embodiment. [Figure 8] FIG. 11 is a diagram showing an example of an image on a display monitor of a radiation irradiation apparatus according to a third embodiment. [Figure 9] 10 is a flowchart showing the operation of a radiation irradiation device according to a fourth embodiment. [Figure 10] FIG. 13 is a diagram showing an example of an image on a display monitor of a radiation irradiation apparatus according to a fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, a radiation irradiation apparatus according to an embodiment of the present invention will be described with reference to the drawings.

[0012] <Embodiment 1> The radiation irradiating device 1 of the embodiment 1 will be described with reference to Fig. 1 to Fig. 3. Fig. 1 is a front view of the radiation irradiating device 1, Fig. 2 is a functional block diagram of the radiation irradiating device 1, and Fig. 3 is a flowchart showing the operation of the radiation irradiating device 1.

[0013] 1, the radiation irradiation device 1 includes a tabletop 80 on which a subject 10 is placed, and a radiation source 11 that irradiates the subject 10 with X-rays. The radiation source 11 is supported on a stand 70 by a radiation source support part 50. The stand 70 also includes a radiation source mechanism part 60 that moves the support part 50 up and down.

[0014] Meanwhile, the tabletop 80 is supported on the stand 70 by a tabletop support part 81. Inside the tabletop support part 81, a mechanism part 82 that moves the position of the tabletop 80 in the longitudinal direction of the tabletop 80 is built in.

[0015] The radiation detector 12 is disposed inside the top board support part 81. The radiation detector 12 has radiation detection elements arranged two-dimensionally, and detects X-rays that are irradiated from the radiation source 11 and transmitted through the subject 10.

[0016] A surveillance camera 21 that captures an image of the subject 10 is supported on the radiation source support part 50. The angle of view of the surveillance camera 21 is set so that the entire body of the subject 10 placed on the top board 80 is captured, as shown in FIG. 4 . The surveillance camera 21 may be fixed to a cover (housing) of the radiation source 11.

[0017] 2, an image processor 13 is connected to the X-ray detector 12, and receives signals output by each radiation detection element of the X-ray detector 12 upon detecting X-rays, and generates an X-ray image (here, a fluoroscopic image) at a predetermined frame rate. A display monitor 14 is connected to the image processor 13.

[0018] 5 shows an example of the display screen of the display monitor 14. The perspective image is displayed in an area 14a of the display screen.

[0019] On the other hand, a motion amount calculation section 20 is connected to the surveillance camera 21. The motion amount calculation section 20 processes the image captured by the surveillance camera 21 and calculates the amount of motion of the subject 10.

[0020] The motion amount calculation section 20 includes an image capture section 22, a posture detection section 23, a motion amount detection section 24, a weight storage section 26, a weighting section 25, a determination section 27, and a motion amount map generation section .

[0021] The image receiving unit 22 captures the images 21a output by the monitoring camera 21 at a predetermined time interval. The predetermined time interval can be set to be approximately the same as the frame rate of the perspective images generated by the image processing unit 13.

[0022] The posture detection unit 23 extracts a plurality of feature points (head, spine, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left ankle, etc.) of predetermined joints of the image of the subject 10 in the image 21a, and detects the two-dimensional coordinates of the extracted feature points, thereby detecting the posture of the subject 10. The posture detection unit 23 can be realized by a known posture detection algorithm. For example, the posture estimation algorithm Open Pose (registered trademark) included in Open VINO (registered trademark) provided by Intel Corp can be used.

[0023] The motion amount detection unit 24 calculates the amount of motion for each of a plurality of predetermined parts (head, torso, right arm, left arm, right leg, left leg) of the subject 10 from the coordinates of the feature points of the subject 10 detected by the posture detection unit 23. For example, the image capture unit 22 calculates the difference in coordinates of a plurality of feature points (head, spine, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left ankle, etc.) between the image 21a captured this time and the image 21a captured last time, and can use this as the amount of motion for each part. For example, the difference in coordinates of the head between the previous image 21a and the current image is used as the amount of motion for the head. The difference in coordinates of the spine between the previous image 21a and the current image is used as the amount of motion for the torso. The maximum value of the differences in coordinates of the right shoulder, right elbow, and right wrist between the previous image 21a and the current image is used as the amount of motion for the right arm, and the maximum value of the differences in coordinates of the left shoulder, left elbow, and left wrist between the previous image 21a and the current image is used as the amount of motion for the left arm. In addition, the maximum value among the differences in the coordinates of the right hip, right knee, and right ankle between the previous and current images 21a is regarded as the amount of movement of the right foot, and the maximum value among the differences in the coordinates of the left hip, left knee, and left ankle is regarded as the amount of movement of the left foot.

[0024] The weighting unit 25 weights the amount of movement of each part (head, torso, right arm, left arm, right leg, left leg) detected by the movement amount detection unit 24 with a weight that is stored in advance for each part in the weight storage unit 26. The weight of each part stored in the weight storage unit 26 is Weight value of leg movement > Weight value of head movement ≥ Weight value of torso movement ≥ Weight value of arm movement As an example, the weight value of the movement of the right leg and the left leg is set to 1.2, the weight value of the movement of the head is set to 0.8, the weight value of the movement of the torso is set to 0.8, and the weight value of the movement of the right arm and the left arm is set to 0.8.

[0025] The weight of the legs is set larger than that of other parts because the closer to the extremities of the body, the larger the amount of movement when the anesthesia is about to wear off. Therefore, by increasing the weight of the extremities (arms, legs), it is possible to easily determine whether the anesthesia is about to wear off. On the other hand, in the intraoperative image 21a, the image of the doctor often overlaps with the image of the subject's 10's arms and torso, and the posture detection unit 23 may not be able to accurately detect the characteristic points of the arms and torso. Therefore, although the arms are extremities, the weight value of the amount of movement of the arms is reduced to the same level as the weight value of the amount of movement of the torso. As a result, the weight value of the amount of movement of the legs is set larger than that of other parts, and the amount of movement of the legs is emphasized to determine whether the anesthesia is about to wear off.

[0026] The weighting unit 25 can correct the weights stored in the weight storage unit 26 according to at least one of the height of the subject 10 currently placed on the top board 80 and a predetermined index representing the body size of the subject 10. For example, in the case of a small child, the weight value of the head movement amount may be corrected to be as large as the foot movement amount. In this case, the height of the subject 10 is detected from the image 21a captured by the camera 21, and if the height is smaller than a predetermined threshold (for example, 130 cm), it is determined that the subject is a child, and the weight value of the head movement amount can be corrected to be large. It is also possible to determine that the subject 10 is a child if not only the height but also the surface area of ​​the subject 10 in the image 21a is smaller than a predetermined threshold. In addition to children, in the case of a subject 10 whose height or body surface area is significantly larger than that of an average adult, for example, the weight value of the feet may be corrected to be lowered to the same level as the arms.

[0027] When one or more of the weighted movement amounts of each part exceeds a predetermined threshold, the determination unit 27 notifies the operator of that fact. For example, the determination unit 27 issues an alarm sound from the connected speaker 29 to alert the operator that the anesthesia is running out. Note that the method of notification is not limited to an alarm sound, and it is of course possible to use other methods such as displaying a warning on the display monitor 14.

[0028] The motion amount map generating unit 28 selects a color that is predetermined to correspond to the amount of motion for each part after weighting calculated by the weighting unit 25, and colors the map showing the shape of the subject 10 for each part. In this way, a map is generated in which the amount of motion of each part is represented by color. The colors selected are colors with different hues or brightness that are predetermined according to the amount of motion.

[0029] The motion amount map generating unit 28 displays the generated map in a predetermined area 14b of the display monitor 14 as shown in Fig. 4. This allows the operator to grasp the amount of motion of each part from the color of the map while looking at the display monitor 14 on which the fluoroscopic image is displayed, without the need to move his or her line of sight.

[0030] Hereinafter, the operation of each unit of the motion amount calculation unit 20 will be described with reference to the flow chart of FIG.

[0031] (Step 101) When the doctor starts inserting the endoscope or catheter into the subject 10, the image capturing unit 22 starts the operation of capturing images from the monitoring camera 21 at a predetermined time interval. The image capturing unit 22 may process the captured images and determine from the doctor's movements that the doctor has started inserting the endoscope or catheter into the subject 10, or may determine that the doctor has started inserting the endoscope or the like when the image capturing unit 22 detects that the radiation source 11 has started emitting X-rays.

[0032] (Step 102) The image capture unit 22 waits until a predetermined time has elapsed since starting image capture in step 101, and when the predetermined time has elapsed, proceeds to step 103. The reason for waiting until the predetermined time has elapsed here is that immediately after starting the insertion of an endoscope or the like, it is immediately after anesthesia has been administered, and therefore there is no need to determine whether anesthesia has worn off.

[0033] (Step 103) After a predetermined time has elapsed, the image capture unit 22 outputs the captured image 21a to the posture detection unit 23. The posture detection unit 23 extracts a plurality of feature points (head, spine, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left ankle, etc.) of the image of the subject 10 in the image 21a, and detects their two-dimensional coordinates.

[0034] (Step 104) The movement amount detection unit 24 detects the amount of movement of each of multiple predetermined parts of the subject 10 (head, torso, right arm, left arm, right leg, left leg) from the coordinates of the feature points of the subject 10 detected by the posture detection unit 23.

[0035] (Steps 105 and 106) Here, the movement amount detection unit 24 detects whether the mechanism unit 82 of the tabletop 80 is operating, and if it is operating, stops detecting the amount of movement and waits until the mechanism unit 82 stops.

[0036] (Step 107) The weighting unit 25 weights the amount of movement of each part (head, torso, right arm, left arm, right leg, left leg) detected by the movement amount detection unit 24 in step 104 using weights pre-stored for each part in the weight memory unit 26.

[0037] At this time, the weighting section 25 may detect the height of the subject 10 and correct the weights in the weight storage section 26 according to the detection results before weighting.

[0038] (Step 108) The motion amount map generating unit 28 generates a map in which the motion amount of each part weighted in step 107 is represented by color.

[0039] (Step 109) The determination unit 27 determines whether the weighted amount of movement of each part exceeds a predetermined threshold value. If one or more of the weighted amounts of movement of each part exceeds the threshold value, the process proceeds to step 110.

[0040] (Step 110) The judgment unit 27 issues an alarm sound from the connected speaker 29. This notifies the operator of the possibility of anesthesia wearoff.

[0041] The above steps 103 to 110 are repeatedly executed every time the image capture unit 22 captures an image at a predetermined rate.

[0042] As described above, by using the radiation imaging apparatus of the first embodiment, a doctor can recognize the end of anesthesia by an alarm sound or a display screen without interrupting the operation of an endoscope or the like.

[0043] Furthermore, the radiation imaging apparatus of the first embodiment can sense the amount of movement of the subject 10 only during a time period when there is a risk of anesthesia wearing off after a predetermined time has elapsed since the start of the IVR, calculate the amount of movement, and display a map, rather than during the entire IVR process. Also, an alarm can be issued when the amount of movement exceeds a threshold. This makes it possible to detect movement specifically related to anesthesia wearing off.

[0044] Furthermore, the doctor can grasp the amount of movement of the patient without moving his / her line of sight from the display monitor on which the fluoroscopic image is displayed.

[0045] In addition, since the amount of movement of the image 21a captured by the monitoring camera 21 is weighted according to the part, the amount of movement of the extremities of the subject 10, especially the legs, can be weighted heavily, and the amount of movement of other parts can also be used as judgment information to judge whether the anesthesia is starting to wear off and notify the doctor. In the image of the monitoring camera 21, the legs are less likely to be covered by the image of the doctor, and the detection accuracy is less likely to decrease compared to the arms, so that the wearoff of the anesthesia can be judged with high accuracy.

[0046] <Embodiment 2> The radiation irradiation apparatus of the second embodiment will be described with reference to the flow chart of FIG.

[0047] The configuration of the radiation irradiation apparatus of the second embodiment is similar to that of the radiation irradiation apparatus of the first embodiment, but some of the operations are different. Only the different parts will be described below with reference to the flow of FIG.

[0048] In embodiment 1, as in steps 105 and 106 of the flow in FIG. 3, the motion amount detection unit 24 detects whether the mechanism unit 82 of the tabletop 80 is operating, and if the mechanism unit 82 is operating, it interrupts detection of the amount of motion and waits until the mechanism unit 82 stops.

[0049] In the second embodiment, in step 105, the movement amount detection unit 24 detects whether the mechanism unit 82 of the tabletop 80 is operating, and if the mechanism unit 82 is operating, the process proceeds to step .

[0050] In step 116, the movement amount detection unit 24 further detects the direction and amount of movement in which the mechanism unit 82 moves the top board 80, and corrects the amount of movement by subtracting the amount of movement of each part of the subject 10 (head, torso, right arm, left arm, right leg, left leg) caused by the movement of the top board 80 from the amount of movement detected in step 104.

[0051] In step 107, weighting section 25 weights the amount of movement of each part corrected by movement amount detection section 24 in step 116 with the weight stored in advance in weight storage section 26 for each part.

[0052] The other steps are the same as those in the first embodiment, so the explanation will be omitted.

[0053] In this way, the radiation detection device of the second embodiment can correct and remove the amount of movement of the subject 10 caused by the movement of the top board 80 without interrupting detection of the amount of movement even while the top board 80 is moving, and can create and display a motion amount map based on the amount of movement after the correction. Furthermore, if the amount of movement after the correction exceeds a threshold, an alarm can be set to notify the doctor of the possibility of anesthesia wearing off.

[0054] Therefore, even during surgery in which the tabletop is moved frequently, the amount of movement of each part can be calculated and displayed without interruption. Also, the possibility of anesthesia wearoff can be notified.

[0055] <Embodiment 3> The radiation irradiation device of the third embodiment will be described with reference to the flow chart of FIG. 7 and the example display screen of FIG.

[0056] The configuration of the radiation irradiation device of the third embodiment is similar to that of the radiation irradiation device of the first embodiment, but differs in the method of displaying the amount of movement on the display monitor 14. Only the different parts will be described below with reference to FIG.

[0057] In the first embodiment, in step 108 of FIG. 3, a map showing the amount of movement of each part with a color is generated and displayed in area 14b of the display monitor 14. In the third embodiment, however, in step 118 instead of step 108, an image of a meter showing the amount of movement after weighting is displayed in area 14b of the display monitor 14 as shown in FIG. 8.

[0058] In the example of Figure 8, the meter image shows a horizontal bar indicating the amount of motion with a vertical line indicating the weighting amount calculated in step 107, but the meter may have a different shape.

[0059] As shown in FIG. 8, only one meter may be displayed on the display monitor 14 to display the maximum value of the amount of movement of each part, or a meter may be displayed for each part.

[0060] As in the third embodiment, by expressing the amount of movement by a meter, there is an advantage that the doctor can easily grasp the magnitude of the amount of movement.

[0061] <Embodiment 4> The radiation irradiation device of the fourth embodiment will be described with reference to the flow chart of FIG. 9 and the example display screen of FIG.

[0062] The configuration of the radiation irradiation apparatus of the fourth embodiment is similar to that of the radiation irradiation apparatus of the first embodiment, but the method of displaying the amount of movement on the display monitor 14 is different.

[0063] In the third embodiment, as shown in the flow of FIG. 9, step 128 is executed instead of step 108 in the first embodiment, and the amount of motion after weighting is displayed as shown in FIG. 10 by the thickness of frame lines 201a, 201b indicating the part of the subject 10 on the image 21a.

[0064] For example, in the example of FIG. 10, the weighted movement amounts of the left arm and left leg are calculated in step 107, and since the movement amount of the left leg is greater than the movement amount of the left arm, a thick frame 201a is displayed at the position of the left leg on image 21a, and a frame 201b thinner than the left leg is displayed at the position of the left arm.

[0065] 10, the image 21a is displayed alone, which has the advantage that the amount of movement of the subject 10 can be easily understood from the large image 21a. Of course, the image 21a with the frame lines 201a and 201b displayed can also be displayed side by side with the perspective images. [Explanation of symbols]

[0066] 1 Radiation irradiation device 10 Subject 11 Radiation source 12 Line Detector 13 Image processing section 14 Display Monitor 14a area 14b area 20 Motion amount calculation unit 21 Camera 21a Image 22 Image capture section 23 Attitude detection unit 24 Motion detection section 25 Weighting section 26 Memory section 27 Judgment section 28 Motion map generator 29 Speaker 50 Radiation source support part 60 Radiation source mechanism section 70 Stand 80 Tabletop 82 Mechanism 201a Border 201b Border

Claims

1. a top plate on which an object is placed, a radiation source which irradiates radiation onto the object, a radiation detector which detects the radiation which is irradiated from the radiation source and which has passed through the object, a camera which images the object, and a motion amount calculation unit which processes the image captured by the camera to calculate a motion amount of the object, The motion amount calculation unit extracts images of a plurality of predetermined parts of the subject from the image captured by the camera, calculates the amount of motion for each part, weights the calculated amount of motion for each part with a weighting factor that has been determined in advance for each of the plurality of parts, and, if the weighted amount of motion exceeds a predetermined threshold, notifies an operator of this.

2. 2. The radiation imaging apparatus according to claim 1, wherein the motion amount calculation unit detects motions of the subject's head, torso, arms, and legs as the plurality of parts of the subject, and the weights to be assigned are Weight value of leg movement amount > Weight value of head movement amount ≥ Weight value of torso movement amount ≥ Weight value of arm movement amount A radiation imaging device equipped with a camera, characterized in that:

3. 2. The radiation imaging device according to claim 1, wherein the motion amount calculation unit corrects the weighting in accordance with at least one of a height of the subject currently placed on the tabletop and a predetermined index representing a body size of the subject.

4. 2. The radiation imaging apparatus according to claim 1, wherein the motion amount calculation unit calculates an index representing a height of the subject or a body size of the subject using an image of the subject captured by the camera.

5. 2. The radiation imaging apparatus according to claim 1, wherein the motion amount calculation unit issues an alarm sound from a connected speaker in order to notify the operator.

6. 2. The radiation imaging device according to claim 1, wherein the motion amount calculation unit displays on a display device a map in which the motion amount of the part multiplied by the weight is represented by color, or an image of a scale representing the motion amount multiplied by the weight, in order to notify the operator.

7. 2. The radiation imaging apparatus according to claim 1, further comprising a mechanism for moving the position of the top plate, 4. A radiation imaging apparatus including a camera, wherein the motion amount calculation section suspends calculation of the motion amount when it detects that the mechanism section is operating.

8. 2. The radiation imaging apparatus according to claim 1, further comprising a mechanism for moving the position of the top plate, A radiation imaging device equipped with a camera, characterized in that when the motion amount calculation unit detects that the mechanism unit is moving the top plate, it corrects the motion amount calculated for the part in accordance with the movement amount of the top plate.

9. 2. The radiation imaging apparatus according to claim 1, further comprising a support portion for supporting the radiation source, A radiation imaging device equipped with a camera, characterized in that the camera is provided on the support part.

10. An image processing method for a radiation imaging device equipped with a camera, comprising: extracting images of a plurality of predetermined parts of the subject from the images captured by the camera and calculating an amount of movement for each of the plurality of parts; weighting the calculated amount of movement for each part with a weight that is predefined for each of the multiple parts; and if the weighted amount of motion exceeds a predetermined threshold, notifying the operator of that fact.

23. A method for processing an image of a radiation imaging apparatus equipped with a camera, comprising:

Images